All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v3 0/8] ARM: KVM: Support for vgic-v3
@ 2016-09-08 16:06 ` Vladimir Murzin
  0 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-08 16:06 UTC (permalink / raw)
  To: kvmarm; +Cc: marc.zyngier, andre.przywara, linux-arm-kernel

Hi,

This is an attempt to make use vgic-v3 under arch/arm since
save-restore functionality got re-written in C and can be shared
between arm/arm64 like it has already been done for vgic-v2 and timer.

With this patches I'm able to get 32 core an AArch32 ARMv8 guest boot:

...
GICv3: CPU31: found redistributor 703 region 0:0x000000003ffd0000
CPU31: thread -1, cpu 3, socket 7, mpidr 80000703
Brought up 32 CPUs
SMP: Total of 32 processors activated (768.00 BogoMIPS).
CPU: All CPU(s) started in SVC mode.
...

Additionally, quite lightweight test based on Self IPI guest test[1]
has been run with up to 255 cpus.

[1] http://www.spinics.net/lists/kvm/msg128974.html

Changelog:

    v2 -> v3
       - rebased on v4.8-rc5
       - commit messages are reworked to be more precise and clear (I
         hope so)
       - MPIDR_HWID_BITMASK is used to discard Aff3 in
         MPIDR_AFFINITY_LEVEL macro
       - cast to u64 is used instead of abuse of GENMASK_ULL while
         building value for typer register
       - static keys are used to select GIC backend
       - config option to guard ITS code is moved to separate patch

    v1 -> v2
       - rebased on v4.8-rc2
       - introduced guard for ITS code

Thanks!

Vladimir Murzin (8):
  arm64: KVM: Use static keys for selecting the GIC backend
  arm64: KVM: Move GIC accessors to arch_gicv3.h
  arm64: KVM: Move vgic-v3 save/restore to virt/kvm/arm/hyp
  KVM: arm64: vgic-its: Introduce config option to guard ITS specific
    code
  KVM: arm: vgic: Support 64-bit data manipulation on 32-bit host
    systems
  ARM: Change MPIDR_AFFINITY_LEVEL to ignore Aff3
  ARM: Move system register accessors to asm/cp15.h
  ARM: KVM: Support vgic-v3

 arch/arm/include/asm/arch_gicv3.h                 |   91 +++++++++++++++++----
 arch/arm/include/asm/cp15.h                       |   15 ++++
 arch/arm/include/asm/cputype.h                    |    3 +-
 arch/arm/include/asm/kvm_asm.h                    |    3 +
 arch/arm/include/asm/kvm_host.h                   |   13 +++
 arch/arm/include/asm/kvm_hyp.h                    |   20 +----
 arch/arm/include/asm/virt.h                       |    8 ++
 arch/arm/include/uapi/asm/kvm.h                   |    7 ++
 arch/arm/kernel/vmlinux.lds.S                     |    6 ++
 arch/arm/kvm/Makefile                             |    2 +
 arch/arm/kvm/arm.c                                |   19 +++++
 arch/arm/kvm/coproc.c                             |   36 ++++++++
 arch/arm/kvm/hyp/Makefile                         |    1 +
 arch/arm/kvm/hyp/switch.c                         |   12 ++-
 arch/arm64/include/asm/arch_gicv3.h               |   13 +++
 arch/arm64/include/asm/kvm_host.h                 |   15 ++++
 arch/arm64/include/asm/kvm_hyp.h                  |    2 -
 arch/arm64/include/asm/virt.h                     |    7 ++
 arch/arm64/kernel/vmlinux.lds.S                   |    6 ++
 arch/arm64/kvm/Kconfig                            |    4 +-
 arch/arm64/kvm/hyp/Makefile                       |    2 +-
 arch/arm64/kvm/hyp/switch.c                       |   19 ++---
 include/kvm/arm_vgic.h                            |    8 --
 {arch/arm64/kvm => virt/kvm/arm}/hyp/vgic-v3-sr.c |   13 ---
 virt/kvm/arm/vgic/vgic-kvm-device.c               |   11 +--
 virt/kvm/arm/vgic/vgic-mmio-v3.c                  |    8 +-
 virt/kvm/arm/vgic/vgic-mmio.c                     |    2 -
 virt/kvm/arm/vgic/vgic-mmio.h                     |    4 +-
 virt/kvm/arm/vgic/vgic.h                          |   54 +-----------
 29 files changed, 265 insertions(+), 139 deletions(-)
 rename {arch/arm64/kvm => virt/kvm/arm}/hyp/vgic-v3-sr.c (96%)

-- 
1.7.9.5

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 0/8] ARM: KVM: Support for vgic-v3
@ 2016-09-08 16:06 ` Vladimir Murzin
  0 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-08 16:06 UTC (permalink / raw)
  To: linux-arm-kernel

Hi,

This is an attempt to make use vgic-v3 under arch/arm since
save-restore functionality got re-written in C and can be shared
between arm/arm64 like it has already been done for vgic-v2 and timer.

With this patches I'm able to get 32 core an AArch32 ARMv8 guest boot:

...
GICv3: CPU31: found redistributor 703 region 0:0x000000003ffd0000
CPU31: thread -1, cpu 3, socket 7, mpidr 80000703
Brought up 32 CPUs
SMP: Total of 32 processors activated (768.00 BogoMIPS).
CPU: All CPU(s) started in SVC mode.
...

Additionally, quite lightweight test based on Self IPI guest test[1]
has been run with up to 255 cpus.

[1] http://www.spinics.net/lists/kvm/msg128974.html

Changelog:

    v2 -> v3
       - rebased on v4.8-rc5
       - commit messages are reworked to be more precise and clear (I
         hope so)
       - MPIDR_HWID_BITMASK is used to discard Aff3 in
         MPIDR_AFFINITY_LEVEL macro
       - cast to u64 is used instead of abuse of GENMASK_ULL while
         building value for typer register
       - static keys are used to select GIC backend
       - config option to guard ITS code is moved to separate patch

    v1 -> v2
       - rebased on v4.8-rc2
       - introduced guard for ITS code

Thanks!

Vladimir Murzin (8):
  arm64: KVM: Use static keys for selecting the GIC backend
  arm64: KVM: Move GIC accessors to arch_gicv3.h
  arm64: KVM: Move vgic-v3 save/restore to virt/kvm/arm/hyp
  KVM: arm64: vgic-its: Introduce config option to guard ITS specific
    code
  KVM: arm: vgic: Support 64-bit data manipulation on 32-bit host
    systems
  ARM: Change MPIDR_AFFINITY_LEVEL to ignore Aff3
  ARM: Move system register accessors to asm/cp15.h
  ARM: KVM: Support vgic-v3

 arch/arm/include/asm/arch_gicv3.h                 |   91 +++++++++++++++++----
 arch/arm/include/asm/cp15.h                       |   15 ++++
 arch/arm/include/asm/cputype.h                    |    3 +-
 arch/arm/include/asm/kvm_asm.h                    |    3 +
 arch/arm/include/asm/kvm_host.h                   |   13 +++
 arch/arm/include/asm/kvm_hyp.h                    |   20 +----
 arch/arm/include/asm/virt.h                       |    8 ++
 arch/arm/include/uapi/asm/kvm.h                   |    7 ++
 arch/arm/kernel/vmlinux.lds.S                     |    6 ++
 arch/arm/kvm/Makefile                             |    2 +
 arch/arm/kvm/arm.c                                |   19 +++++
 arch/arm/kvm/coproc.c                             |   36 ++++++++
 arch/arm/kvm/hyp/Makefile                         |    1 +
 arch/arm/kvm/hyp/switch.c                         |   12 ++-
 arch/arm64/include/asm/arch_gicv3.h               |   13 +++
 arch/arm64/include/asm/kvm_host.h                 |   15 ++++
 arch/arm64/include/asm/kvm_hyp.h                  |    2 -
 arch/arm64/include/asm/virt.h                     |    7 ++
 arch/arm64/kernel/vmlinux.lds.S                   |    6 ++
 arch/arm64/kvm/Kconfig                            |    4 +-
 arch/arm64/kvm/hyp/Makefile                       |    2 +-
 arch/arm64/kvm/hyp/switch.c                       |   19 ++---
 include/kvm/arm_vgic.h                            |    8 --
 {arch/arm64/kvm => virt/kvm/arm}/hyp/vgic-v3-sr.c |   13 ---
 virt/kvm/arm/vgic/vgic-kvm-device.c               |   11 +--
 virt/kvm/arm/vgic/vgic-mmio-v3.c                  |    8 +-
 virt/kvm/arm/vgic/vgic-mmio.c                     |    2 -
 virt/kvm/arm/vgic/vgic-mmio.h                     |    4 +-
 virt/kvm/arm/vgic/vgic.h                          |   54 +-----------
 29 files changed, 265 insertions(+), 139 deletions(-)
 rename {arch/arm64/kvm => virt/kvm/arm}/hyp/vgic-v3-sr.c (96%)

-- 
1.7.9.5

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 1/8] arm64: KVM: Use static keys for selecting the GIC backend
  2016-09-08 16:06 ` Vladimir Murzin
@ 2016-09-08 16:06   ` Vladimir Murzin
  -1 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-08 16:06 UTC (permalink / raw)
  To: kvmarm; +Cc: marc.zyngier, andre.przywara, linux-arm-kernel

Currently GIC backend is selected via alternative framework and this
is fine. We are going to introduce vgic-v3 to 32-bit world and there
we don't have patching framework in hand, so we can either check
support for GICv3 every time we need to choose which backend to use or
try to optimise it by using static keys. The later looks quite
promising because we can share logic involved in selecting GIC backend
between architectures if both uses static keys.

This patch moves arm64 from alternative to static keys framework for
selecting GIC backend. To make static keys work on hyp side we need to
make sure that hyp can access to the key which is RW data. For that
purpose introduce __hyp_data section we can map to hyp and place the
key there.

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 arch/arm/include/asm/kvm_host.h   |   13 +++++++++++++
 arch/arm/include/asm/kvm_hyp.h    |    2 --
 arch/arm/include/asm/virt.h       |    8 ++++++++
 arch/arm/kernel/vmlinux.lds.S     |    6 ++++++
 arch/arm/kvm/arm.c                |   19 +++++++++++++++++++
 arch/arm64/include/asm/kvm_host.h |   15 +++++++++++++++
 arch/arm64/include/asm/kvm_hyp.h  |    2 --
 arch/arm64/include/asm/virt.h     |    7 +++++++
 arch/arm64/kernel/vmlinux.lds.S   |    6 ++++++
 arch/arm64/kvm/hyp/switch.c       |   19 +++++++++----------
 10 files changed, 83 insertions(+), 14 deletions(-)

diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
index de338d9..bfa6eec 100644
--- a/arch/arm/include/asm/kvm_host.h
+++ b/arch/arm/include/asm/kvm_host.h
@@ -21,11 +21,14 @@
 
 #include <linux/types.h>
 #include <linux/kvm_types.h>
+#include <linux/jump_label.h>
+
 #include <asm/kvm.h>
 #include <asm/kvm_asm.h>
 #include <asm/kvm_mmio.h>
 #include <asm/fpstate.h>
 #include <kvm/arm_arch_timer.h>
+#include <asm/cputype.h>
 
 #define __KVM_HAVE_ARCH_INTC_INITIALIZED
 
@@ -310,4 +313,14 @@ static inline int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu,
 	return -ENXIO;
 }
 
+extern struct static_key_false kvm_gicv3_cpuif;
+
+static inline bool kvm_arm_support_gicv3_cpuif(void)
+{
+	if (IS_ENABLED(CONFIG_ARM_GIC_V3))
+		return !!cpuid_feature_extract(CPUID_EXT_PFR1, 28);
+	else
+		return false;
+}
+
 #endif /* __ARM_KVM_HOST_H__ */
diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
index 6eaff28..bd9434e 100644
--- a/arch/arm/include/asm/kvm_hyp.h
+++ b/arch/arm/include/asm/kvm_hyp.h
@@ -23,8 +23,6 @@
 #include <asm/kvm_mmu.h>
 #include <asm/vfp.h>
 
-#define __hyp_text __section(.hyp.text) notrace
-
 #define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
 	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
 #define __ACCESS_CP15_64(Op1, CRm)		\
diff --git a/arch/arm/include/asm/virt.h b/arch/arm/include/asm/virt.h
index a2e75b8..e61a809 100644
--- a/arch/arm/include/asm/virt.h
+++ b/arch/arm/include/asm/virt.h
@@ -28,6 +28,9 @@
  */
 #define BOOT_CPU_MODE_MISMATCH	PSR_N_BIT
 
+#define __hyp_text __section(.hyp.text) notrace
+#define __hyp_data __section(.hyp.data)
+
 #ifndef __ASSEMBLY__
 #include <asm/cacheflush.h>
 
@@ -87,6 +90,11 @@ extern char __hyp_idmap_text_end[];
 /* The section containing the hypervisor text */
 extern char __hyp_text_start[];
 extern char __hyp_text_end[];
+
+/* The section containing the hypervisor data */
+extern char __hyp_data_start[];
+extern char __hyp_data_end[];
+
 #endif
 
 #endif /* __ASSEMBLY__ */
diff --git a/arch/arm/kernel/vmlinux.lds.S b/arch/arm/kernel/vmlinux.lds.S
index d24e5dd..6d53824 100644
--- a/arch/arm/kernel/vmlinux.lds.S
+++ b/arch/arm/kernel/vmlinux.lds.S
@@ -25,6 +25,11 @@
 	*(.hyp.text)							\
 	VMLINUX_SYMBOL(__hyp_text_end) = .;
 
+#define HYPERVISOR_DATA							\
+	VMLINUX_SYMBOL(__hyp_data_start) = .;				\
+	*(.hyp.data)							\
+	VMLINUX_SYMBOL(__hyp_data_end) = .;
+
 #define IDMAP_TEXT							\
 	ALIGN_FUNCTION();						\
 	VMLINUX_SYMBOL(__idmap_text_start) = .;				\
@@ -256,6 +261,7 @@ SECTIONS
 		 */
 		DATA_DATA
 		CONSTRUCTORS
+		HYPERVISOR_DATA
 
 		_edata = .;
 	}
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index 75f130e..f966763 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -27,6 +27,7 @@
 #include <linux/mman.h>
 #include <linux/sched.h>
 #include <linux/kvm.h>
+#include <linux/irqchip/arm-gic-v3.h>
 #include <trace/events/kvm.h>
 #include <kvm/arm_pmu.h>
 
@@ -68,6 +69,9 @@ static bool vgic_present;
 
 static DEFINE_PER_CPU(unsigned char, kvm_arm_hardware_enabled);
 
+/* GIC system register CPU interface */
+__hyp_data DEFINE_STATIC_KEY_FALSE(kvm_gicv3_cpuif);
+
 static void kvm_arm_set_running_vcpu(struct kvm_vcpu *vcpu)
 {
 	BUG_ON(preemptible());
@@ -1178,6 +1182,14 @@ static int init_common_resources(void)
 		return -ENOMEM;
 	}
 
+	if (kvm_arm_support_gicv3_cpuif()) {
+		if (!gic_enable_sre())
+			kvm_info("GIC CPU interface present but disabled by higher exception level\n");
+
+		static_branch_enable(&kvm_gicv3_cpuif);
+		kvm_info("GIC system register CPU interface\n");
+	}
+
 	return 0;
 }
 
@@ -1297,6 +1309,13 @@ static int init_hyp_mode(void)
 		goto out_err;
 	}
 
+	err = create_hyp_mappings(kvm_ksym_ref(__hyp_data_start),
+				  kvm_ksym_ref(__hyp_data_end), PAGE_HYP);
+	if (err) {
+		kvm_err("Cannot map hyp data section\n");
+		goto out_err;
+	}
+
 	err = create_hyp_mappings(kvm_ksym_ref(__start_rodata),
 				  kvm_ksym_ref(__end_rodata), PAGE_HYP_RO);
 	if (err) {
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 3eda975..1da74e8 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -24,6 +24,9 @@
 
 #include <linux/types.h>
 #include <linux/kvm_types.h>
+#include <linux/jump_label.h>
+
+#include <asm/cpufeature.h>
 #include <asm/kvm.h>
 #include <asm/kvm_asm.h>
 #include <asm/kvm_mmio.h>
@@ -390,4 +393,16 @@ static inline void __cpu_init_stage2(void)
 		  "PARange is %d bits, unsupported configuration!", parange);
 }
 
+extern struct static_key_false kvm_gicv3_cpuif;
+
+static inline bool kvm_arm_support_gicv3_cpuif(void)
+{
+	int reg = read_system_reg(SYS_ID_AA64PFR0_EL1);
+
+	if (IS_ENABLED(CONFIG_ARM_GIC_V3))
+		return !!cpuid_feature_extract_unsigned_field(reg, ID_AA64PFR0_GIC_SHIFT);
+
+	return false;
+}
+
 #endif /* __ARM64_KVM_HOST_H__ */
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index cff5105..5c4ac82 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -23,8 +23,6 @@
 #include <asm/kvm_mmu.h>
 #include <asm/sysreg.h>
 
-#define __hyp_text __section(.hyp.text) notrace
-
 #define read_sysreg_elx(r,nvh,vh)					\
 	({								\
 		u64 reg;						\
diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
index 1788545..c49426e 100644
--- a/arch/arm64/include/asm/virt.h
+++ b/arch/arm64/include/asm/virt.h
@@ -42,6 +42,9 @@
 #define BOOT_CPU_MODE_EL1	(0xe11)
 #define BOOT_CPU_MODE_EL2	(0xe12)
 
+#define __hyp_text __section(.hyp.text) notrace
+#define __hyp_data __section(.hyp.data)
+
 #ifndef __ASSEMBLY__
 
 #include <asm/ptrace.h>
@@ -95,6 +98,10 @@ extern char __hyp_idmap_text_end[];
 extern char __hyp_text_start[];
 extern char __hyp_text_end[];
 
+/* The section containing the hypervisor data */
+extern char __hyp_data_start[];
+extern char __hyp_data_end[];
+
 #endif /* __ASSEMBLY__ */
 
 #endif /* ! __ASM__VIRT_H */
diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
index 659963d..ea94a10 100644
--- a/arch/arm64/kernel/vmlinux.lds.S
+++ b/arch/arm64/kernel/vmlinux.lds.S
@@ -40,6 +40,11 @@ jiffies = jiffies_64;
 	*(.hyp.text)					\
 	VMLINUX_SYMBOL(__hyp_text_end) = .;
 
+#define HYPERVISOR_DATA					\
+	VMLINUX_SYMBOL(__hyp_data_start) = .;		\
+	.hyp.data : {*(.hyp.data)}			\
+	VMLINUX_SYMBOL(__hyp_data_end) = .;
+
 #define IDMAP_TEXT					\
 	. = ALIGN(SZ_4K);				\
 	VMLINUX_SYMBOL(__idmap_text_start) = .;		\
@@ -185,6 +190,7 @@ SECTIONS
 	_data = .;
 	_sdata = .;
 	RW_DATA_SECTION(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
+	HYPERVISOR_DATA
 	PECOFF_EDATA_PADDING
 	_edata = .;
 
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 5a84b45..cdc1360 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -126,17 +126,13 @@ static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
 	write_sysreg(0, vttbr_el2);
 }
 
-static hyp_alternate_select(__vgic_call_save_state,
-			    __vgic_v2_save_state, __vgic_v3_save_state,
-			    ARM64_HAS_SYSREG_GIC_CPUIF);
-
-static hyp_alternate_select(__vgic_call_restore_state,
-			    __vgic_v2_restore_state, __vgic_v3_restore_state,
-			    ARM64_HAS_SYSREG_GIC_CPUIF);
-
 static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
 {
-	__vgic_call_save_state()(vcpu);
+	if (static_branch_unlikely(&kvm_gicv3_cpuif))
+		__vgic_v3_save_state(vcpu);
+	else
+		__vgic_v2_save_state(vcpu);
+
 	write_sysreg(read_sysreg(hcr_el2) & ~HCR_INT_OVERRIDE, hcr_el2);
 }
 
@@ -149,7 +145,10 @@ static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
 	val |= vcpu->arch.irq_lines;
 	write_sysreg(val, hcr_el2);
 
-	__vgic_call_restore_state()(vcpu);
+	if (static_branch_unlikely(&kvm_gicv3_cpuif))
+		__vgic_v3_restore_state(vcpu);
+	else
+		__vgic_v2_restore_state(vcpu);
 }
 
 static bool __hyp_text __true_value(void)
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 72+ messages in thread
* [PATCH v3 1/8] arm64: KVM: Use static keys for selecting the GIC backend
@ 2016-09-08 16:06   ` Vladimir Murzin
  0 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-08 16:06 UTC (permalink / raw)
  To: linux-arm-kernel

Currently GIC backend is selected via alternative framework and this
is fine. We are going to introduce vgic-v3 to 32-bit world and there
we don't have patching framework in hand, so we can either check
support for GICv3 every time we need to choose which backend to use or
try to optimise it by using static keys. The later looks quite
promising because we can share logic involved in selecting GIC backend
between architectures if both uses static keys.

This patch moves arm64 from alternative to static keys framework for
selecting GIC backend. To make static keys work on hyp side we need to
make sure that hyp can access to the key which is RW data. For that
purpose introduce __hyp_data section we can map to hyp and place the
key there.

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 arch/arm/include/asm/kvm_host.h   |   13 +++++++++++++
 arch/arm/include/asm/kvm_hyp.h    |    2 --
 arch/arm/include/asm/virt.h       |    8 ++++++++
 arch/arm/kernel/vmlinux.lds.S     |    6 ++++++
 arch/arm/kvm/arm.c                |   19 +++++++++++++++++++
 arch/arm64/include/asm/kvm_host.h |   15 +++++++++++++++
 arch/arm64/include/asm/kvm_hyp.h  |    2 --
 arch/arm64/include/asm/virt.h     |    7 +++++++
 arch/arm64/kernel/vmlinux.lds.S   |    6 ++++++
 arch/arm64/kvm/hyp/switch.c       |   19 +++++++++----------
 10 files changed, 83 insertions(+), 14 deletions(-)

diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
index de338d9..bfa6eec 100644
--- a/arch/arm/include/asm/kvm_host.h
+++ b/arch/arm/include/asm/kvm_host.h
@@ -21,11 +21,14 @@
 
 #include <linux/types.h>
 #include <linux/kvm_types.h>
+#include <linux/jump_label.h>
+
 #include <asm/kvm.h>
 #include <asm/kvm_asm.h>
 #include <asm/kvm_mmio.h>
 #include <asm/fpstate.h>
 #include <kvm/arm_arch_timer.h>
+#include <asm/cputype.h>
 
 #define __KVM_HAVE_ARCH_INTC_INITIALIZED
 
@@ -310,4 +313,14 @@ static inline int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu,
 	return -ENXIO;
 }
 
+extern struct static_key_false kvm_gicv3_cpuif;
+
+static inline bool kvm_arm_support_gicv3_cpuif(void)
+{
+	if (IS_ENABLED(CONFIG_ARM_GIC_V3))
+		return !!cpuid_feature_extract(CPUID_EXT_PFR1, 28);
+	else
+		return false;
+}
+
 #endif /* __ARM_KVM_HOST_H__ */
diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
index 6eaff28..bd9434e 100644
--- a/arch/arm/include/asm/kvm_hyp.h
+++ b/arch/arm/include/asm/kvm_hyp.h
@@ -23,8 +23,6 @@
 #include <asm/kvm_mmu.h>
 #include <asm/vfp.h>
 
-#define __hyp_text __section(.hyp.text) notrace
-
 #define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
 	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
 #define __ACCESS_CP15_64(Op1, CRm)		\
diff --git a/arch/arm/include/asm/virt.h b/arch/arm/include/asm/virt.h
index a2e75b8..e61a809 100644
--- a/arch/arm/include/asm/virt.h
+++ b/arch/arm/include/asm/virt.h
@@ -28,6 +28,9 @@
  */
 #define BOOT_CPU_MODE_MISMATCH	PSR_N_BIT
 
+#define __hyp_text __section(.hyp.text) notrace
+#define __hyp_data __section(.hyp.data)
+
 #ifndef __ASSEMBLY__
 #include <asm/cacheflush.h>
 
@@ -87,6 +90,11 @@ extern char __hyp_idmap_text_end[];
 /* The section containing the hypervisor text */
 extern char __hyp_text_start[];
 extern char __hyp_text_end[];
+
+/* The section containing the hypervisor data */
+extern char __hyp_data_start[];
+extern char __hyp_data_end[];
+
 #endif
 
 #endif /* __ASSEMBLY__ */
diff --git a/arch/arm/kernel/vmlinux.lds.S b/arch/arm/kernel/vmlinux.lds.S
index d24e5dd..6d53824 100644
--- a/arch/arm/kernel/vmlinux.lds.S
+++ b/arch/arm/kernel/vmlinux.lds.S
@@ -25,6 +25,11 @@
 	*(.hyp.text)							\
 	VMLINUX_SYMBOL(__hyp_text_end) = .;
 
+#define HYPERVISOR_DATA							\
+	VMLINUX_SYMBOL(__hyp_data_start) = .;				\
+	*(.hyp.data)							\
+	VMLINUX_SYMBOL(__hyp_data_end) = .;
+
 #define IDMAP_TEXT							\
 	ALIGN_FUNCTION();						\
 	VMLINUX_SYMBOL(__idmap_text_start) = .;				\
@@ -256,6 +261,7 @@ SECTIONS
 		 */
 		DATA_DATA
 		CONSTRUCTORS
+		HYPERVISOR_DATA
 
 		_edata = .;
 	}
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index 75f130e..f966763 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -27,6 +27,7 @@
 #include <linux/mman.h>
 #include <linux/sched.h>
 #include <linux/kvm.h>
+#include <linux/irqchip/arm-gic-v3.h>
 #include <trace/events/kvm.h>
 #include <kvm/arm_pmu.h>
 
@@ -68,6 +69,9 @@ static bool vgic_present;
 
 static DEFINE_PER_CPU(unsigned char, kvm_arm_hardware_enabled);
 
+/* GIC system register CPU interface */
+__hyp_data DEFINE_STATIC_KEY_FALSE(kvm_gicv3_cpuif);
+
 static void kvm_arm_set_running_vcpu(struct kvm_vcpu *vcpu)
 {
 	BUG_ON(preemptible());
@@ -1178,6 +1182,14 @@ static int init_common_resources(void)
 		return -ENOMEM;
 	}
 
+	if (kvm_arm_support_gicv3_cpuif()) {
+		if (!gic_enable_sre())
+			kvm_info("GIC CPU interface present but disabled by higher exception level\n");
+
+		static_branch_enable(&kvm_gicv3_cpuif);
+		kvm_info("GIC system register CPU interface\n");
+	}
+
 	return 0;
 }
 
@@ -1297,6 +1309,13 @@ static int init_hyp_mode(void)
 		goto out_err;
 	}
 
+	err = create_hyp_mappings(kvm_ksym_ref(__hyp_data_start),
+				  kvm_ksym_ref(__hyp_data_end), PAGE_HYP);
+	if (err) {
+		kvm_err("Cannot map hyp data section\n");
+		goto out_err;
+	}
+
 	err = create_hyp_mappings(kvm_ksym_ref(__start_rodata),
 				  kvm_ksym_ref(__end_rodata), PAGE_HYP_RO);
 	if (err) {
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 3eda975..1da74e8 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -24,6 +24,9 @@
 
 #include <linux/types.h>
 #include <linux/kvm_types.h>
+#include <linux/jump_label.h>
+
+#include <asm/cpufeature.h>
 #include <asm/kvm.h>
 #include <asm/kvm_asm.h>
 #include <asm/kvm_mmio.h>
@@ -390,4 +393,16 @@ static inline void __cpu_init_stage2(void)
 		  "PARange is %d bits, unsupported configuration!", parange);
 }
 
+extern struct static_key_false kvm_gicv3_cpuif;
+
+static inline bool kvm_arm_support_gicv3_cpuif(void)
+{
+	int reg = read_system_reg(SYS_ID_AA64PFR0_EL1);
+
+	if (IS_ENABLED(CONFIG_ARM_GIC_V3))
+		return !!cpuid_feature_extract_unsigned_field(reg, ID_AA64PFR0_GIC_SHIFT);
+
+	return false;
+}
+
 #endif /* __ARM64_KVM_HOST_H__ */
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index cff5105..5c4ac82 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -23,8 +23,6 @@
 #include <asm/kvm_mmu.h>
 #include <asm/sysreg.h>
 
-#define __hyp_text __section(.hyp.text) notrace
-
 #define read_sysreg_elx(r,nvh,vh)					\
 	({								\
 		u64 reg;						\
diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
index 1788545..c49426e 100644
--- a/arch/arm64/include/asm/virt.h
+++ b/arch/arm64/include/asm/virt.h
@@ -42,6 +42,9 @@
 #define BOOT_CPU_MODE_EL1	(0xe11)
 #define BOOT_CPU_MODE_EL2	(0xe12)
 
+#define __hyp_text __section(.hyp.text) notrace
+#define __hyp_data __section(.hyp.data)
+
 #ifndef __ASSEMBLY__
 
 #include <asm/ptrace.h>
@@ -95,6 +98,10 @@ extern char __hyp_idmap_text_end[];
 extern char __hyp_text_start[];
 extern char __hyp_text_end[];
 
+/* The section containing the hypervisor data */
+extern char __hyp_data_start[];
+extern char __hyp_data_end[];
+
 #endif /* __ASSEMBLY__ */
 
 #endif /* ! __ASM__VIRT_H */
diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
index 659963d..ea94a10 100644
--- a/arch/arm64/kernel/vmlinux.lds.S
+++ b/arch/arm64/kernel/vmlinux.lds.S
@@ -40,6 +40,11 @@ jiffies = jiffies_64;
 	*(.hyp.text)					\
 	VMLINUX_SYMBOL(__hyp_text_end) = .;
 
+#define HYPERVISOR_DATA					\
+	VMLINUX_SYMBOL(__hyp_data_start) = .;		\
+	.hyp.data : {*(.hyp.data)}			\
+	VMLINUX_SYMBOL(__hyp_data_end) = .;
+
 #define IDMAP_TEXT					\
 	. = ALIGN(SZ_4K);				\
 	VMLINUX_SYMBOL(__idmap_text_start) = .;		\
@@ -185,6 +190,7 @@ SECTIONS
 	_data = .;
 	_sdata = .;
 	RW_DATA_SECTION(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
+	HYPERVISOR_DATA
 	PECOFF_EDATA_PADDING
 	_edata = .;
 
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 5a84b45..cdc1360 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -126,17 +126,13 @@ static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
 	write_sysreg(0, vttbr_el2);
 }
 
-static hyp_alternate_select(__vgic_call_save_state,
-			    __vgic_v2_save_state, __vgic_v3_save_state,
-			    ARM64_HAS_SYSREG_GIC_CPUIF);
-
-static hyp_alternate_select(__vgic_call_restore_state,
-			    __vgic_v2_restore_state, __vgic_v3_restore_state,
-			    ARM64_HAS_SYSREG_GIC_CPUIF);
-
 static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
 {
-	__vgic_call_save_state()(vcpu);
+	if (static_branch_unlikely(&kvm_gicv3_cpuif))
+		__vgic_v3_save_state(vcpu);
+	else
+		__vgic_v2_save_state(vcpu);
+
 	write_sysreg(read_sysreg(hcr_el2) & ~HCR_INT_OVERRIDE, hcr_el2);
 }
 
@@ -149,7 +145,10 @@ static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
 	val |= vcpu->arch.irq_lines;
 	write_sysreg(val, hcr_el2);
 
-	__vgic_call_restore_state()(vcpu);
+	if (static_branch_unlikely(&kvm_gicv3_cpuif))
+		__vgic_v3_restore_state(vcpu);
+	else
+		__vgic_v2_restore_state(vcpu);
 }
 
 static bool __hyp_text __true_value(void)
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 72+ messages in thread
* [PATCH v3 2/8] arm64: KVM: Move GIC accessors to arch_gicv3.h
  2016-09-08 16:06 ` Vladimir Murzin
@ 2016-09-08 16:06   ` Vladimir Murzin
  -1 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-08 16:06 UTC (permalink / raw)
  To: kvmarm; +Cc: marc.zyngier, andre.przywara, linux-arm-kernel

Since we are going to share vgic-v3 save/restore code with ARM keep
arch specific accessors separately.

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
---
 arch/arm64/include/asm/arch_gicv3.h |   13 +++++++++++++
 arch/arm64/kvm/hyp/vgic-v3-sr.c     |   13 -------------
 2 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h
index 8ec88e5..ae7dbd7 100644
--- a/arch/arm64/include/asm/arch_gicv3.h
+++ b/arch/arm64/include/asm/arch_gicv3.h
@@ -79,6 +79,19 @@
 #include <linux/stringify.h>
 #include <asm/barrier.h>
 
+#define read_gicreg(r)							\
+	({								\
+		u64 reg;						\
+		asm volatile("mrs_s %0, " __stringify(r) : "=r" (reg));	\
+		reg;							\
+	})
+
+#define write_gicreg(v,r)						\
+	do {								\
+		u64 __val = (v);					\
+		asm volatile("msr_s " __stringify(r) ", %0" : : "r" (__val));\
+	} while (0)
+
 /*
  * Low-level accessors
  *
diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
index 5f8f80b..f2dbd2e 100644
--- a/arch/arm64/kvm/hyp/vgic-v3-sr.c
+++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
@@ -24,19 +24,6 @@
 #define vtr_to_max_lr_idx(v)		((v) & 0xf)
 #define vtr_to_nr_pri_bits(v)		(((u32)(v) >> 29) + 1)
 
-#define read_gicreg(r)							\
-	({								\
-		u64 reg;						\
-		asm volatile("mrs_s %0, " __stringify(r) : "=r" (reg));	\
-		reg;							\
-	})
-
-#define write_gicreg(v,r)						\
-	do {								\
-		u64 __val = (v);					\
-		asm volatile("msr_s " __stringify(r) ", %0" : : "r" (__val));\
-	} while (0)
-
 static u64 __hyp_text __gic_v3_get_lr(unsigned int lr)
 {
 	switch (lr & 0xf) {
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 72+ messages in thread
* [PATCH v3 2/8] arm64: KVM: Move GIC accessors to arch_gicv3.h
@ 2016-09-08 16:06   ` Vladimir Murzin
  0 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-08 16:06 UTC (permalink / raw)
  To: linux-arm-kernel

Since we are going to share vgic-v3 save/restore code with ARM keep
arch specific accessors separately.

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
---
 arch/arm64/include/asm/arch_gicv3.h |   13 +++++++++++++
 arch/arm64/kvm/hyp/vgic-v3-sr.c     |   13 -------------
 2 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h
index 8ec88e5..ae7dbd7 100644
--- a/arch/arm64/include/asm/arch_gicv3.h
+++ b/arch/arm64/include/asm/arch_gicv3.h
@@ -79,6 +79,19 @@
 #include <linux/stringify.h>
 #include <asm/barrier.h>
 
+#define read_gicreg(r)							\
+	({								\
+		u64 reg;						\
+		asm volatile("mrs_s %0, " __stringify(r) : "=r" (reg));	\
+		reg;							\
+	})
+
+#define write_gicreg(v,r)						\
+	do {								\
+		u64 __val = (v);					\
+		asm volatile("msr_s " __stringify(r) ", %0" : : "r" (__val));\
+	} while (0)
+
 /*
  * Low-level accessors
  *
diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
index 5f8f80b..f2dbd2e 100644
--- a/arch/arm64/kvm/hyp/vgic-v3-sr.c
+++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
@@ -24,19 +24,6 @@
 #define vtr_to_max_lr_idx(v)		((v) & 0xf)
 #define vtr_to_nr_pri_bits(v)		(((u32)(v) >> 29) + 1)
 
-#define read_gicreg(r)							\
-	({								\
-		u64 reg;						\
-		asm volatile("mrs_s %0, " __stringify(r) : "=r" (reg));	\
-		reg;							\
-	})
-
-#define write_gicreg(v,r)						\
-	do {								\
-		u64 __val = (v);					\
-		asm volatile("msr_s " __stringify(r) ", %0" : : "r" (__val));\
-	} while (0)
-
 static u64 __hyp_text __gic_v3_get_lr(unsigned int lr)
 {
 	switch (lr & 0xf) {
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 72+ messages in thread
* [PATCH v3 3/8] arm64: KVM: Move vgic-v3 save/restore to virt/kvm/arm/hyp
  2016-09-08 16:06 ` Vladimir Murzin
@ 2016-09-08 16:06   ` Vladimir Murzin
  -1 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-08 16:06 UTC (permalink / raw)
  To: kvmarm; +Cc: marc.zyngier, andre.przywara, linux-arm-kernel

So we can reuse the code under arch/arm

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 arch/arm64/kvm/hyp/Makefile                       |    2 +-
 {arch/arm64/kvm => virt/kvm/arm}/hyp/vgic-v3-sr.c |    0
 2 files changed, 1 insertion(+), 1 deletion(-)
 rename {arch/arm64/kvm => virt/kvm/arm}/hyp/vgic-v3-sr.c (100%)

diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
index 0c85feb..aaf42ae 100644
--- a/arch/arm64/kvm/hyp/Makefile
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -5,9 +5,9 @@
 KVM=../../../../virt/kvm
 
 obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
+obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
 
-obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += entry.o
diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/virt/kvm/arm/hyp/vgic-v3-sr.c
similarity index 100%
rename from arch/arm64/kvm/hyp/vgic-v3-sr.c
rename to virt/kvm/arm/hyp/vgic-v3-sr.c
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 72+ messages in thread
* [PATCH v3 3/8] arm64: KVM: Move vgic-v3 save/restore to virt/kvm/arm/hyp
@ 2016-09-08 16:06   ` Vladimir Murzin
  0 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-08 16:06 UTC (permalink / raw)
  To: linux-arm-kernel

So we can reuse the code under arch/arm

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 arch/arm64/kvm/hyp/Makefile                       |    2 +-
 {arch/arm64/kvm => virt/kvm/arm}/hyp/vgic-v3-sr.c |    0
 2 files changed, 1 insertion(+), 1 deletion(-)
 rename {arch/arm64/kvm => virt/kvm/arm}/hyp/vgic-v3-sr.c (100%)

diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
index 0c85feb..aaf42ae 100644
--- a/arch/arm64/kvm/hyp/Makefile
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -5,9 +5,9 @@
 KVM=../../../../virt/kvm
 
 obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
+obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
 
-obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += entry.o
diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/virt/kvm/arm/hyp/vgic-v3-sr.c
similarity index 100%
rename from arch/arm64/kvm/hyp/vgic-v3-sr.c
rename to virt/kvm/arm/hyp/vgic-v3-sr.c
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 72+ messages in thread
* [PATCH v3 4/8] KVM: arm64: vgic-its: Introduce config option to guard ITS specific code
  2016-09-08 16:06 ` Vladimir Murzin
@ 2016-09-08 16:06   ` Vladimir Murzin
  -1 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-08 16:06 UTC (permalink / raw)
  To: kvmarm; +Cc: marc.zyngier, andre.przywara, linux-arm-kernel

By now ITS code guarded with KVM_ARM_VGIC_V3 config option which was
introduced to hide everything specific to vgic-v3 from 32-bit world.
We are going to support vgic-v3 in 32-bit world and KVM_ARM_VGIC_V3
will gone, but we don't have support for ITS there yet and we need to
continue keeping ITS away.
Introduce the new config option to prevent ITS code being build in
32-bit mode when support for vgic-v3 is done.

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 arch/arm64/kvm/Kconfig              |    4 ++++
 virt/kvm/arm/vgic/vgic-kvm-device.c |    3 +++
 virt/kvm/arm/vgic/vgic-mmio-v3.c    |    2 ++
 virt/kvm/arm/vgic/vgic.h            |    4 ++++
 4 files changed, 13 insertions(+)

diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index 9c9edc9..7ba9164 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -16,6 +16,9 @@ menuconfig VIRTUALIZATION
 
 if VIRTUALIZATION
 
+config KVM_ARM_VGIC_V3_ITS
+	bool
+
 config KVM_ARM_VGIC_V3
 	bool
 
@@ -35,6 +38,7 @@ config KVM
 	select HAVE_KVM_EVENTFD
 	select HAVE_KVM_IRQFD
 	select KVM_ARM_VGIC_V3
+	select KVM_ARM_VGIC_V3_ITS
 	select KVM_ARM_PMU if HW_PERF_EVENTS
 	select HAVE_KVM_MSI
 	select HAVE_KVM_IRQCHIP
diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
index 1813f93..f3811b3 100644
--- a/virt/kvm/arm/vgic/vgic-kvm-device.c
+++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
@@ -223,9 +223,12 @@ int kvm_register_vgic_device(unsigned long type)
 	case KVM_DEV_TYPE_ARM_VGIC_V3:
 		ret = kvm_register_device_ops(&kvm_arm_vgic_v3_ops,
 					      KVM_DEV_TYPE_ARM_VGIC_V3);
+
+#ifdef CONFIG_KVM_ARM_VGIC_V3_ITS
 		if (ret)
 			break;
 		ret = kvm_vgic_register_its_device();
+#endif
 		break;
 #endif
 	}
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index 90d8181..acbe691 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -42,6 +42,7 @@ u64 update_64bit_reg(u64 reg, unsigned int offset, unsigned int len,
 	return reg | ((u64)val << lower);
 }
 
+#ifdef CONFIG_KVM_ARM_VGIC_V3_ITS
 bool vgic_has_its(struct kvm *kvm)
 {
 	struct vgic_dist *dist = &kvm->arch.vgic;
@@ -51,6 +52,7 @@ bool vgic_has_its(struct kvm *kvm)
 
 	return dist->has_its;
 }
+#endif
 
 static unsigned long vgic_mmio_read_v3_misc(struct kvm_vcpu *vcpu,
 					    gpa_t addr, unsigned int len)
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index 6c4625c..100045f 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -84,11 +84,15 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu);
 int vgic_v3_probe(const struct gic_kvm_info *info);
 int vgic_v3_map_resources(struct kvm *kvm);
 int vgic_register_redist_iodevs(struct kvm *kvm, gpa_t dist_base_address);
+
+#ifdef CONFIG_KVM_ARM_VGIC_V3_ITS
 int vgic_register_its_iodevs(struct kvm *kvm);
 bool vgic_has_its(struct kvm *kvm);
 int kvm_vgic_register_its_device(void);
 void vgic_enable_lpis(struct kvm_vcpu *vcpu);
 int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi);
+#endif
+
 #else
 static inline void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu)
 {
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 72+ messages in thread
* [PATCH v3 4/8] KVM: arm64: vgic-its: Introduce config option to guard ITS specific code
@ 2016-09-08 16:06   ` Vladimir Murzin
  0 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-08 16:06 UTC (permalink / raw)
  To: linux-arm-kernel

By now ITS code guarded with KVM_ARM_VGIC_V3 config option which was
introduced to hide everything specific to vgic-v3 from 32-bit world.
We are going to support vgic-v3 in 32-bit world and KVM_ARM_VGIC_V3
will gone, but we don't have support for ITS there yet and we need to
continue keeping ITS away.
Introduce the new config option to prevent ITS code being build in
32-bit mode when support for vgic-v3 is done.

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 arch/arm64/kvm/Kconfig              |    4 ++++
 virt/kvm/arm/vgic/vgic-kvm-device.c |    3 +++
 virt/kvm/arm/vgic/vgic-mmio-v3.c    |    2 ++
 virt/kvm/arm/vgic/vgic.h            |    4 ++++
 4 files changed, 13 insertions(+)

diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index 9c9edc9..7ba9164 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -16,6 +16,9 @@ menuconfig VIRTUALIZATION
 
 if VIRTUALIZATION
 
+config KVM_ARM_VGIC_V3_ITS
+	bool
+
 config KVM_ARM_VGIC_V3
 	bool
 
@@ -35,6 +38,7 @@ config KVM
 	select HAVE_KVM_EVENTFD
 	select HAVE_KVM_IRQFD
 	select KVM_ARM_VGIC_V3
+	select KVM_ARM_VGIC_V3_ITS
 	select KVM_ARM_PMU if HW_PERF_EVENTS
 	select HAVE_KVM_MSI
 	select HAVE_KVM_IRQCHIP
diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
index 1813f93..f3811b3 100644
--- a/virt/kvm/arm/vgic/vgic-kvm-device.c
+++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
@@ -223,9 +223,12 @@ int kvm_register_vgic_device(unsigned long type)
 	case KVM_DEV_TYPE_ARM_VGIC_V3:
 		ret = kvm_register_device_ops(&kvm_arm_vgic_v3_ops,
 					      KVM_DEV_TYPE_ARM_VGIC_V3);
+
+#ifdef CONFIG_KVM_ARM_VGIC_V3_ITS
 		if (ret)
 			break;
 		ret = kvm_vgic_register_its_device();
+#endif
 		break;
 #endif
 	}
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index 90d8181..acbe691 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -42,6 +42,7 @@ u64 update_64bit_reg(u64 reg, unsigned int offset, unsigned int len,
 	return reg | ((u64)val << lower);
 }
 
+#ifdef CONFIG_KVM_ARM_VGIC_V3_ITS
 bool vgic_has_its(struct kvm *kvm)
 {
 	struct vgic_dist *dist = &kvm->arch.vgic;
@@ -51,6 +52,7 @@ bool vgic_has_its(struct kvm *kvm)
 
 	return dist->has_its;
 }
+#endif
 
 static unsigned long vgic_mmio_read_v3_misc(struct kvm_vcpu *vcpu,
 					    gpa_t addr, unsigned int len)
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index 6c4625c..100045f 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -84,11 +84,15 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu);
 int vgic_v3_probe(const struct gic_kvm_info *info);
 int vgic_v3_map_resources(struct kvm *kvm);
 int vgic_register_redist_iodevs(struct kvm *kvm, gpa_t dist_base_address);
+
+#ifdef CONFIG_KVM_ARM_VGIC_V3_ITS
 int vgic_register_its_iodevs(struct kvm *kvm);
 bool vgic_has_its(struct kvm *kvm);
 int kvm_vgic_register_its_device(void);
 void vgic_enable_lpis(struct kvm_vcpu *vcpu);
 int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi);
+#endif
+
 #else
 static inline void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu)
 {
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 72+ messages in thread
* [PATCH v3 5/8] KVM: arm: vgic: Support 64-bit data manipulation on 32-bit host systems
  2016-09-08 16:06 ` Vladimir Murzin
@ 2016-09-08 16:06   ` Vladimir Murzin
  -1 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-08 16:06 UTC (permalink / raw)
  To: kvmarm; +Cc: marc.zyngier, andre.przywara, linux-arm-kernel

We have couple of 64-bit registers defined in GICv3 architecture, so
unsigned long accesses to these registers will only access a single
32-bit part of that regitser. On the other hand these registers can't
be accessed as 64-bit with a single instruction like ldrd/strd or
ldmia/stmia if we run a 32-bit host because KVM does not support
access to MMIO space done by these instructions.

It means that a 32-bit guest accesses these registers in 32-bit
chunks, so the only thing we need to do is to ensure that
extract_bytes() always takes 64-bit data.

Since we are here fix couple of other width related issues catched by
gcc

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 virt/kvm/arm/vgic/vgic-mmio-v3.c |    6 +++---
 virt/kvm/arm/vgic/vgic-mmio.h    |    2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index acbe691..0d3c76a 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -23,7 +23,7 @@
 #include "vgic-mmio.h"
 
 /* extract @num bytes at @offset bytes offset in data */
-unsigned long extract_bytes(unsigned long data, unsigned int offset,
+unsigned long extract_bytes(u64 data, unsigned int offset,
 			    unsigned int num)
 {
 	return (data >> (offset * 8)) & GENMASK_ULL(num * 8 - 1, 0);
@@ -181,7 +181,7 @@ static unsigned long vgic_mmio_read_v3r_typer(struct kvm_vcpu *vcpu,
 	int target_vcpu_id = vcpu->vcpu_id;
 	u64 value;
 
-	value = (mpidr & GENMASK(23, 0)) << 32;
+	value = (u64)(mpidr & GENMASK(23, 0)) << 32;
 	value |= ((target_vcpu_id & 0xffff) << 8);
 	if (target_vcpu_id == atomic_read(&vcpu->kvm->online_vcpus) - 1)
 		value |= GICR_TYPER_LAST;
@@ -611,7 +611,7 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)
 	bool broadcast;
 
 	sgi = (reg & ICC_SGI1R_SGI_ID_MASK) >> ICC_SGI1R_SGI_ID_SHIFT;
-	broadcast = reg & BIT(ICC_SGI1R_IRQ_ROUTING_MODE_BIT);
+	broadcast = reg & BIT_ULL(ICC_SGI1R_IRQ_ROUTING_MODE_BIT);
 	target_cpus = (reg & ICC_SGI1R_TARGET_LIST_MASK) >> ICC_SGI1R_TARGET_LIST_SHIFT;
 	mpidr = SGI_AFFINITY_LEVEL(reg, 3);
 	mpidr |= SGI_AFFINITY_LEVEL(reg, 2);
diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
index 0b3ecf9..80f92ce 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.h
+++ b/virt/kvm/arm/vgic/vgic-mmio.h
@@ -96,7 +96,7 @@ unsigned long vgic_data_mmio_bus_to_host(const void *val, unsigned int len);
 void vgic_data_host_to_mmio_bus(void *buf, unsigned int len,
 				unsigned long data);
 
-unsigned long extract_bytes(unsigned long data, unsigned int offset,
+unsigned long extract_bytes(u64 data, unsigned int offset,
 			    unsigned int num);
 
 u64 update_64bit_reg(u64 reg, unsigned int offset, unsigned int len,
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 72+ messages in thread
* [PATCH v3 5/8] KVM: arm: vgic: Support 64-bit data manipulation on 32-bit host systems
@ 2016-09-08 16:06   ` Vladimir Murzin
  0 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-08 16:06 UTC (permalink / raw)
  To: linux-arm-kernel

We have couple of 64-bit registers defined in GICv3 architecture, so
unsigned long accesses to these registers will only access a single
32-bit part of that regitser. On the other hand these registers can't
be accessed as 64-bit with a single instruction like ldrd/strd or
ldmia/stmia if we run a 32-bit host because KVM does not support
access to MMIO space done by these instructions.

It means that a 32-bit guest accesses these registers in 32-bit
chunks, so the only thing we need to do is to ensure that
extract_bytes() always takes 64-bit data.

Since we are here fix couple of other width related issues catched by
gcc

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 virt/kvm/arm/vgic/vgic-mmio-v3.c |    6 +++---
 virt/kvm/arm/vgic/vgic-mmio.h    |    2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index acbe691..0d3c76a 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -23,7 +23,7 @@
 #include "vgic-mmio.h"
 
 /* extract @num bytes at @offset bytes offset in data */
-unsigned long extract_bytes(unsigned long data, unsigned int offset,
+unsigned long extract_bytes(u64 data, unsigned int offset,
 			    unsigned int num)
 {
 	return (data >> (offset * 8)) & GENMASK_ULL(num * 8 - 1, 0);
@@ -181,7 +181,7 @@ static unsigned long vgic_mmio_read_v3r_typer(struct kvm_vcpu *vcpu,
 	int target_vcpu_id = vcpu->vcpu_id;
 	u64 value;
 
-	value = (mpidr & GENMASK(23, 0)) << 32;
+	value = (u64)(mpidr & GENMASK(23, 0)) << 32;
 	value |= ((target_vcpu_id & 0xffff) << 8);
 	if (target_vcpu_id == atomic_read(&vcpu->kvm->online_vcpus) - 1)
 		value |= GICR_TYPER_LAST;
@@ -611,7 +611,7 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)
 	bool broadcast;
 
 	sgi = (reg & ICC_SGI1R_SGI_ID_MASK) >> ICC_SGI1R_SGI_ID_SHIFT;
-	broadcast = reg & BIT(ICC_SGI1R_IRQ_ROUTING_MODE_BIT);
+	broadcast = reg & BIT_ULL(ICC_SGI1R_IRQ_ROUTING_MODE_BIT);
 	target_cpus = (reg & ICC_SGI1R_TARGET_LIST_MASK) >> ICC_SGI1R_TARGET_LIST_SHIFT;
 	mpidr = SGI_AFFINITY_LEVEL(reg, 3);
 	mpidr |= SGI_AFFINITY_LEVEL(reg, 2);
diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
index 0b3ecf9..80f92ce 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.h
+++ b/virt/kvm/arm/vgic/vgic-mmio.h
@@ -96,7 +96,7 @@ unsigned long vgic_data_mmio_bus_to_host(const void *val, unsigned int len);
 void vgic_data_host_to_mmio_bus(void *buf, unsigned int len,
 				unsigned long data);
 
-unsigned long extract_bytes(unsigned long data, unsigned int offset,
+unsigned long extract_bytes(u64 data, unsigned int offset,
 			    unsigned int num);
 
 u64 update_64bit_reg(u64 reg, unsigned int offset, unsigned int len,
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 72+ messages in thread
* [PATCH v3 6/8] ARM: Change MPIDR_AFFINITY_LEVEL to ignore Aff3
  2016-09-08 16:06 ` Vladimir Murzin
@ 2016-09-08 16:06   ` Vladimir Murzin
  -1 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-08 16:06 UTC (permalink / raw)
  To: kvmarm; +Cc: marc.zyngier, andre.przywara, linux-arm-kernel

vgic-v3 driver queries CPU affinity level up to Aff3, which is valid
for arm64.  However, for arm up to Aff2 levels are supported, so
querying for third level ends with upper bits of MPIDR are treated as
valid affinity level which is not true. Make sure we report zero for
any affinity level above two.

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 arch/arm/include/asm/cputype.h |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/arm/include/asm/cputype.h b/arch/arm/include/asm/cputype.h
index 1ee94c7..f08fac4 100644
--- a/arch/arm/include/asm/cputype.h
+++ b/arch/arm/include/asm/cputype.h
@@ -55,9 +55,10 @@
 
 #define MPIDR_LEVEL_BITS 8
 #define MPIDR_LEVEL_MASK ((1 << MPIDR_LEVEL_BITS) - 1)
+#define MPIDR_LEVEL_SHIFT(level) (MPIDR_LEVEL_BITS * level)
 
 #define MPIDR_AFFINITY_LEVEL(mpidr, level) \
-	((mpidr >> (MPIDR_LEVEL_BITS * level)) & MPIDR_LEVEL_MASK)
+	(((mpidr & MPIDR_HWID_BITMASK) >> MPIDR_LEVEL_SHIFT(level)) & MPIDR_LEVEL_MASK)
 
 #define ARM_CPU_IMP_ARM			0x41
 #define ARM_CPU_IMP_INTEL		0x69
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 72+ messages in thread
* [PATCH v3 6/8] ARM: Change MPIDR_AFFINITY_LEVEL to ignore Aff3
@ 2016-09-08 16:06   ` Vladimir Murzin
  0 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-08 16:06 UTC (permalink / raw)
  To: linux-arm-kernel

vgic-v3 driver queries CPU affinity level up to Aff3, which is valid
for arm64.  However, for arm up to Aff2 levels are supported, so
querying for third level ends with upper bits of MPIDR are treated as
valid affinity level which is not true. Make sure we report zero for
any affinity level above two.

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 arch/arm/include/asm/cputype.h |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/arm/include/asm/cputype.h b/arch/arm/include/asm/cputype.h
index 1ee94c7..f08fac4 100644
--- a/arch/arm/include/asm/cputype.h
+++ b/arch/arm/include/asm/cputype.h
@@ -55,9 +55,10 @@
 
 #define MPIDR_LEVEL_BITS 8
 #define MPIDR_LEVEL_MASK ((1 << MPIDR_LEVEL_BITS) - 1)
+#define MPIDR_LEVEL_SHIFT(level) (MPIDR_LEVEL_BITS * level)
 
 #define MPIDR_AFFINITY_LEVEL(mpidr, level) \
-	((mpidr >> (MPIDR_LEVEL_BITS * level)) & MPIDR_LEVEL_MASK)
+	(((mpidr & MPIDR_HWID_BITMASK) >> MPIDR_LEVEL_SHIFT(level)) & MPIDR_LEVEL_MASK)
 
 #define ARM_CPU_IMP_ARM			0x41
 #define ARM_CPU_IMP_INTEL		0x69
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 72+ messages in thread
* [PATCH v3 7/8] ARM: Move system register accessors to asm/cp15.h
  2016-09-08 16:06 ` Vladimir Murzin
@ 2016-09-08 16:06   ` Vladimir Murzin
  -1 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-08 16:06 UTC (permalink / raw)
  To: kvmarm; +Cc: marc.zyngier, andre.przywara, linux-arm-kernel

Headers linux/irqchip/arm-gic.v3.h and arch/arm/include/asm/kvm_hyp.h
are included in virt/kvm/arm/hyp/vgic-v3-sr.c and both define macros
called __ACCESS_CP15 and __ACCESS_CP15_64 which obviously creates a
conflict. These macros were introduced independently for GIC and KVM
and, in fact, do the same thing.

As an option we could add prefixes to KVM and GIC version of macros so
they won't clash, but it'd introduce code duplication.  Alternatively,
we could keep macro in, say, GIC header and include it in KVM one (or
vice versa), but such dependency would not look nicer.

So we follow arm64 way (it handles this via sysreg.h) and move only
single set of macros to asm/cp15.h

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 arch/arm/include/asm/arch_gicv3.h |   27 +++++++++++----------------
 arch/arm/include/asm/cp15.h       |   15 +++++++++++++++
 arch/arm/include/asm/kvm_hyp.h    |   15 +--------------
 3 files changed, 27 insertions(+), 30 deletions(-)

diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
index e08d151..af25c32 100644
--- a/arch/arm/include/asm/arch_gicv3.h
+++ b/arch/arm/include/asm/arch_gicv3.h
@@ -22,9 +22,7 @@
 
 #include <linux/io.h>
 #include <asm/barrier.h>
-
-#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
-#define __ACCESS_CP15_64(Op1, CRm)		p15, Op1, %Q0, %R0, CRm
+#include <asm/cp15.h>
 
 #define ICC_EOIR1			__ACCESS_CP15(c12, 0, c12, 1)
 #define ICC_DIR				__ACCESS_CP15(c12, 0, c11, 1)
@@ -102,58 +100,55 @@
 
 static inline void gic_write_eoir(u32 irq)
 {
-	asm volatile("mcr " __stringify(ICC_EOIR1) : : "r" (irq));
+	write_sysreg(irq, ICC_EOIR1);
 	isb();
 }
 
 static inline void gic_write_dir(u32 val)
 {
-	asm volatile("mcr " __stringify(ICC_DIR) : : "r" (val));
+	write_sysreg(val, ICC_DIR);
 	isb();
 }
 
 static inline u32 gic_read_iar(void)
 {
-	u32 irqstat;
+	u32 irqstat = read_sysreg(ICC_IAR1);
 
-	asm volatile("mrc " __stringify(ICC_IAR1) : "=r" (irqstat));
 	dsb(sy);
+
 	return irqstat;
 }
 
 static inline void gic_write_pmr(u32 val)
 {
-	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
+	write_sysreg(val, ICC_PMR);
 }
 
 static inline void gic_write_ctlr(u32 val)
 {
-	asm volatile("mcr " __stringify(ICC_CTLR) : : "r" (val));
+	write_sysreg(val, ICC_CTLR);
 	isb();
 }
 
 static inline void gic_write_grpen1(u32 val)
 {
-	asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
+	write_sysreg(val, ICC_IGRPEN1);
 	isb();
 }
 
 static inline void gic_write_sgi1r(u64 val)
 {
-	asm volatile("mcrr " __stringify(ICC_SGI1R) : : "r" (val));
+	write_sysreg(val, ICC_SGI1R);
 }
 
 static inline u32 gic_read_sre(void)
 {
-	u32 val;
-
-	asm volatile("mrc " __stringify(ICC_SRE) : "=r" (val));
-	return val;
+	return read_sysreg(ICC_SRE);
 }
 
 static inline void gic_write_sre(u32 val)
 {
-	asm volatile("mcr " __stringify(ICC_SRE) : : "r" (val));
+	write_sysreg(val, ICC_SRE);
 	isb();
 }
 
diff --git a/arch/arm/include/asm/cp15.h b/arch/arm/include/asm/cp15.h
index c3f1152..f661732 100644
--- a/arch/arm/include/asm/cp15.h
+++ b/arch/arm/include/asm/cp15.h
@@ -47,6 +47,21 @@
 #define vectors_high()	(0)
 #endif
 
+#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
+	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
+#define __ACCESS_CP15_64(Op1, CRm)		\
+	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
+
+#define __read_sysreg(r, w, c, t) ({				\
+	t __val;						\
+	asm volatile(r " " c : "=r" (__val));			\
+	__val;							\
+})
+#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
+
+#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
+#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
+
 #ifdef CONFIG_CPU_CP15
 
 extern unsigned long cr_alignment;	/* defined in entry-armv.S */
diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
index bd9434e..0b475d2 100644
--- a/arch/arm/include/asm/kvm_hyp.h
+++ b/arch/arm/include/asm/kvm_hyp.h
@@ -20,26 +20,13 @@
 
 #include <linux/compiler.h>
 #include <linux/kvm_host.h>
+#include <asm/cp15.h>
 #include <asm/kvm_mmu.h>
 #include <asm/vfp.h>
 
-#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
-	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
-#define __ACCESS_CP15_64(Op1, CRm)		\
-	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
 #define __ACCESS_VFP(CRn)			\
 	"mrc", "mcr", __stringify(p10, 7, %0, CRn, cr0, 0), u32
 
-#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
-#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
-
-#define __read_sysreg(r, w, c, t) ({				\
-	t __val;						\
-	asm volatile(r " " c : "=r" (__val));			\
-	__val;							\
-})
-#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
-
 #define write_special(v, r)					\
 	asm volatile("msr " __stringify(r) ", %0" : : "r" (v))
 #define read_special(r) ({					\
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 72+ messages in thread
* [PATCH v3 7/8] ARM: Move system register accessors to asm/cp15.h
@ 2016-09-08 16:06   ` Vladimir Murzin
  0 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-08 16:06 UTC (permalink / raw)
  To: linux-arm-kernel

Headers linux/irqchip/arm-gic.v3.h and arch/arm/include/asm/kvm_hyp.h
are included in virt/kvm/arm/hyp/vgic-v3-sr.c and both define macros
called __ACCESS_CP15 and __ACCESS_CP15_64 which obviously creates a
conflict. These macros were introduced independently for GIC and KVM
and, in fact, do the same thing.

As an option we could add prefixes to KVM and GIC version of macros so
they won't clash, but it'd introduce code duplication.  Alternatively,
we could keep macro in, say, GIC header and include it in KVM one (or
vice versa), but such dependency would not look nicer.

So we follow arm64 way (it handles this via sysreg.h) and move only
single set of macros to asm/cp15.h

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 arch/arm/include/asm/arch_gicv3.h |   27 +++++++++++----------------
 arch/arm/include/asm/cp15.h       |   15 +++++++++++++++
 arch/arm/include/asm/kvm_hyp.h    |   15 +--------------
 3 files changed, 27 insertions(+), 30 deletions(-)

diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
index e08d151..af25c32 100644
--- a/arch/arm/include/asm/arch_gicv3.h
+++ b/arch/arm/include/asm/arch_gicv3.h
@@ -22,9 +22,7 @@
 
 #include <linux/io.h>
 #include <asm/barrier.h>
-
-#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
-#define __ACCESS_CP15_64(Op1, CRm)		p15, Op1, %Q0, %R0, CRm
+#include <asm/cp15.h>
 
 #define ICC_EOIR1			__ACCESS_CP15(c12, 0, c12, 1)
 #define ICC_DIR				__ACCESS_CP15(c12, 0, c11, 1)
@@ -102,58 +100,55 @@
 
 static inline void gic_write_eoir(u32 irq)
 {
-	asm volatile("mcr " __stringify(ICC_EOIR1) : : "r" (irq));
+	write_sysreg(irq, ICC_EOIR1);
 	isb();
 }
 
 static inline void gic_write_dir(u32 val)
 {
-	asm volatile("mcr " __stringify(ICC_DIR) : : "r" (val));
+	write_sysreg(val, ICC_DIR);
 	isb();
 }
 
 static inline u32 gic_read_iar(void)
 {
-	u32 irqstat;
+	u32 irqstat = read_sysreg(ICC_IAR1);
 
-	asm volatile("mrc " __stringify(ICC_IAR1) : "=r" (irqstat));
 	dsb(sy);
+
 	return irqstat;
 }
 
 static inline void gic_write_pmr(u32 val)
 {
-	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
+	write_sysreg(val, ICC_PMR);
 }
 
 static inline void gic_write_ctlr(u32 val)
 {
-	asm volatile("mcr " __stringify(ICC_CTLR) : : "r" (val));
+	write_sysreg(val, ICC_CTLR);
 	isb();
 }
 
 static inline void gic_write_grpen1(u32 val)
 {
-	asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
+	write_sysreg(val, ICC_IGRPEN1);
 	isb();
 }
 
 static inline void gic_write_sgi1r(u64 val)
 {
-	asm volatile("mcrr " __stringify(ICC_SGI1R) : : "r" (val));
+	write_sysreg(val, ICC_SGI1R);
 }
 
 static inline u32 gic_read_sre(void)
 {
-	u32 val;
-
-	asm volatile("mrc " __stringify(ICC_SRE) : "=r" (val));
-	return val;
+	return read_sysreg(ICC_SRE);
 }
 
 static inline void gic_write_sre(u32 val)
 {
-	asm volatile("mcr " __stringify(ICC_SRE) : : "r" (val));
+	write_sysreg(val, ICC_SRE);
 	isb();
 }
 
diff --git a/arch/arm/include/asm/cp15.h b/arch/arm/include/asm/cp15.h
index c3f1152..f661732 100644
--- a/arch/arm/include/asm/cp15.h
+++ b/arch/arm/include/asm/cp15.h
@@ -47,6 +47,21 @@
 #define vectors_high()	(0)
 #endif
 
+#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
+	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
+#define __ACCESS_CP15_64(Op1, CRm)		\
+	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
+
+#define __read_sysreg(r, w, c, t) ({				\
+	t __val;						\
+	asm volatile(r " " c : "=r" (__val));			\
+	__val;							\
+})
+#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
+
+#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
+#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
+
 #ifdef CONFIG_CPU_CP15
 
 extern unsigned long cr_alignment;	/* defined in entry-armv.S */
diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
index bd9434e..0b475d2 100644
--- a/arch/arm/include/asm/kvm_hyp.h
+++ b/arch/arm/include/asm/kvm_hyp.h
@@ -20,26 +20,13 @@
 
 #include <linux/compiler.h>
 #include <linux/kvm_host.h>
+#include <asm/cp15.h>
 #include <asm/kvm_mmu.h>
 #include <asm/vfp.h>
 
-#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
-	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
-#define __ACCESS_CP15_64(Op1, CRm)		\
-	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
 #define __ACCESS_VFP(CRn)			\
 	"mrc", "mcr", __stringify(p10, 7, %0, CRn, cr0, 0), u32
 
-#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
-#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
-
-#define __read_sysreg(r, w, c, t) ({				\
-	t __val;						\
-	asm volatile(r " " c : "=r" (__val));			\
-	__val;							\
-})
-#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
-
 #define write_special(v, r)					\
 	asm volatile("msr " __stringify(r) ", %0" : : "r" (v))
 #define read_special(r) ({					\
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 72+ messages in thread
* [PATCH v3 8/8] ARM: KVM: Support vgic-v3
  2016-09-08 16:06 ` Vladimir Murzin
@ 2016-09-08 16:06   ` Vladimir Murzin
  -1 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-08 16:06 UTC (permalink / raw)
  To: kvmarm; +Cc: marc.zyngier, andre.przywara, linux-arm-kernel

This patch allows to build and use vgic-v3 in 32-bit mode.

Unfortunately, it can not be split in several steps without extra
stubs to keep patches independent and bisectable.
For instance, virt/kvm/arm/vgic/vgic-v3.c uses function from
vgic-v3-sr.c, handling access to GICv3 cpu interface from the guest
requires vgic_v3.vgic_sre to be already defined.

It is how support has been done:

* vgic-v3 save/restore routines are written in such way that they map
  arm64 system register naming nicely, but it does not fit to arm
  world. To keep virt/kvm/arm/hyp/vgic-v3-sr.c untouched we create a
  mapping with a function for each register mapping the 32-bit to the
  64-bit accessors

* report configured SRE on access to GICv3 cpu interface from the guest

* required vgic-v3 macros are provided via uapi.h

* static keys are used to select GIC backend

* to make vgic-v3 build KVM_ARM_VGIC_V3 guard is removed along with
  the static inlines

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 arch/arm/include/asm/arch_gicv3.h   |   64 +++++++++++++++++++++++++++++++++++
 arch/arm/include/asm/kvm_asm.h      |    3 ++
 arch/arm/include/asm/kvm_hyp.h      |    3 ++
 arch/arm/include/uapi/asm/kvm.h     |    7 ++++
 arch/arm/kvm/Makefile               |    2 ++
 arch/arm/kvm/coproc.c               |   36 ++++++++++++++++++++
 arch/arm/kvm/hyp/Makefile           |    1 +
 arch/arm/kvm/hyp/switch.c           |   12 +++++--
 arch/arm64/kvm/Kconfig              |    4 ---
 include/kvm/arm_vgic.h              |    8 -----
 virt/kvm/arm/vgic/vgic-kvm-device.c |    8 -----
 virt/kvm/arm/vgic/vgic-mmio.c       |    2 --
 virt/kvm/arm/vgic/vgic-mmio.h       |    2 --
 virt/kvm/arm/vgic/vgic.h            |   54 -----------------------------
 14 files changed, 126 insertions(+), 80 deletions(-)

diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
index af25c32..f93f6bd 100644
--- a/arch/arm/include/asm/arch_gicv3.h
+++ b/arch/arm/include/asm/arch_gicv3.h
@@ -96,6 +96,70 @@
 #define ICH_AP1R2			__AP1Rx(2)
 #define ICH_AP1R3			__AP1Rx(3)
 
+/* A32-to-A64 mappings used by VGIC save/restore */
+
+#define CPUIF_MAP(a32, a64)			\
+static inline void write_ ## a64(u32 val)	\
+{						\
+	write_sysreg(val, a32);			\
+}						\
+static inline u32 read_ ## a64(void)		\
+{						\
+	return read_sysreg(a32); 		\
+}						\
+
+#define CPUIF_MAP_LO_HI(a32lo, a32hi, a64)	\
+static inline void write_ ## a64(u64 val)	\
+{						\
+	write_sysreg((u32)val, a32lo);		\
+	write_sysreg((u32)(val >> 32), a32hi);	\
+}						\
+static inline u64 read_ ## a64(void)		\
+{						\
+	u64 val = read_sysreg(a32lo);		\
+						\
+	val |=	(u64)read_sysreg(a32hi) << 32;	\
+						\
+	return val; 				\
+}
+
+CPUIF_MAP(ICH_HCR, ICH_HCR_EL2)
+CPUIF_MAP(ICH_VTR, ICH_VTR_EL2)
+CPUIF_MAP(ICH_MISR, ICH_MISR_EL2)
+CPUIF_MAP(ICH_EISR, ICH_EISR_EL2)
+CPUIF_MAP(ICH_ELSR, ICH_ELSR_EL2)
+CPUIF_MAP(ICH_VMCR, ICH_VMCR_EL2)
+CPUIF_MAP(ICH_AP0R3, ICH_AP0R3_EL2)
+CPUIF_MAP(ICH_AP0R2, ICH_AP0R2_EL2)
+CPUIF_MAP(ICH_AP0R1, ICH_AP0R1_EL2)
+CPUIF_MAP(ICH_AP0R0, ICH_AP0R0_EL2)
+CPUIF_MAP(ICH_AP1R3, ICH_AP1R3_EL2)
+CPUIF_MAP(ICH_AP1R2, ICH_AP1R2_EL2)
+CPUIF_MAP(ICH_AP1R1, ICH_AP1R1_EL2)
+CPUIF_MAP(ICH_AP1R0, ICH_AP1R0_EL2)
+CPUIF_MAP(ICC_HSRE, ICC_SRE_EL2)
+CPUIF_MAP(ICC_SRE, ICC_SRE_EL1)
+
+CPUIF_MAP_LO_HI(ICH_LR15, ICH_LRC15, ICH_LR15_EL2)
+CPUIF_MAP_LO_HI(ICH_LR14, ICH_LRC14, ICH_LR14_EL2)
+CPUIF_MAP_LO_HI(ICH_LR13, ICH_LRC13, ICH_LR13_EL2)
+CPUIF_MAP_LO_HI(ICH_LR12, ICH_LRC12, ICH_LR12_EL2)
+CPUIF_MAP_LO_HI(ICH_LR11, ICH_LRC11, ICH_LR11_EL2)
+CPUIF_MAP_LO_HI(ICH_LR10, ICH_LRC10, ICH_LR10_EL2)
+CPUIF_MAP_LO_HI(ICH_LR9, ICH_LRC9, ICH_LR9_EL2)
+CPUIF_MAP_LO_HI(ICH_LR8, ICH_LRC8, ICH_LR8_EL2)
+CPUIF_MAP_LO_HI(ICH_LR7, ICH_LRC7, ICH_LR7_EL2)
+CPUIF_MAP_LO_HI(ICH_LR6, ICH_LRC6, ICH_LR6_EL2)
+CPUIF_MAP_LO_HI(ICH_LR5, ICH_LRC5, ICH_LR5_EL2)
+CPUIF_MAP_LO_HI(ICH_LR4, ICH_LRC4, ICH_LR4_EL2)
+CPUIF_MAP_LO_HI(ICH_LR3, ICH_LRC3, ICH_LR3_EL2)
+CPUIF_MAP_LO_HI(ICH_LR2, ICH_LRC2, ICH_LR2_EL2)
+CPUIF_MAP_LO_HI(ICH_LR1, ICH_LRC1, ICH_LR1_EL2)
+CPUIF_MAP_LO_HI(ICH_LR0, ICH_LRC0, ICH_LR0_EL2)
+
+#define read_gicreg(r)                 read_##r()
+#define write_gicreg(v, r)             write_##r(v)
+
 /* Low-level accessors */
 
 static inline void gic_write_eoir(u32 irq)
diff --git a/arch/arm/include/asm/kvm_asm.h b/arch/arm/include/asm/kvm_asm.h
index 58faff5..dfccf94 100644
--- a/arch/arm/include/asm/kvm_asm.h
+++ b/arch/arm/include/asm/kvm_asm.h
@@ -68,6 +68,9 @@ extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
 extern void __init_stage2_translation(void);
 
 extern void __kvm_hyp_reset(unsigned long);
+
+extern u64 __vgic_v3_get_ich_vtr_el2(void);
+extern void __vgic_v3_init_lrs(void);
 #endif
 
 #endif /* __ARM_KVM_ASM_H__ */
diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
index 0b475d2..ed96cf9 100644
--- a/arch/arm/include/asm/kvm_hyp.h
+++ b/arch/arm/include/asm/kvm_hyp.h
@@ -104,6 +104,9 @@ void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
 void __sysreg_save_state(struct kvm_cpu_context *ctxt);
 void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
 
+void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
+void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
+
 void asmlinkage __vfp_save_state(struct vfp_hard_struct *vfp);
 void asmlinkage __vfp_restore_state(struct vfp_hard_struct *vfp);
 static inline bool __vfp_enabled(void)
diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h
index a2b3eb3..b38c10c 100644
--- a/arch/arm/include/uapi/asm/kvm.h
+++ b/arch/arm/include/uapi/asm/kvm.h
@@ -84,6 +84,13 @@ struct kvm_regs {
 #define KVM_VGIC_V2_DIST_SIZE		0x1000
 #define KVM_VGIC_V2_CPU_SIZE		0x2000
 
+/* Supported VGICv3 address types  */
+#define KVM_VGIC_V3_ADDR_TYPE_DIST	2
+#define KVM_VGIC_V3_ADDR_TYPE_REDIST	3
+
+#define KVM_VGIC_V3_DIST_SIZE		SZ_64K
+#define KVM_VGIC_V3_REDIST_SIZE		(2 * SZ_64K)
+
 #define KVM_ARM_VCPU_POWER_OFF		0 /* CPU is started in OFF state */
 #define KVM_ARM_VCPU_PSCI_0_2		1 /* CPU uses PSCI v0.2 */
 
diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
index 10d77a6..043d817f 100644
--- a/arch/arm/kvm/Makefile
+++ b/arch/arm/kvm/Makefile
@@ -26,8 +26,10 @@ obj-y += $(KVM)/arm/vgic/vgic.o
 obj-y += $(KVM)/arm/vgic/vgic-init.o
 obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
 obj-y += $(KVM)/arm/vgic/vgic-v2.o
+obj-y += $(KVM)/arm/vgic/vgic-v3.o
 obj-y += $(KVM)/arm/vgic/vgic-mmio.o
 obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
+obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
 obj-y += $(KVM)/arm/vgic/vgic-kvm-device.o
 obj-y += $(KVM)/irqchip.o
 obj-y += $(KVM)/arm/arch_timer.o
diff --git a/arch/arm/kvm/coproc.c b/arch/arm/kvm/coproc.c
index 1bb2b79..10c0244 100644
--- a/arch/arm/kvm/coproc.c
+++ b/arch/arm/kvm/coproc.c
@@ -228,6 +228,36 @@ bool access_vm_reg(struct kvm_vcpu *vcpu,
 	return true;
 }
 
+static bool access_gic_sgi(struct kvm_vcpu *vcpu,
+			   const struct coproc_params *p,
+			   const struct coproc_reg *r)
+{
+	u64 reg;
+
+	if (!p->is_write)
+		return read_from_write_only(vcpu, p);
+
+	reg = *vcpu_reg(vcpu, p->Rt2);
+	reg <<= 32;
+	reg |= *vcpu_reg(vcpu, p->Rt1) ;
+
+	vgic_v3_dispatch_sgi(vcpu, reg);
+
+	return true;
+}
+
+static bool access_gic_sre(struct kvm_vcpu *vcpu,
+			   const struct coproc_params *p,
+			   const struct coproc_reg *r)
+{
+	if (p->is_write)
+		return ignore_write(vcpu, p);
+
+	*vcpu_reg(vcpu, p->Rt1) = vcpu->arch.vgic_cpu.vgic_v3.vgic_sre;
+
+	return true;
+}
+
 /*
  * We could trap ID_DFR0 and tell the guest we don't support performance
  * monitoring.  Unfortunately the patch to make the kernel check ID_DFR0 was
@@ -361,10 +391,16 @@ static const struct coproc_reg cp15_regs[] = {
 	{ CRn(10), CRm( 3), Op1( 0), Op2( 1), is32,
 			access_vm_reg, reset_unknown, c10_AMAIR1},
 
+	/* ICC_SGI1R */
+	{ CRm64(12), Op1( 0), is64, access_gic_sgi},
+
 	/* VBAR: swapped by interrupt.S. */
 	{ CRn(12), CRm( 0), Op1( 0), Op2( 0), is32,
 			NULL, reset_val, c12_VBAR, 0x00000000 },
 
+	/* ICC_SRE */
+	{ CRn(12), CRm(12), Op1( 0), Op2(5), is32, access_gic_sre },
+
 	/* CONTEXTIDR/TPIDRURW/TPIDRURO/TPIDRPRW: swapped by interrupt.S. */
 	{ CRn(13), CRm( 0), Op1( 0), Op2( 1), is32,
 			access_vm_reg, reset_val, c13_CID, 0x00000000 },
diff --git a/arch/arm/kvm/hyp/Makefile b/arch/arm/kvm/hyp/Makefile
index 8dfa5f7..3023bb5 100644
--- a/arch/arm/kvm/hyp/Makefile
+++ b/arch/arm/kvm/hyp/Makefile
@@ -5,6 +5,7 @@
 KVM=../../../../virt/kvm
 
 obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
+obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
 
 obj-$(CONFIG_KVM_ARM_HOST) += tlb.o
diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
index b13caa9..8409dd5 100644
--- a/arch/arm/kvm/hyp/switch.c
+++ b/arch/arm/kvm/hyp/switch.c
@@ -14,6 +14,7 @@
  * You should have received a copy of the GNU General Public License
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
+#include <linux/jump_label.h>
 
 #include <asm/kvm_asm.h>
 #include <asm/kvm_hyp.h>
@@ -74,14 +75,21 @@ static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
 	write_sysreg(read_sysreg(MIDR), VPIDR);
 }
 
+
 static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
 {
-	__vgic_v2_save_state(vcpu);
+	if (static_branch_unlikely(&kvm_gicv3_cpuif))
+		__vgic_v3_save_state(vcpu);
+	else
+		__vgic_v2_save_state(vcpu);
 }
 
 static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
 {
-	__vgic_v2_restore_state(vcpu);
+	if (static_branch_unlikely(&kvm_gicv3_cpuif))
+		__vgic_v3_restore_state(vcpu);
+	else
+		__vgic_v2_restore_state(vcpu);
 }
 
 static bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu)
diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index 7ba9164..6eaf12c 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -19,9 +19,6 @@ if VIRTUALIZATION
 config KVM_ARM_VGIC_V3_ITS
 	bool
 
-config KVM_ARM_VGIC_V3
-	bool
-
 config KVM
 	bool "Kernel-based Virtual Machine (KVM) support"
 	depends on OF
@@ -37,7 +34,6 @@ config KVM
 	select KVM_VFIO
 	select HAVE_KVM_EVENTFD
 	select HAVE_KVM_IRQFD
-	select KVM_ARM_VGIC_V3
 	select KVM_ARM_VGIC_V3_ITS
 	select KVM_ARM_PMU if HW_PERF_EVENTS
 	select HAVE_KVM_MSI
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 19b698e..7462138 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -217,7 +217,6 @@ struct vgic_v2_cpu_if {
 };
 
 struct vgic_v3_cpu_if {
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 	u32		vgic_hcr;
 	u32		vgic_vmcr;
 	u32		vgic_sre;	/* Restored only, change ignored */
@@ -227,7 +226,6 @@ struct vgic_v3_cpu_if {
 	u32		vgic_ap0r[4];
 	u32		vgic_ap1r[4];
 	u64		vgic_lr[VGIC_V3_MAX_LRS];
-#endif
 };
 
 struct vgic_cpu {
@@ -294,13 +292,7 @@ bool kvm_vcpu_has_pending_irqs(struct kvm_vcpu *vcpu);
 void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu);
 void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu);
 
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg);
-#else
-static inline void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)
-{
-}
-#endif
 
 /**
  * kvm_vgic_get_max_vcpus - Get the maximum number of VCPUs allowed by HW
diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
index f3811b3..4dc026a 100644
--- a/virt/kvm/arm/vgic/vgic-kvm-device.c
+++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
@@ -71,7 +71,6 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write)
 		addr_ptr = &vgic->vgic_cpu_base;
 		alignment = SZ_4K;
 		break;
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 	case KVM_VGIC_V3_ADDR_TYPE_DIST:
 		type_needed = KVM_DEV_TYPE_ARM_VGIC_V3;
 		addr_ptr = &vgic->vgic_dist_base;
@@ -82,7 +81,6 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write)
 		addr_ptr = &vgic->vgic_redist_base;
 		alignment = SZ_64K;
 		break;
-#endif
 	default:
 		r = -ENODEV;
 		goto out;
@@ -219,7 +217,6 @@ int kvm_register_vgic_device(unsigned long type)
 		ret = kvm_register_device_ops(&kvm_arm_vgic_v2_ops,
 					      KVM_DEV_TYPE_ARM_VGIC_V2);
 		break;
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 	case KVM_DEV_TYPE_ARM_VGIC_V3:
 		ret = kvm_register_device_ops(&kvm_arm_vgic_v3_ops,
 					      KVM_DEV_TYPE_ARM_VGIC_V3);
@@ -230,7 +227,6 @@ int kvm_register_vgic_device(unsigned long type)
 		ret = kvm_vgic_register_its_device();
 #endif
 		break;
-#endif
 	}
 
 	return ret;
@@ -392,8 +388,6 @@ struct kvm_device_ops kvm_arm_vgic_v2_ops = {
 
 /* V3 ops */
 
-#ifdef CONFIG_KVM_ARM_VGIC_V3
-
 static int vgic_v3_set_attr(struct kvm_device *dev,
 			    struct kvm_device_attr *attr)
 {
@@ -436,5 +430,3 @@ struct kvm_device_ops kvm_arm_vgic_v3_ops = {
 	.get_attr = vgic_v3_get_attr,
 	.has_attr = vgic_v3_has_attr,
 };
-
-#endif /* CONFIG_KVM_ARM_VGIC_V3 */
diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
index 3bad3c5..e18b30d 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.c
+++ b/virt/kvm/arm/vgic/vgic-mmio.c
@@ -550,11 +550,9 @@ int vgic_register_dist_iodev(struct kvm *kvm, gpa_t dist_base_address,
 	case VGIC_V2:
 		len = vgic_v2_init_dist_iodev(io_device);
 		break;
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 	case VGIC_V3:
 		len = vgic_v3_init_dist_iodev(io_device);
 		break;
-#endif
 	default:
 		BUG_ON(1);
 	}
diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
index 80f92ce..4c34d39 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.h
+++ b/virt/kvm/arm/vgic/vgic-mmio.h
@@ -162,12 +162,10 @@ unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
 
 unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
 
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 u64 vgic_sanitise_outer_cacheability(u64 reg);
 u64 vgic_sanitise_inner_cacheability(u64 reg);
 u64 vgic_sanitise_shareability(u64 reg);
 u64 vgic_sanitise_field(u64 reg, u64 field_mask, int field_shift,
 			u64 (*sanitise_fn)(u64));
-#endif
 
 #endif
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index 100045f..9d9e014 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -72,7 +72,6 @@ static inline void vgic_get_irq_kref(struct vgic_irq *irq)
 	kref_get(&irq->refcount);
 }
 
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu);
 void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu);
 void vgic_v3_populate_lr(struct kvm_vcpu *vcpu, struct vgic_irq *irq, int lr);
@@ -91,60 +90,7 @@ bool vgic_has_its(struct kvm *kvm);
 int kvm_vgic_register_its_device(void);
 void vgic_enable_lpis(struct kvm_vcpu *vcpu);
 int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi);
-#endif
-
 #else
-static inline void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu)
-{
-}
-
-static inline void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu)
-{
-}
-
-static inline void vgic_v3_populate_lr(struct kvm_vcpu *vcpu,
-				       struct vgic_irq *irq, int lr)
-{
-}
-
-static inline void vgic_v3_clear_lr(struct kvm_vcpu *vcpu, int lr)
-{
-}
-
-static inline void vgic_v3_set_underflow(struct kvm_vcpu *vcpu)
-{
-}
-
-static inline
-void vgic_v3_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
-{
-}
-
-static inline
-void vgic_v3_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
-{
-}
-
-static inline void vgic_v3_enable(struct kvm_vcpu *vcpu)
-{
-}
-
-static inline int vgic_v3_probe(const struct gic_kvm_info *info)
-{
-	return -ENODEV;
-}
-
-static inline int vgic_v3_map_resources(struct kvm *kvm)
-{
-	return -ENODEV;
-}
-
-static inline int vgic_register_redist_iodevs(struct kvm *kvm,
-					      gpa_t dist_base_address)
-{
-	return -ENODEV;
-}
-
 static inline int vgic_register_its_iodevs(struct kvm *kvm)
 {
 	return -ENODEV;
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 72+ messages in thread
* [PATCH v3 8/8] ARM: KVM: Support vgic-v3
@ 2016-09-08 16:06   ` Vladimir Murzin
  0 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-08 16:06 UTC (permalink / raw)
  To: linux-arm-kernel

This patch allows to build and use vgic-v3 in 32-bit mode.

Unfortunately, it can not be split in several steps without extra
stubs to keep patches independent and bisectable.
For instance, virt/kvm/arm/vgic/vgic-v3.c uses function from
vgic-v3-sr.c, handling access to GICv3 cpu interface from the guest
requires vgic_v3.vgic_sre to be already defined.

It is how support has been done:

* vgic-v3 save/restore routines are written in such way that they map
  arm64 system register naming nicely, but it does not fit to arm
  world. To keep virt/kvm/arm/hyp/vgic-v3-sr.c untouched we create a
  mapping with a function for each register mapping the 32-bit to the
  64-bit accessors

* report configured SRE on access to GICv3 cpu interface from the guest

* required vgic-v3 macros are provided via uapi.h

* static keys are used to select GIC backend

* to make vgic-v3 build KVM_ARM_VGIC_V3 guard is removed along with
  the static inlines

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 arch/arm/include/asm/arch_gicv3.h   |   64 +++++++++++++++++++++++++++++++++++
 arch/arm/include/asm/kvm_asm.h      |    3 ++
 arch/arm/include/asm/kvm_hyp.h      |    3 ++
 arch/arm/include/uapi/asm/kvm.h     |    7 ++++
 arch/arm/kvm/Makefile               |    2 ++
 arch/arm/kvm/coproc.c               |   36 ++++++++++++++++++++
 arch/arm/kvm/hyp/Makefile           |    1 +
 arch/arm/kvm/hyp/switch.c           |   12 +++++--
 arch/arm64/kvm/Kconfig              |    4 ---
 include/kvm/arm_vgic.h              |    8 -----
 virt/kvm/arm/vgic/vgic-kvm-device.c |    8 -----
 virt/kvm/arm/vgic/vgic-mmio.c       |    2 --
 virt/kvm/arm/vgic/vgic-mmio.h       |    2 --
 virt/kvm/arm/vgic/vgic.h            |   54 -----------------------------
 14 files changed, 126 insertions(+), 80 deletions(-)

diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
index af25c32..f93f6bd 100644
--- a/arch/arm/include/asm/arch_gicv3.h
+++ b/arch/arm/include/asm/arch_gicv3.h
@@ -96,6 +96,70 @@
 #define ICH_AP1R2			__AP1Rx(2)
 #define ICH_AP1R3			__AP1Rx(3)
 
+/* A32-to-A64 mappings used by VGIC save/restore */
+
+#define CPUIF_MAP(a32, a64)			\
+static inline void write_ ## a64(u32 val)	\
+{						\
+	write_sysreg(val, a32);			\
+}						\
+static inline u32 read_ ## a64(void)		\
+{						\
+	return read_sysreg(a32); 		\
+}						\
+
+#define CPUIF_MAP_LO_HI(a32lo, a32hi, a64)	\
+static inline void write_ ## a64(u64 val)	\
+{						\
+	write_sysreg((u32)val, a32lo);		\
+	write_sysreg((u32)(val >> 32), a32hi);	\
+}						\
+static inline u64 read_ ## a64(void)		\
+{						\
+	u64 val = read_sysreg(a32lo);		\
+						\
+	val |=	(u64)read_sysreg(a32hi) << 32;	\
+						\
+	return val; 				\
+}
+
+CPUIF_MAP(ICH_HCR, ICH_HCR_EL2)
+CPUIF_MAP(ICH_VTR, ICH_VTR_EL2)
+CPUIF_MAP(ICH_MISR, ICH_MISR_EL2)
+CPUIF_MAP(ICH_EISR, ICH_EISR_EL2)
+CPUIF_MAP(ICH_ELSR, ICH_ELSR_EL2)
+CPUIF_MAP(ICH_VMCR, ICH_VMCR_EL2)
+CPUIF_MAP(ICH_AP0R3, ICH_AP0R3_EL2)
+CPUIF_MAP(ICH_AP0R2, ICH_AP0R2_EL2)
+CPUIF_MAP(ICH_AP0R1, ICH_AP0R1_EL2)
+CPUIF_MAP(ICH_AP0R0, ICH_AP0R0_EL2)
+CPUIF_MAP(ICH_AP1R3, ICH_AP1R3_EL2)
+CPUIF_MAP(ICH_AP1R2, ICH_AP1R2_EL2)
+CPUIF_MAP(ICH_AP1R1, ICH_AP1R1_EL2)
+CPUIF_MAP(ICH_AP1R0, ICH_AP1R0_EL2)
+CPUIF_MAP(ICC_HSRE, ICC_SRE_EL2)
+CPUIF_MAP(ICC_SRE, ICC_SRE_EL1)
+
+CPUIF_MAP_LO_HI(ICH_LR15, ICH_LRC15, ICH_LR15_EL2)
+CPUIF_MAP_LO_HI(ICH_LR14, ICH_LRC14, ICH_LR14_EL2)
+CPUIF_MAP_LO_HI(ICH_LR13, ICH_LRC13, ICH_LR13_EL2)
+CPUIF_MAP_LO_HI(ICH_LR12, ICH_LRC12, ICH_LR12_EL2)
+CPUIF_MAP_LO_HI(ICH_LR11, ICH_LRC11, ICH_LR11_EL2)
+CPUIF_MAP_LO_HI(ICH_LR10, ICH_LRC10, ICH_LR10_EL2)
+CPUIF_MAP_LO_HI(ICH_LR9, ICH_LRC9, ICH_LR9_EL2)
+CPUIF_MAP_LO_HI(ICH_LR8, ICH_LRC8, ICH_LR8_EL2)
+CPUIF_MAP_LO_HI(ICH_LR7, ICH_LRC7, ICH_LR7_EL2)
+CPUIF_MAP_LO_HI(ICH_LR6, ICH_LRC6, ICH_LR6_EL2)
+CPUIF_MAP_LO_HI(ICH_LR5, ICH_LRC5, ICH_LR5_EL2)
+CPUIF_MAP_LO_HI(ICH_LR4, ICH_LRC4, ICH_LR4_EL2)
+CPUIF_MAP_LO_HI(ICH_LR3, ICH_LRC3, ICH_LR3_EL2)
+CPUIF_MAP_LO_HI(ICH_LR2, ICH_LRC2, ICH_LR2_EL2)
+CPUIF_MAP_LO_HI(ICH_LR1, ICH_LRC1, ICH_LR1_EL2)
+CPUIF_MAP_LO_HI(ICH_LR0, ICH_LRC0, ICH_LR0_EL2)
+
+#define read_gicreg(r)                 read_##r()
+#define write_gicreg(v, r)             write_##r(v)
+
 /* Low-level accessors */
 
 static inline void gic_write_eoir(u32 irq)
diff --git a/arch/arm/include/asm/kvm_asm.h b/arch/arm/include/asm/kvm_asm.h
index 58faff5..dfccf94 100644
--- a/arch/arm/include/asm/kvm_asm.h
+++ b/arch/arm/include/asm/kvm_asm.h
@@ -68,6 +68,9 @@ extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
 extern void __init_stage2_translation(void);
 
 extern void __kvm_hyp_reset(unsigned long);
+
+extern u64 __vgic_v3_get_ich_vtr_el2(void);
+extern void __vgic_v3_init_lrs(void);
 #endif
 
 #endif /* __ARM_KVM_ASM_H__ */
diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
index 0b475d2..ed96cf9 100644
--- a/arch/arm/include/asm/kvm_hyp.h
+++ b/arch/arm/include/asm/kvm_hyp.h
@@ -104,6 +104,9 @@ void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
 void __sysreg_save_state(struct kvm_cpu_context *ctxt);
 void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
 
+void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
+void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
+
 void asmlinkage __vfp_save_state(struct vfp_hard_struct *vfp);
 void asmlinkage __vfp_restore_state(struct vfp_hard_struct *vfp);
 static inline bool __vfp_enabled(void)
diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h
index a2b3eb3..b38c10c 100644
--- a/arch/arm/include/uapi/asm/kvm.h
+++ b/arch/arm/include/uapi/asm/kvm.h
@@ -84,6 +84,13 @@ struct kvm_regs {
 #define KVM_VGIC_V2_DIST_SIZE		0x1000
 #define KVM_VGIC_V2_CPU_SIZE		0x2000
 
+/* Supported VGICv3 address types  */
+#define KVM_VGIC_V3_ADDR_TYPE_DIST	2
+#define KVM_VGIC_V3_ADDR_TYPE_REDIST	3
+
+#define KVM_VGIC_V3_DIST_SIZE		SZ_64K
+#define KVM_VGIC_V3_REDIST_SIZE		(2 * SZ_64K)
+
 #define KVM_ARM_VCPU_POWER_OFF		0 /* CPU is started in OFF state */
 #define KVM_ARM_VCPU_PSCI_0_2		1 /* CPU uses PSCI v0.2 */
 
diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
index 10d77a6..043d817f 100644
--- a/arch/arm/kvm/Makefile
+++ b/arch/arm/kvm/Makefile
@@ -26,8 +26,10 @@ obj-y += $(KVM)/arm/vgic/vgic.o
 obj-y += $(KVM)/arm/vgic/vgic-init.o
 obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
 obj-y += $(KVM)/arm/vgic/vgic-v2.o
+obj-y += $(KVM)/arm/vgic/vgic-v3.o
 obj-y += $(KVM)/arm/vgic/vgic-mmio.o
 obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
+obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
 obj-y += $(KVM)/arm/vgic/vgic-kvm-device.o
 obj-y += $(KVM)/irqchip.o
 obj-y += $(KVM)/arm/arch_timer.o
diff --git a/arch/arm/kvm/coproc.c b/arch/arm/kvm/coproc.c
index 1bb2b79..10c0244 100644
--- a/arch/arm/kvm/coproc.c
+++ b/arch/arm/kvm/coproc.c
@@ -228,6 +228,36 @@ bool access_vm_reg(struct kvm_vcpu *vcpu,
 	return true;
 }
 
+static bool access_gic_sgi(struct kvm_vcpu *vcpu,
+			   const struct coproc_params *p,
+			   const struct coproc_reg *r)
+{
+	u64 reg;
+
+	if (!p->is_write)
+		return read_from_write_only(vcpu, p);
+
+	reg = *vcpu_reg(vcpu, p->Rt2);
+	reg <<= 32;
+	reg |= *vcpu_reg(vcpu, p->Rt1) ;
+
+	vgic_v3_dispatch_sgi(vcpu, reg);
+
+	return true;
+}
+
+static bool access_gic_sre(struct kvm_vcpu *vcpu,
+			   const struct coproc_params *p,
+			   const struct coproc_reg *r)
+{
+	if (p->is_write)
+		return ignore_write(vcpu, p);
+
+	*vcpu_reg(vcpu, p->Rt1) = vcpu->arch.vgic_cpu.vgic_v3.vgic_sre;
+
+	return true;
+}
+
 /*
  * We could trap ID_DFR0 and tell the guest we don't support performance
  * monitoring.  Unfortunately the patch to make the kernel check ID_DFR0 was
@@ -361,10 +391,16 @@ static const struct coproc_reg cp15_regs[] = {
 	{ CRn(10), CRm( 3), Op1( 0), Op2( 1), is32,
 			access_vm_reg, reset_unknown, c10_AMAIR1},
 
+	/* ICC_SGI1R */
+	{ CRm64(12), Op1( 0), is64, access_gic_sgi},
+
 	/* VBAR: swapped by interrupt.S. */
 	{ CRn(12), CRm( 0), Op1( 0), Op2( 0), is32,
 			NULL, reset_val, c12_VBAR, 0x00000000 },
 
+	/* ICC_SRE */
+	{ CRn(12), CRm(12), Op1( 0), Op2(5), is32, access_gic_sre },
+
 	/* CONTEXTIDR/TPIDRURW/TPIDRURO/TPIDRPRW: swapped by interrupt.S. */
 	{ CRn(13), CRm( 0), Op1( 0), Op2( 1), is32,
 			access_vm_reg, reset_val, c13_CID, 0x00000000 },
diff --git a/arch/arm/kvm/hyp/Makefile b/arch/arm/kvm/hyp/Makefile
index 8dfa5f7..3023bb5 100644
--- a/arch/arm/kvm/hyp/Makefile
+++ b/arch/arm/kvm/hyp/Makefile
@@ -5,6 +5,7 @@
 KVM=../../../../virt/kvm
 
 obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
+obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
 
 obj-$(CONFIG_KVM_ARM_HOST) += tlb.o
diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
index b13caa9..8409dd5 100644
--- a/arch/arm/kvm/hyp/switch.c
+++ b/arch/arm/kvm/hyp/switch.c
@@ -14,6 +14,7 @@
  * You should have received a copy of the GNU General Public License
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
+#include <linux/jump_label.h>
 
 #include <asm/kvm_asm.h>
 #include <asm/kvm_hyp.h>
@@ -74,14 +75,21 @@ static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
 	write_sysreg(read_sysreg(MIDR), VPIDR);
 }
 
+
 static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
 {
-	__vgic_v2_save_state(vcpu);
+	if (static_branch_unlikely(&kvm_gicv3_cpuif))
+		__vgic_v3_save_state(vcpu);
+	else
+		__vgic_v2_save_state(vcpu);
 }
 
 static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
 {
-	__vgic_v2_restore_state(vcpu);
+	if (static_branch_unlikely(&kvm_gicv3_cpuif))
+		__vgic_v3_restore_state(vcpu);
+	else
+		__vgic_v2_restore_state(vcpu);
 }
 
 static bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu)
diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index 7ba9164..6eaf12c 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -19,9 +19,6 @@ if VIRTUALIZATION
 config KVM_ARM_VGIC_V3_ITS
 	bool
 
-config KVM_ARM_VGIC_V3
-	bool
-
 config KVM
 	bool "Kernel-based Virtual Machine (KVM) support"
 	depends on OF
@@ -37,7 +34,6 @@ config KVM
 	select KVM_VFIO
 	select HAVE_KVM_EVENTFD
 	select HAVE_KVM_IRQFD
-	select KVM_ARM_VGIC_V3
 	select KVM_ARM_VGIC_V3_ITS
 	select KVM_ARM_PMU if HW_PERF_EVENTS
 	select HAVE_KVM_MSI
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 19b698e..7462138 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -217,7 +217,6 @@ struct vgic_v2_cpu_if {
 };
 
 struct vgic_v3_cpu_if {
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 	u32		vgic_hcr;
 	u32		vgic_vmcr;
 	u32		vgic_sre;	/* Restored only, change ignored */
@@ -227,7 +226,6 @@ struct vgic_v3_cpu_if {
 	u32		vgic_ap0r[4];
 	u32		vgic_ap1r[4];
 	u64		vgic_lr[VGIC_V3_MAX_LRS];
-#endif
 };
 
 struct vgic_cpu {
@@ -294,13 +292,7 @@ bool kvm_vcpu_has_pending_irqs(struct kvm_vcpu *vcpu);
 void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu);
 void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu);
 
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg);
-#else
-static inline void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)
-{
-}
-#endif
 
 /**
  * kvm_vgic_get_max_vcpus - Get the maximum number of VCPUs allowed by HW
diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
index f3811b3..4dc026a 100644
--- a/virt/kvm/arm/vgic/vgic-kvm-device.c
+++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
@@ -71,7 +71,6 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write)
 		addr_ptr = &vgic->vgic_cpu_base;
 		alignment = SZ_4K;
 		break;
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 	case KVM_VGIC_V3_ADDR_TYPE_DIST:
 		type_needed = KVM_DEV_TYPE_ARM_VGIC_V3;
 		addr_ptr = &vgic->vgic_dist_base;
@@ -82,7 +81,6 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write)
 		addr_ptr = &vgic->vgic_redist_base;
 		alignment = SZ_64K;
 		break;
-#endif
 	default:
 		r = -ENODEV;
 		goto out;
@@ -219,7 +217,6 @@ int kvm_register_vgic_device(unsigned long type)
 		ret = kvm_register_device_ops(&kvm_arm_vgic_v2_ops,
 					      KVM_DEV_TYPE_ARM_VGIC_V2);
 		break;
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 	case KVM_DEV_TYPE_ARM_VGIC_V3:
 		ret = kvm_register_device_ops(&kvm_arm_vgic_v3_ops,
 					      KVM_DEV_TYPE_ARM_VGIC_V3);
@@ -230,7 +227,6 @@ int kvm_register_vgic_device(unsigned long type)
 		ret = kvm_vgic_register_its_device();
 #endif
 		break;
-#endif
 	}
 
 	return ret;
@@ -392,8 +388,6 @@ struct kvm_device_ops kvm_arm_vgic_v2_ops = {
 
 /* V3 ops */
 
-#ifdef CONFIG_KVM_ARM_VGIC_V3
-
 static int vgic_v3_set_attr(struct kvm_device *dev,
 			    struct kvm_device_attr *attr)
 {
@@ -436,5 +430,3 @@ struct kvm_device_ops kvm_arm_vgic_v3_ops = {
 	.get_attr = vgic_v3_get_attr,
 	.has_attr = vgic_v3_has_attr,
 };
-
-#endif /* CONFIG_KVM_ARM_VGIC_V3 */
diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
index 3bad3c5..e18b30d 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.c
+++ b/virt/kvm/arm/vgic/vgic-mmio.c
@@ -550,11 +550,9 @@ int vgic_register_dist_iodev(struct kvm *kvm, gpa_t dist_base_address,
 	case VGIC_V2:
 		len = vgic_v2_init_dist_iodev(io_device);
 		break;
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 	case VGIC_V3:
 		len = vgic_v3_init_dist_iodev(io_device);
 		break;
-#endif
 	default:
 		BUG_ON(1);
 	}
diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
index 80f92ce..4c34d39 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.h
+++ b/virt/kvm/arm/vgic/vgic-mmio.h
@@ -162,12 +162,10 @@ unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
 
 unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
 
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 u64 vgic_sanitise_outer_cacheability(u64 reg);
 u64 vgic_sanitise_inner_cacheability(u64 reg);
 u64 vgic_sanitise_shareability(u64 reg);
 u64 vgic_sanitise_field(u64 reg, u64 field_mask, int field_shift,
 			u64 (*sanitise_fn)(u64));
-#endif
 
 #endif
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index 100045f..9d9e014 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -72,7 +72,6 @@ static inline void vgic_get_irq_kref(struct vgic_irq *irq)
 	kref_get(&irq->refcount);
 }
 
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu);
 void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu);
 void vgic_v3_populate_lr(struct kvm_vcpu *vcpu, struct vgic_irq *irq, int lr);
@@ -91,60 +90,7 @@ bool vgic_has_its(struct kvm *kvm);
 int kvm_vgic_register_its_device(void);
 void vgic_enable_lpis(struct kvm_vcpu *vcpu);
 int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi);
-#endif
-
 #else
-static inline void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu)
-{
-}
-
-static inline void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu)
-{
-}
-
-static inline void vgic_v3_populate_lr(struct kvm_vcpu *vcpu,
-				       struct vgic_irq *irq, int lr)
-{
-}
-
-static inline void vgic_v3_clear_lr(struct kvm_vcpu *vcpu, int lr)
-{
-}
-
-static inline void vgic_v3_set_underflow(struct kvm_vcpu *vcpu)
-{
-}
-
-static inline
-void vgic_v3_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
-{
-}
-
-static inline
-void vgic_v3_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
-{
-}
-
-static inline void vgic_v3_enable(struct kvm_vcpu *vcpu)
-{
-}
-
-static inline int vgic_v3_probe(const struct gic_kvm_info *info)
-{
-	return -ENODEV;
-}
-
-static inline int vgic_v3_map_resources(struct kvm *kvm)
-{
-	return -ENODEV;
-}
-
-static inline int vgic_register_redist_iodevs(struct kvm *kvm,
-					      gpa_t dist_base_address)
-{
-	return -ENODEV;
-}
-
 static inline int vgic_register_its_iodevs(struct kvm *kvm)
 {
 	return -ENODEV;
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 0/8] ARM: KVM: Support for vgic-v3
  2016-09-08 16:06 ` Vladimir Murzin
@ 2016-09-09  7:58   ` Vladimir Murzin
  -1 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-09  7:58 UTC (permalink / raw)
  To: kvmarm; +Cc: marc.zyngier, andre.przywara, linux-arm-kernel

+Christoffer

I should have checked my send-email command twice...

Vladimir

On 08/09/16 17:06, Vladimir Murzin wrote:
> Hi,
> 
> This is an attempt to make use vgic-v3 under arch/arm since
> save-restore functionality got re-written in C and can be shared
> between arm/arm64 like it has already been done for vgic-v2 and timer.
> 
> With this patches I'm able to get 32 core an AArch32 ARMv8 guest boot:
> 
> ...
> GICv3: CPU31: found redistributor 703 region 0:0x000000003ffd0000
> CPU31: thread -1, cpu 3, socket 7, mpidr 80000703
> Brought up 32 CPUs
> SMP: Total of 32 processors activated (768.00 BogoMIPS).
> CPU: All CPU(s) started in SVC mode.
> ...
> 
> Additionally, quite lightweight test based on Self IPI guest test[1]
> has been run with up to 255 cpus.
> 
> [1] http://www.spinics.net/lists/kvm/msg128974.html
> 
> Changelog:
> 
>     v2 -> v3
>        - rebased on v4.8-rc5
>        - commit messages are reworked to be more precise and clear (I
>          hope so)
>        - MPIDR_HWID_BITMASK is used to discard Aff3 in
>          MPIDR_AFFINITY_LEVEL macro
>        - cast to u64 is used instead of abuse of GENMASK_ULL while
>          building value for typer register
>        - static keys are used to select GIC backend
>        - config option to guard ITS code is moved to separate patch
> 
>     v1 -> v2
>        - rebased on v4.8-rc2
>        - introduced guard for ITS code
> 
> Thanks!
> 
> Vladimir Murzin (8):
>   arm64: KVM: Use static keys for selecting the GIC backend
>   arm64: KVM: Move GIC accessors to arch_gicv3.h
>   arm64: KVM: Move vgic-v3 save/restore to virt/kvm/arm/hyp
>   KVM: arm64: vgic-its: Introduce config option to guard ITS specific
>     code
>   KVM: arm: vgic: Support 64-bit data manipulation on 32-bit host
>     systems
>   ARM: Change MPIDR_AFFINITY_LEVEL to ignore Aff3
>   ARM: Move system register accessors to asm/cp15.h
>   ARM: KVM: Support vgic-v3
> 
>  arch/arm/include/asm/arch_gicv3.h                 |   91 +++++++++++++++++----
>  arch/arm/include/asm/cp15.h                       |   15 ++++
>  arch/arm/include/asm/cputype.h                    |    3 +-
>  arch/arm/include/asm/kvm_asm.h                    |    3 +
>  arch/arm/include/asm/kvm_host.h                   |   13 +++
>  arch/arm/include/asm/kvm_hyp.h                    |   20 +----
>  arch/arm/include/asm/virt.h                       |    8 ++
>  arch/arm/include/uapi/asm/kvm.h                   |    7 ++
>  arch/arm/kernel/vmlinux.lds.S                     |    6 ++
>  arch/arm/kvm/Makefile                             |    2 +
>  arch/arm/kvm/arm.c                                |   19 +++++
>  arch/arm/kvm/coproc.c                             |   36 ++++++++
>  arch/arm/kvm/hyp/Makefile                         |    1 +
>  arch/arm/kvm/hyp/switch.c                         |   12 ++-
>  arch/arm64/include/asm/arch_gicv3.h               |   13 +++
>  arch/arm64/include/asm/kvm_host.h                 |   15 ++++
>  arch/arm64/include/asm/kvm_hyp.h                  |    2 -
>  arch/arm64/include/asm/virt.h                     |    7 ++
>  arch/arm64/kernel/vmlinux.lds.S                   |    6 ++
>  arch/arm64/kvm/Kconfig                            |    4 +-
>  arch/arm64/kvm/hyp/Makefile                       |    2 +-
>  arch/arm64/kvm/hyp/switch.c                       |   19 ++---
>  include/kvm/arm_vgic.h                            |    8 --
>  {arch/arm64/kvm => virt/kvm/arm}/hyp/vgic-v3-sr.c |   13 ---
>  virt/kvm/arm/vgic/vgic-kvm-device.c               |   11 +--
>  virt/kvm/arm/vgic/vgic-mmio-v3.c                  |    8 +-
>  virt/kvm/arm/vgic/vgic-mmio.c                     |    2 -
>  virt/kvm/arm/vgic/vgic-mmio.h                     |    4 +-
>  virt/kvm/arm/vgic/vgic.h                          |   54 +-----------
>  29 files changed, 265 insertions(+), 139 deletions(-)
>  rename {arch/arm64/kvm => virt/kvm/arm}/hyp/vgic-v3-sr.c (96%)
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 0/8] ARM: KVM: Support for vgic-v3
@ 2016-09-09  7:58   ` Vladimir Murzin
  0 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-09  7:58 UTC (permalink / raw)
  To: linux-arm-kernel

+Christoffer

I should have checked my send-email command twice...

Vladimir

On 08/09/16 17:06, Vladimir Murzin wrote:
> Hi,
> 
> This is an attempt to make use vgic-v3 under arch/arm since
> save-restore functionality got re-written in C and can be shared
> between arm/arm64 like it has already been done for vgic-v2 and timer.
> 
> With this patches I'm able to get 32 core an AArch32 ARMv8 guest boot:
> 
> ...
> GICv3: CPU31: found redistributor 703 region 0:0x000000003ffd0000
> CPU31: thread -1, cpu 3, socket 7, mpidr 80000703
> Brought up 32 CPUs
> SMP: Total of 32 processors activated (768.00 BogoMIPS).
> CPU: All CPU(s) started in SVC mode.
> ...
> 
> Additionally, quite lightweight test based on Self IPI guest test[1]
> has been run with up to 255 cpus.
> 
> [1] http://www.spinics.net/lists/kvm/msg128974.html
> 
> Changelog:
> 
>     v2 -> v3
>        - rebased on v4.8-rc5
>        - commit messages are reworked to be more precise and clear (I
>          hope so)
>        - MPIDR_HWID_BITMASK is used to discard Aff3 in
>          MPIDR_AFFINITY_LEVEL macro
>        - cast to u64 is used instead of abuse of GENMASK_ULL while
>          building value for typer register
>        - static keys are used to select GIC backend
>        - config option to guard ITS code is moved to separate patch
> 
>     v1 -> v2
>        - rebased on v4.8-rc2
>        - introduced guard for ITS code
> 
> Thanks!
> 
> Vladimir Murzin (8):
>   arm64: KVM: Use static keys for selecting the GIC backend
>   arm64: KVM: Move GIC accessors to arch_gicv3.h
>   arm64: KVM: Move vgic-v3 save/restore to virt/kvm/arm/hyp
>   KVM: arm64: vgic-its: Introduce config option to guard ITS specific
>     code
>   KVM: arm: vgic: Support 64-bit data manipulation on 32-bit host
>     systems
>   ARM: Change MPIDR_AFFINITY_LEVEL to ignore Aff3
>   ARM: Move system register accessors to asm/cp15.h
>   ARM: KVM: Support vgic-v3
> 
>  arch/arm/include/asm/arch_gicv3.h                 |   91 +++++++++++++++++----
>  arch/arm/include/asm/cp15.h                       |   15 ++++
>  arch/arm/include/asm/cputype.h                    |    3 +-
>  arch/arm/include/asm/kvm_asm.h                    |    3 +
>  arch/arm/include/asm/kvm_host.h                   |   13 +++
>  arch/arm/include/asm/kvm_hyp.h                    |   20 +----
>  arch/arm/include/asm/virt.h                       |    8 ++
>  arch/arm/include/uapi/asm/kvm.h                   |    7 ++
>  arch/arm/kernel/vmlinux.lds.S                     |    6 ++
>  arch/arm/kvm/Makefile                             |    2 +
>  arch/arm/kvm/arm.c                                |   19 +++++
>  arch/arm/kvm/coproc.c                             |   36 ++++++++
>  arch/arm/kvm/hyp/Makefile                         |    1 +
>  arch/arm/kvm/hyp/switch.c                         |   12 ++-
>  arch/arm64/include/asm/arch_gicv3.h               |   13 +++
>  arch/arm64/include/asm/kvm_host.h                 |   15 ++++
>  arch/arm64/include/asm/kvm_hyp.h                  |    2 -
>  arch/arm64/include/asm/virt.h                     |    7 ++
>  arch/arm64/kernel/vmlinux.lds.S                   |    6 ++
>  arch/arm64/kvm/Kconfig                            |    4 +-
>  arch/arm64/kvm/hyp/Makefile                       |    2 +-
>  arch/arm64/kvm/hyp/switch.c                       |   19 ++---
>  include/kvm/arm_vgic.h                            |    8 --
>  {arch/arm64/kvm => virt/kvm/arm}/hyp/vgic-v3-sr.c |   13 ---
>  virt/kvm/arm/vgic/vgic-kvm-device.c               |   11 +--
>  virt/kvm/arm/vgic/vgic-mmio-v3.c                  |    8 +-
>  virt/kvm/arm/vgic/vgic-mmio.c                     |    2 -
>  virt/kvm/arm/vgic/vgic-mmio.h                     |    4 +-
>  virt/kvm/arm/vgic/vgic.h                          |   54 +-----------
>  29 files changed, 265 insertions(+), 139 deletions(-)
>  rename {arch/arm64/kvm => virt/kvm/arm}/hyp/vgic-v3-sr.c (96%)
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 1/8] arm64: KVM: Use static keys for selecting the GIC backend
  2016-09-08 16:06   ` Vladimir Murzin
@ 2016-09-09  9:19     ` Marc Zyngier
  -1 siblings, 0 replies; 72+ messages in thread
From: Marc Zyngier @ 2016-09-09  9:19 UTC (permalink / raw)
  To: Vladimir Murzin, kvmarm; +Cc: andre.przywara, linux-arm-kernel

Hi Vladimir,

On 08/09/16 17:06, Vladimir Murzin wrote:
> Currently GIC backend is selected via alternative framework and this
> is fine. We are going to introduce vgic-v3 to 32-bit world and there
> we don't have patching framework in hand, so we can either check
> support for GICv3 every time we need to choose which backend to use or
> try to optimise it by using static keys. The later looks quite
> promising because we can share logic involved in selecting GIC backend
> between architectures if both uses static keys.
> 
> This patch moves arm64 from alternative to static keys framework for
> selecting GIC backend. To make static keys work on hyp side we need to
> make sure that hyp can access to the key which is RW data. For that
> purpose introduce __hyp_data section we can map to hyp and place the
> key there.
> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> ---
>  arch/arm/include/asm/kvm_host.h   |   13 +++++++++++++
>  arch/arm/include/asm/kvm_hyp.h    |    2 --
>  arch/arm/include/asm/virt.h       |    8 ++++++++
>  arch/arm/kernel/vmlinux.lds.S     |    6 ++++++
>  arch/arm/kvm/arm.c                |   19 +++++++++++++++++++
>  arch/arm64/include/asm/kvm_host.h |   15 +++++++++++++++
>  arch/arm64/include/asm/kvm_hyp.h  |    2 --
>  arch/arm64/include/asm/virt.h     |    7 +++++++
>  arch/arm64/kernel/vmlinux.lds.S   |    6 ++++++
>  arch/arm64/kvm/hyp/switch.c       |   19 +++++++++----------
>  10 files changed, 83 insertions(+), 14 deletions(-)
> 
> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> index de338d9..bfa6eec 100644
> --- a/arch/arm/include/asm/kvm_host.h
> +++ b/arch/arm/include/asm/kvm_host.h
> @@ -21,11 +21,14 @@
>  
>  #include <linux/types.h>
>  #include <linux/kvm_types.h>
> +#include <linux/jump_label.h>
> +
>  #include <asm/kvm.h>
>  #include <asm/kvm_asm.h>
>  #include <asm/kvm_mmio.h>
>  #include <asm/fpstate.h>
>  #include <kvm/arm_arch_timer.h>
> +#include <asm/cputype.h>
>  
>  #define __KVM_HAVE_ARCH_INTC_INITIALIZED
>  
> @@ -310,4 +313,14 @@ static inline int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu,
>  	return -ENXIO;
>  }
>  
> +extern struct static_key_false kvm_gicv3_cpuif;

I think we should follow the model set by kvm_vgic_global_state, which
is declared in arm_vgic.h. Even better, we should *embed* the static key
in this structure. This will reduce the clutter and we wouldn't have to
deal with all the section stuff (the hyp_data thing is a good cleanup,
but I'd like to see it as a separate patch if possible).

> +
> +static inline bool kvm_arm_support_gicv3_cpuif(void)
> +{
> +	if (IS_ENABLED(CONFIG_ARM_GIC_V3))
> +		return !!cpuid_feature_extract(CPUID_EXT_PFR1, 28);
> +	else
> +		return false;
> +}
> +
>  #endif /* __ARM_KVM_HOST_H__ */
> diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
> index 6eaff28..bd9434e 100644
> --- a/arch/arm/include/asm/kvm_hyp.h
> +++ b/arch/arm/include/asm/kvm_hyp.h
> @@ -23,8 +23,6 @@
>  #include <asm/kvm_mmu.h>
>  #include <asm/vfp.h>
>  
> -#define __hyp_text __section(.hyp.text) notrace
> -
>  #define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
>  	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
>  #define __ACCESS_CP15_64(Op1, CRm)		\
> diff --git a/arch/arm/include/asm/virt.h b/arch/arm/include/asm/virt.h
> index a2e75b8..e61a809 100644
> --- a/arch/arm/include/asm/virt.h
> +++ b/arch/arm/include/asm/virt.h
> @@ -28,6 +28,9 @@
>   */
>  #define BOOT_CPU_MODE_MISMATCH	PSR_N_BIT
>  
> +#define __hyp_text __section(.hyp.text) notrace
> +#define __hyp_data __section(.hyp.data)
> +
>  #ifndef __ASSEMBLY__
>  #include <asm/cacheflush.h>
>  
> @@ -87,6 +90,11 @@ extern char __hyp_idmap_text_end[];
>  /* The section containing the hypervisor text */
>  extern char __hyp_text_start[];
>  extern char __hyp_text_end[];
> +
> +/* The section containing the hypervisor data */
> +extern char __hyp_data_start[];
> +extern char __hyp_data_end[];
> +
>  #endif
>  
>  #endif /* __ASSEMBLY__ */
> diff --git a/arch/arm/kernel/vmlinux.lds.S b/arch/arm/kernel/vmlinux.lds.S
> index d24e5dd..6d53824 100644
> --- a/arch/arm/kernel/vmlinux.lds.S
> +++ b/arch/arm/kernel/vmlinux.lds.S
> @@ -25,6 +25,11 @@
>  	*(.hyp.text)							\
>  	VMLINUX_SYMBOL(__hyp_text_end) = .;
>  
> +#define HYPERVISOR_DATA							\
> +	VMLINUX_SYMBOL(__hyp_data_start) = .;				\
> +	*(.hyp.data)							\
> +	VMLINUX_SYMBOL(__hyp_data_end) = .;
> +
>  #define IDMAP_TEXT							\
>  	ALIGN_FUNCTION();						\
>  	VMLINUX_SYMBOL(__idmap_text_start) = .;				\
> @@ -256,6 +261,7 @@ SECTIONS
>  		 */
>  		DATA_DATA
>  		CONSTRUCTORS
> +		HYPERVISOR_DATA

If you follow my idea of of embedding the static key, we can defer all
of this to another patch set.

>  
>  		_edata = .;
>  	}
> diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
> index 75f130e..f966763 100644
> --- a/arch/arm/kvm/arm.c
> +++ b/arch/arm/kvm/arm.c
> @@ -27,6 +27,7 @@
>  #include <linux/mman.h>
>  #include <linux/sched.h>
>  #include <linux/kvm.h>
> +#include <linux/irqchip/arm-gic-v3.h>
>  #include <trace/events/kvm.h>
>  #include <kvm/arm_pmu.h>
>  
> @@ -68,6 +69,9 @@ static bool vgic_present;
>  
>  static DEFINE_PER_CPU(unsigned char, kvm_arm_hardware_enabled);
>  
> +/* GIC system register CPU interface */
> +__hyp_data DEFINE_STATIC_KEY_FALSE(kvm_gicv3_cpuif);
> +
>  static void kvm_arm_set_running_vcpu(struct kvm_vcpu *vcpu)
>  {
>  	BUG_ON(preemptible());
> @@ -1178,6 +1182,14 @@ static int init_common_resources(void)
>  		return -ENOMEM;
>  	}
>  
> +	if (kvm_arm_support_gicv3_cpuif()) {

Why do we need to check this? If we identify a GICv3 (as exposed by the
host GIC driver), let's just use that.

> +		if (!gic_enable_sre())
> +			kvm_info("GIC CPU interface present but disabled by higher exception level\n");

Do you really want to try and enable SRE there? It feels wrong, as it
will already have been enabled by the host driver. Also, we don't
support having GICv3 in a non-SRE configuration (firmware must expose a
GICv2 in that case).

> +
> +		static_branch_enable(&kvm_gicv3_cpuif);
> +		kvm_info("GIC system register CPU interface\n");
> +	}

Can this whole hunk be moved to the vgic initialization instead?

> +
>  	return 0;
>  }
>  
> @@ -1297,6 +1309,13 @@ static int init_hyp_mode(void)
>  		goto out_err;
>  	}
>  
> +	err = create_hyp_mappings(kvm_ksym_ref(__hyp_data_start),
> +				  kvm_ksym_ref(__hyp_data_end), PAGE_HYP);
> +	if (err) {
> +		kvm_err("Cannot map hyp data section\n");
> +		goto out_err;
> +	}
> +
>  	err = create_hyp_mappings(kvm_ksym_ref(__start_rodata),
>  				  kvm_ksym_ref(__end_rodata), PAGE_HYP_RO);
>  	if (err) {
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 3eda975..1da74e8 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -24,6 +24,9 @@
>  
>  #include <linux/types.h>
>  #include <linux/kvm_types.h>
> +#include <linux/jump_label.h>
> +
> +#include <asm/cpufeature.h>
>  #include <asm/kvm.h>
>  #include <asm/kvm_asm.h>
>  #include <asm/kvm_mmio.h>
> @@ -390,4 +393,16 @@ static inline void __cpu_init_stage2(void)
>  		  "PARange is %d bits, unsupported configuration!", parange);
>  }
>  
> +extern struct static_key_false kvm_gicv3_cpuif;
> +
> +static inline bool kvm_arm_support_gicv3_cpuif(void)
> +{
> +	int reg = read_system_reg(SYS_ID_AA64PFR0_EL1);
> +
> +	if (IS_ENABLED(CONFIG_ARM_GIC_V3))
> +		return !!cpuid_feature_extract_unsigned_field(reg, ID_AA64PFR0_GIC_SHIFT);
> +
> +	return false;
> +}
> +
>  #endif /* __ARM64_KVM_HOST_H__ */
> diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
> index cff5105..5c4ac82 100644
> --- a/arch/arm64/include/asm/kvm_hyp.h
> +++ b/arch/arm64/include/asm/kvm_hyp.h
> @@ -23,8 +23,6 @@
>  #include <asm/kvm_mmu.h>
>  #include <asm/sysreg.h>
>  
> -#define __hyp_text __section(.hyp.text) notrace
> -
>  #define read_sysreg_elx(r,nvh,vh)					\
>  	({								\
>  		u64 reg;						\
> diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
> index 1788545..c49426e 100644
> --- a/arch/arm64/include/asm/virt.h
> +++ b/arch/arm64/include/asm/virt.h
> @@ -42,6 +42,9 @@
>  #define BOOT_CPU_MODE_EL1	(0xe11)
>  #define BOOT_CPU_MODE_EL2	(0xe12)
>  
> +#define __hyp_text __section(.hyp.text) notrace
> +#define __hyp_data __section(.hyp.data)
> +
>  #ifndef __ASSEMBLY__
>  
>  #include <asm/ptrace.h>
> @@ -95,6 +98,10 @@ extern char __hyp_idmap_text_end[];
>  extern char __hyp_text_start[];
>  extern char __hyp_text_end[];
>  
> +/* The section containing the hypervisor data */
> +extern char __hyp_data_start[];
> +extern char __hyp_data_end[];
> +
>  #endif /* __ASSEMBLY__ */
>  
>  #endif /* ! __ASM__VIRT_H */
> diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
> index 659963d..ea94a10 100644
> --- a/arch/arm64/kernel/vmlinux.lds.S
> +++ b/arch/arm64/kernel/vmlinux.lds.S
> @@ -40,6 +40,11 @@ jiffies = jiffies_64;
>  	*(.hyp.text)					\
>  	VMLINUX_SYMBOL(__hyp_text_end) = .;
>  
> +#define HYPERVISOR_DATA					\
> +	VMLINUX_SYMBOL(__hyp_data_start) = .;		\
> +	.hyp.data : {*(.hyp.data)}			\
> +	VMLINUX_SYMBOL(__hyp_data_end) = .;
> +
>  #define IDMAP_TEXT					\
>  	. = ALIGN(SZ_4K);				\
>  	VMLINUX_SYMBOL(__idmap_text_start) = .;		\
> @@ -185,6 +190,7 @@ SECTIONS
>  	_data = .;
>  	_sdata = .;
>  	RW_DATA_SECTION(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
> +	HYPERVISOR_DATA
>  	PECOFF_EDATA_PADDING
>  	_edata = .;
>  
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 5a84b45..cdc1360 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -126,17 +126,13 @@ static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
>  	write_sysreg(0, vttbr_el2);
>  }
>  
> -static hyp_alternate_select(__vgic_call_save_state,
> -			    __vgic_v2_save_state, __vgic_v3_save_state,
> -			    ARM64_HAS_SYSREG_GIC_CPUIF);
> -
> -static hyp_alternate_select(__vgic_call_restore_state,
> -			    __vgic_v2_restore_state, __vgic_v3_restore_state,
> -			    ARM64_HAS_SYSREG_GIC_CPUIF);
> -
>  static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
>  {
> -	__vgic_call_save_state()(vcpu);
> +	if (static_branch_unlikely(&kvm_gicv3_cpuif))
> +		__vgic_v3_save_state(vcpu);
> +	else
> +		__vgic_v2_save_state(vcpu);
> +
>  	write_sysreg(read_sysreg(hcr_el2) & ~HCR_INT_OVERRIDE, hcr_el2);
>  }
>  
> @@ -149,7 +145,10 @@ static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
>  	val |= vcpu->arch.irq_lines;
>  	write_sysreg(val, hcr_el2);
>  
> -	__vgic_call_restore_state()(vcpu);
> +	if (static_branch_unlikely(&kvm_gicv3_cpuif))
> +		__vgic_v3_restore_state(vcpu);
> +	else
> +		__vgic_v2_restore_state(vcpu);
>  }
>  
>  static bool __hyp_text __true_value(void)
> 

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 1/8] arm64: KVM: Use static keys for selecting the GIC backend
@ 2016-09-09  9:19     ` Marc Zyngier
  0 siblings, 0 replies; 72+ messages in thread
From: Marc Zyngier @ 2016-09-09  9:19 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Vladimir,

On 08/09/16 17:06, Vladimir Murzin wrote:
> Currently GIC backend is selected via alternative framework and this
> is fine. We are going to introduce vgic-v3 to 32-bit world and there
> we don't have patching framework in hand, so we can either check
> support for GICv3 every time we need to choose which backend to use or
> try to optimise it by using static keys. The later looks quite
> promising because we can share logic involved in selecting GIC backend
> between architectures if both uses static keys.
> 
> This patch moves arm64 from alternative to static keys framework for
> selecting GIC backend. To make static keys work on hyp side we need to
> make sure that hyp can access to the key which is RW data. For that
> purpose introduce __hyp_data section we can map to hyp and place the
> key there.
> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> ---
>  arch/arm/include/asm/kvm_host.h   |   13 +++++++++++++
>  arch/arm/include/asm/kvm_hyp.h    |    2 --
>  arch/arm/include/asm/virt.h       |    8 ++++++++
>  arch/arm/kernel/vmlinux.lds.S     |    6 ++++++
>  arch/arm/kvm/arm.c                |   19 +++++++++++++++++++
>  arch/arm64/include/asm/kvm_host.h |   15 +++++++++++++++
>  arch/arm64/include/asm/kvm_hyp.h  |    2 --
>  arch/arm64/include/asm/virt.h     |    7 +++++++
>  arch/arm64/kernel/vmlinux.lds.S   |    6 ++++++
>  arch/arm64/kvm/hyp/switch.c       |   19 +++++++++----------
>  10 files changed, 83 insertions(+), 14 deletions(-)
> 
> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> index de338d9..bfa6eec 100644
> --- a/arch/arm/include/asm/kvm_host.h
> +++ b/arch/arm/include/asm/kvm_host.h
> @@ -21,11 +21,14 @@
>  
>  #include <linux/types.h>
>  #include <linux/kvm_types.h>
> +#include <linux/jump_label.h>
> +
>  #include <asm/kvm.h>
>  #include <asm/kvm_asm.h>
>  #include <asm/kvm_mmio.h>
>  #include <asm/fpstate.h>
>  #include <kvm/arm_arch_timer.h>
> +#include <asm/cputype.h>
>  
>  #define __KVM_HAVE_ARCH_INTC_INITIALIZED
>  
> @@ -310,4 +313,14 @@ static inline int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu,
>  	return -ENXIO;
>  }
>  
> +extern struct static_key_false kvm_gicv3_cpuif;

I think we should follow the model set by kvm_vgic_global_state, which
is declared in arm_vgic.h. Even better, we should *embed* the static key
in this structure. This will reduce the clutter and we wouldn't have to
deal with all the section stuff (the hyp_data thing is a good cleanup,
but I'd like to see it as a separate patch if possible).

> +
> +static inline bool kvm_arm_support_gicv3_cpuif(void)
> +{
> +	if (IS_ENABLED(CONFIG_ARM_GIC_V3))
> +		return !!cpuid_feature_extract(CPUID_EXT_PFR1, 28);
> +	else
> +		return false;
> +}
> +
>  #endif /* __ARM_KVM_HOST_H__ */
> diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
> index 6eaff28..bd9434e 100644
> --- a/arch/arm/include/asm/kvm_hyp.h
> +++ b/arch/arm/include/asm/kvm_hyp.h
> @@ -23,8 +23,6 @@
>  #include <asm/kvm_mmu.h>
>  #include <asm/vfp.h>
>  
> -#define __hyp_text __section(.hyp.text) notrace
> -
>  #define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
>  	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
>  #define __ACCESS_CP15_64(Op1, CRm)		\
> diff --git a/arch/arm/include/asm/virt.h b/arch/arm/include/asm/virt.h
> index a2e75b8..e61a809 100644
> --- a/arch/arm/include/asm/virt.h
> +++ b/arch/arm/include/asm/virt.h
> @@ -28,6 +28,9 @@
>   */
>  #define BOOT_CPU_MODE_MISMATCH	PSR_N_BIT
>  
> +#define __hyp_text __section(.hyp.text) notrace
> +#define __hyp_data __section(.hyp.data)
> +
>  #ifndef __ASSEMBLY__
>  #include <asm/cacheflush.h>
>  
> @@ -87,6 +90,11 @@ extern char __hyp_idmap_text_end[];
>  /* The section containing the hypervisor text */
>  extern char __hyp_text_start[];
>  extern char __hyp_text_end[];
> +
> +/* The section containing the hypervisor data */
> +extern char __hyp_data_start[];
> +extern char __hyp_data_end[];
> +
>  #endif
>  
>  #endif /* __ASSEMBLY__ */
> diff --git a/arch/arm/kernel/vmlinux.lds.S b/arch/arm/kernel/vmlinux.lds.S
> index d24e5dd..6d53824 100644
> --- a/arch/arm/kernel/vmlinux.lds.S
> +++ b/arch/arm/kernel/vmlinux.lds.S
> @@ -25,6 +25,11 @@
>  	*(.hyp.text)							\
>  	VMLINUX_SYMBOL(__hyp_text_end) = .;
>  
> +#define HYPERVISOR_DATA							\
> +	VMLINUX_SYMBOL(__hyp_data_start) = .;				\
> +	*(.hyp.data)							\
> +	VMLINUX_SYMBOL(__hyp_data_end) = .;
> +
>  #define IDMAP_TEXT							\
>  	ALIGN_FUNCTION();						\
>  	VMLINUX_SYMBOL(__idmap_text_start) = .;				\
> @@ -256,6 +261,7 @@ SECTIONS
>  		 */
>  		DATA_DATA
>  		CONSTRUCTORS
> +		HYPERVISOR_DATA

If you follow my idea of of embedding the static key, we can defer all
of this to another patch set.

>  
>  		_edata = .;
>  	}
> diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
> index 75f130e..f966763 100644
> --- a/arch/arm/kvm/arm.c
> +++ b/arch/arm/kvm/arm.c
> @@ -27,6 +27,7 @@
>  #include <linux/mman.h>
>  #include <linux/sched.h>
>  #include <linux/kvm.h>
> +#include <linux/irqchip/arm-gic-v3.h>
>  #include <trace/events/kvm.h>
>  #include <kvm/arm_pmu.h>
>  
> @@ -68,6 +69,9 @@ static bool vgic_present;
>  
>  static DEFINE_PER_CPU(unsigned char, kvm_arm_hardware_enabled);
>  
> +/* GIC system register CPU interface */
> +__hyp_data DEFINE_STATIC_KEY_FALSE(kvm_gicv3_cpuif);
> +
>  static void kvm_arm_set_running_vcpu(struct kvm_vcpu *vcpu)
>  {
>  	BUG_ON(preemptible());
> @@ -1178,6 +1182,14 @@ static int init_common_resources(void)
>  		return -ENOMEM;
>  	}
>  
> +	if (kvm_arm_support_gicv3_cpuif()) {

Why do we need to check this? If we identify a GICv3 (as exposed by the
host GIC driver), let's just use that.

> +		if (!gic_enable_sre())
> +			kvm_info("GIC CPU interface present but disabled by higher exception level\n");

Do you really want to try and enable SRE there? It feels wrong, as it
will already have been enabled by the host driver. Also, we don't
support having GICv3 in a non-SRE configuration (firmware must expose a
GICv2 in that case).

> +
> +		static_branch_enable(&kvm_gicv3_cpuif);
> +		kvm_info("GIC system register CPU interface\n");
> +	}

Can this whole hunk be moved to the vgic initialization instead?

> +
>  	return 0;
>  }
>  
> @@ -1297,6 +1309,13 @@ static int init_hyp_mode(void)
>  		goto out_err;
>  	}
>  
> +	err = create_hyp_mappings(kvm_ksym_ref(__hyp_data_start),
> +				  kvm_ksym_ref(__hyp_data_end), PAGE_HYP);
> +	if (err) {
> +		kvm_err("Cannot map hyp data section\n");
> +		goto out_err;
> +	}
> +
>  	err = create_hyp_mappings(kvm_ksym_ref(__start_rodata),
>  				  kvm_ksym_ref(__end_rodata), PAGE_HYP_RO);
>  	if (err) {
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 3eda975..1da74e8 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -24,6 +24,9 @@
>  
>  #include <linux/types.h>
>  #include <linux/kvm_types.h>
> +#include <linux/jump_label.h>
> +
> +#include <asm/cpufeature.h>
>  #include <asm/kvm.h>
>  #include <asm/kvm_asm.h>
>  #include <asm/kvm_mmio.h>
> @@ -390,4 +393,16 @@ static inline void __cpu_init_stage2(void)
>  		  "PARange is %d bits, unsupported configuration!", parange);
>  }
>  
> +extern struct static_key_false kvm_gicv3_cpuif;
> +
> +static inline bool kvm_arm_support_gicv3_cpuif(void)
> +{
> +	int reg = read_system_reg(SYS_ID_AA64PFR0_EL1);
> +
> +	if (IS_ENABLED(CONFIG_ARM_GIC_V3))
> +		return !!cpuid_feature_extract_unsigned_field(reg, ID_AA64PFR0_GIC_SHIFT);
> +
> +	return false;
> +}
> +
>  #endif /* __ARM64_KVM_HOST_H__ */
> diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
> index cff5105..5c4ac82 100644
> --- a/arch/arm64/include/asm/kvm_hyp.h
> +++ b/arch/arm64/include/asm/kvm_hyp.h
> @@ -23,8 +23,6 @@
>  #include <asm/kvm_mmu.h>
>  #include <asm/sysreg.h>
>  
> -#define __hyp_text __section(.hyp.text) notrace
> -
>  #define read_sysreg_elx(r,nvh,vh)					\
>  	({								\
>  		u64 reg;						\
> diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
> index 1788545..c49426e 100644
> --- a/arch/arm64/include/asm/virt.h
> +++ b/arch/arm64/include/asm/virt.h
> @@ -42,6 +42,9 @@
>  #define BOOT_CPU_MODE_EL1	(0xe11)
>  #define BOOT_CPU_MODE_EL2	(0xe12)
>  
> +#define __hyp_text __section(.hyp.text) notrace
> +#define __hyp_data __section(.hyp.data)
> +
>  #ifndef __ASSEMBLY__
>  
>  #include <asm/ptrace.h>
> @@ -95,6 +98,10 @@ extern char __hyp_idmap_text_end[];
>  extern char __hyp_text_start[];
>  extern char __hyp_text_end[];
>  
> +/* The section containing the hypervisor data */
> +extern char __hyp_data_start[];
> +extern char __hyp_data_end[];
> +
>  #endif /* __ASSEMBLY__ */
>  
>  #endif /* ! __ASM__VIRT_H */
> diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
> index 659963d..ea94a10 100644
> --- a/arch/arm64/kernel/vmlinux.lds.S
> +++ b/arch/arm64/kernel/vmlinux.lds.S
> @@ -40,6 +40,11 @@ jiffies = jiffies_64;
>  	*(.hyp.text)					\
>  	VMLINUX_SYMBOL(__hyp_text_end) = .;
>  
> +#define HYPERVISOR_DATA					\
> +	VMLINUX_SYMBOL(__hyp_data_start) = .;		\
> +	.hyp.data : {*(.hyp.data)}			\
> +	VMLINUX_SYMBOL(__hyp_data_end) = .;
> +
>  #define IDMAP_TEXT					\
>  	. = ALIGN(SZ_4K);				\
>  	VMLINUX_SYMBOL(__idmap_text_start) = .;		\
> @@ -185,6 +190,7 @@ SECTIONS
>  	_data = .;
>  	_sdata = .;
>  	RW_DATA_SECTION(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
> +	HYPERVISOR_DATA
>  	PECOFF_EDATA_PADDING
>  	_edata = .;
>  
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 5a84b45..cdc1360 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -126,17 +126,13 @@ static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
>  	write_sysreg(0, vttbr_el2);
>  }
>  
> -static hyp_alternate_select(__vgic_call_save_state,
> -			    __vgic_v2_save_state, __vgic_v3_save_state,
> -			    ARM64_HAS_SYSREG_GIC_CPUIF);
> -
> -static hyp_alternate_select(__vgic_call_restore_state,
> -			    __vgic_v2_restore_state, __vgic_v3_restore_state,
> -			    ARM64_HAS_SYSREG_GIC_CPUIF);
> -
>  static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
>  {
> -	__vgic_call_save_state()(vcpu);
> +	if (static_branch_unlikely(&kvm_gicv3_cpuif))
> +		__vgic_v3_save_state(vcpu);
> +	else
> +		__vgic_v2_save_state(vcpu);
> +
>  	write_sysreg(read_sysreg(hcr_el2) & ~HCR_INT_OVERRIDE, hcr_el2);
>  }
>  
> @@ -149,7 +145,10 @@ static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
>  	val |= vcpu->arch.irq_lines;
>  	write_sysreg(val, hcr_el2);
>  
> -	__vgic_call_restore_state()(vcpu);
> +	if (static_branch_unlikely(&kvm_gicv3_cpuif))
> +		__vgic_v3_restore_state(vcpu);
> +	else
> +		__vgic_v2_restore_state(vcpu);
>  }
>  
>  static bool __hyp_text __true_value(void)
> 

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 1/8] arm64: KVM: Use static keys for selecting the GIC backend
  2016-09-09  9:19     ` Marc Zyngier
@ 2016-09-09  9:33       ` Vladimir Murzin
  -1 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-09  9:33 UTC (permalink / raw)
  To: Marc Zyngier, kvmarm; +Cc: andre.przywara, linux-arm-kernel

Hi Marc,

On 09/09/16 10:19, Marc Zyngier wrote:
> Hi Vladimir,
> 
...
>>  
>> +extern struct static_key_false kvm_gicv3_cpuif;
> 
> I think we should follow the model set by kvm_vgic_global_state, which
> is declared in arm_vgic.h. Even better, we should *embed* the static key
> in this structure. This will reduce the clutter and we wouldn't have to
> deal with all the section stuff (the hyp_data thing is a good cleanup,
> but I'd like to see it as a separate patch if possible).

Yes, it is what I was thinking about too, but was not sure about which
way to go, so hyp_data seemed me something we might reuse latter.
However, I agree that we can defer hyp_data thing...

> 
>> +
>> +static inline bool kvm_arm_support_gicv3_cpuif(void)
>> +{
>> +	if (IS_ENABLED(CONFIG_ARM_GIC_V3))
>> +		return !!cpuid_feature_extract(CPUID_EXT_PFR1, 28);
>> +	else
>> +		return false;
>> +}
>> +
>>  #endif /* __ARM_KVM_HOST_H__ */
>> diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
>> index 6eaff28..bd9434e 100644
>> --- a/arch/arm/include/asm/kvm_hyp.h
>> +++ b/arch/arm/include/asm/kvm_hyp.h
>> @@ -23,8 +23,6 @@
>>  #include <asm/kvm_mmu.h>
>>  #include <asm/vfp.h>
>>  
>> -#define __hyp_text __section(.hyp.text) notrace
>> -
>>  #define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
>>  	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
>>  #define __ACCESS_CP15_64(Op1, CRm)		\
>> diff --git a/arch/arm/include/asm/virt.h b/arch/arm/include/asm/virt.h
>> index a2e75b8..e61a809 100644
>> --- a/arch/arm/include/asm/virt.h
>> +++ b/arch/arm/include/asm/virt.h
>> @@ -28,6 +28,9 @@
>>   */
>>  #define BOOT_CPU_MODE_MISMATCH	PSR_N_BIT
>>  
>> +#define __hyp_text __section(.hyp.text) notrace
>> +#define __hyp_data __section(.hyp.data)
>> +
>>  #ifndef __ASSEMBLY__
>>  #include <asm/cacheflush.h>
>>  
>> @@ -87,6 +90,11 @@ extern char __hyp_idmap_text_end[];
>>  /* The section containing the hypervisor text */
>>  extern char __hyp_text_start[];
>>  extern char __hyp_text_end[];
>> +
>> +/* The section containing the hypervisor data */
>> +extern char __hyp_data_start[];
>> +extern char __hyp_data_end[];
>> +
>>  #endif
>>  
>>  #endif /* __ASSEMBLY__ */
>> diff --git a/arch/arm/kernel/vmlinux.lds.S b/arch/arm/kernel/vmlinux.lds.S
>> index d24e5dd..6d53824 100644
>> --- a/arch/arm/kernel/vmlinux.lds.S
>> +++ b/arch/arm/kernel/vmlinux.lds.S
>> @@ -25,6 +25,11 @@
>>  	*(.hyp.text)							\
>>  	VMLINUX_SYMBOL(__hyp_text_end) = .;
>>  
>> +#define HYPERVISOR_DATA							\
>> +	VMLINUX_SYMBOL(__hyp_data_start) = .;				\
>> +	*(.hyp.data)							\
>> +	VMLINUX_SYMBOL(__hyp_data_end) = .;
>> +
>>  #define IDMAP_TEXT							\
>>  	ALIGN_FUNCTION();						\
>>  	VMLINUX_SYMBOL(__idmap_text_start) = .;				\
>> @@ -256,6 +261,7 @@ SECTIONS
>>  		 */
>>  		DATA_DATA
>>  		CONSTRUCTORS
>> +		HYPERVISOR_DATA
> 
> If you follow my idea of of embedding the static key, we can defer all
> of this to another patch set.

I'm happy with your idea, so I'll drop this hunk.

> 
>>  
>>  		_edata = .;
>>  	}
>> diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
>> index 75f130e..f966763 100644
>> --- a/arch/arm/kvm/arm.c
>> +++ b/arch/arm/kvm/arm.c
>> @@ -27,6 +27,7 @@
>>  #include <linux/mman.h>
>>  #include <linux/sched.h>
>>  #include <linux/kvm.h>
>> +#include <linux/irqchip/arm-gic-v3.h>
>>  #include <trace/events/kvm.h>
>>  #include <kvm/arm_pmu.h>
>>  
>> @@ -68,6 +69,9 @@ static bool vgic_present;
>>  
>>  static DEFINE_PER_CPU(unsigned char, kvm_arm_hardware_enabled);
>>  
>> +/* GIC system register CPU interface */
>> +__hyp_data DEFINE_STATIC_KEY_FALSE(kvm_gicv3_cpuif);
>> +
>>  static void kvm_arm_set_running_vcpu(struct kvm_vcpu *vcpu)
>>  {
>>  	BUG_ON(preemptible());
>> @@ -1178,6 +1182,14 @@ static int init_common_resources(void)
>>  		return -ENOMEM;
>>  	}
>>  
>> +	if (kvm_arm_support_gicv3_cpuif()) {
> 
> Why do we need to check this? If we identify a GICv3 (as exposed by the
> host GIC driver), let's just use that.

True, will drop this.

> 
>> +		if (!gic_enable_sre())
>> +			kvm_info("GIC CPU interface present but disabled by higher exception level\n");
> 
> Do you really want to try and enable SRE there? It feels wrong, as it
> will already have been enabled by the host driver. Also, we don't
> support having GICv3 in a non-SRE configuration (firmware must expose a
> GICv2 in that case).
> 
>> +
>> +		static_branch_enable(&kvm_gicv3_cpuif);
>> +		kvm_info("GIC system register CPU interface\n");
>> +	}
> 
> Can this whole hunk be moved to the vgic initialization instead?
> 

I'll move it there.

>> +
>>  	return 0;
>>  }
>>  
>> @@ -1297,6 +1309,13 @@ static int init_hyp_mode(void)
>>  		goto out_err;
>>  	}
>>  
>> +	err = create_hyp_mappings(kvm_ksym_ref(__hyp_data_start),
>> +				  kvm_ksym_ref(__hyp_data_end), PAGE_HYP);
>> +	if (err) {
>> +		kvm_err("Cannot map hyp data section\n");
>> +		goto out_err;
>> +	}
>> +
>>  	err = create_hyp_mappings(kvm_ksym_ref(__start_rodata),
>>  				  kvm_ksym_ref(__end_rodata), PAGE_HYP_RO);
>>  	if (err) {
>> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
>> index 3eda975..1da74e8 100644
>> --- a/arch/arm64/include/asm/kvm_host.h
>> +++ b/arch/arm64/include/asm/kvm_host.h
>> @@ -24,6 +24,9 @@
>>  
>>  #include <linux/types.h>
>>  #include <linux/kvm_types.h>
>> +#include <linux/jump_label.h>
>> +
>> +#include <asm/cpufeature.h>
>>  #include <asm/kvm.h>
>>  #include <asm/kvm_asm.h>
>>  #include <asm/kvm_mmio.h>
>> @@ -390,4 +393,16 @@ static inline void __cpu_init_stage2(void)
>>  		  "PARange is %d bits, unsupported configuration!", parange);
>>  }
>>  
>> +extern struct static_key_false kvm_gicv3_cpuif;
>> +
>> +static inline bool kvm_arm_support_gicv3_cpuif(void)
>> +{
>> +	int reg = read_system_reg(SYS_ID_AA64PFR0_EL1);
>> +
>> +	if (IS_ENABLED(CONFIG_ARM_GIC_V3))
>> +		return !!cpuid_feature_extract_unsigned_field(reg, ID_AA64PFR0_GIC_SHIFT);
>> +
>> +	return false;
>> +}
>> +
>>  #endif /* __ARM64_KVM_HOST_H__ */
>> diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
>> index cff5105..5c4ac82 100644
>> --- a/arch/arm64/include/asm/kvm_hyp.h
>> +++ b/arch/arm64/include/asm/kvm_hyp.h
>> @@ -23,8 +23,6 @@
>>  #include <asm/kvm_mmu.h>
>>  #include <asm/sysreg.h>
>>  
>> -#define __hyp_text __section(.hyp.text) notrace
>> -
>>  #define read_sysreg_elx(r,nvh,vh)					\
>>  	({								\
>>  		u64 reg;						\
>> diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
>> index 1788545..c49426e 100644
>> --- a/arch/arm64/include/asm/virt.h
>> +++ b/arch/arm64/include/asm/virt.h
>> @@ -42,6 +42,9 @@
>>  #define BOOT_CPU_MODE_EL1	(0xe11)
>>  #define BOOT_CPU_MODE_EL2	(0xe12)
>>  
>> +#define __hyp_text __section(.hyp.text) notrace
>> +#define __hyp_data __section(.hyp.data)
>> +
>>  #ifndef __ASSEMBLY__
>>  
>>  #include <asm/ptrace.h>
>> @@ -95,6 +98,10 @@ extern char __hyp_idmap_text_end[];
>>  extern char __hyp_text_start[];
>>  extern char __hyp_text_end[];
>>  
>> +/* The section containing the hypervisor data */
>> +extern char __hyp_data_start[];
>> +extern char __hyp_data_end[];
>> +
>>  #endif /* __ASSEMBLY__ */
>>  
>>  #endif /* ! __ASM__VIRT_H */
>> diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
>> index 659963d..ea94a10 100644
>> --- a/arch/arm64/kernel/vmlinux.lds.S
>> +++ b/arch/arm64/kernel/vmlinux.lds.S
>> @@ -40,6 +40,11 @@ jiffies = jiffies_64;
>>  	*(.hyp.text)					\
>>  	VMLINUX_SYMBOL(__hyp_text_end) = .;
>>  
>> +#define HYPERVISOR_DATA					\
>> +	VMLINUX_SYMBOL(__hyp_data_start) = .;		\
>> +	.hyp.data : {*(.hyp.data)}			\
>> +	VMLINUX_SYMBOL(__hyp_data_end) = .;
>> +
>>  #define IDMAP_TEXT					\
>>  	. = ALIGN(SZ_4K);				\
>>  	VMLINUX_SYMBOL(__idmap_text_start) = .;		\
>> @@ -185,6 +190,7 @@ SECTIONS
>>  	_data = .;
>>  	_sdata = .;
>>  	RW_DATA_SECTION(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
>> +	HYPERVISOR_DATA
>>  	PECOFF_EDATA_PADDING
>>  	_edata = .;
>>  
>> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
>> index 5a84b45..cdc1360 100644
>> --- a/arch/arm64/kvm/hyp/switch.c
>> +++ b/arch/arm64/kvm/hyp/switch.c
>> @@ -126,17 +126,13 @@ static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
>>  	write_sysreg(0, vttbr_el2);
>>  }
>>  
>> -static hyp_alternate_select(__vgic_call_save_state,
>> -			    __vgic_v2_save_state, __vgic_v3_save_state,
>> -			    ARM64_HAS_SYSREG_GIC_CPUIF);
>> -
>> -static hyp_alternate_select(__vgic_call_restore_state,
>> -			    __vgic_v2_restore_state, __vgic_v3_restore_state,
>> -			    ARM64_HAS_SYSREG_GIC_CPUIF);
>> -
>>  static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
>>  {
>> -	__vgic_call_save_state()(vcpu);
>> +	if (static_branch_unlikely(&kvm_gicv3_cpuif))
>> +		__vgic_v3_save_state(vcpu);
>> +	else
>> +		__vgic_v2_save_state(vcpu);
>> +
>>  	write_sysreg(read_sysreg(hcr_el2) & ~HCR_INT_OVERRIDE, hcr_el2);
>>  }
>>  
>> @@ -149,7 +145,10 @@ static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
>>  	val |= vcpu->arch.irq_lines;
>>  	write_sysreg(val, hcr_el2);
>>  
>> -	__vgic_call_restore_state()(vcpu);
>> +	if (static_branch_unlikely(&kvm_gicv3_cpuif))
>> +		__vgic_v3_restore_state(vcpu);
>> +	else
>> +		__vgic_v2_restore_state(vcpu);
>>  }
>>  
>>  static bool __hyp_text __true_value(void)
>>
> 
> Thanks,

Thanks for feedback!

Vladimir

> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 1/8] arm64: KVM: Use static keys for selecting the GIC backend
@ 2016-09-09  9:33       ` Vladimir Murzin
  0 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-09  9:33 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Marc,

On 09/09/16 10:19, Marc Zyngier wrote:
> Hi Vladimir,
> 
...
>>  
>> +extern struct static_key_false kvm_gicv3_cpuif;
> 
> I think we should follow the model set by kvm_vgic_global_state, which
> is declared in arm_vgic.h. Even better, we should *embed* the static key
> in this structure. This will reduce the clutter and we wouldn't have to
> deal with all the section stuff (the hyp_data thing is a good cleanup,
> but I'd like to see it as a separate patch if possible).

Yes, it is what I was thinking about too, but was not sure about which
way to go, so hyp_data seemed me something we might reuse latter.
However, I agree that we can defer hyp_data thing...

> 
>> +
>> +static inline bool kvm_arm_support_gicv3_cpuif(void)
>> +{
>> +	if (IS_ENABLED(CONFIG_ARM_GIC_V3))
>> +		return !!cpuid_feature_extract(CPUID_EXT_PFR1, 28);
>> +	else
>> +		return false;
>> +}
>> +
>>  #endif /* __ARM_KVM_HOST_H__ */
>> diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
>> index 6eaff28..bd9434e 100644
>> --- a/arch/arm/include/asm/kvm_hyp.h
>> +++ b/arch/arm/include/asm/kvm_hyp.h
>> @@ -23,8 +23,6 @@
>>  #include <asm/kvm_mmu.h>
>>  #include <asm/vfp.h>
>>  
>> -#define __hyp_text __section(.hyp.text) notrace
>> -
>>  #define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
>>  	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
>>  #define __ACCESS_CP15_64(Op1, CRm)		\
>> diff --git a/arch/arm/include/asm/virt.h b/arch/arm/include/asm/virt.h
>> index a2e75b8..e61a809 100644
>> --- a/arch/arm/include/asm/virt.h
>> +++ b/arch/arm/include/asm/virt.h
>> @@ -28,6 +28,9 @@
>>   */
>>  #define BOOT_CPU_MODE_MISMATCH	PSR_N_BIT
>>  
>> +#define __hyp_text __section(.hyp.text) notrace
>> +#define __hyp_data __section(.hyp.data)
>> +
>>  #ifndef __ASSEMBLY__
>>  #include <asm/cacheflush.h>
>>  
>> @@ -87,6 +90,11 @@ extern char __hyp_idmap_text_end[];
>>  /* The section containing the hypervisor text */
>>  extern char __hyp_text_start[];
>>  extern char __hyp_text_end[];
>> +
>> +/* The section containing the hypervisor data */
>> +extern char __hyp_data_start[];
>> +extern char __hyp_data_end[];
>> +
>>  #endif
>>  
>>  #endif /* __ASSEMBLY__ */
>> diff --git a/arch/arm/kernel/vmlinux.lds.S b/arch/arm/kernel/vmlinux.lds.S
>> index d24e5dd..6d53824 100644
>> --- a/arch/arm/kernel/vmlinux.lds.S
>> +++ b/arch/arm/kernel/vmlinux.lds.S
>> @@ -25,6 +25,11 @@
>>  	*(.hyp.text)							\
>>  	VMLINUX_SYMBOL(__hyp_text_end) = .;
>>  
>> +#define HYPERVISOR_DATA							\
>> +	VMLINUX_SYMBOL(__hyp_data_start) = .;				\
>> +	*(.hyp.data)							\
>> +	VMLINUX_SYMBOL(__hyp_data_end) = .;
>> +
>>  #define IDMAP_TEXT							\
>>  	ALIGN_FUNCTION();						\
>>  	VMLINUX_SYMBOL(__idmap_text_start) = .;				\
>> @@ -256,6 +261,7 @@ SECTIONS
>>  		 */
>>  		DATA_DATA
>>  		CONSTRUCTORS
>> +		HYPERVISOR_DATA
> 
> If you follow my idea of of embedding the static key, we can defer all
> of this to another patch set.

I'm happy with your idea, so I'll drop this hunk.

> 
>>  
>>  		_edata = .;
>>  	}
>> diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
>> index 75f130e..f966763 100644
>> --- a/arch/arm/kvm/arm.c
>> +++ b/arch/arm/kvm/arm.c
>> @@ -27,6 +27,7 @@
>>  #include <linux/mman.h>
>>  #include <linux/sched.h>
>>  #include <linux/kvm.h>
>> +#include <linux/irqchip/arm-gic-v3.h>
>>  #include <trace/events/kvm.h>
>>  #include <kvm/arm_pmu.h>
>>  
>> @@ -68,6 +69,9 @@ static bool vgic_present;
>>  
>>  static DEFINE_PER_CPU(unsigned char, kvm_arm_hardware_enabled);
>>  
>> +/* GIC system register CPU interface */
>> +__hyp_data DEFINE_STATIC_KEY_FALSE(kvm_gicv3_cpuif);
>> +
>>  static void kvm_arm_set_running_vcpu(struct kvm_vcpu *vcpu)
>>  {
>>  	BUG_ON(preemptible());
>> @@ -1178,6 +1182,14 @@ static int init_common_resources(void)
>>  		return -ENOMEM;
>>  	}
>>  
>> +	if (kvm_arm_support_gicv3_cpuif()) {
> 
> Why do we need to check this? If we identify a GICv3 (as exposed by the
> host GIC driver), let's just use that.

True, will drop this.

> 
>> +		if (!gic_enable_sre())
>> +			kvm_info("GIC CPU interface present but disabled by higher exception level\n");
> 
> Do you really want to try and enable SRE there? It feels wrong, as it
> will already have been enabled by the host driver. Also, we don't
> support having GICv3 in a non-SRE configuration (firmware must expose a
> GICv2 in that case).
> 
>> +
>> +		static_branch_enable(&kvm_gicv3_cpuif);
>> +		kvm_info("GIC system register CPU interface\n");
>> +	}
> 
> Can this whole hunk be moved to the vgic initialization instead?
> 

I'll move it there.

>> +
>>  	return 0;
>>  }
>>  
>> @@ -1297,6 +1309,13 @@ static int init_hyp_mode(void)
>>  		goto out_err;
>>  	}
>>  
>> +	err = create_hyp_mappings(kvm_ksym_ref(__hyp_data_start),
>> +				  kvm_ksym_ref(__hyp_data_end), PAGE_HYP);
>> +	if (err) {
>> +		kvm_err("Cannot map hyp data section\n");
>> +		goto out_err;
>> +	}
>> +
>>  	err = create_hyp_mappings(kvm_ksym_ref(__start_rodata),
>>  				  kvm_ksym_ref(__end_rodata), PAGE_HYP_RO);
>>  	if (err) {
>> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
>> index 3eda975..1da74e8 100644
>> --- a/arch/arm64/include/asm/kvm_host.h
>> +++ b/arch/arm64/include/asm/kvm_host.h
>> @@ -24,6 +24,9 @@
>>  
>>  #include <linux/types.h>
>>  #include <linux/kvm_types.h>
>> +#include <linux/jump_label.h>
>> +
>> +#include <asm/cpufeature.h>
>>  #include <asm/kvm.h>
>>  #include <asm/kvm_asm.h>
>>  #include <asm/kvm_mmio.h>
>> @@ -390,4 +393,16 @@ static inline void __cpu_init_stage2(void)
>>  		  "PARange is %d bits, unsupported configuration!", parange);
>>  }
>>  
>> +extern struct static_key_false kvm_gicv3_cpuif;
>> +
>> +static inline bool kvm_arm_support_gicv3_cpuif(void)
>> +{
>> +	int reg = read_system_reg(SYS_ID_AA64PFR0_EL1);
>> +
>> +	if (IS_ENABLED(CONFIG_ARM_GIC_V3))
>> +		return !!cpuid_feature_extract_unsigned_field(reg, ID_AA64PFR0_GIC_SHIFT);
>> +
>> +	return false;
>> +}
>> +
>>  #endif /* __ARM64_KVM_HOST_H__ */
>> diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
>> index cff5105..5c4ac82 100644
>> --- a/arch/arm64/include/asm/kvm_hyp.h
>> +++ b/arch/arm64/include/asm/kvm_hyp.h
>> @@ -23,8 +23,6 @@
>>  #include <asm/kvm_mmu.h>
>>  #include <asm/sysreg.h>
>>  
>> -#define __hyp_text __section(.hyp.text) notrace
>> -
>>  #define read_sysreg_elx(r,nvh,vh)					\
>>  	({								\
>>  		u64 reg;						\
>> diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
>> index 1788545..c49426e 100644
>> --- a/arch/arm64/include/asm/virt.h
>> +++ b/arch/arm64/include/asm/virt.h
>> @@ -42,6 +42,9 @@
>>  #define BOOT_CPU_MODE_EL1	(0xe11)
>>  #define BOOT_CPU_MODE_EL2	(0xe12)
>>  
>> +#define __hyp_text __section(.hyp.text) notrace
>> +#define __hyp_data __section(.hyp.data)
>> +
>>  #ifndef __ASSEMBLY__
>>  
>>  #include <asm/ptrace.h>
>> @@ -95,6 +98,10 @@ extern char __hyp_idmap_text_end[];
>>  extern char __hyp_text_start[];
>>  extern char __hyp_text_end[];
>>  
>> +/* The section containing the hypervisor data */
>> +extern char __hyp_data_start[];
>> +extern char __hyp_data_end[];
>> +
>>  #endif /* __ASSEMBLY__ */
>>  
>>  #endif /* ! __ASM__VIRT_H */
>> diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
>> index 659963d..ea94a10 100644
>> --- a/arch/arm64/kernel/vmlinux.lds.S
>> +++ b/arch/arm64/kernel/vmlinux.lds.S
>> @@ -40,6 +40,11 @@ jiffies = jiffies_64;
>>  	*(.hyp.text)					\
>>  	VMLINUX_SYMBOL(__hyp_text_end) = .;
>>  
>> +#define HYPERVISOR_DATA					\
>> +	VMLINUX_SYMBOL(__hyp_data_start) = .;		\
>> +	.hyp.data : {*(.hyp.data)}			\
>> +	VMLINUX_SYMBOL(__hyp_data_end) = .;
>> +
>>  #define IDMAP_TEXT					\
>>  	. = ALIGN(SZ_4K);				\
>>  	VMLINUX_SYMBOL(__idmap_text_start) = .;		\
>> @@ -185,6 +190,7 @@ SECTIONS
>>  	_data = .;
>>  	_sdata = .;
>>  	RW_DATA_SECTION(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
>> +	HYPERVISOR_DATA
>>  	PECOFF_EDATA_PADDING
>>  	_edata = .;
>>  
>> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
>> index 5a84b45..cdc1360 100644
>> --- a/arch/arm64/kvm/hyp/switch.c
>> +++ b/arch/arm64/kvm/hyp/switch.c
>> @@ -126,17 +126,13 @@ static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
>>  	write_sysreg(0, vttbr_el2);
>>  }
>>  
>> -static hyp_alternate_select(__vgic_call_save_state,
>> -			    __vgic_v2_save_state, __vgic_v3_save_state,
>> -			    ARM64_HAS_SYSREG_GIC_CPUIF);
>> -
>> -static hyp_alternate_select(__vgic_call_restore_state,
>> -			    __vgic_v2_restore_state, __vgic_v3_restore_state,
>> -			    ARM64_HAS_SYSREG_GIC_CPUIF);
>> -
>>  static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
>>  {
>> -	__vgic_call_save_state()(vcpu);
>> +	if (static_branch_unlikely(&kvm_gicv3_cpuif))
>> +		__vgic_v3_save_state(vcpu);
>> +	else
>> +		__vgic_v2_save_state(vcpu);
>> +
>>  	write_sysreg(read_sysreg(hcr_el2) & ~HCR_INT_OVERRIDE, hcr_el2);
>>  }
>>  
>> @@ -149,7 +145,10 @@ static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
>>  	val |= vcpu->arch.irq_lines;
>>  	write_sysreg(val, hcr_el2);
>>  
>> -	__vgic_call_restore_state()(vcpu);
>> +	if (static_branch_unlikely(&kvm_gicv3_cpuif))
>> +		__vgic_v3_restore_state(vcpu);
>> +	else
>> +		__vgic_v2_restore_state(vcpu);
>>  }
>>  
>>  static bool __hyp_text __true_value(void)
>>
> 
> Thanks,

Thanks for feedback!

Vladimir

> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 0/8] ARM: KVM: Support for vgic-v3
  2016-09-09  7:58   ` Vladimir Murzin
@ 2016-09-09 11:26     ` Christoffer Dall
  -1 siblings, 0 replies; 72+ messages in thread
From: Christoffer Dall @ 2016-09-09 11:26 UTC (permalink / raw)
  To: Vladimir Murzin; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

On Fri, Sep 09, 2016 at 08:58:23AM +0100, Vladimir Murzin wrote:
> +Christoffer
> 
> I should have checked my send-email command twice...

No worries, as long as you cc the kvmarm list, I'm happy.

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 0/8] ARM: KVM: Support for vgic-v3
@ 2016-09-09 11:26     ` Christoffer Dall
  0 siblings, 0 replies; 72+ messages in thread
From: Christoffer Dall @ 2016-09-09 11:26 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Sep 09, 2016 at 08:58:23AM +0100, Vladimir Murzin wrote:
> +Christoffer
> 
> I should have checked my send-email command twice...

No worries, as long as you cc the kvmarm list, I'm happy.

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 1/8] arm64: KVM: Use static keys for selecting the GIC backend
  2016-09-09  9:33       ` Vladimir Murzin
@ 2016-09-09 13:45         ` Vladimir Murzin
  -1 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-09 13:45 UTC (permalink / raw)
  To: Marc Zyngier, kvmarm; +Cc: andre.przywara, linux-arm-kernel

On 09/09/16 10:33, Vladimir Murzin wrote:
> Hi Marc,
> 
> On 09/09/16 10:19, Marc Zyngier wrote:
>> > Hi Vladimir,
>> > 
> ...
>>> >>  
>>> >> +extern struct static_key_false kvm_gicv3_cpuif;
>> > 
>> > I think we should follow the model set by kvm_vgic_global_state, which
>> > is declared in arm_vgic.h. Even better, we should *embed* the static key
>> > in this structure. This will reduce the clutter and we wouldn't have to
>> > deal with all the section stuff (the hyp_data thing is a good cleanup,
>> > but I'd like to see it as a separate patch if possible).
> Yes, it is what I was thinking about too, but was not sure about which
> way to go, so hyp_data seemed me something we might reuse latter.
> However, I agree that we can defer hyp_data thing...
> 

I've just tried it out and it seems that static keys are not happy to
accept a key after kern_hyp_va is applied at &kvm_vgic_global_state:

> In file included from ./include/linux/jump_label.h:105:0,
>                  from arch/arm64/kvm/hyp/switch.c:19:
> ./arch/arm64/include/asm/jump_label.h: In function ‘__guest_run’:
> ./arch/arm64/include/asm/jump_label.h:31:2: warning: asm operand 0 probably doesn’t match constraints
>   asm goto("1: nop\n\t"
>   ^
> ./arch/arm64/include/asm/jump_label.h:31:2: warning: asm operand 0 probably doesn’t match constraints
>   asm goto("1: nop\n\t"
>   ^
> ./arch/arm64/include/asm/jump_label.h:31:2: error: impossible constraint in ‘asm’
>   asm goto("1: nop\n\t"
>   ^
> ./arch/arm64/include/asm/jump_label.h:31:2: error: impossible constraint in ‘asm’
>   asm goto("1: nop\n\t"
>   ^
> make[1]: *** [arch/arm64/kvm/hyp/switch.o] Error 1
> make: *** [arch/arm64/kvm/hyp/switch.o] Error 2

it looks like we cannot avoid hyp_data thing... if you don't mind I can
do hyp_data clean-up in separate patch. Alternatively, we can do
conversion to static keys for both architectures later as an
optimisation step.

Cheers
Vladimir

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 1/8] arm64: KVM: Use static keys for selecting the GIC backend
@ 2016-09-09 13:45         ` Vladimir Murzin
  0 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-09 13:45 UTC (permalink / raw)
  To: linux-arm-kernel

On 09/09/16 10:33, Vladimir Murzin wrote:
> Hi Marc,
> 
> On 09/09/16 10:19, Marc Zyngier wrote:
>> > Hi Vladimir,
>> > 
> ...
>>> >>  
>>> >> +extern struct static_key_false kvm_gicv3_cpuif;
>> > 
>> > I think we should follow the model set by kvm_vgic_global_state, which
>> > is declared in arm_vgic.h. Even better, we should *embed* the static key
>> > in this structure. This will reduce the clutter and we wouldn't have to
>> > deal with all the section stuff (the hyp_data thing is a good cleanup,
>> > but I'd like to see it as a separate patch if possible).
> Yes, it is what I was thinking about too, but was not sure about which
> way to go, so hyp_data seemed me something we might reuse latter.
> However, I agree that we can defer hyp_data thing...
> 

I've just tried it out and it seems that static keys are not happy to
accept a key after kern_hyp_va is applied at &kvm_vgic_global_state:

> In file included from ./include/linux/jump_label.h:105:0,
>                  from arch/arm64/kvm/hyp/switch.c:19:
> ./arch/arm64/include/asm/jump_label.h: In function ?__guest_run?:
> ./arch/arm64/include/asm/jump_label.h:31:2: warning: asm operand 0 probably doesn?t match constraints
>   asm goto("1: nop\n\t"
>   ^
> ./arch/arm64/include/asm/jump_label.h:31:2: warning: asm operand 0 probably doesn?t match constraints
>   asm goto("1: nop\n\t"
>   ^
> ./arch/arm64/include/asm/jump_label.h:31:2: error: impossible constraint in ?asm?
>   asm goto("1: nop\n\t"
>   ^
> ./arch/arm64/include/asm/jump_label.h:31:2: error: impossible constraint in ?asm?
>   asm goto("1: nop\n\t"
>   ^
> make[1]: *** [arch/arm64/kvm/hyp/switch.o] Error 1
> make: *** [arch/arm64/kvm/hyp/switch.o] Error 2

it looks like we cannot avoid hyp_data thing... if you don't mind I can
do hyp_data clean-up in separate patch. Alternatively, we can do
conversion to static keys for both architectures later as an
optimisation step.

Cheers
Vladimir

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 1/8] arm64: KVM: Use static keys for selecting the GIC backend
  2016-09-09 13:45         ` Vladimir Murzin
@ 2016-09-09 14:17           ` Marc Zyngier
  -1 siblings, 0 replies; 72+ messages in thread
From: Marc Zyngier @ 2016-09-09 14:17 UTC (permalink / raw)
  To: Vladimir Murzin, kvmarm; +Cc: andre.przywara, linux-arm-kernel

On 09/09/16 14:45, Vladimir Murzin wrote:
> On 09/09/16 10:33, Vladimir Murzin wrote:
>> Hi Marc,
>>
>> On 09/09/16 10:19, Marc Zyngier wrote:
>>>> Hi Vladimir,
>>>>
>> ...
>>>>>>  
>>>>>> +extern struct static_key_false kvm_gicv3_cpuif;
>>>>
>>>> I think we should follow the model set by kvm_vgic_global_state, which
>>>> is declared in arm_vgic.h. Even better, we should *embed* the static key
>>>> in this structure. This will reduce the clutter and we wouldn't have to
>>>> deal with all the section stuff (the hyp_data thing is a good cleanup,
>>>> but I'd like to see it as a separate patch if possible).
>> Yes, it is what I was thinking about too, but was not sure about which
>> way to go, so hyp_data seemed me something we might reuse latter.
>> However, I agree that we can defer hyp_data thing...
>>
> 
> I've just tried it out and it seems that static keys are not happy to
> accept a key after kern_hyp_va is applied at &kvm_vgic_global_state:

Ah, there is a trick. You do not need kern_hyp_va at all, because this
is not evaluated as an expression at runtime (so the pointer doesn't matter).

>> In file included from ./include/linux/jump_label.h:105:0,
>>                  from arch/arm64/kvm/hyp/switch.c:19:
>> ./arch/arm64/include/asm/jump_label.h: In function ‘__guest_run’:
>> ./arch/arm64/include/asm/jump_label.h:31:2: warning: asm operand 0 probably doesn’t match constraints
>>   asm goto("1: nop\n\t"
>>   ^
>> ./arch/arm64/include/asm/jump_label.h:31:2: warning: asm operand 0 probably doesn’t match constraints
>>   asm goto("1: nop\n\t"
>>   ^
>> ./arch/arm64/include/asm/jump_label.h:31:2: error: impossible constraint in ‘asm’
>>   asm goto("1: nop\n\t"
>>   ^
>> ./arch/arm64/include/asm/jump_label.h:31:2: error: impossible constraint in ‘asm’
>>   asm goto("1: nop\n\t"
>>   ^
>> make[1]: *** [arch/arm64/kvm/hyp/switch.o] Error 1
>> make: *** [arch/arm64/kvm/hyp/switch.o] Error 2
> 
> it looks like we cannot avoid hyp_data thing... if you don't mind I can
> do hyp_data clean-up in separate patch. Alternatively, we can do
> conversion to static keys for both architectures later as an
> optimisation step.

Can you try the above first? I've just tried the same approach with my
vgic-trap series, and it compiles fine (untested though):

diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 194184c..397e240 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -301,7 +301,7 @@ again:
 	if (exit_code == ARM_EXCEPTION_TRAP && !__populate_fault_info(vcpu))
 		goto again;
 
-	if (static_branch_unlikely(&vgic_v2_cpuif_trap) &&
+	if (static_branch_unlikely(&kvm_vgic_global_state.vgic_v2_cpuif_trap) &&
 	    exit_code == ARM_EXCEPTION_TRAP) {
 		bool valid;
 
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index bb46c03..cc59a28 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -67,6 +67,9 @@ struct vgic_global {
 
 	/* Only needed for the legacy KVM_CREATE_IRQCHIP */
 	bool			can_emulate_gicv2;
+
+	/* Trapping GICv2 GICV region? */
+	struct static_key_false	vgic_v2_cpuif_trap;
 };
 
 extern struct vgic_global kvm_vgic_global_state;
@@ -269,8 +272,6 @@ struct vgic_cpu {
 	bool lpis_enabled;
 };
 
-extern struct static_key_false vgic_v2_cpuif_trap;
-
 int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write);
 void kvm_vgic_early_init(struct kvm *kvm);
 int kvm_vgic_create(struct kvm *kvm, u32 type);
diff --git a/virt/kvm/arm/vgic/vgic-v2.c b/virt/kvm/arm/vgic/vgic-v2.c
index 0a063af..4af600e 100644
--- a/virt/kvm/arm/vgic/vgic-v2.c
+++ b/virt/kvm/arm/vgic/vgic-v2.c
@@ -278,7 +278,7 @@ int vgic_v2_map_resources(struct kvm *kvm)
 		goto out;
 	}
 
-	if (!static_branch_unlikely(&vgic_v2_cpuif_trap)) {
+	if (!static_branch_unlikely(&kvm_vgic_global_state.vgic_v2_cpuif_trap)) {
 		ret = kvm_phys_addr_ioremap(kvm, dist->vgic_cpu_base,
 					    kvm_vgic_global_state.vcpu_base,
 					    KVM_VGIC_V2_CPU_SIZE, true);
@@ -296,8 +296,6 @@ out:
 	return ret;
 }
 
-DEFINE_STATIC_KEY_FALSE(vgic_v2_cpuif_trap);
-
 /**
  * vgic_v2_probe - probe for a GICv2 compatible interrupt controller in DT
  * @node:	pointer to the DT node
@@ -314,6 +312,8 @@ int vgic_v2_probe(const struct gic_kvm_info *info)
 		return -ENXIO;
 	}
 
+	kvm_vgic_global_state.vgic_v2_cpuif_trap = STATIC_KEY_FALSE_INIT;
+
 	if (!PAGE_ALIGNED(info->vcpu.start) ||
 	    !PAGE_ALIGNED(resource_size(&info->vcpu))) {
 		kvm_info("GICV region size/alignment is unsafe, using trapping (reduced performance)\n");
@@ -332,7 +332,7 @@ int vgic_v2_probe(const struct gic_kvm_info *info)
 			goto out;
 		}
 
-		static_branch_enable(&vgic_v2_cpuif_trap);
+		static_branch_enable(&kvm_vgic_global_state.vgic_v2_cpuif_trap);
 	}
 
 	kvm_vgic_global_state.vctrl_base = ioremap(info->vctrl.start,


Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply related	[flat|nested] 72+ messages in thread
* [PATCH v3 1/8] arm64: KVM: Use static keys for selecting the GIC backend
@ 2016-09-09 14:17           ` Marc Zyngier
  0 siblings, 0 replies; 72+ messages in thread
From: Marc Zyngier @ 2016-09-09 14:17 UTC (permalink / raw)
  To: linux-arm-kernel

On 09/09/16 14:45, Vladimir Murzin wrote:
> On 09/09/16 10:33, Vladimir Murzin wrote:
>> Hi Marc,
>>
>> On 09/09/16 10:19, Marc Zyngier wrote:
>>>> Hi Vladimir,
>>>>
>> ...
>>>>>>  
>>>>>> +extern struct static_key_false kvm_gicv3_cpuif;
>>>>
>>>> I think we should follow the model set by kvm_vgic_global_state, which
>>>> is declared in arm_vgic.h. Even better, we should *embed* the static key
>>>> in this structure. This will reduce the clutter and we wouldn't have to
>>>> deal with all the section stuff (the hyp_data thing is a good cleanup,
>>>> but I'd like to see it as a separate patch if possible).
>> Yes, it is what I was thinking about too, but was not sure about which
>> way to go, so hyp_data seemed me something we might reuse latter.
>> However, I agree that we can defer hyp_data thing...
>>
> 
> I've just tried it out and it seems that static keys are not happy to
> accept a key after kern_hyp_va is applied at &kvm_vgic_global_state:

Ah, there is a trick. You do not need kern_hyp_va at all, because this
is not evaluated as an expression at runtime (so the pointer doesn't matter).

>> In file included from ./include/linux/jump_label.h:105:0,
>>                  from arch/arm64/kvm/hyp/switch.c:19:
>> ./arch/arm64/include/asm/jump_label.h: In function ?__guest_run?:
>> ./arch/arm64/include/asm/jump_label.h:31:2: warning: asm operand 0 probably doesn?t match constraints
>>   asm goto("1: nop\n\t"
>>   ^
>> ./arch/arm64/include/asm/jump_label.h:31:2: warning: asm operand 0 probably doesn?t match constraints
>>   asm goto("1: nop\n\t"
>>   ^
>> ./arch/arm64/include/asm/jump_label.h:31:2: error: impossible constraint in ?asm?
>>   asm goto("1: nop\n\t"
>>   ^
>> ./arch/arm64/include/asm/jump_label.h:31:2: error: impossible constraint in ?asm?
>>   asm goto("1: nop\n\t"
>>   ^
>> make[1]: *** [arch/arm64/kvm/hyp/switch.o] Error 1
>> make: *** [arch/arm64/kvm/hyp/switch.o] Error 2
> 
> it looks like we cannot avoid hyp_data thing... if you don't mind I can
> do hyp_data clean-up in separate patch. Alternatively, we can do
> conversion to static keys for both architectures later as an
> optimisation step.

Can you try the above first? I've just tried the same approach with my
vgic-trap series, and it compiles fine (untested though):

diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 194184c..397e240 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -301,7 +301,7 @@ again:
 	if (exit_code == ARM_EXCEPTION_TRAP && !__populate_fault_info(vcpu))
 		goto again;
 
-	if (static_branch_unlikely(&vgic_v2_cpuif_trap) &&
+	if (static_branch_unlikely(&kvm_vgic_global_state.vgic_v2_cpuif_trap) &&
 	    exit_code == ARM_EXCEPTION_TRAP) {
 		bool valid;
 
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index bb46c03..cc59a28 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -67,6 +67,9 @@ struct vgic_global {
 
 	/* Only needed for the legacy KVM_CREATE_IRQCHIP */
 	bool			can_emulate_gicv2;
+
+	/* Trapping GICv2 GICV region? */
+	struct static_key_false	vgic_v2_cpuif_trap;
 };
 
 extern struct vgic_global kvm_vgic_global_state;
@@ -269,8 +272,6 @@ struct vgic_cpu {
 	bool lpis_enabled;
 };
 
-extern struct static_key_false vgic_v2_cpuif_trap;
-
 int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write);
 void kvm_vgic_early_init(struct kvm *kvm);
 int kvm_vgic_create(struct kvm *kvm, u32 type);
diff --git a/virt/kvm/arm/vgic/vgic-v2.c b/virt/kvm/arm/vgic/vgic-v2.c
index 0a063af..4af600e 100644
--- a/virt/kvm/arm/vgic/vgic-v2.c
+++ b/virt/kvm/arm/vgic/vgic-v2.c
@@ -278,7 +278,7 @@ int vgic_v2_map_resources(struct kvm *kvm)
 		goto out;
 	}
 
-	if (!static_branch_unlikely(&vgic_v2_cpuif_trap)) {
+	if (!static_branch_unlikely(&kvm_vgic_global_state.vgic_v2_cpuif_trap)) {
 		ret = kvm_phys_addr_ioremap(kvm, dist->vgic_cpu_base,
 					    kvm_vgic_global_state.vcpu_base,
 					    KVM_VGIC_V2_CPU_SIZE, true);
@@ -296,8 +296,6 @@ out:
 	return ret;
 }
 
-DEFINE_STATIC_KEY_FALSE(vgic_v2_cpuif_trap);
-
 /**
  * vgic_v2_probe - probe for a GICv2 compatible interrupt controller in DT
  * @node:	pointer to the DT node
@@ -314,6 +312,8 @@ int vgic_v2_probe(const struct gic_kvm_info *info)
 		return -ENXIO;
 	}
 
+	kvm_vgic_global_state.vgic_v2_cpuif_trap = STATIC_KEY_FALSE_INIT;
+
 	if (!PAGE_ALIGNED(info->vcpu.start) ||
 	    !PAGE_ALIGNED(resource_size(&info->vcpu))) {
 		kvm_info("GICV region size/alignment is unsafe, using trapping (reduced performance)\n");
@@ -332,7 +332,7 @@ int vgic_v2_probe(const struct gic_kvm_info *info)
 			goto out;
 		}
 
-		static_branch_enable(&vgic_v2_cpuif_trap);
+		static_branch_enable(&kvm_vgic_global_state.vgic_v2_cpuif_trap);
 	}
 
 	kvm_vgic_global_state.vctrl_base = ioremap(info->vctrl.start,


Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply related	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 1/8] arm64: KVM: Use static keys for selecting the GIC backend
  2016-09-09 14:17           ` Marc Zyngier
@ 2016-09-09 15:14             ` Vladimir Murzin
  -1 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-09 15:14 UTC (permalink / raw)
  To: Marc Zyngier, kvmarm; +Cc: andre.przywara, linux-arm-kernel

On 09/09/16 15:17, Marc Zyngier wrote:
> On 09/09/16 14:45, Vladimir Murzin wrote:
>> On 09/09/16 10:33, Vladimir Murzin wrote:
>>> Hi Marc,
>>>
>>> On 09/09/16 10:19, Marc Zyngier wrote:
>>>>> Hi Vladimir,
>>>>>
>>> ...
>>>>>>>  
>>>>>>> +extern struct static_key_false kvm_gicv3_cpuif;
>>>>>
>>>>> I think we should follow the model set by kvm_vgic_global_state, which
>>>>> is declared in arm_vgic.h. Even better, we should *embed* the static key
>>>>> in this structure. This will reduce the clutter and we wouldn't have to
>>>>> deal with all the section stuff (the hyp_data thing is a good cleanup,
>>>>> but I'd like to see it as a separate patch if possible).
>>> Yes, it is what I was thinking about too, but was not sure about which
>>> way to go, so hyp_data seemed me something we might reuse latter.
>>> However, I agree that we can defer hyp_data thing...
>>>
>>
>> I've just tried it out and it seems that static keys are not happy to
>> accept a key after kern_hyp_va is applied at &kvm_vgic_global_state:
> 
> Ah, there is a trick. You do not need kern_hyp_va at all, because this
> is not evaluated as an expression at runtime (so the pointer doesn't matter).
> 

Ah, right, thank for a tip! ;)

>>> In file included from ./include/linux/jump_label.h:105:0,
>>>                  from arch/arm64/kvm/hyp/switch.c:19:
>>> ./arch/arm64/include/asm/jump_label.h: In function ‘__guest_run’:
>>> ./arch/arm64/include/asm/jump_label.h:31:2: warning: asm operand 0 probably doesn’t match constraints
>>>   asm goto("1: nop\n\t"
>>>   ^
>>> ./arch/arm64/include/asm/jump_label.h:31:2: warning: asm operand 0 probably doesn’t match constraints
>>>   asm goto("1: nop\n\t"
>>>   ^
>>> ./arch/arm64/include/asm/jump_label.h:31:2: error: impossible constraint in ‘asm’
>>>   asm goto("1: nop\n\t"
>>>   ^
>>> ./arch/arm64/include/asm/jump_label.h:31:2: error: impossible constraint in ‘asm’
>>>   asm goto("1: nop\n\t"
>>>   ^
>>> make[1]: *** [arch/arm64/kvm/hyp/switch.o] Error 1
>>> make: *** [arch/arm64/kvm/hyp/switch.o] Error 2
>>
>> it looks like we cannot avoid hyp_data thing... if you don't mind I can
>> do hyp_data clean-up in separate patch. Alternatively, we can do
>> conversion to static keys for both architectures later as an
>> optimisation step.
> 
> Can you try the above first? I've just tried the same approach with my
> vgic-trap series, and it compiles fine (untested though):

I was about to try it out, but didn't manage to find a branch with
vgic-trap series, so I did a quick fixup for my series and now it is
running non-VHE boot tests and I don't expect issues with VHE one. I
think diff bellow should work runtime too, but if you do want me to give
it a try it'd be handy to have a branch I can pull from ;)

> 
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 194184c..397e240 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -301,7 +301,7 @@ again:
>  	if (exit_code == ARM_EXCEPTION_TRAP && !__populate_fault_info(vcpu))
>  		goto again;
>  
> -	if (static_branch_unlikely(&vgic_v2_cpuif_trap) &&
> +	if (static_branch_unlikely(&kvm_vgic_global_state.vgic_v2_cpuif_trap) &&
>  	    exit_code == ARM_EXCEPTION_TRAP) {
>  		bool valid;
>  
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index bb46c03..cc59a28 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -67,6 +67,9 @@ struct vgic_global {
>  
>  	/* Only needed for the legacy KVM_CREATE_IRQCHIP */
>  	bool			can_emulate_gicv2;
> +
> +	/* Trapping GICv2 GICV region? */
> +	struct static_key_false	vgic_v2_cpuif_trap;
>  };
>  
>  extern struct vgic_global kvm_vgic_global_state;
> @@ -269,8 +272,6 @@ struct vgic_cpu {
>  	bool lpis_enabled;
>  };
>  
> -extern struct static_key_false vgic_v2_cpuif_trap;
> -
>  int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write);
>  void kvm_vgic_early_init(struct kvm *kvm);
>  int kvm_vgic_create(struct kvm *kvm, u32 type);
> diff --git a/virt/kvm/arm/vgic/vgic-v2.c b/virt/kvm/arm/vgic/vgic-v2.c
> index 0a063af..4af600e 100644
> --- a/virt/kvm/arm/vgic/vgic-v2.c
> +++ b/virt/kvm/arm/vgic/vgic-v2.c
> @@ -278,7 +278,7 @@ int vgic_v2_map_resources(struct kvm *kvm)
>  		goto out;
>  	}
>  
> -	if (!static_branch_unlikely(&vgic_v2_cpuif_trap)) {
> +	if (!static_branch_unlikely(&kvm_vgic_global_state.vgic_v2_cpuif_trap)) {
>  		ret = kvm_phys_addr_ioremap(kvm, dist->vgic_cpu_base,
>  					    kvm_vgic_global_state.vcpu_base,
>  					    KVM_VGIC_V2_CPU_SIZE, true);
> @@ -296,8 +296,6 @@ out:
>  	return ret;
>  }
>  
> -DEFINE_STATIC_KEY_FALSE(vgic_v2_cpuif_trap);
> -
>  /**
>   * vgic_v2_probe - probe for a GICv2 compatible interrupt controller in DT
>   * @node:	pointer to the DT node
> @@ -314,6 +312,8 @@ int vgic_v2_probe(const struct gic_kvm_info *info)
>  		return -ENXIO;
>  	}
>  
> +	kvm_vgic_global_state.vgic_v2_cpuif_trap = STATIC_KEY_FALSE_INIT;
> +

I did this a bit different:

+struct vgic_global __section(.hyp.text) kvm_vgic_global_state =
{.gicv3_cpuif = STATIC_KEY_FALSE_INIT,};

Thanks
Vladimir

>  	if (!PAGE_ALIGNED(info->vcpu.start) ||
>  	    !PAGE_ALIGNED(resource_size(&info->vcpu))) {
>  		kvm_info("GICV region size/alignment is unsafe, using trapping (reduced performance)\n");
> @@ -332,7 +332,7 @@ int vgic_v2_probe(const struct gic_kvm_info *info)
>  			goto out;
>  		}
>  
> -		static_branch_enable(&vgic_v2_cpuif_trap);
> +		static_branch_enable(&kvm_vgic_global_state.vgic_v2_cpuif_trap);
>  	}
>  
>  	kvm_vgic_global_state.vctrl_base = ioremap(info->vctrl.start,
> 
> 
> Thanks,
> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 1/8] arm64: KVM: Use static keys for selecting the GIC backend
@ 2016-09-09 15:14             ` Vladimir Murzin
  0 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-09 15:14 UTC (permalink / raw)
  To: linux-arm-kernel

On 09/09/16 15:17, Marc Zyngier wrote:
> On 09/09/16 14:45, Vladimir Murzin wrote:
>> On 09/09/16 10:33, Vladimir Murzin wrote:
>>> Hi Marc,
>>>
>>> On 09/09/16 10:19, Marc Zyngier wrote:
>>>>> Hi Vladimir,
>>>>>
>>> ...
>>>>>>>  
>>>>>>> +extern struct static_key_false kvm_gicv3_cpuif;
>>>>>
>>>>> I think we should follow the model set by kvm_vgic_global_state, which
>>>>> is declared in arm_vgic.h. Even better, we should *embed* the static key
>>>>> in this structure. This will reduce the clutter and we wouldn't have to
>>>>> deal with all the section stuff (the hyp_data thing is a good cleanup,
>>>>> but I'd like to see it as a separate patch if possible).
>>> Yes, it is what I was thinking about too, but was not sure about which
>>> way to go, so hyp_data seemed me something we might reuse latter.
>>> However, I agree that we can defer hyp_data thing...
>>>
>>
>> I've just tried it out and it seems that static keys are not happy to
>> accept a key after kern_hyp_va is applied at &kvm_vgic_global_state:
> 
> Ah, there is a trick. You do not need kern_hyp_va at all, because this
> is not evaluated as an expression at runtime (so the pointer doesn't matter).
> 

Ah, right, thank for a tip! ;)

>>> In file included from ./include/linux/jump_label.h:105:0,
>>>                  from arch/arm64/kvm/hyp/switch.c:19:
>>> ./arch/arm64/include/asm/jump_label.h: In function ?__guest_run?:
>>> ./arch/arm64/include/asm/jump_label.h:31:2: warning: asm operand 0 probably doesn?t match constraints
>>>   asm goto("1: nop\n\t"
>>>   ^
>>> ./arch/arm64/include/asm/jump_label.h:31:2: warning: asm operand 0 probably doesn?t match constraints
>>>   asm goto("1: nop\n\t"
>>>   ^
>>> ./arch/arm64/include/asm/jump_label.h:31:2: error: impossible constraint in ?asm?
>>>   asm goto("1: nop\n\t"
>>>   ^
>>> ./arch/arm64/include/asm/jump_label.h:31:2: error: impossible constraint in ?asm?
>>>   asm goto("1: nop\n\t"
>>>   ^
>>> make[1]: *** [arch/arm64/kvm/hyp/switch.o] Error 1
>>> make: *** [arch/arm64/kvm/hyp/switch.o] Error 2
>>
>> it looks like we cannot avoid hyp_data thing... if you don't mind I can
>> do hyp_data clean-up in separate patch. Alternatively, we can do
>> conversion to static keys for both architectures later as an
>> optimisation step.
> 
> Can you try the above first? I've just tried the same approach with my
> vgic-trap series, and it compiles fine (untested though):

I was about to try it out, but didn't manage to find a branch with
vgic-trap series, so I did a quick fixup for my series and now it is
running non-VHE boot tests and I don't expect issues with VHE one. I
think diff bellow should work runtime too, but if you do want me to give
it a try it'd be handy to have a branch I can pull from ;)

> 
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 194184c..397e240 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -301,7 +301,7 @@ again:
>  	if (exit_code == ARM_EXCEPTION_TRAP && !__populate_fault_info(vcpu))
>  		goto again;
>  
> -	if (static_branch_unlikely(&vgic_v2_cpuif_trap) &&
> +	if (static_branch_unlikely(&kvm_vgic_global_state.vgic_v2_cpuif_trap) &&
>  	    exit_code == ARM_EXCEPTION_TRAP) {
>  		bool valid;
>  
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index bb46c03..cc59a28 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -67,6 +67,9 @@ struct vgic_global {
>  
>  	/* Only needed for the legacy KVM_CREATE_IRQCHIP */
>  	bool			can_emulate_gicv2;
> +
> +	/* Trapping GICv2 GICV region? */
> +	struct static_key_false	vgic_v2_cpuif_trap;
>  };
>  
>  extern struct vgic_global kvm_vgic_global_state;
> @@ -269,8 +272,6 @@ struct vgic_cpu {
>  	bool lpis_enabled;
>  };
>  
> -extern struct static_key_false vgic_v2_cpuif_trap;
> -
>  int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write);
>  void kvm_vgic_early_init(struct kvm *kvm);
>  int kvm_vgic_create(struct kvm *kvm, u32 type);
> diff --git a/virt/kvm/arm/vgic/vgic-v2.c b/virt/kvm/arm/vgic/vgic-v2.c
> index 0a063af..4af600e 100644
> --- a/virt/kvm/arm/vgic/vgic-v2.c
> +++ b/virt/kvm/arm/vgic/vgic-v2.c
> @@ -278,7 +278,7 @@ int vgic_v2_map_resources(struct kvm *kvm)
>  		goto out;
>  	}
>  
> -	if (!static_branch_unlikely(&vgic_v2_cpuif_trap)) {
> +	if (!static_branch_unlikely(&kvm_vgic_global_state.vgic_v2_cpuif_trap)) {
>  		ret = kvm_phys_addr_ioremap(kvm, dist->vgic_cpu_base,
>  					    kvm_vgic_global_state.vcpu_base,
>  					    KVM_VGIC_V2_CPU_SIZE, true);
> @@ -296,8 +296,6 @@ out:
>  	return ret;
>  }
>  
> -DEFINE_STATIC_KEY_FALSE(vgic_v2_cpuif_trap);
> -
>  /**
>   * vgic_v2_probe - probe for a GICv2 compatible interrupt controller in DT
>   * @node:	pointer to the DT node
> @@ -314,6 +312,8 @@ int vgic_v2_probe(const struct gic_kvm_info *info)
>  		return -ENXIO;
>  	}
>  
> +	kvm_vgic_global_state.vgic_v2_cpuif_trap = STATIC_KEY_FALSE_INIT;
> +

I did this a bit different:

+struct vgic_global __section(.hyp.text) kvm_vgic_global_state =
{.gicv3_cpuif = STATIC_KEY_FALSE_INIT,};

Thanks
Vladimir

>  	if (!PAGE_ALIGNED(info->vcpu.start) ||
>  	    !PAGE_ALIGNED(resource_size(&info->vcpu))) {
>  		kvm_info("GICV region size/alignment is unsafe, using trapping (reduced performance)\n");
> @@ -332,7 +332,7 @@ int vgic_v2_probe(const struct gic_kvm_info *info)
>  			goto out;
>  		}
>  
> -		static_branch_enable(&vgic_v2_cpuif_trap);
> +		static_branch_enable(&kvm_vgic_global_state.vgic_v2_cpuif_trap);
>  	}
>  
>  	kvm_vgic_global_state.vctrl_base = ioremap(info->vctrl.start,
> 
> 
> Thanks,
> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 1/8] arm64: KVM: Use static keys for selecting the GIC backend
  2016-09-09 15:14             ` Vladimir Murzin
@ 2016-09-09 15:25               ` Marc Zyngier
  -1 siblings, 0 replies; 72+ messages in thread
From: Marc Zyngier @ 2016-09-09 15:25 UTC (permalink / raw)
  To: Vladimir Murzin, kvmarm; +Cc: andre.przywara, linux-arm-kernel

On 09/09/16 16:14, Vladimir Murzin wrote:
> On 09/09/16 15:17, Marc Zyngier wrote:
>> On 09/09/16 14:45, Vladimir Murzin wrote:
>>> On 09/09/16 10:33, Vladimir Murzin wrote:
>>>> Hi Marc,
>>>>
>>>> On 09/09/16 10:19, Marc Zyngier wrote:
>>>>>> Hi Vladimir,
>>>>>>
>>>> ...
>>>>>>>>  
>>>>>>>> +extern struct static_key_false kvm_gicv3_cpuif;
>>>>>>
>>>>>> I think we should follow the model set by kvm_vgic_global_state, which
>>>>>> is declared in arm_vgic.h. Even better, we should *embed* the static key
>>>>>> in this structure. This will reduce the clutter and we wouldn't have to
>>>>>> deal with all the section stuff (the hyp_data thing is a good cleanup,
>>>>>> but I'd like to see it as a separate patch if possible).
>>>> Yes, it is what I was thinking about too, but was not sure about which
>>>> way to go, so hyp_data seemed me something we might reuse latter.
>>>> However, I agree that we can defer hyp_data thing...
>>>>
>>>
>>> I've just tried it out and it seems that static keys are not happy to
>>> accept a key after kern_hyp_va is applied at &kvm_vgic_global_state:
>>
>> Ah, there is a trick. You do not need kern_hyp_va at all, because this
>> is not evaluated as an expression at runtime (so the pointer doesn't matter).
>>
> 
> Ah, right, thank for a tip! ;)
> 
>>>> In file included from ./include/linux/jump_label.h:105:0,
>>>>                  from arch/arm64/kvm/hyp/switch.c:19:
>>>> ./arch/arm64/include/asm/jump_label.h: In function ‘__guest_run’:
>>>> ./arch/arm64/include/asm/jump_label.h:31:2: warning: asm operand 0 probably doesn’t match constraints
>>>>   asm goto("1: nop\n\t"
>>>>   ^
>>>> ./arch/arm64/include/asm/jump_label.h:31:2: warning: asm operand 0 probably doesn’t match constraints
>>>>   asm goto("1: nop\n\t"
>>>>   ^
>>>> ./arch/arm64/include/asm/jump_label.h:31:2: error: impossible constraint in ‘asm’
>>>>   asm goto("1: nop\n\t"
>>>>   ^
>>>> ./arch/arm64/include/asm/jump_label.h:31:2: error: impossible constraint in ‘asm’
>>>>   asm goto("1: nop\n\t"
>>>>   ^
>>>> make[1]: *** [arch/arm64/kvm/hyp/switch.o] Error 1
>>>> make: *** [arch/arm64/kvm/hyp/switch.o] Error 2
>>>
>>> it looks like we cannot avoid hyp_data thing... if you don't mind I can
>>> do hyp_data clean-up in separate patch. Alternatively, we can do
>>> conversion to static keys for both architectures later as an
>>> optimisation step.
>>
>> Can you try the above first? I've just tried the same approach with my
>> vgic-trap series, and it compiles fine (untested though):
> 
> I was about to try it out, but didn't manage to find a branch with
> vgic-trap series, so I did a quick fixup for my series and now it is
> running non-VHE boot tests and I don't expect issues with VHE one. I
> think diff bellow should work runtime too, but if you do want me to give
> it a try it'd be handy to have a branch I can pull from ;)

I think Christoffer has pulled it into kvmarm/queue.

>> +	kvm_vgic_global_state.vgic_v2_cpuif_trap = STATIC_KEY_FALSE_INIT;
>> +
> 
> I did this a bit different:
> 
> +struct vgic_global __section(.hyp.text) kvm_vgic_global_state =
> {.gicv3_cpuif = STATIC_KEY_FALSE_INIT,};

Yeah, that's actually nicer.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 1/8] arm64: KVM: Use static keys for selecting the GIC backend
@ 2016-09-09 15:25               ` Marc Zyngier
  0 siblings, 0 replies; 72+ messages in thread
From: Marc Zyngier @ 2016-09-09 15:25 UTC (permalink / raw)
  To: linux-arm-kernel

On 09/09/16 16:14, Vladimir Murzin wrote:
> On 09/09/16 15:17, Marc Zyngier wrote:
>> On 09/09/16 14:45, Vladimir Murzin wrote:
>>> On 09/09/16 10:33, Vladimir Murzin wrote:
>>>> Hi Marc,
>>>>
>>>> On 09/09/16 10:19, Marc Zyngier wrote:
>>>>>> Hi Vladimir,
>>>>>>
>>>> ...
>>>>>>>>  
>>>>>>>> +extern struct static_key_false kvm_gicv3_cpuif;
>>>>>>
>>>>>> I think we should follow the model set by kvm_vgic_global_state, which
>>>>>> is declared in arm_vgic.h. Even better, we should *embed* the static key
>>>>>> in this structure. This will reduce the clutter and we wouldn't have to
>>>>>> deal with all the section stuff (the hyp_data thing is a good cleanup,
>>>>>> but I'd like to see it as a separate patch if possible).
>>>> Yes, it is what I was thinking about too, but was not sure about which
>>>> way to go, so hyp_data seemed me something we might reuse latter.
>>>> However, I agree that we can defer hyp_data thing...
>>>>
>>>
>>> I've just tried it out and it seems that static keys are not happy to
>>> accept a key after kern_hyp_va is applied at &kvm_vgic_global_state:
>>
>> Ah, there is a trick. You do not need kern_hyp_va at all, because this
>> is not evaluated as an expression at runtime (so the pointer doesn't matter).
>>
> 
> Ah, right, thank for a tip! ;)
> 
>>>> In file included from ./include/linux/jump_label.h:105:0,
>>>>                  from arch/arm64/kvm/hyp/switch.c:19:
>>>> ./arch/arm64/include/asm/jump_label.h: In function ?__guest_run?:
>>>> ./arch/arm64/include/asm/jump_label.h:31:2: warning: asm operand 0 probably doesn?t match constraints
>>>>   asm goto("1: nop\n\t"
>>>>   ^
>>>> ./arch/arm64/include/asm/jump_label.h:31:2: warning: asm operand 0 probably doesn?t match constraints
>>>>   asm goto("1: nop\n\t"
>>>>   ^
>>>> ./arch/arm64/include/asm/jump_label.h:31:2: error: impossible constraint in ?asm?
>>>>   asm goto("1: nop\n\t"
>>>>   ^
>>>> ./arch/arm64/include/asm/jump_label.h:31:2: error: impossible constraint in ?asm?
>>>>   asm goto("1: nop\n\t"
>>>>   ^
>>>> make[1]: *** [arch/arm64/kvm/hyp/switch.o] Error 1
>>>> make: *** [arch/arm64/kvm/hyp/switch.o] Error 2
>>>
>>> it looks like we cannot avoid hyp_data thing... if you don't mind I can
>>> do hyp_data clean-up in separate patch. Alternatively, we can do
>>> conversion to static keys for both architectures later as an
>>> optimisation step.
>>
>> Can you try the above first? I've just tried the same approach with my
>> vgic-trap series, and it compiles fine (untested though):
> 
> I was about to try it out, but didn't manage to find a branch with
> vgic-trap series, so I did a quick fixup for my series and now it is
> running non-VHE boot tests and I don't expect issues with VHE one. I
> think diff bellow should work runtime too, but if you do want me to give
> it a try it'd be handy to have a branch I can pull from ;)

I think Christoffer has pulled it into kvmarm/queue.

>> +	kvm_vgic_global_state.vgic_v2_cpuif_trap = STATIC_KEY_FALSE_INIT;
>> +
> 
> I did this a bit different:
> 
> +struct vgic_global __section(.hyp.text) kvm_vgic_global_state =
> {.gicv3_cpuif = STATIC_KEY_FALSE_INIT,};

Yeah, that's actually nicer.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 1/8] arm64: KVM: Use static keys for selecting the GIC backend
  2016-09-09 15:25               ` Marc Zyngier
@ 2016-09-09 16:18                 ` Vladimir Murzin
  -1 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-09 16:18 UTC (permalink / raw)
  To: Marc Zyngier, kvmarm; +Cc: andre.przywara, linux-arm-kernel

On 09/09/16 16:25, Marc Zyngier wrote:
> On 09/09/16 16:14, Vladimir Murzin wrote:
>> On 09/09/16 15:17, Marc Zyngier wrote:
>>> On 09/09/16 14:45, Vladimir Murzin wrote:
>>>> On 09/09/16 10:33, Vladimir Murzin wrote:
>>>>> Hi Marc,
>>>>>
>>>>> On 09/09/16 10:19, Marc Zyngier wrote:
>>>>>>> Hi Vladimir,
>>>>>>>
>>>>> ...
>>>>>>>>>  
>>>>>>>>> +extern struct static_key_false kvm_gicv3_cpuif;
>>>>>>>
>>>>>>> I think we should follow the model set by kvm_vgic_global_state, which
>>>>>>> is declared in arm_vgic.h. Even better, we should *embed* the static key
>>>>>>> in this structure. This will reduce the clutter and we wouldn't have to
>>>>>>> deal with all the section stuff (the hyp_data thing is a good cleanup,
>>>>>>> but I'd like to see it as a separate patch if possible).
>>>>> Yes, it is what I was thinking about too, but was not sure about which
>>>>> way to go, so hyp_data seemed me something we might reuse latter.
>>>>> However, I agree that we can defer hyp_data thing...
>>>>>
>>>>
>>>> I've just tried it out and it seems that static keys are not happy to
>>>> accept a key after kern_hyp_va is applied at &kvm_vgic_global_state:
>>>
>>> Ah, there is a trick. You do not need kern_hyp_va at all, because this
>>> is not evaluated as an expression at runtime (so the pointer doesn't matter).
>>>
>>
>> Ah, right, thank for a tip! ;)
>>
>>>>> In file included from ./include/linux/jump_label.h:105:0,
>>>>>                  from arch/arm64/kvm/hyp/switch.c:19:
>>>>> ./arch/arm64/include/asm/jump_label.h: In function ‘__guest_run’:
>>>>> ./arch/arm64/include/asm/jump_label.h:31:2: warning: asm operand 0 probably doesn’t match constraints
>>>>>   asm goto("1: nop\n\t"
>>>>>   ^
>>>>> ./arch/arm64/include/asm/jump_label.h:31:2: warning: asm operand 0 probably doesn’t match constraints
>>>>>   asm goto("1: nop\n\t"
>>>>>   ^
>>>>> ./arch/arm64/include/asm/jump_label.h:31:2: error: impossible constraint in ‘asm’
>>>>>   asm goto("1: nop\n\t"
>>>>>   ^
>>>>> ./arch/arm64/include/asm/jump_label.h:31:2: error: impossible constraint in ‘asm’
>>>>>   asm goto("1: nop\n\t"
>>>>>   ^
>>>>> make[1]: *** [arch/arm64/kvm/hyp/switch.o] Error 1
>>>>> make: *** [arch/arm64/kvm/hyp/switch.o] Error 2
>>>>
>>>> it looks like we cannot avoid hyp_data thing... if you don't mind I can
>>>> do hyp_data clean-up in separate patch. Alternatively, we can do
>>>> conversion to static keys for both architectures later as an
>>>> optimisation step.
>>>
>>> Can you try the above first? I've just tried the same approach with my
>>> vgic-trap series, and it compiles fine (untested though):
>>
>> I was about to try it out, but didn't manage to find a branch with
>> vgic-trap series, so I did a quick fixup for my series and now it is
>> running non-VHE boot tests and I don't expect issues with VHE one. I
>> think diff bellow should work runtime too, but if you do want me to give
>> it a try it'd be handy to have a branch I can pull from ;)
> 
> I think Christoffer has pulled it into kvmarm/queue.

Nothing has exploded so far for both architectures.

Cheers
Vladimir

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 1/8] arm64: KVM: Use static keys for selecting the GIC backend
@ 2016-09-09 16:18                 ` Vladimir Murzin
  0 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-09 16:18 UTC (permalink / raw)
  To: linux-arm-kernel

On 09/09/16 16:25, Marc Zyngier wrote:
> On 09/09/16 16:14, Vladimir Murzin wrote:
>> On 09/09/16 15:17, Marc Zyngier wrote:
>>> On 09/09/16 14:45, Vladimir Murzin wrote:
>>>> On 09/09/16 10:33, Vladimir Murzin wrote:
>>>>> Hi Marc,
>>>>>
>>>>> On 09/09/16 10:19, Marc Zyngier wrote:
>>>>>>> Hi Vladimir,
>>>>>>>
>>>>> ...
>>>>>>>>>  
>>>>>>>>> +extern struct static_key_false kvm_gicv3_cpuif;
>>>>>>>
>>>>>>> I think we should follow the model set by kvm_vgic_global_state, which
>>>>>>> is declared in arm_vgic.h. Even better, we should *embed* the static key
>>>>>>> in this structure. This will reduce the clutter and we wouldn't have to
>>>>>>> deal with all the section stuff (the hyp_data thing is a good cleanup,
>>>>>>> but I'd like to see it as a separate patch if possible).
>>>>> Yes, it is what I was thinking about too, but was not sure about which
>>>>> way to go, so hyp_data seemed me something we might reuse latter.
>>>>> However, I agree that we can defer hyp_data thing...
>>>>>
>>>>
>>>> I've just tried it out and it seems that static keys are not happy to
>>>> accept a key after kern_hyp_va is applied at &kvm_vgic_global_state:
>>>
>>> Ah, there is a trick. You do not need kern_hyp_va at all, because this
>>> is not evaluated as an expression at runtime (so the pointer doesn't matter).
>>>
>>
>> Ah, right, thank for a tip! ;)
>>
>>>>> In file included from ./include/linux/jump_label.h:105:0,
>>>>>                  from arch/arm64/kvm/hyp/switch.c:19:
>>>>> ./arch/arm64/include/asm/jump_label.h: In function ?__guest_run?:
>>>>> ./arch/arm64/include/asm/jump_label.h:31:2: warning: asm operand 0 probably doesn?t match constraints
>>>>>   asm goto("1: nop\n\t"
>>>>>   ^
>>>>> ./arch/arm64/include/asm/jump_label.h:31:2: warning: asm operand 0 probably doesn?t match constraints
>>>>>   asm goto("1: nop\n\t"
>>>>>   ^
>>>>> ./arch/arm64/include/asm/jump_label.h:31:2: error: impossible constraint in ?asm?
>>>>>   asm goto("1: nop\n\t"
>>>>>   ^
>>>>> ./arch/arm64/include/asm/jump_label.h:31:2: error: impossible constraint in ?asm?
>>>>>   asm goto("1: nop\n\t"
>>>>>   ^
>>>>> make[1]: *** [arch/arm64/kvm/hyp/switch.o] Error 1
>>>>> make: *** [arch/arm64/kvm/hyp/switch.o] Error 2
>>>>
>>>> it looks like we cannot avoid hyp_data thing... if you don't mind I can
>>>> do hyp_data clean-up in separate patch. Alternatively, we can do
>>>> conversion to static keys for both architectures later as an
>>>> optimisation step.
>>>
>>> Can you try the above first? I've just tried the same approach with my
>>> vgic-trap series, and it compiles fine (untested though):
>>
>> I was about to try it out, but didn't manage to find a branch with
>> vgic-trap series, so I did a quick fixup for my series and now it is
>> running non-VHE boot tests and I don't expect issues with VHE one. I
>> think diff bellow should work runtime too, but if you do want me to give
>> it a try it'd be handy to have a branch I can pull from ;)
> 
> I think Christoffer has pulled it into kvmarm/queue.

Nothing has exploded so far for both architectures.

Cheers
Vladimir

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 2/8] arm64: KVM: Move GIC accessors to arch_gicv3.h
  2016-09-08 16:06   ` Vladimir Murzin
@ 2016-09-09 16:32     ` Marc Zyngier
  -1 siblings, 0 replies; 72+ messages in thread
From: Marc Zyngier @ 2016-09-09 16:32 UTC (permalink / raw)
  To: Vladimir Murzin, kvmarm; +Cc: andre.przywara, linux-arm-kernel

On 08/09/16 17:06, Vladimir Murzin wrote:
> Since we are going to share vgic-v3 save/restore code with ARM keep
> arch specific accessors separately.
> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

Acked-by: Marc Zyngier <marc.zyngier@arm.com>

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 2/8] arm64: KVM: Move GIC accessors to arch_gicv3.h
@ 2016-09-09 16:32     ` Marc Zyngier
  0 siblings, 0 replies; 72+ messages in thread
From: Marc Zyngier @ 2016-09-09 16:32 UTC (permalink / raw)
  To: linux-arm-kernel

On 08/09/16 17:06, Vladimir Murzin wrote:
> Since we are going to share vgic-v3 save/restore code with ARM keep
> arch specific accessors separately.
> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

Acked-by: Marc Zyngier <marc.zyngier@arm.com>

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 3/8] arm64: KVM: Move vgic-v3 save/restore to virt/kvm/arm/hyp
  2016-09-08 16:06   ` Vladimir Murzin
@ 2016-09-09 16:33     ` Marc Zyngier
  -1 siblings, 0 replies; 72+ messages in thread
From: Marc Zyngier @ 2016-09-09 16:33 UTC (permalink / raw)
  To: Vladimir Murzin, kvmarm; +Cc: andre.przywara, linux-arm-kernel

On 08/09/16 17:06, Vladimir Murzin wrote:
> So we can reuse the code under arch/arm
> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>

Acked-by: Marc Zyngier <marc.zyngier@arm.com>

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 3/8] arm64: KVM: Move vgic-v3 save/restore to virt/kvm/arm/hyp
@ 2016-09-09 16:33     ` Marc Zyngier
  0 siblings, 0 replies; 72+ messages in thread
From: Marc Zyngier @ 2016-09-09 16:33 UTC (permalink / raw)
  To: linux-arm-kernel

On 08/09/16 17:06, Vladimir Murzin wrote:
> So we can reuse the code under arch/arm
> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>

Acked-by: Marc Zyngier <marc.zyngier@arm.com>

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 8/8] ARM: KVM: Support vgic-v3
  2016-09-08 16:06   ` Vladimir Murzin
@ 2016-09-09 16:45     ` Marc Zyngier
  -1 siblings, 0 replies; 72+ messages in thread
From: Marc Zyngier @ 2016-09-09 16:45 UTC (permalink / raw)
  To: Vladimir Murzin, kvmarm; +Cc: andre.przywara, linux-arm-kernel

On 08/09/16 17:06, Vladimir Murzin wrote:
> This patch allows to build and use vgic-v3 in 32-bit mode.
> 
> Unfortunately, it can not be split in several steps without extra
> stubs to keep patches independent and bisectable.
> For instance, virt/kvm/arm/vgic/vgic-v3.c uses function from
> vgic-v3-sr.c, handling access to GICv3 cpu interface from the guest
> requires vgic_v3.vgic_sre to be already defined.
> 
> It is how support has been done:
> 
> * vgic-v3 save/restore routines are written in such way that they map
>   arm64 system register naming nicely, but it does not fit to arm
>   world. To keep virt/kvm/arm/hyp/vgic-v3-sr.c untouched we create a
>   mapping with a function for each register mapping the 32-bit to the
>   64-bit accessors
> 
> * report configured SRE on access to GICv3 cpu interface from the guest
> 
> * required vgic-v3 macros are provided via uapi.h
> 
> * static keys are used to select GIC backend
> 
> * to make vgic-v3 build KVM_ARM_VGIC_V3 guard is removed along with
>   the static inlines
> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> ---
>  arch/arm/include/asm/arch_gicv3.h   |   64 +++++++++++++++++++++++++++++++++++
>  arch/arm/include/asm/kvm_asm.h      |    3 ++
>  arch/arm/include/asm/kvm_hyp.h      |    3 ++
>  arch/arm/include/uapi/asm/kvm.h     |    7 ++++
>  arch/arm/kvm/Makefile               |    2 ++
>  arch/arm/kvm/coproc.c               |   36 ++++++++++++++++++++
>  arch/arm/kvm/hyp/Makefile           |    1 +
>  arch/arm/kvm/hyp/switch.c           |   12 +++++--
>  arch/arm64/kvm/Kconfig              |    4 ---
>  include/kvm/arm_vgic.h              |    8 -----
>  virt/kvm/arm/vgic/vgic-kvm-device.c |    8 -----
>  virt/kvm/arm/vgic/vgic-mmio.c       |    2 --
>  virt/kvm/arm/vgic/vgic-mmio.h       |    2 --
>  virt/kvm/arm/vgic/vgic.h            |   54 -----------------------------
>  14 files changed, 126 insertions(+), 80 deletions(-)
> 
> diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
> index af25c32..f93f6bd 100644
> --- a/arch/arm/include/asm/arch_gicv3.h
> +++ b/arch/arm/include/asm/arch_gicv3.h
> @@ -96,6 +96,70 @@
>  #define ICH_AP1R2			__AP1Rx(2)
>  #define ICH_AP1R3			__AP1Rx(3)
>  
> +/* A32-to-A64 mappings used by VGIC save/restore */
> +
> +#define CPUIF_MAP(a32, a64)			\
> +static inline void write_ ## a64(u32 val)	\
> +{						\
> +	write_sysreg(val, a32);			\
> +}						\
> +static inline u32 read_ ## a64(void)		\
> +{						\
> +	return read_sysreg(a32); 		\
> +}						\
> +
> +#define CPUIF_MAP_LO_HI(a32lo, a32hi, a64)	\
> +static inline void write_ ## a64(u64 val)	\
> +{						\
> +	write_sysreg((u32)val, a32lo);		\
> +	write_sysreg((u32)(val >> 32), a32hi);	\

Please use {lower,upper}_32_bits, which make the casting/shifting go away.

> +}						\
> +static inline u64 read_ ## a64(void)		\
> +{						\
> +	u64 val = read_sysreg(a32lo);		\
> +						\
> +	val |=	(u64)read_sysreg(a32hi) << 32;	\
> +						\
> +	return val; 				\
> +}
> +
> +CPUIF_MAP(ICH_HCR, ICH_HCR_EL2)
> +CPUIF_MAP(ICH_VTR, ICH_VTR_EL2)
> +CPUIF_MAP(ICH_MISR, ICH_MISR_EL2)
> +CPUIF_MAP(ICH_EISR, ICH_EISR_EL2)
> +CPUIF_MAP(ICH_ELSR, ICH_ELSR_EL2)
> +CPUIF_MAP(ICH_VMCR, ICH_VMCR_EL2)
> +CPUIF_MAP(ICH_AP0R3, ICH_AP0R3_EL2)
> +CPUIF_MAP(ICH_AP0R2, ICH_AP0R2_EL2)
> +CPUIF_MAP(ICH_AP0R1, ICH_AP0R1_EL2)
> +CPUIF_MAP(ICH_AP0R0, ICH_AP0R0_EL2)
> +CPUIF_MAP(ICH_AP1R3, ICH_AP1R3_EL2)
> +CPUIF_MAP(ICH_AP1R2, ICH_AP1R2_EL2)
> +CPUIF_MAP(ICH_AP1R1, ICH_AP1R1_EL2)
> +CPUIF_MAP(ICH_AP1R0, ICH_AP1R0_EL2)
> +CPUIF_MAP(ICC_HSRE, ICC_SRE_EL2)
> +CPUIF_MAP(ICC_SRE, ICC_SRE_EL1)
> +
> +CPUIF_MAP_LO_HI(ICH_LR15, ICH_LRC15, ICH_LR15_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR14, ICH_LRC14, ICH_LR14_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR13, ICH_LRC13, ICH_LR13_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR12, ICH_LRC12, ICH_LR12_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR11, ICH_LRC11, ICH_LR11_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR10, ICH_LRC10, ICH_LR10_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR9, ICH_LRC9, ICH_LR9_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR8, ICH_LRC8, ICH_LR8_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR7, ICH_LRC7, ICH_LR7_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR6, ICH_LRC6, ICH_LR6_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR5, ICH_LRC5, ICH_LR5_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR4, ICH_LRC4, ICH_LR4_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR3, ICH_LRC3, ICH_LR3_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR2, ICH_LRC2, ICH_LR2_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR1, ICH_LRC1, ICH_LR1_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR0, ICH_LRC0, ICH_LR0_EL2)
> +
> +#define read_gicreg(r)                 read_##r()
> +#define write_gicreg(v, r)             write_##r(v)
> +

Can you make this change a separate patch? It will make it easier to
merge if I can ack it as a standalone change. It will also give the last
patch a fantastic diffstat... ;-)

>  /* Low-level accessors */
>  
>  static inline void gic_write_eoir(u32 irq)
> diff --git a/arch/arm/include/asm/kvm_asm.h b/arch/arm/include/asm/kvm_asm.h
> index 58faff5..dfccf94 100644
> --- a/arch/arm/include/asm/kvm_asm.h
> +++ b/arch/arm/include/asm/kvm_asm.h
> @@ -68,6 +68,9 @@ extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
>  extern void __init_stage2_translation(void);
>  
>  extern void __kvm_hyp_reset(unsigned long);
> +
> +extern u64 __vgic_v3_get_ich_vtr_el2(void);
> +extern void __vgic_v3_init_lrs(void);
>  #endif
>  
>  #endif /* __ARM_KVM_ASM_H__ */
> diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
> index 0b475d2..ed96cf9 100644
> --- a/arch/arm/include/asm/kvm_hyp.h
> +++ b/arch/arm/include/asm/kvm_hyp.h
> @@ -104,6 +104,9 @@ void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
>  void __sysreg_save_state(struct kvm_cpu_context *ctxt);
>  void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
>  
> +void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
> +void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
> +
>  void asmlinkage __vfp_save_state(struct vfp_hard_struct *vfp);
>  void asmlinkage __vfp_restore_state(struct vfp_hard_struct *vfp);
>  static inline bool __vfp_enabled(void)
> diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h
> index a2b3eb3..b38c10c 100644
> --- a/arch/arm/include/uapi/asm/kvm.h
> +++ b/arch/arm/include/uapi/asm/kvm.h
> @@ -84,6 +84,13 @@ struct kvm_regs {
>  #define KVM_VGIC_V2_DIST_SIZE		0x1000
>  #define KVM_VGIC_V2_CPU_SIZE		0x2000
>  
> +/* Supported VGICv3 address types  */
> +#define KVM_VGIC_V3_ADDR_TYPE_DIST	2
> +#define KVM_VGIC_V3_ADDR_TYPE_REDIST	3
> +
> +#define KVM_VGIC_V3_DIST_SIZE		SZ_64K
> +#define KVM_VGIC_V3_REDIST_SIZE		(2 * SZ_64K)
> +
>  #define KVM_ARM_VCPU_POWER_OFF		0 /* CPU is started in OFF state */
>  #define KVM_ARM_VCPU_PSCI_0_2		1 /* CPU uses PSCI v0.2 */
>  
> diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
> index 10d77a6..043d817f 100644
> --- a/arch/arm/kvm/Makefile
> +++ b/arch/arm/kvm/Makefile
> @@ -26,8 +26,10 @@ obj-y += $(KVM)/arm/vgic/vgic.o
>  obj-y += $(KVM)/arm/vgic/vgic-init.o
>  obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
>  obj-y += $(KVM)/arm/vgic/vgic-v2.o
> +obj-y += $(KVM)/arm/vgic/vgic-v3.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
> +obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
>  obj-y += $(KVM)/arm/vgic/vgic-kvm-device.o
>  obj-y += $(KVM)/irqchip.o
>  obj-y += $(KVM)/arm/arch_timer.o
> diff --git a/arch/arm/kvm/coproc.c b/arch/arm/kvm/coproc.c
> index 1bb2b79..10c0244 100644
> --- a/arch/arm/kvm/coproc.c
> +++ b/arch/arm/kvm/coproc.c
> @@ -228,6 +228,36 @@ bool access_vm_reg(struct kvm_vcpu *vcpu,
>  	return true;
>  }
>  
> +static bool access_gic_sgi(struct kvm_vcpu *vcpu,
> +			   const struct coproc_params *p,
> +			   const struct coproc_reg *r)
> +{
> +	u64 reg;
> +
> +	if (!p->is_write)
> +		return read_from_write_only(vcpu, p);
> +
> +	reg = *vcpu_reg(vcpu, p->Rt2);
> +	reg <<= 32;

nit: can you write this as

	reg = (u64)*vcpu_reg(vcpu, p->Rt2) << 32;

which I find easier to read...

> +	reg |= *vcpu_reg(vcpu, p->Rt1) ;
> +
> +	vgic_v3_dispatch_sgi(vcpu, reg);
> +
> +	return true;
> +}
> +
> +static bool access_gic_sre(struct kvm_vcpu *vcpu,
> +			   const struct coproc_params *p,
> +			   const struct coproc_reg *r)
> +{
> +	if (p->is_write)
> +		return ignore_write(vcpu, p);
> +
> +	*vcpu_reg(vcpu, p->Rt1) = vcpu->arch.vgic_cpu.vgic_v3.vgic_sre;
> +
> +	return true;
> +}
> +
>  /*
>   * We could trap ID_DFR0 and tell the guest we don't support performance
>   * monitoring.  Unfortunately the patch to make the kernel check ID_DFR0 was
> @@ -361,10 +391,16 @@ static const struct coproc_reg cp15_regs[] = {
>  	{ CRn(10), CRm( 3), Op1( 0), Op2( 1), is32,
>  			access_vm_reg, reset_unknown, c10_AMAIR1},
>  
> +	/* ICC_SGI1R */
> +	{ CRm64(12), Op1( 0), is64, access_gic_sgi},
> +
>  	/* VBAR: swapped by interrupt.S. */
>  	{ CRn(12), CRm( 0), Op1( 0), Op2( 0), is32,
>  			NULL, reset_val, c12_VBAR, 0x00000000 },
>  
> +	/* ICC_SRE */
> +	{ CRn(12), CRm(12), Op1( 0), Op2(5), is32, access_gic_sre },
> +
>  	/* CONTEXTIDR/TPIDRURW/TPIDRURO/TPIDRPRW: swapped by interrupt.S. */
>  	{ CRn(13), CRm( 0), Op1( 0), Op2( 1), is32,
>  			access_vm_reg, reset_val, c13_CID, 0x00000000 },
> diff --git a/arch/arm/kvm/hyp/Makefile b/arch/arm/kvm/hyp/Makefile
> index 8dfa5f7..3023bb5 100644
> --- a/arch/arm/kvm/hyp/Makefile
> +++ b/arch/arm/kvm/hyp/Makefile
> @@ -5,6 +5,7 @@
>  KVM=../../../../virt/kvm
>  
>  obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
> +obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
>  obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
>  
>  obj-$(CONFIG_KVM_ARM_HOST) += tlb.o
> diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
> index b13caa9..8409dd5 100644
> --- a/arch/arm/kvm/hyp/switch.c
> +++ b/arch/arm/kvm/hyp/switch.c
> @@ -14,6 +14,7 @@
>   * You should have received a copy of the GNU General Public License
>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>   */
> +#include <linux/jump_label.h>
>  
>  #include <asm/kvm_asm.h>
>  #include <asm/kvm_hyp.h>
> @@ -74,14 +75,21 @@ static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
>  	write_sysreg(read_sysreg(MIDR), VPIDR);
>  }
>  
> +
>  static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
>  {
> -	__vgic_v2_save_state(vcpu);
> +	if (static_branch_unlikely(&kvm_gicv3_cpuif))
> +		__vgic_v3_save_state(vcpu);
> +	else
> +		__vgic_v2_save_state(vcpu);
>  }
>  
>  static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
>  {
> -	__vgic_v2_restore_state(vcpu);
> +	if (static_branch_unlikely(&kvm_gicv3_cpuif))
> +		__vgic_v3_restore_state(vcpu);
> +	else
> +		__vgic_v2_restore_state(vcpu);
>  }
>  
>  static bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu)
> diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
> index 7ba9164..6eaf12c 100644
> --- a/arch/arm64/kvm/Kconfig
> +++ b/arch/arm64/kvm/Kconfig
> @@ -19,9 +19,6 @@ if VIRTUALIZATION
>  config KVM_ARM_VGIC_V3_ITS
>  	bool
>  
> -config KVM_ARM_VGIC_V3
> -	bool
> -
>  config KVM
>  	bool "Kernel-based Virtual Machine (KVM) support"
>  	depends on OF
> @@ -37,7 +34,6 @@ config KVM
>  	select KVM_VFIO
>  	select HAVE_KVM_EVENTFD
>  	select HAVE_KVM_IRQFD
> -	select KVM_ARM_VGIC_V3
>  	select KVM_ARM_VGIC_V3_ITS
>  	select KVM_ARM_PMU if HW_PERF_EVENTS
>  	select HAVE_KVM_MSI
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index 19b698e..7462138 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -217,7 +217,6 @@ struct vgic_v2_cpu_if {
>  };
>  
>  struct vgic_v3_cpu_if {
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  	u32		vgic_hcr;
>  	u32		vgic_vmcr;
>  	u32		vgic_sre;	/* Restored only, change ignored */
> @@ -227,7 +226,6 @@ struct vgic_v3_cpu_if {
>  	u32		vgic_ap0r[4];
>  	u32		vgic_ap1r[4];
>  	u64		vgic_lr[VGIC_V3_MAX_LRS];
> -#endif
>  };
>  
>  struct vgic_cpu {
> @@ -294,13 +292,7 @@ bool kvm_vcpu_has_pending_irqs(struct kvm_vcpu *vcpu);
>  void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu);
>  void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu);
>  
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg);
> -#else
> -static inline void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)
> -{
> -}
> -#endif
>  
>  /**
>   * kvm_vgic_get_max_vcpus - Get the maximum number of VCPUs allowed by HW
> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
> index f3811b3..4dc026a 100644
> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
> @@ -71,7 +71,6 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write)
>  		addr_ptr = &vgic->vgic_cpu_base;
>  		alignment = SZ_4K;
>  		break;
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  	case KVM_VGIC_V3_ADDR_TYPE_DIST:
>  		type_needed = KVM_DEV_TYPE_ARM_VGIC_V3;
>  		addr_ptr = &vgic->vgic_dist_base;
> @@ -82,7 +81,6 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write)
>  		addr_ptr = &vgic->vgic_redist_base;
>  		alignment = SZ_64K;
>  		break;
> -#endif
>  	default:
>  		r = -ENODEV;
>  		goto out;
> @@ -219,7 +217,6 @@ int kvm_register_vgic_device(unsigned long type)
>  		ret = kvm_register_device_ops(&kvm_arm_vgic_v2_ops,
>  					      KVM_DEV_TYPE_ARM_VGIC_V2);
>  		break;
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  	case KVM_DEV_TYPE_ARM_VGIC_V3:
>  		ret = kvm_register_device_ops(&kvm_arm_vgic_v3_ops,
>  					      KVM_DEV_TYPE_ARM_VGIC_V3);
> @@ -230,7 +227,6 @@ int kvm_register_vgic_device(unsigned long type)
>  		ret = kvm_vgic_register_its_device();
>  #endif
>  		break;
> -#endif
>  	}
>  
>  	return ret;
> @@ -392,8 +388,6 @@ struct kvm_device_ops kvm_arm_vgic_v2_ops = {
>  
>  /* V3 ops */
>  
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
> -
>  static int vgic_v3_set_attr(struct kvm_device *dev,
>  			    struct kvm_device_attr *attr)
>  {
> @@ -436,5 +430,3 @@ struct kvm_device_ops kvm_arm_vgic_v3_ops = {
>  	.get_attr = vgic_v3_get_attr,
>  	.has_attr = vgic_v3_has_attr,
>  };
> -
> -#endif /* CONFIG_KVM_ARM_VGIC_V3 */
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
> index 3bad3c5..e18b30d 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio.c
> @@ -550,11 +550,9 @@ int vgic_register_dist_iodev(struct kvm *kvm, gpa_t dist_base_address,
>  	case VGIC_V2:
>  		len = vgic_v2_init_dist_iodev(io_device);
>  		break;
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  	case VGIC_V3:
>  		len = vgic_v3_init_dist_iodev(io_device);
>  		break;
> -#endif
>  	default:
>  		BUG_ON(1);
>  	}
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
> index 80f92ce..4c34d39 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.h
> +++ b/virt/kvm/arm/vgic/vgic-mmio.h
> @@ -162,12 +162,10 @@ unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
>  
>  unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
>  
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  u64 vgic_sanitise_outer_cacheability(u64 reg);
>  u64 vgic_sanitise_inner_cacheability(u64 reg);
>  u64 vgic_sanitise_shareability(u64 reg);
>  u64 vgic_sanitise_field(u64 reg, u64 field_mask, int field_shift,
>  			u64 (*sanitise_fn)(u64));
> -#endif
>  
>  #endif
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index 100045f..9d9e014 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -72,7 +72,6 @@ static inline void vgic_get_irq_kref(struct vgic_irq *irq)
>  	kref_get(&irq->refcount);
>  }
>  
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu);
>  void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu);
>  void vgic_v3_populate_lr(struct kvm_vcpu *vcpu, struct vgic_irq *irq, int lr);
> @@ -91,60 +90,7 @@ bool vgic_has_its(struct kvm *kvm);
>  int kvm_vgic_register_its_device(void);
>  void vgic_enable_lpis(struct kvm_vcpu *vcpu);
>  int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi);
> -#endif
> -
>  #else
> -static inline void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu)
> -{
> -}
> -
> -static inline void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu)
> -{
> -}
> -
> -static inline void vgic_v3_populate_lr(struct kvm_vcpu *vcpu,
> -				       struct vgic_irq *irq, int lr)
> -{
> -}
> -
> -static inline void vgic_v3_clear_lr(struct kvm_vcpu *vcpu, int lr)
> -{
> -}
> -
> -static inline void vgic_v3_set_underflow(struct kvm_vcpu *vcpu)
> -{
> -}
> -
> -static inline
> -void vgic_v3_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
> -{
> -}
> -
> -static inline
> -void vgic_v3_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
> -{
> -}
> -
> -static inline void vgic_v3_enable(struct kvm_vcpu *vcpu)
> -{
> -}
> -
> -static inline int vgic_v3_probe(const struct gic_kvm_info *info)
> -{
> -	return -ENODEV;
> -}
> -
> -static inline int vgic_v3_map_resources(struct kvm *kvm)
> -{
> -	return -ENODEV;
> -}
> -
> -static inline int vgic_register_redist_iodevs(struct kvm *kvm,
> -					      gpa_t dist_base_address)
> -{
> -	return -ENODEV;
> -}
> -
>  static inline int vgic_register_its_iodevs(struct kvm *kvm)
>  {
>  	return -ENODEV;
> 

It otherwise looks good to me.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 8/8] ARM: KVM: Support vgic-v3
@ 2016-09-09 16:45     ` Marc Zyngier
  0 siblings, 0 replies; 72+ messages in thread
From: Marc Zyngier @ 2016-09-09 16:45 UTC (permalink / raw)
  To: linux-arm-kernel

On 08/09/16 17:06, Vladimir Murzin wrote:
> This patch allows to build and use vgic-v3 in 32-bit mode.
> 
> Unfortunately, it can not be split in several steps without extra
> stubs to keep patches independent and bisectable.
> For instance, virt/kvm/arm/vgic/vgic-v3.c uses function from
> vgic-v3-sr.c, handling access to GICv3 cpu interface from the guest
> requires vgic_v3.vgic_sre to be already defined.
> 
> It is how support has been done:
> 
> * vgic-v3 save/restore routines are written in such way that they map
>   arm64 system register naming nicely, but it does not fit to arm
>   world. To keep virt/kvm/arm/hyp/vgic-v3-sr.c untouched we create a
>   mapping with a function for each register mapping the 32-bit to the
>   64-bit accessors
> 
> * report configured SRE on access to GICv3 cpu interface from the guest
> 
> * required vgic-v3 macros are provided via uapi.h
> 
> * static keys are used to select GIC backend
> 
> * to make vgic-v3 build KVM_ARM_VGIC_V3 guard is removed along with
>   the static inlines
> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> ---
>  arch/arm/include/asm/arch_gicv3.h   |   64 +++++++++++++++++++++++++++++++++++
>  arch/arm/include/asm/kvm_asm.h      |    3 ++
>  arch/arm/include/asm/kvm_hyp.h      |    3 ++
>  arch/arm/include/uapi/asm/kvm.h     |    7 ++++
>  arch/arm/kvm/Makefile               |    2 ++
>  arch/arm/kvm/coproc.c               |   36 ++++++++++++++++++++
>  arch/arm/kvm/hyp/Makefile           |    1 +
>  arch/arm/kvm/hyp/switch.c           |   12 +++++--
>  arch/arm64/kvm/Kconfig              |    4 ---
>  include/kvm/arm_vgic.h              |    8 -----
>  virt/kvm/arm/vgic/vgic-kvm-device.c |    8 -----
>  virt/kvm/arm/vgic/vgic-mmio.c       |    2 --
>  virt/kvm/arm/vgic/vgic-mmio.h       |    2 --
>  virt/kvm/arm/vgic/vgic.h            |   54 -----------------------------
>  14 files changed, 126 insertions(+), 80 deletions(-)
> 
> diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
> index af25c32..f93f6bd 100644
> --- a/arch/arm/include/asm/arch_gicv3.h
> +++ b/arch/arm/include/asm/arch_gicv3.h
> @@ -96,6 +96,70 @@
>  #define ICH_AP1R2			__AP1Rx(2)
>  #define ICH_AP1R3			__AP1Rx(3)
>  
> +/* A32-to-A64 mappings used by VGIC save/restore */
> +
> +#define CPUIF_MAP(a32, a64)			\
> +static inline void write_ ## a64(u32 val)	\
> +{						\
> +	write_sysreg(val, a32);			\
> +}						\
> +static inline u32 read_ ## a64(void)		\
> +{						\
> +	return read_sysreg(a32); 		\
> +}						\
> +
> +#define CPUIF_MAP_LO_HI(a32lo, a32hi, a64)	\
> +static inline void write_ ## a64(u64 val)	\
> +{						\
> +	write_sysreg((u32)val, a32lo);		\
> +	write_sysreg((u32)(val >> 32), a32hi);	\

Please use {lower,upper}_32_bits, which make the casting/shifting go away.

> +}						\
> +static inline u64 read_ ## a64(void)		\
> +{						\
> +	u64 val = read_sysreg(a32lo);		\
> +						\
> +	val |=	(u64)read_sysreg(a32hi) << 32;	\
> +						\
> +	return val; 				\
> +}
> +
> +CPUIF_MAP(ICH_HCR, ICH_HCR_EL2)
> +CPUIF_MAP(ICH_VTR, ICH_VTR_EL2)
> +CPUIF_MAP(ICH_MISR, ICH_MISR_EL2)
> +CPUIF_MAP(ICH_EISR, ICH_EISR_EL2)
> +CPUIF_MAP(ICH_ELSR, ICH_ELSR_EL2)
> +CPUIF_MAP(ICH_VMCR, ICH_VMCR_EL2)
> +CPUIF_MAP(ICH_AP0R3, ICH_AP0R3_EL2)
> +CPUIF_MAP(ICH_AP0R2, ICH_AP0R2_EL2)
> +CPUIF_MAP(ICH_AP0R1, ICH_AP0R1_EL2)
> +CPUIF_MAP(ICH_AP0R0, ICH_AP0R0_EL2)
> +CPUIF_MAP(ICH_AP1R3, ICH_AP1R3_EL2)
> +CPUIF_MAP(ICH_AP1R2, ICH_AP1R2_EL2)
> +CPUIF_MAP(ICH_AP1R1, ICH_AP1R1_EL2)
> +CPUIF_MAP(ICH_AP1R0, ICH_AP1R0_EL2)
> +CPUIF_MAP(ICC_HSRE, ICC_SRE_EL2)
> +CPUIF_MAP(ICC_SRE, ICC_SRE_EL1)
> +
> +CPUIF_MAP_LO_HI(ICH_LR15, ICH_LRC15, ICH_LR15_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR14, ICH_LRC14, ICH_LR14_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR13, ICH_LRC13, ICH_LR13_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR12, ICH_LRC12, ICH_LR12_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR11, ICH_LRC11, ICH_LR11_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR10, ICH_LRC10, ICH_LR10_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR9, ICH_LRC9, ICH_LR9_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR8, ICH_LRC8, ICH_LR8_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR7, ICH_LRC7, ICH_LR7_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR6, ICH_LRC6, ICH_LR6_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR5, ICH_LRC5, ICH_LR5_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR4, ICH_LRC4, ICH_LR4_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR3, ICH_LRC3, ICH_LR3_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR2, ICH_LRC2, ICH_LR2_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR1, ICH_LRC1, ICH_LR1_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR0, ICH_LRC0, ICH_LR0_EL2)
> +
> +#define read_gicreg(r)                 read_##r()
> +#define write_gicreg(v, r)             write_##r(v)
> +

Can you make this change a separate patch? It will make it easier to
merge if I can ack it as a standalone change. It will also give the last
patch a fantastic diffstat... ;-)

>  /* Low-level accessors */
>  
>  static inline void gic_write_eoir(u32 irq)
> diff --git a/arch/arm/include/asm/kvm_asm.h b/arch/arm/include/asm/kvm_asm.h
> index 58faff5..dfccf94 100644
> --- a/arch/arm/include/asm/kvm_asm.h
> +++ b/arch/arm/include/asm/kvm_asm.h
> @@ -68,6 +68,9 @@ extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
>  extern void __init_stage2_translation(void);
>  
>  extern void __kvm_hyp_reset(unsigned long);
> +
> +extern u64 __vgic_v3_get_ich_vtr_el2(void);
> +extern void __vgic_v3_init_lrs(void);
>  #endif
>  
>  #endif /* __ARM_KVM_ASM_H__ */
> diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
> index 0b475d2..ed96cf9 100644
> --- a/arch/arm/include/asm/kvm_hyp.h
> +++ b/arch/arm/include/asm/kvm_hyp.h
> @@ -104,6 +104,9 @@ void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
>  void __sysreg_save_state(struct kvm_cpu_context *ctxt);
>  void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
>  
> +void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
> +void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
> +
>  void asmlinkage __vfp_save_state(struct vfp_hard_struct *vfp);
>  void asmlinkage __vfp_restore_state(struct vfp_hard_struct *vfp);
>  static inline bool __vfp_enabled(void)
> diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h
> index a2b3eb3..b38c10c 100644
> --- a/arch/arm/include/uapi/asm/kvm.h
> +++ b/arch/arm/include/uapi/asm/kvm.h
> @@ -84,6 +84,13 @@ struct kvm_regs {
>  #define KVM_VGIC_V2_DIST_SIZE		0x1000
>  #define KVM_VGIC_V2_CPU_SIZE		0x2000
>  
> +/* Supported VGICv3 address types  */
> +#define KVM_VGIC_V3_ADDR_TYPE_DIST	2
> +#define KVM_VGIC_V3_ADDR_TYPE_REDIST	3
> +
> +#define KVM_VGIC_V3_DIST_SIZE		SZ_64K
> +#define KVM_VGIC_V3_REDIST_SIZE		(2 * SZ_64K)
> +
>  #define KVM_ARM_VCPU_POWER_OFF		0 /* CPU is started in OFF state */
>  #define KVM_ARM_VCPU_PSCI_0_2		1 /* CPU uses PSCI v0.2 */
>  
> diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
> index 10d77a6..043d817f 100644
> --- a/arch/arm/kvm/Makefile
> +++ b/arch/arm/kvm/Makefile
> @@ -26,8 +26,10 @@ obj-y += $(KVM)/arm/vgic/vgic.o
>  obj-y += $(KVM)/arm/vgic/vgic-init.o
>  obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
>  obj-y += $(KVM)/arm/vgic/vgic-v2.o
> +obj-y += $(KVM)/arm/vgic/vgic-v3.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
> +obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
>  obj-y += $(KVM)/arm/vgic/vgic-kvm-device.o
>  obj-y += $(KVM)/irqchip.o
>  obj-y += $(KVM)/arm/arch_timer.o
> diff --git a/arch/arm/kvm/coproc.c b/arch/arm/kvm/coproc.c
> index 1bb2b79..10c0244 100644
> --- a/arch/arm/kvm/coproc.c
> +++ b/arch/arm/kvm/coproc.c
> @@ -228,6 +228,36 @@ bool access_vm_reg(struct kvm_vcpu *vcpu,
>  	return true;
>  }
>  
> +static bool access_gic_sgi(struct kvm_vcpu *vcpu,
> +			   const struct coproc_params *p,
> +			   const struct coproc_reg *r)
> +{
> +	u64 reg;
> +
> +	if (!p->is_write)
> +		return read_from_write_only(vcpu, p);
> +
> +	reg = *vcpu_reg(vcpu, p->Rt2);
> +	reg <<= 32;

nit: can you write this as

	reg = (u64)*vcpu_reg(vcpu, p->Rt2) << 32;

which I find easier to read...

> +	reg |= *vcpu_reg(vcpu, p->Rt1) ;
> +
> +	vgic_v3_dispatch_sgi(vcpu, reg);
> +
> +	return true;
> +}
> +
> +static bool access_gic_sre(struct kvm_vcpu *vcpu,
> +			   const struct coproc_params *p,
> +			   const struct coproc_reg *r)
> +{
> +	if (p->is_write)
> +		return ignore_write(vcpu, p);
> +
> +	*vcpu_reg(vcpu, p->Rt1) = vcpu->arch.vgic_cpu.vgic_v3.vgic_sre;
> +
> +	return true;
> +}
> +
>  /*
>   * We could trap ID_DFR0 and tell the guest we don't support performance
>   * monitoring.  Unfortunately the patch to make the kernel check ID_DFR0 was
> @@ -361,10 +391,16 @@ static const struct coproc_reg cp15_regs[] = {
>  	{ CRn(10), CRm( 3), Op1( 0), Op2( 1), is32,
>  			access_vm_reg, reset_unknown, c10_AMAIR1},
>  
> +	/* ICC_SGI1R */
> +	{ CRm64(12), Op1( 0), is64, access_gic_sgi},
> +
>  	/* VBAR: swapped by interrupt.S. */
>  	{ CRn(12), CRm( 0), Op1( 0), Op2( 0), is32,
>  			NULL, reset_val, c12_VBAR, 0x00000000 },
>  
> +	/* ICC_SRE */
> +	{ CRn(12), CRm(12), Op1( 0), Op2(5), is32, access_gic_sre },
> +
>  	/* CONTEXTIDR/TPIDRURW/TPIDRURO/TPIDRPRW: swapped by interrupt.S. */
>  	{ CRn(13), CRm( 0), Op1( 0), Op2( 1), is32,
>  			access_vm_reg, reset_val, c13_CID, 0x00000000 },
> diff --git a/arch/arm/kvm/hyp/Makefile b/arch/arm/kvm/hyp/Makefile
> index 8dfa5f7..3023bb5 100644
> --- a/arch/arm/kvm/hyp/Makefile
> +++ b/arch/arm/kvm/hyp/Makefile
> @@ -5,6 +5,7 @@
>  KVM=../../../../virt/kvm
>  
>  obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
> +obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
>  obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
>  
>  obj-$(CONFIG_KVM_ARM_HOST) += tlb.o
> diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
> index b13caa9..8409dd5 100644
> --- a/arch/arm/kvm/hyp/switch.c
> +++ b/arch/arm/kvm/hyp/switch.c
> @@ -14,6 +14,7 @@
>   * You should have received a copy of the GNU General Public License
>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>   */
> +#include <linux/jump_label.h>
>  
>  #include <asm/kvm_asm.h>
>  #include <asm/kvm_hyp.h>
> @@ -74,14 +75,21 @@ static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
>  	write_sysreg(read_sysreg(MIDR), VPIDR);
>  }
>  
> +
>  static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
>  {
> -	__vgic_v2_save_state(vcpu);
> +	if (static_branch_unlikely(&kvm_gicv3_cpuif))
> +		__vgic_v3_save_state(vcpu);
> +	else
> +		__vgic_v2_save_state(vcpu);
>  }
>  
>  static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
>  {
> -	__vgic_v2_restore_state(vcpu);
> +	if (static_branch_unlikely(&kvm_gicv3_cpuif))
> +		__vgic_v3_restore_state(vcpu);
> +	else
> +		__vgic_v2_restore_state(vcpu);
>  }
>  
>  static bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu)
> diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
> index 7ba9164..6eaf12c 100644
> --- a/arch/arm64/kvm/Kconfig
> +++ b/arch/arm64/kvm/Kconfig
> @@ -19,9 +19,6 @@ if VIRTUALIZATION
>  config KVM_ARM_VGIC_V3_ITS
>  	bool
>  
> -config KVM_ARM_VGIC_V3
> -	bool
> -
>  config KVM
>  	bool "Kernel-based Virtual Machine (KVM) support"
>  	depends on OF
> @@ -37,7 +34,6 @@ config KVM
>  	select KVM_VFIO
>  	select HAVE_KVM_EVENTFD
>  	select HAVE_KVM_IRQFD
> -	select KVM_ARM_VGIC_V3
>  	select KVM_ARM_VGIC_V3_ITS
>  	select KVM_ARM_PMU if HW_PERF_EVENTS
>  	select HAVE_KVM_MSI
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index 19b698e..7462138 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -217,7 +217,6 @@ struct vgic_v2_cpu_if {
>  };
>  
>  struct vgic_v3_cpu_if {
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  	u32		vgic_hcr;
>  	u32		vgic_vmcr;
>  	u32		vgic_sre;	/* Restored only, change ignored */
> @@ -227,7 +226,6 @@ struct vgic_v3_cpu_if {
>  	u32		vgic_ap0r[4];
>  	u32		vgic_ap1r[4];
>  	u64		vgic_lr[VGIC_V3_MAX_LRS];
> -#endif
>  };
>  
>  struct vgic_cpu {
> @@ -294,13 +292,7 @@ bool kvm_vcpu_has_pending_irqs(struct kvm_vcpu *vcpu);
>  void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu);
>  void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu);
>  
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg);
> -#else
> -static inline void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)
> -{
> -}
> -#endif
>  
>  /**
>   * kvm_vgic_get_max_vcpus - Get the maximum number of VCPUs allowed by HW
> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
> index f3811b3..4dc026a 100644
> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
> @@ -71,7 +71,6 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write)
>  		addr_ptr = &vgic->vgic_cpu_base;
>  		alignment = SZ_4K;
>  		break;
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  	case KVM_VGIC_V3_ADDR_TYPE_DIST:
>  		type_needed = KVM_DEV_TYPE_ARM_VGIC_V3;
>  		addr_ptr = &vgic->vgic_dist_base;
> @@ -82,7 +81,6 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write)
>  		addr_ptr = &vgic->vgic_redist_base;
>  		alignment = SZ_64K;
>  		break;
> -#endif
>  	default:
>  		r = -ENODEV;
>  		goto out;
> @@ -219,7 +217,6 @@ int kvm_register_vgic_device(unsigned long type)
>  		ret = kvm_register_device_ops(&kvm_arm_vgic_v2_ops,
>  					      KVM_DEV_TYPE_ARM_VGIC_V2);
>  		break;
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  	case KVM_DEV_TYPE_ARM_VGIC_V3:
>  		ret = kvm_register_device_ops(&kvm_arm_vgic_v3_ops,
>  					      KVM_DEV_TYPE_ARM_VGIC_V3);
> @@ -230,7 +227,6 @@ int kvm_register_vgic_device(unsigned long type)
>  		ret = kvm_vgic_register_its_device();
>  #endif
>  		break;
> -#endif
>  	}
>  
>  	return ret;
> @@ -392,8 +388,6 @@ struct kvm_device_ops kvm_arm_vgic_v2_ops = {
>  
>  /* V3 ops */
>  
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
> -
>  static int vgic_v3_set_attr(struct kvm_device *dev,
>  			    struct kvm_device_attr *attr)
>  {
> @@ -436,5 +430,3 @@ struct kvm_device_ops kvm_arm_vgic_v3_ops = {
>  	.get_attr = vgic_v3_get_attr,
>  	.has_attr = vgic_v3_has_attr,
>  };
> -
> -#endif /* CONFIG_KVM_ARM_VGIC_V3 */
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
> index 3bad3c5..e18b30d 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio.c
> @@ -550,11 +550,9 @@ int vgic_register_dist_iodev(struct kvm *kvm, gpa_t dist_base_address,
>  	case VGIC_V2:
>  		len = vgic_v2_init_dist_iodev(io_device);
>  		break;
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  	case VGIC_V3:
>  		len = vgic_v3_init_dist_iodev(io_device);
>  		break;
> -#endif
>  	default:
>  		BUG_ON(1);
>  	}
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
> index 80f92ce..4c34d39 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.h
> +++ b/virt/kvm/arm/vgic/vgic-mmio.h
> @@ -162,12 +162,10 @@ unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
>  
>  unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
>  
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  u64 vgic_sanitise_outer_cacheability(u64 reg);
>  u64 vgic_sanitise_inner_cacheability(u64 reg);
>  u64 vgic_sanitise_shareability(u64 reg);
>  u64 vgic_sanitise_field(u64 reg, u64 field_mask, int field_shift,
>  			u64 (*sanitise_fn)(u64));
> -#endif
>  
>  #endif
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index 100045f..9d9e014 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -72,7 +72,6 @@ static inline void vgic_get_irq_kref(struct vgic_irq *irq)
>  	kref_get(&irq->refcount);
>  }
>  
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu);
>  void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu);
>  void vgic_v3_populate_lr(struct kvm_vcpu *vcpu, struct vgic_irq *irq, int lr);
> @@ -91,60 +90,7 @@ bool vgic_has_its(struct kvm *kvm);
>  int kvm_vgic_register_its_device(void);
>  void vgic_enable_lpis(struct kvm_vcpu *vcpu);
>  int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi);
> -#endif
> -
>  #else
> -static inline void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu)
> -{
> -}
> -
> -static inline void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu)
> -{
> -}
> -
> -static inline void vgic_v3_populate_lr(struct kvm_vcpu *vcpu,
> -				       struct vgic_irq *irq, int lr)
> -{
> -}
> -
> -static inline void vgic_v3_clear_lr(struct kvm_vcpu *vcpu, int lr)
> -{
> -}
> -
> -static inline void vgic_v3_set_underflow(struct kvm_vcpu *vcpu)
> -{
> -}
> -
> -static inline
> -void vgic_v3_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
> -{
> -}
> -
> -static inline
> -void vgic_v3_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
> -{
> -}
> -
> -static inline void vgic_v3_enable(struct kvm_vcpu *vcpu)
> -{
> -}
> -
> -static inline int vgic_v3_probe(const struct gic_kvm_info *info)
> -{
> -	return -ENODEV;
> -}
> -
> -static inline int vgic_v3_map_resources(struct kvm *kvm)
> -{
> -	return -ENODEV;
> -}
> -
> -static inline int vgic_register_redist_iodevs(struct kvm *kvm,
> -					      gpa_t dist_base_address)
> -{
> -	return -ENODEV;
> -}
> -
>  static inline int vgic_register_its_iodevs(struct kvm *kvm)
>  {
>  	return -ENODEV;
> 

It otherwise looks good to me.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 4/8] KVM: arm64: vgic-its: Introduce config option to guard ITS specific code
  2016-09-08 16:06   ` Vladimir Murzin
@ 2016-09-09 16:46     ` Marc Zyngier
  -1 siblings, 0 replies; 72+ messages in thread
From: Marc Zyngier @ 2016-09-09 16:46 UTC (permalink / raw)
  To: Vladimir Murzin, kvmarm; +Cc: andre.przywara, linux-arm-kernel

On 08/09/16 17:06, Vladimir Murzin wrote:
> By now ITS code guarded with KVM_ARM_VGIC_V3 config option which was
> introduced to hide everything specific to vgic-v3 from 32-bit world.
> We are going to support vgic-v3 in 32-bit world and KVM_ARM_VGIC_V3
> will gone, but we don't have support for ITS there yet and we need to
> continue keeping ITS away.
> Introduce the new config option to prevent ITS code being build in
> 32-bit mode when support for vgic-v3 is done.
> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>

Acked-by: Marc Zyngier <marc.zyngier@arm.com>

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 4/8] KVM: arm64: vgic-its: Introduce config option to guard ITS specific code
@ 2016-09-09 16:46     ` Marc Zyngier
  0 siblings, 0 replies; 72+ messages in thread
From: Marc Zyngier @ 2016-09-09 16:46 UTC (permalink / raw)
  To: linux-arm-kernel

On 08/09/16 17:06, Vladimir Murzin wrote:
> By now ITS code guarded with KVM_ARM_VGIC_V3 config option which was
> introduced to hide everything specific to vgic-v3 from 32-bit world.
> We are going to support vgic-v3 in 32-bit world and KVM_ARM_VGIC_V3
> will gone, but we don't have support for ITS there yet and we need to
> continue keeping ITS away.
> Introduce the new config option to prevent ITS code being build in
> 32-bit mode when support for vgic-v3 is done.
> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>

Acked-by: Marc Zyngier <marc.zyngier@arm.com>

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 5/8] KVM: arm: vgic: Support 64-bit data manipulation on 32-bit host systems
  2016-09-08 16:06   ` Vladimir Murzin
@ 2016-09-09 16:55     ` Marc Zyngier
  -1 siblings, 0 replies; 72+ messages in thread
From: Marc Zyngier @ 2016-09-09 16:55 UTC (permalink / raw)
  To: Vladimir Murzin, kvmarm; +Cc: andre.przywara, linux-arm-kernel

On 08/09/16 17:06, Vladimir Murzin wrote:
> We have couple of 64-bit registers defined in GICv3 architecture, so
> unsigned long accesses to these registers will only access a single
> 32-bit part of that regitser. On the other hand these registers can't
> be accessed as 64-bit with a single instruction like ldrd/strd or
> ldmia/stmia if we run a 32-bit host because KVM does not support
> access to MMIO space done by these instructions.
> 
> It means that a 32-bit guest accesses these registers in 32-bit
> chunks, so the only thing we need to do is to ensure that
> extract_bytes() always takes 64-bit data.
> 
> Since we are here fix couple of other width related issues catched by
> gcc
> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-mmio-v3.c |    6 +++---
>  virt/kvm/arm/vgic/vgic-mmio.h    |    2 +-
>  2 files changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> index acbe691..0d3c76a 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> @@ -23,7 +23,7 @@
>  #include "vgic-mmio.h"
>  
>  /* extract @num bytes at @offset bytes offset in data */
> -unsigned long extract_bytes(unsigned long data, unsigned int offset,
> +unsigned long extract_bytes(u64 data, unsigned int offset,
>  			    unsigned int num)
>  {
>  	return (data >> (offset * 8)) & GENMASK_ULL(num * 8 - 1, 0);
> @@ -181,7 +181,7 @@ static unsigned long vgic_mmio_read_v3r_typer(struct kvm_vcpu *vcpu,
>  	int target_vcpu_id = vcpu->vcpu_id;
>  	u64 value;
>  
> -	value = (mpidr & GENMASK(23, 0)) << 32;
> +	value = (u64)(mpidr & GENMASK(23, 0)) << 32;
>  	value |= ((target_vcpu_id & 0xffff) << 8);
>  	if (target_vcpu_id == atomic_read(&vcpu->kvm->online_vcpus) - 1)
>  		value |= GICR_TYPER_LAST;
> @@ -611,7 +611,7 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)
>  	bool broadcast;
>  
>  	sgi = (reg & ICC_SGI1R_SGI_ID_MASK) >> ICC_SGI1R_SGI_ID_SHIFT;
> -	broadcast = reg & BIT(ICC_SGI1R_IRQ_ROUTING_MODE_BIT);
> +	broadcast = reg & BIT_ULL(ICC_SGI1R_IRQ_ROUTING_MODE_BIT);
>  	target_cpus = (reg & ICC_SGI1R_TARGET_LIST_MASK) >> ICC_SGI1R_TARGET_LIST_SHIFT;
>  	mpidr = SGI_AFFINITY_LEVEL(reg, 3);
>  	mpidr |= SGI_AFFINITY_LEVEL(reg, 2);
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
> index 0b3ecf9..80f92ce 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.h
> +++ b/virt/kvm/arm/vgic/vgic-mmio.h
> @@ -96,7 +96,7 @@ unsigned long vgic_data_mmio_bus_to_host(const void *val, unsigned int len);
>  void vgic_data_host_to_mmio_bus(void *buf, unsigned int len,
>  				unsigned long data);
>  
> -unsigned long extract_bytes(unsigned long data, unsigned int offset,
> +unsigned long extract_bytes(u64 data, unsigned int offset,
>  			    unsigned int num);
>  
>  u64 update_64bit_reg(u64 reg, unsigned int offset, unsigned int len,
> 

My personal preference would be to split this in two patches. One that
changes extract_bytes to work on 64bit quantities, and another one that
addresses the 64bit issues. Not a big deal though.

Acked-by: Marc Zyngier <marc.zyngier@arm.com>

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 5/8] KVM: arm: vgic: Support 64-bit data manipulation on 32-bit host systems
@ 2016-09-09 16:55     ` Marc Zyngier
  0 siblings, 0 replies; 72+ messages in thread
From: Marc Zyngier @ 2016-09-09 16:55 UTC (permalink / raw)
  To: linux-arm-kernel

On 08/09/16 17:06, Vladimir Murzin wrote:
> We have couple of 64-bit registers defined in GICv3 architecture, so
> unsigned long accesses to these registers will only access a single
> 32-bit part of that regitser. On the other hand these registers can't
> be accessed as 64-bit with a single instruction like ldrd/strd or
> ldmia/stmia if we run a 32-bit host because KVM does not support
> access to MMIO space done by these instructions.
> 
> It means that a 32-bit guest accesses these registers in 32-bit
> chunks, so the only thing we need to do is to ensure that
> extract_bytes() always takes 64-bit data.
> 
> Since we are here fix couple of other width related issues catched by
> gcc
> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-mmio-v3.c |    6 +++---
>  virt/kvm/arm/vgic/vgic-mmio.h    |    2 +-
>  2 files changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> index acbe691..0d3c76a 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> @@ -23,7 +23,7 @@
>  #include "vgic-mmio.h"
>  
>  /* extract @num bytes at @offset bytes offset in data */
> -unsigned long extract_bytes(unsigned long data, unsigned int offset,
> +unsigned long extract_bytes(u64 data, unsigned int offset,
>  			    unsigned int num)
>  {
>  	return (data >> (offset * 8)) & GENMASK_ULL(num * 8 - 1, 0);
> @@ -181,7 +181,7 @@ static unsigned long vgic_mmio_read_v3r_typer(struct kvm_vcpu *vcpu,
>  	int target_vcpu_id = vcpu->vcpu_id;
>  	u64 value;
>  
> -	value = (mpidr & GENMASK(23, 0)) << 32;
> +	value = (u64)(mpidr & GENMASK(23, 0)) << 32;
>  	value |= ((target_vcpu_id & 0xffff) << 8);
>  	if (target_vcpu_id == atomic_read(&vcpu->kvm->online_vcpus) - 1)
>  		value |= GICR_TYPER_LAST;
> @@ -611,7 +611,7 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)
>  	bool broadcast;
>  
>  	sgi = (reg & ICC_SGI1R_SGI_ID_MASK) >> ICC_SGI1R_SGI_ID_SHIFT;
> -	broadcast = reg & BIT(ICC_SGI1R_IRQ_ROUTING_MODE_BIT);
> +	broadcast = reg & BIT_ULL(ICC_SGI1R_IRQ_ROUTING_MODE_BIT);
>  	target_cpus = (reg & ICC_SGI1R_TARGET_LIST_MASK) >> ICC_SGI1R_TARGET_LIST_SHIFT;
>  	mpidr = SGI_AFFINITY_LEVEL(reg, 3);
>  	mpidr |= SGI_AFFINITY_LEVEL(reg, 2);
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
> index 0b3ecf9..80f92ce 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.h
> +++ b/virt/kvm/arm/vgic/vgic-mmio.h
> @@ -96,7 +96,7 @@ unsigned long vgic_data_mmio_bus_to_host(const void *val, unsigned int len);
>  void vgic_data_host_to_mmio_bus(void *buf, unsigned int len,
>  				unsigned long data);
>  
> -unsigned long extract_bytes(unsigned long data, unsigned int offset,
> +unsigned long extract_bytes(u64 data, unsigned int offset,
>  			    unsigned int num);
>  
>  u64 update_64bit_reg(u64 reg, unsigned int offset, unsigned int len,
> 

My personal preference would be to split this in two patches. One that
changes extract_bytes to work on 64bit quantities, and another one that
addresses the 64bit issues. Not a big deal though.

Acked-by: Marc Zyngier <marc.zyngier@arm.com>

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 6/8] ARM: Change MPIDR_AFFINITY_LEVEL to ignore Aff3
  2016-09-08 16:06   ` Vladimir Murzin
@ 2016-09-09 16:59     ` Marc Zyngier
  -1 siblings, 0 replies; 72+ messages in thread
From: Marc Zyngier @ 2016-09-09 16:59 UTC (permalink / raw)
  To: Vladimir Murzin, kvmarm; +Cc: andre.przywara, linux-arm-kernel

On 08/09/16 17:06, Vladimir Murzin wrote:
> vgic-v3 driver queries CPU affinity level up to Aff3, which is valid
> for arm64.  However, for arm up to Aff2 levels are supported, so
> querying for third level ends with upper bits of MPIDR are treated as
> valid affinity level which is not true. Make sure we report zero for
> any affinity level above two.
> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> ---
>  arch/arm/include/asm/cputype.h |    3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/arm/include/asm/cputype.h b/arch/arm/include/asm/cputype.h
> index 1ee94c7..f08fac4 100644
> --- a/arch/arm/include/asm/cputype.h
> +++ b/arch/arm/include/asm/cputype.h
> @@ -55,9 +55,10 @@
>  
>  #define MPIDR_LEVEL_BITS 8
>  #define MPIDR_LEVEL_MASK ((1 << MPIDR_LEVEL_BITS) - 1)
> +#define MPIDR_LEVEL_SHIFT(level) (MPIDR_LEVEL_BITS * level)
>  
>  #define MPIDR_AFFINITY_LEVEL(mpidr, level) \
> -	((mpidr >> (MPIDR_LEVEL_BITS * level)) & MPIDR_LEVEL_MASK)
> +	(((mpidr & MPIDR_HWID_BITMASK) >> MPIDR_LEVEL_SHIFT(level)) & MPIDR_LEVEL_MASK)
>  
>  #define ARM_CPU_IMP_ARM			0x41
>  #define ARM_CPU_IMP_INTEL		0x69
> 

There is something I don't quite get. Is this patch really necessary?
Are there cases where we construct a MPIDR that can have Aff3 set on 32bit?

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 6/8] ARM: Change MPIDR_AFFINITY_LEVEL to ignore Aff3
@ 2016-09-09 16:59     ` Marc Zyngier
  0 siblings, 0 replies; 72+ messages in thread
From: Marc Zyngier @ 2016-09-09 16:59 UTC (permalink / raw)
  To: linux-arm-kernel

On 08/09/16 17:06, Vladimir Murzin wrote:
> vgic-v3 driver queries CPU affinity level up to Aff3, which is valid
> for arm64.  However, for arm up to Aff2 levels are supported, so
> querying for third level ends with upper bits of MPIDR are treated as
> valid affinity level which is not true. Make sure we report zero for
> any affinity level above two.
> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> ---
>  arch/arm/include/asm/cputype.h |    3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/arm/include/asm/cputype.h b/arch/arm/include/asm/cputype.h
> index 1ee94c7..f08fac4 100644
> --- a/arch/arm/include/asm/cputype.h
> +++ b/arch/arm/include/asm/cputype.h
> @@ -55,9 +55,10 @@
>  
>  #define MPIDR_LEVEL_BITS 8
>  #define MPIDR_LEVEL_MASK ((1 << MPIDR_LEVEL_BITS) - 1)
> +#define MPIDR_LEVEL_SHIFT(level) (MPIDR_LEVEL_BITS * level)
>  
>  #define MPIDR_AFFINITY_LEVEL(mpidr, level) \
> -	((mpidr >> (MPIDR_LEVEL_BITS * level)) & MPIDR_LEVEL_MASK)
> +	(((mpidr & MPIDR_HWID_BITMASK) >> MPIDR_LEVEL_SHIFT(level)) & MPIDR_LEVEL_MASK)
>  
>  #define ARM_CPU_IMP_ARM			0x41
>  #define ARM_CPU_IMP_INTEL		0x69
> 

There is something I don't quite get. Is this patch really necessary?
Are there cases where we construct a MPIDR that can have Aff3 set on 32bit?

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 7/8] ARM: Move system register accessors to asm/cp15.h
  2016-09-08 16:06   ` Vladimir Murzin
@ 2016-09-09 17:05     ` Marc Zyngier
  -1 siblings, 0 replies; 72+ messages in thread
From: Marc Zyngier @ 2016-09-09 17:05 UTC (permalink / raw)
  To: Vladimir Murzin, kvmarm; +Cc: andre.przywara, linux-arm-kernel

On 08/09/16 17:06, Vladimir Murzin wrote:
> Headers linux/irqchip/arm-gic.v3.h and arch/arm/include/asm/kvm_hyp.h
> are included in virt/kvm/arm/hyp/vgic-v3-sr.c and both define macros
> called __ACCESS_CP15 and __ACCESS_CP15_64 which obviously creates a
> conflict. These macros were introduced independently for GIC and KVM
> and, in fact, do the same thing.
> 
> As an option we could add prefixes to KVM and GIC version of macros so
> they won't clash, but it'd introduce code duplication.  Alternatively,
> we could keep macro in, say, GIC header and include it in KVM one (or
> vice versa), but such dependency would not look nicer.
> 
> So we follow arm64 way (it handles this via sysreg.h) and move only
> single set of macros to asm/cp15.h
> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> ---
>  arch/arm/include/asm/arch_gicv3.h |   27 +++++++++++----------------
>  arch/arm/include/asm/cp15.h       |   15 +++++++++++++++
>  arch/arm/include/asm/kvm_hyp.h    |   15 +--------------
>  3 files changed, 27 insertions(+), 30 deletions(-)
> 
> diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
> index e08d151..af25c32 100644
> --- a/arch/arm/include/asm/arch_gicv3.h
> +++ b/arch/arm/include/asm/arch_gicv3.h
> @@ -22,9 +22,7 @@
>  
>  #include <linux/io.h>
>  #include <asm/barrier.h>
> -
> -#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
> -#define __ACCESS_CP15_64(Op1, CRm)		p15, Op1, %Q0, %R0, CRm
> +#include <asm/cp15.h>
>  
>  #define ICC_EOIR1			__ACCESS_CP15(c12, 0, c12, 1)
>  #define ICC_DIR				__ACCESS_CP15(c12, 0, c11, 1)
> @@ -102,58 +100,55 @@
>  
>  static inline void gic_write_eoir(u32 irq)
>  {
> -	asm volatile("mcr " __stringify(ICC_EOIR1) : : "r" (irq));
> +	write_sysreg(irq, ICC_EOIR1);
>  	isb();
>  }
>  
>  static inline void gic_write_dir(u32 val)
>  {
> -	asm volatile("mcr " __stringify(ICC_DIR) : : "r" (val));
> +	write_sysreg(val, ICC_DIR);
>  	isb();
>  }
>  
>  static inline u32 gic_read_iar(void)
>  {
> -	u32 irqstat;
> +	u32 irqstat = read_sysreg(ICC_IAR1);
>  
> -	asm volatile("mrc " __stringify(ICC_IAR1) : "=r" (irqstat));
>  	dsb(sy);
> +
>  	return irqstat;
>  }
>  
>  static inline void gic_write_pmr(u32 val)
>  {
> -	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
> +	write_sysreg(val, ICC_PMR);
>  }
>  
>  static inline void gic_write_ctlr(u32 val)
>  {
> -	asm volatile("mcr " __stringify(ICC_CTLR) : : "r" (val));
> +	write_sysreg(val, ICC_CTLR);
>  	isb();
>  }
>  
>  static inline void gic_write_grpen1(u32 val)
>  {
> -	asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
> +	write_sysreg(val, ICC_IGRPEN1);
>  	isb();
>  }
>  
>  static inline void gic_write_sgi1r(u64 val)
>  {
> -	asm volatile("mcrr " __stringify(ICC_SGI1R) : : "r" (val));
> +	write_sysreg(val, ICC_SGI1R);
>  }
>  
>  static inline u32 gic_read_sre(void)
>  {
> -	u32 val;
> -
> -	asm volatile("mrc " __stringify(ICC_SRE) : "=r" (val));
> -	return val;
> +	return read_sysreg(ICC_SRE);
>  }
>  
>  static inline void gic_write_sre(u32 val)
>  {
> -	asm volatile("mcr " __stringify(ICC_SRE) : : "r" (val));
> +	write_sysreg(val, ICC_SRE);
>  	isb();
>  }
>  
> diff --git a/arch/arm/include/asm/cp15.h b/arch/arm/include/asm/cp15.h
> index c3f1152..f661732 100644
> --- a/arch/arm/include/asm/cp15.h
> +++ b/arch/arm/include/asm/cp15.h
> @@ -47,6 +47,21 @@
>  #define vectors_high()	(0)
>  #endif
>  
> +#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
> +	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
> +#define __ACCESS_CP15_64(Op1, CRm)		\
> +	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
> +
> +#define __read_sysreg(r, w, c, t) ({				\
> +	t __val;						\
> +	asm volatile(r " " c : "=r" (__val));			\
> +	__val;							\
> +})
> +#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
> +
> +#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
> +#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
> +

Shouldn't that be placed after the #ifdef below?

>  #ifdef CONFIG_CPU_CP15
>  
>  extern unsigned long cr_alignment;	/* defined in entry-armv.S */
> diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
> index bd9434e..0b475d2 100644
> --- a/arch/arm/include/asm/kvm_hyp.h
> +++ b/arch/arm/include/asm/kvm_hyp.h
> @@ -20,26 +20,13 @@
>  
>  #include <linux/compiler.h>
>  #include <linux/kvm_host.h>
> +#include <asm/cp15.h>
>  #include <asm/kvm_mmu.h>
>  #include <asm/vfp.h>
>  
> -#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
> -	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
> -#define __ACCESS_CP15_64(Op1, CRm)		\
> -	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
>  #define __ACCESS_VFP(CRn)			\
>  	"mrc", "mcr", __stringify(p10, 7, %0, CRn, cr0, 0), u32
>  
> -#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
> -#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
> -
> -#define __read_sysreg(r, w, c, t) ({				\
> -	t __val;						\
> -	asm volatile(r " " c : "=r" (__val));			\
> -	__val;							\
> -})
> -#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
> -
>  #define write_special(v, r)					\
>  	asm volatile("msr " __stringify(r) ", %0" : : "r" (v))
>  #define read_special(r) ({					\
> 

Could you please cc RMK on this, given that this touches a core arch/arm
file?

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 7/8] ARM: Move system register accessors to asm/cp15.h
@ 2016-09-09 17:05     ` Marc Zyngier
  0 siblings, 0 replies; 72+ messages in thread
From: Marc Zyngier @ 2016-09-09 17:05 UTC (permalink / raw)
  To: linux-arm-kernel

On 08/09/16 17:06, Vladimir Murzin wrote:
> Headers linux/irqchip/arm-gic.v3.h and arch/arm/include/asm/kvm_hyp.h
> are included in virt/kvm/arm/hyp/vgic-v3-sr.c and both define macros
> called __ACCESS_CP15 and __ACCESS_CP15_64 which obviously creates a
> conflict. These macros were introduced independently for GIC and KVM
> and, in fact, do the same thing.
> 
> As an option we could add prefixes to KVM and GIC version of macros so
> they won't clash, but it'd introduce code duplication.  Alternatively,
> we could keep macro in, say, GIC header and include it in KVM one (or
> vice versa), but such dependency would not look nicer.
> 
> So we follow arm64 way (it handles this via sysreg.h) and move only
> single set of macros to asm/cp15.h
> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> ---
>  arch/arm/include/asm/arch_gicv3.h |   27 +++++++++++----------------
>  arch/arm/include/asm/cp15.h       |   15 +++++++++++++++
>  arch/arm/include/asm/kvm_hyp.h    |   15 +--------------
>  3 files changed, 27 insertions(+), 30 deletions(-)
> 
> diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
> index e08d151..af25c32 100644
> --- a/arch/arm/include/asm/arch_gicv3.h
> +++ b/arch/arm/include/asm/arch_gicv3.h
> @@ -22,9 +22,7 @@
>  
>  #include <linux/io.h>
>  #include <asm/barrier.h>
> -
> -#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
> -#define __ACCESS_CP15_64(Op1, CRm)		p15, Op1, %Q0, %R0, CRm
> +#include <asm/cp15.h>
>  
>  #define ICC_EOIR1			__ACCESS_CP15(c12, 0, c12, 1)
>  #define ICC_DIR				__ACCESS_CP15(c12, 0, c11, 1)
> @@ -102,58 +100,55 @@
>  
>  static inline void gic_write_eoir(u32 irq)
>  {
> -	asm volatile("mcr " __stringify(ICC_EOIR1) : : "r" (irq));
> +	write_sysreg(irq, ICC_EOIR1);
>  	isb();
>  }
>  
>  static inline void gic_write_dir(u32 val)
>  {
> -	asm volatile("mcr " __stringify(ICC_DIR) : : "r" (val));
> +	write_sysreg(val, ICC_DIR);
>  	isb();
>  }
>  
>  static inline u32 gic_read_iar(void)
>  {
> -	u32 irqstat;
> +	u32 irqstat = read_sysreg(ICC_IAR1);
>  
> -	asm volatile("mrc " __stringify(ICC_IAR1) : "=r" (irqstat));
>  	dsb(sy);
> +
>  	return irqstat;
>  }
>  
>  static inline void gic_write_pmr(u32 val)
>  {
> -	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
> +	write_sysreg(val, ICC_PMR);
>  }
>  
>  static inline void gic_write_ctlr(u32 val)
>  {
> -	asm volatile("mcr " __stringify(ICC_CTLR) : : "r" (val));
> +	write_sysreg(val, ICC_CTLR);
>  	isb();
>  }
>  
>  static inline void gic_write_grpen1(u32 val)
>  {
> -	asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
> +	write_sysreg(val, ICC_IGRPEN1);
>  	isb();
>  }
>  
>  static inline void gic_write_sgi1r(u64 val)
>  {
> -	asm volatile("mcrr " __stringify(ICC_SGI1R) : : "r" (val));
> +	write_sysreg(val, ICC_SGI1R);
>  }
>  
>  static inline u32 gic_read_sre(void)
>  {
> -	u32 val;
> -
> -	asm volatile("mrc " __stringify(ICC_SRE) : "=r" (val));
> -	return val;
> +	return read_sysreg(ICC_SRE);
>  }
>  
>  static inline void gic_write_sre(u32 val)
>  {
> -	asm volatile("mcr " __stringify(ICC_SRE) : : "r" (val));
> +	write_sysreg(val, ICC_SRE);
>  	isb();
>  }
>  
> diff --git a/arch/arm/include/asm/cp15.h b/arch/arm/include/asm/cp15.h
> index c3f1152..f661732 100644
> --- a/arch/arm/include/asm/cp15.h
> +++ b/arch/arm/include/asm/cp15.h
> @@ -47,6 +47,21 @@
>  #define vectors_high()	(0)
>  #endif
>  
> +#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
> +	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
> +#define __ACCESS_CP15_64(Op1, CRm)		\
> +	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
> +
> +#define __read_sysreg(r, w, c, t) ({				\
> +	t __val;						\
> +	asm volatile(r " " c : "=r" (__val));			\
> +	__val;							\
> +})
> +#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
> +
> +#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
> +#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
> +

Shouldn't that be placed after the #ifdef below?

>  #ifdef CONFIG_CPU_CP15
>  
>  extern unsigned long cr_alignment;	/* defined in entry-armv.S */
> diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
> index bd9434e..0b475d2 100644
> --- a/arch/arm/include/asm/kvm_hyp.h
> +++ b/arch/arm/include/asm/kvm_hyp.h
> @@ -20,26 +20,13 @@
>  
>  #include <linux/compiler.h>
>  #include <linux/kvm_host.h>
> +#include <asm/cp15.h>
>  #include <asm/kvm_mmu.h>
>  #include <asm/vfp.h>
>  
> -#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
> -	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
> -#define __ACCESS_CP15_64(Op1, CRm)		\
> -	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
>  #define __ACCESS_VFP(CRn)			\
>  	"mrc", "mcr", __stringify(p10, 7, %0, CRn, cr0, 0), u32
>  
> -#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
> -#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
> -
> -#define __read_sysreg(r, w, c, t) ({				\
> -	t __val;						\
> -	asm volatile(r " " c : "=r" (__val));			\
> -	__val;							\
> -})
> -#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
> -
>  #define write_special(v, r)					\
>  	asm volatile("msr " __stringify(r) ", %0" : : "r" (v))
>  #define read_special(r) ({					\
> 

Could you please cc RMK on this, given that this touches a core arch/arm
file?

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 1/8] arm64: KVM: Use static keys for selecting the GIC backend
  2016-09-09 16:18                 ` Vladimir Murzin
@ 2016-09-09 17:06                   ` Marc Zyngier
  -1 siblings, 0 replies; 72+ messages in thread
From: Marc Zyngier @ 2016-09-09 17:06 UTC (permalink / raw)
  To: Vladimir Murzin, kvmarm; +Cc: andre.przywara, linux-arm-kernel

On 09/09/16 17:18, Vladimir Murzin wrote:
> On 09/09/16 16:25, Marc Zyngier wrote:
>> On 09/09/16 16:14, Vladimir Murzin wrote:
>>> On 09/09/16 15:17, Marc Zyngier wrote:
>>>> On 09/09/16 14:45, Vladimir Murzin wrote:
>>>>> On 09/09/16 10:33, Vladimir Murzin wrote:
>>>>>> Hi Marc,
>>>>>>
>>>>>> On 09/09/16 10:19, Marc Zyngier wrote:
>>>>>>>> Hi Vladimir,
>>>>>>>>
>>>>>> ...
>>>>>>>>>>  
>>>>>>>>>> +extern struct static_key_false kvm_gicv3_cpuif;
>>>>>>>>
>>>>>>>> I think we should follow the model set by kvm_vgic_global_state, which
>>>>>>>> is declared in arm_vgic.h. Even better, we should *embed* the static key
>>>>>>>> in this structure. This will reduce the clutter and we wouldn't have to
>>>>>>>> deal with all the section stuff (the hyp_data thing is a good cleanup,
>>>>>>>> but I'd like to see it as a separate patch if possible).
>>>>>> Yes, it is what I was thinking about too, but was not sure about which
>>>>>> way to go, so hyp_data seemed me something we might reuse latter.
>>>>>> However, I agree that we can defer hyp_data thing...
>>>>>>
>>>>>
>>>>> I've just tried it out and it seems that static keys are not happy to
>>>>> accept a key after kern_hyp_va is applied at &kvm_vgic_global_state:
>>>>
>>>> Ah, there is a trick. You do not need kern_hyp_va at all, because this
>>>> is not evaluated as an expression at runtime (so the pointer doesn't matter).
>>>>
>>>
>>> Ah, right, thank for a tip! ;)
>>>
>>>>>> In file included from ./include/linux/jump_label.h:105:0,
>>>>>>                  from arch/arm64/kvm/hyp/switch.c:19:
>>>>>> ./arch/arm64/include/asm/jump_label.h: In function ‘__guest_run’:
>>>>>> ./arch/arm64/include/asm/jump_label.h:31:2: warning: asm operand 0 probably doesn’t match constraints
>>>>>>   asm goto("1: nop\n\t"
>>>>>>   ^
>>>>>> ./arch/arm64/include/asm/jump_label.h:31:2: warning: asm operand 0 probably doesn’t match constraints
>>>>>>   asm goto("1: nop\n\t"
>>>>>>   ^
>>>>>> ./arch/arm64/include/asm/jump_label.h:31:2: error: impossible constraint in ‘asm’
>>>>>>   asm goto("1: nop\n\t"
>>>>>>   ^
>>>>>> ./arch/arm64/include/asm/jump_label.h:31:2: error: impossible constraint in ‘asm’
>>>>>>   asm goto("1: nop\n\t"
>>>>>>   ^
>>>>>> make[1]: *** [arch/arm64/kvm/hyp/switch.o] Error 1
>>>>>> make: *** [arch/arm64/kvm/hyp/switch.o] Error 2
>>>>>
>>>>> it looks like we cannot avoid hyp_data thing... if you don't mind I can
>>>>> do hyp_data clean-up in separate patch. Alternatively, we can do
>>>>> conversion to static keys for both architectures later as an
>>>>> optimisation step.
>>>>
>>>> Can you try the above first? I've just tried the same approach with my
>>>> vgic-trap series, and it compiles fine (untested though):
>>>
>>> I was about to try it out, but didn't manage to find a branch with
>>> vgic-trap series, so I did a quick fixup for my series and now it is
>>> running non-VHE boot tests and I don't expect issues with VHE one. I
>>> think diff bellow should work runtime too, but if you do want me to give
>>> it a try it'd be handy to have a branch I can pull from ;)
>>
>> I think Christoffer has pulled it into kvmarm/queue.
> 
> Nothing has exploded so far for both architectures.

Great. Hopefully you can respin the series next week.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 1/8] arm64: KVM: Use static keys for selecting the GIC backend
@ 2016-09-09 17:06                   ` Marc Zyngier
  0 siblings, 0 replies; 72+ messages in thread
From: Marc Zyngier @ 2016-09-09 17:06 UTC (permalink / raw)
  To: linux-arm-kernel

On 09/09/16 17:18, Vladimir Murzin wrote:
> On 09/09/16 16:25, Marc Zyngier wrote:
>> On 09/09/16 16:14, Vladimir Murzin wrote:
>>> On 09/09/16 15:17, Marc Zyngier wrote:
>>>> On 09/09/16 14:45, Vladimir Murzin wrote:
>>>>> On 09/09/16 10:33, Vladimir Murzin wrote:
>>>>>> Hi Marc,
>>>>>>
>>>>>> On 09/09/16 10:19, Marc Zyngier wrote:
>>>>>>>> Hi Vladimir,
>>>>>>>>
>>>>>> ...
>>>>>>>>>>  
>>>>>>>>>> +extern struct static_key_false kvm_gicv3_cpuif;
>>>>>>>>
>>>>>>>> I think we should follow the model set by kvm_vgic_global_state, which
>>>>>>>> is declared in arm_vgic.h. Even better, we should *embed* the static key
>>>>>>>> in this structure. This will reduce the clutter and we wouldn't have to
>>>>>>>> deal with all the section stuff (the hyp_data thing is a good cleanup,
>>>>>>>> but I'd like to see it as a separate patch if possible).
>>>>>> Yes, it is what I was thinking about too, but was not sure about which
>>>>>> way to go, so hyp_data seemed me something we might reuse latter.
>>>>>> However, I agree that we can defer hyp_data thing...
>>>>>>
>>>>>
>>>>> I've just tried it out and it seems that static keys are not happy to
>>>>> accept a key after kern_hyp_va is applied at &kvm_vgic_global_state:
>>>>
>>>> Ah, there is a trick. You do not need kern_hyp_va at all, because this
>>>> is not evaluated as an expression at runtime (so the pointer doesn't matter).
>>>>
>>>
>>> Ah, right, thank for a tip! ;)
>>>
>>>>>> In file included from ./include/linux/jump_label.h:105:0,
>>>>>>                  from arch/arm64/kvm/hyp/switch.c:19:
>>>>>> ./arch/arm64/include/asm/jump_label.h: In function ?__guest_run?:
>>>>>> ./arch/arm64/include/asm/jump_label.h:31:2: warning: asm operand 0 probably doesn?t match constraints
>>>>>>   asm goto("1: nop\n\t"
>>>>>>   ^
>>>>>> ./arch/arm64/include/asm/jump_label.h:31:2: warning: asm operand 0 probably doesn?t match constraints
>>>>>>   asm goto("1: nop\n\t"
>>>>>>   ^
>>>>>> ./arch/arm64/include/asm/jump_label.h:31:2: error: impossible constraint in ?asm?
>>>>>>   asm goto("1: nop\n\t"
>>>>>>   ^
>>>>>> ./arch/arm64/include/asm/jump_label.h:31:2: error: impossible constraint in ?asm?
>>>>>>   asm goto("1: nop\n\t"
>>>>>>   ^
>>>>>> make[1]: *** [arch/arm64/kvm/hyp/switch.o] Error 1
>>>>>> make: *** [arch/arm64/kvm/hyp/switch.o] Error 2
>>>>>
>>>>> it looks like we cannot avoid hyp_data thing... if you don't mind I can
>>>>> do hyp_data clean-up in separate patch. Alternatively, we can do
>>>>> conversion to static keys for both architectures later as an
>>>>> optimisation step.
>>>>
>>>> Can you try the above first? I've just tried the same approach with my
>>>> vgic-trap series, and it compiles fine (untested though):
>>>
>>> I was about to try it out, but didn't manage to find a branch with
>>> vgic-trap series, so I did a quick fixup for my series and now it is
>>> running non-VHE boot tests and I don't expect issues with VHE one. I
>>> think diff bellow should work runtime too, but if you do want me to give
>>> it a try it'd be handy to have a branch I can pull from ;)
>>
>> I think Christoffer has pulled it into kvmarm/queue.
> 
> Nothing has exploded so far for both architectures.

Great. Hopefully you can respin the series next week.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 2/8] arm64: KVM: Move GIC accessors to arch_gicv3.h
  2016-09-09 16:32     ` Marc Zyngier
@ 2016-09-12  9:18       ` Vladimir Murzin
  -1 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-12  9:18 UTC (permalink / raw)
  To: Marc Zyngier, kvmarm; +Cc: andre.przywara, linux-arm-kernel

On 09/09/16 17:32, Marc Zyngier wrote:
> On 08/09/16 17:06, Vladimir Murzin wrote:
>> Since we are going to share vgic-v3 save/restore code with ARM keep
>> arch specific accessors separately.
>>
>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>> Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
> 
> Acked-by: Marc Zyngier <marc.zyngier@arm.com>
> 

Thanks!

Vladimir

> 	M.
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 2/8] arm64: KVM: Move GIC accessors to arch_gicv3.h
@ 2016-09-12  9:18       ` Vladimir Murzin
  0 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-12  9:18 UTC (permalink / raw)
  To: linux-arm-kernel

On 09/09/16 17:32, Marc Zyngier wrote:
> On 08/09/16 17:06, Vladimir Murzin wrote:
>> Since we are going to share vgic-v3 save/restore code with ARM keep
>> arch specific accessors separately.
>>
>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>> Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
> 
> Acked-by: Marc Zyngier <marc.zyngier@arm.com>
> 

Thanks!

Vladimir

> 	M.
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 3/8] arm64: KVM: Move vgic-v3 save/restore to virt/kvm/arm/hyp
  2016-09-09 16:33     ` Marc Zyngier
@ 2016-09-12  9:18       ` Vladimir Murzin
  -1 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-12  9:18 UTC (permalink / raw)
  To: Marc Zyngier, kvmarm; +Cc: andre.przywara, linux-arm-kernel

On 09/09/16 17:33, Marc Zyngier wrote:
> On 08/09/16 17:06, Vladimir Murzin wrote:
>> So we can reuse the code under arch/arm
>>
>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> 
> Acked-by: Marc Zyngier <marc.zyngier@arm.com>
> 

Thanks!

Vladimir

> 	M.
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 3/8] arm64: KVM: Move vgic-v3 save/restore to virt/kvm/arm/hyp
@ 2016-09-12  9:18       ` Vladimir Murzin
  0 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-12  9:18 UTC (permalink / raw)
  To: linux-arm-kernel

On 09/09/16 17:33, Marc Zyngier wrote:
> On 08/09/16 17:06, Vladimir Murzin wrote:
>> So we can reuse the code under arch/arm
>>
>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> 
> Acked-by: Marc Zyngier <marc.zyngier@arm.com>
> 

Thanks!

Vladimir

> 	M.
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 8/8] ARM: KVM: Support vgic-v3
  2016-09-09 16:45     ` Marc Zyngier
@ 2016-09-12  9:23       ` Vladimir Murzin
  -1 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-12  9:23 UTC (permalink / raw)
  To: Marc Zyngier, kvmarm; +Cc: andre.przywara, linux-arm-kernel

On 09/09/16 17:45, Marc Zyngier wrote:
> On 08/09/16 17:06, Vladimir Murzin wrote:
>> This patch allows to build and use vgic-v3 in 32-bit mode.
>>

snip...

>> diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
>> index af25c32..f93f6bd 100644
>> --- a/arch/arm/include/asm/arch_gicv3.h
>> +++ b/arch/arm/include/asm/arch_gicv3.h
>> @@ -96,6 +96,70 @@
>>  #define ICH_AP1R2			__AP1Rx(2)
>>  #define ICH_AP1R3			__AP1Rx(3)
>>  
>> +/* A32-to-A64 mappings used by VGIC save/restore */
>> +
>> +#define CPUIF_MAP(a32, a64)			\
>> +static inline void write_ ## a64(u32 val)	\
>> +{						\
>> +	write_sysreg(val, a32);			\
>> +}						\
>> +static inline u32 read_ ## a64(void)		\
>> +{						\
>> +	return read_sysreg(a32); 		\
>> +}						\
>> +
>> +#define CPUIF_MAP_LO_HI(a32lo, a32hi, a64)	\
>> +static inline void write_ ## a64(u64 val)	\
>> +{						\
>> +	write_sysreg((u32)val, a32lo);		\
>> +	write_sysreg((u32)(val >> 32), a32hi);	\
> 
> Please use {lower,upper}_32_bits, which make the casting/shifting go away.
> 

Will do.

>> +}						\
>> +static inline u64 read_ ## a64(void)		\
>> +{						\
>> +	u64 val = read_sysreg(a32lo);		\
>> +						\
>> +	val |=	(u64)read_sysreg(a32hi) << 32;	\
>> +						\
>> +	return val; 				\
>> +}
>> +
>> +CPUIF_MAP(ICH_HCR, ICH_HCR_EL2)
>> +CPUIF_MAP(ICH_VTR, ICH_VTR_EL2)
>> +CPUIF_MAP(ICH_MISR, ICH_MISR_EL2)
>> +CPUIF_MAP(ICH_EISR, ICH_EISR_EL2)
>> +CPUIF_MAP(ICH_ELSR, ICH_ELSR_EL2)
>> +CPUIF_MAP(ICH_VMCR, ICH_VMCR_EL2)
>> +CPUIF_MAP(ICH_AP0R3, ICH_AP0R3_EL2)
>> +CPUIF_MAP(ICH_AP0R2, ICH_AP0R2_EL2)
>> +CPUIF_MAP(ICH_AP0R1, ICH_AP0R1_EL2)
>> +CPUIF_MAP(ICH_AP0R0, ICH_AP0R0_EL2)
>> +CPUIF_MAP(ICH_AP1R3, ICH_AP1R3_EL2)
>> +CPUIF_MAP(ICH_AP1R2, ICH_AP1R2_EL2)
>> +CPUIF_MAP(ICH_AP1R1, ICH_AP1R1_EL2)
>> +CPUIF_MAP(ICH_AP1R0, ICH_AP1R0_EL2)
>> +CPUIF_MAP(ICC_HSRE, ICC_SRE_EL2)
>> +CPUIF_MAP(ICC_SRE, ICC_SRE_EL1)
>> +
>> +CPUIF_MAP_LO_HI(ICH_LR15, ICH_LRC15, ICH_LR15_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR14, ICH_LRC14, ICH_LR14_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR13, ICH_LRC13, ICH_LR13_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR12, ICH_LRC12, ICH_LR12_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR11, ICH_LRC11, ICH_LR11_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR10, ICH_LRC10, ICH_LR10_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR9, ICH_LRC9, ICH_LR9_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR8, ICH_LRC8, ICH_LR8_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR7, ICH_LRC7, ICH_LR7_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR6, ICH_LRC6, ICH_LR6_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR5, ICH_LRC5, ICH_LR5_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR4, ICH_LRC4, ICH_LR4_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR3, ICH_LRC3, ICH_LR3_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR2, ICH_LRC2, ICH_LR2_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR1, ICH_LRC1, ICH_LR1_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR0, ICH_LRC0, ICH_LR0_EL2)
>> +
>> +#define read_gicreg(r)                 read_##r()
>> +#define write_gicreg(v, r)             write_##r(v)
>> +
> 
> Can you make this change a separate patch? It will make it easier to
> merge if I can ack it as a standalone change. It will also give the last
> patch a fantastic diffstat... ;-)
> 

Yes, I can ;)

snip...

>> diff --git a/arch/arm/kvm/coproc.c b/arch/arm/kvm/coproc.c
>> index 1bb2b79..10c0244 100644
>> --- a/arch/arm/kvm/coproc.c
>> +++ b/arch/arm/kvm/coproc.c
>> @@ -228,6 +228,36 @@ bool access_vm_reg(struct kvm_vcpu *vcpu,
>>  	return true;
>>  }
>>  
>> +static bool access_gic_sgi(struct kvm_vcpu *vcpu,
>> +			   const struct coproc_params *p,
>> +			   const struct coproc_reg *r)
>> +{
>> +	u64 reg;
>> +
>> +	if (!p->is_write)
>> +		return read_from_write_only(vcpu, p);
>> +
>> +	reg = *vcpu_reg(vcpu, p->Rt2);
>> +	reg <<= 32;
> 
> nit: can you write this as
> 
> 	reg = (u64)*vcpu_reg(vcpu, p->Rt2) << 32;
> 
> which I find easier to read...
> 

I'll rewrite it.

snip...

>> diff --git a/arch/arm/kvm/hyp/Makefile b/arch/arm/kvm/hyp/Makefile
>> index 8dfa5f7..3023bb5 100644
>> --- a/arch/arm/kvm/hyp/Makefile
>> +++ b/arch/arm/kvm/hyp/Makefile
>> @@ -5,6 +5,7 @@
>>  KVM=../../../../virt/kvm
>>  
>>  obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
>> +obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
>>  obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
>>  
>>  obj-$(CONFIG_KVM_ARM_HOST) += tlb.o
>> diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
>> index b13caa9..8409dd5 100644
>> --- a/arch/arm/kvm/hyp/switch.c
>> +++ b/arch/arm/kvm/hyp/switch.c
>> @@ -14,6 +14,7 @@

> 
> It otherwise looks good to me.
> 

Thanks for feedback!

Cheers
Vladimir

> Thanks,
> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 8/8] ARM: KVM: Support vgic-v3
@ 2016-09-12  9:23       ` Vladimir Murzin
  0 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-12  9:23 UTC (permalink / raw)
  To: linux-arm-kernel

On 09/09/16 17:45, Marc Zyngier wrote:
> On 08/09/16 17:06, Vladimir Murzin wrote:
>> This patch allows to build and use vgic-v3 in 32-bit mode.
>>

snip...

>> diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
>> index af25c32..f93f6bd 100644
>> --- a/arch/arm/include/asm/arch_gicv3.h
>> +++ b/arch/arm/include/asm/arch_gicv3.h
>> @@ -96,6 +96,70 @@
>>  #define ICH_AP1R2			__AP1Rx(2)
>>  #define ICH_AP1R3			__AP1Rx(3)
>>  
>> +/* A32-to-A64 mappings used by VGIC save/restore */
>> +
>> +#define CPUIF_MAP(a32, a64)			\
>> +static inline void write_ ## a64(u32 val)	\
>> +{						\
>> +	write_sysreg(val, a32);			\
>> +}						\
>> +static inline u32 read_ ## a64(void)		\
>> +{						\
>> +	return read_sysreg(a32); 		\
>> +}						\
>> +
>> +#define CPUIF_MAP_LO_HI(a32lo, a32hi, a64)	\
>> +static inline void write_ ## a64(u64 val)	\
>> +{						\
>> +	write_sysreg((u32)val, a32lo);		\
>> +	write_sysreg((u32)(val >> 32), a32hi);	\
> 
> Please use {lower,upper}_32_bits, which make the casting/shifting go away.
> 

Will do.

>> +}						\
>> +static inline u64 read_ ## a64(void)		\
>> +{						\
>> +	u64 val = read_sysreg(a32lo);		\
>> +						\
>> +	val |=	(u64)read_sysreg(a32hi) << 32;	\
>> +						\
>> +	return val; 				\
>> +}
>> +
>> +CPUIF_MAP(ICH_HCR, ICH_HCR_EL2)
>> +CPUIF_MAP(ICH_VTR, ICH_VTR_EL2)
>> +CPUIF_MAP(ICH_MISR, ICH_MISR_EL2)
>> +CPUIF_MAP(ICH_EISR, ICH_EISR_EL2)
>> +CPUIF_MAP(ICH_ELSR, ICH_ELSR_EL2)
>> +CPUIF_MAP(ICH_VMCR, ICH_VMCR_EL2)
>> +CPUIF_MAP(ICH_AP0R3, ICH_AP0R3_EL2)
>> +CPUIF_MAP(ICH_AP0R2, ICH_AP0R2_EL2)
>> +CPUIF_MAP(ICH_AP0R1, ICH_AP0R1_EL2)
>> +CPUIF_MAP(ICH_AP0R0, ICH_AP0R0_EL2)
>> +CPUIF_MAP(ICH_AP1R3, ICH_AP1R3_EL2)
>> +CPUIF_MAP(ICH_AP1R2, ICH_AP1R2_EL2)
>> +CPUIF_MAP(ICH_AP1R1, ICH_AP1R1_EL2)
>> +CPUIF_MAP(ICH_AP1R0, ICH_AP1R0_EL2)
>> +CPUIF_MAP(ICC_HSRE, ICC_SRE_EL2)
>> +CPUIF_MAP(ICC_SRE, ICC_SRE_EL1)
>> +
>> +CPUIF_MAP_LO_HI(ICH_LR15, ICH_LRC15, ICH_LR15_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR14, ICH_LRC14, ICH_LR14_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR13, ICH_LRC13, ICH_LR13_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR12, ICH_LRC12, ICH_LR12_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR11, ICH_LRC11, ICH_LR11_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR10, ICH_LRC10, ICH_LR10_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR9, ICH_LRC9, ICH_LR9_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR8, ICH_LRC8, ICH_LR8_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR7, ICH_LRC7, ICH_LR7_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR6, ICH_LRC6, ICH_LR6_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR5, ICH_LRC5, ICH_LR5_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR4, ICH_LRC4, ICH_LR4_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR3, ICH_LRC3, ICH_LR3_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR2, ICH_LRC2, ICH_LR2_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR1, ICH_LRC1, ICH_LR1_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR0, ICH_LRC0, ICH_LR0_EL2)
>> +
>> +#define read_gicreg(r)                 read_##r()
>> +#define write_gicreg(v, r)             write_##r(v)
>> +
> 
> Can you make this change a separate patch? It will make it easier to
> merge if I can ack it as a standalone change. It will also give the last
> patch a fantastic diffstat... ;-)
> 

Yes, I can ;)

snip...

>> diff --git a/arch/arm/kvm/coproc.c b/arch/arm/kvm/coproc.c
>> index 1bb2b79..10c0244 100644
>> --- a/arch/arm/kvm/coproc.c
>> +++ b/arch/arm/kvm/coproc.c
>> @@ -228,6 +228,36 @@ bool access_vm_reg(struct kvm_vcpu *vcpu,
>>  	return true;
>>  }
>>  
>> +static bool access_gic_sgi(struct kvm_vcpu *vcpu,
>> +			   const struct coproc_params *p,
>> +			   const struct coproc_reg *r)
>> +{
>> +	u64 reg;
>> +
>> +	if (!p->is_write)
>> +		return read_from_write_only(vcpu, p);
>> +
>> +	reg = *vcpu_reg(vcpu, p->Rt2);
>> +	reg <<= 32;
> 
> nit: can you write this as
> 
> 	reg = (u64)*vcpu_reg(vcpu, p->Rt2) << 32;
> 
> which I find easier to read...
> 

I'll rewrite it.

snip...

>> diff --git a/arch/arm/kvm/hyp/Makefile b/arch/arm/kvm/hyp/Makefile
>> index 8dfa5f7..3023bb5 100644
>> --- a/arch/arm/kvm/hyp/Makefile
>> +++ b/arch/arm/kvm/hyp/Makefile
>> @@ -5,6 +5,7 @@
>>  KVM=../../../../virt/kvm
>>  
>>  obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
>> +obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
>>  obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
>>  
>>  obj-$(CONFIG_KVM_ARM_HOST) += tlb.o
>> diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
>> index b13caa9..8409dd5 100644
>> --- a/arch/arm/kvm/hyp/switch.c
>> +++ b/arch/arm/kvm/hyp/switch.c
>> @@ -14,6 +14,7 @@

> 
> It otherwise looks good to me.
> 

Thanks for feedback!

Cheers
Vladimir

> Thanks,
> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 4/8] KVM: arm64: vgic-its: Introduce config option to guard ITS specific code
  2016-09-09 16:46     ` Marc Zyngier
@ 2016-09-12  9:23       ` Vladimir Murzin
  -1 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-12  9:23 UTC (permalink / raw)
  To: Marc Zyngier, kvmarm; +Cc: andre.przywara, linux-arm-kernel

On 09/09/16 17:46, Marc Zyngier wrote:
> On 08/09/16 17:06, Vladimir Murzin wrote:
>> By now ITS code guarded with KVM_ARM_VGIC_V3 config option which was
>> introduced to hide everything specific to vgic-v3 from 32-bit world.
>> We are going to support vgic-v3 in 32-bit world and KVM_ARM_VGIC_V3
>> will gone, but we don't have support for ITS there yet and we need to
>> continue keeping ITS away.
>> Introduce the new config option to prevent ITS code being build in
>> 32-bit mode when support for vgic-v3 is done.
>>
>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> 
> Acked-by: Marc Zyngier <marc.zyngier@arm.com>

Thanks!

Vladimir

> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 4/8] KVM: arm64: vgic-its: Introduce config option to guard ITS specific code
@ 2016-09-12  9:23       ` Vladimir Murzin
  0 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-12  9:23 UTC (permalink / raw)
  To: linux-arm-kernel

On 09/09/16 17:46, Marc Zyngier wrote:
> On 08/09/16 17:06, Vladimir Murzin wrote:
>> By now ITS code guarded with KVM_ARM_VGIC_V3 config option which was
>> introduced to hide everything specific to vgic-v3 from 32-bit world.
>> We are going to support vgic-v3 in 32-bit world and KVM_ARM_VGIC_V3
>> will gone, but we don't have support for ITS there yet and we need to
>> continue keeping ITS away.
>> Introduce the new config option to prevent ITS code being build in
>> 32-bit mode when support for vgic-v3 is done.
>>
>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> 
> Acked-by: Marc Zyngier <marc.zyngier@arm.com>

Thanks!

Vladimir

> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 5/8] KVM: arm: vgic: Support 64-bit data manipulation on 32-bit host systems
  2016-09-09 16:55     ` Marc Zyngier
@ 2016-09-12  9:25       ` Vladimir Murzin
  -1 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-12  9:25 UTC (permalink / raw)
  To: Marc Zyngier, kvmarm; +Cc: andre.przywara, linux-arm-kernel

On 09/09/16 17:55, Marc Zyngier wrote:
> On 08/09/16 17:06, Vladimir Murzin wrote:
>> We have couple of 64-bit registers defined in GICv3 architecture, so
>> unsigned long accesses to these registers will only access a single
>> 32-bit part of that regitser. On the other hand these registers can't
>> be accessed as 64-bit with a single instruction like ldrd/strd or
>> ldmia/stmia if we run a 32-bit host because KVM does not support
>> access to MMIO space done by these instructions.
>>
>> It means that a 32-bit guest accesses these registers in 32-bit
>> chunks, so the only thing we need to do is to ensure that
>> extract_bytes() always takes 64-bit data.
>>
>> Since we are here fix couple of other width related issues catched by
>> gcc
>>
>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>> ---
>>  virt/kvm/arm/vgic/vgic-mmio-v3.c |    6 +++---
>>  virt/kvm/arm/vgic/vgic-mmio.h    |    2 +-
>>  2 files changed, 4 insertions(+), 4 deletions(-)
>>
>> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> index acbe691..0d3c76a 100644
>> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> @@ -23,7 +23,7 @@
>>  #include "vgic-mmio.h"
>>  
>>  /* extract @num bytes at @offset bytes offset in data */
>> -unsigned long extract_bytes(unsigned long data, unsigned int offset,
>> +unsigned long extract_bytes(u64 data, unsigned int offset,
>>  			    unsigned int num)
>>  {
>>  	return (data >> (offset * 8)) & GENMASK_ULL(num * 8 - 1, 0);
>> @@ -181,7 +181,7 @@ static unsigned long vgic_mmio_read_v3r_typer(struct kvm_vcpu *vcpu,
>>  	int target_vcpu_id = vcpu->vcpu_id;
>>  	u64 value;
>>  
>> -	value = (mpidr & GENMASK(23, 0)) << 32;
>> +	value = (u64)(mpidr & GENMASK(23, 0)) << 32;
>>  	value |= ((target_vcpu_id & 0xffff) << 8);
>>  	if (target_vcpu_id == atomic_read(&vcpu->kvm->online_vcpus) - 1)
>>  		value |= GICR_TYPER_LAST;
>> @@ -611,7 +611,7 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)
>>  	bool broadcast;
>>  
>>  	sgi = (reg & ICC_SGI1R_SGI_ID_MASK) >> ICC_SGI1R_SGI_ID_SHIFT;
>> -	broadcast = reg & BIT(ICC_SGI1R_IRQ_ROUTING_MODE_BIT);
>> +	broadcast = reg & BIT_ULL(ICC_SGI1R_IRQ_ROUTING_MODE_BIT);
>>  	target_cpus = (reg & ICC_SGI1R_TARGET_LIST_MASK) >> ICC_SGI1R_TARGET_LIST_SHIFT;
>>  	mpidr = SGI_AFFINITY_LEVEL(reg, 3);
>>  	mpidr |= SGI_AFFINITY_LEVEL(reg, 2);
>> diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
>> index 0b3ecf9..80f92ce 100644
>> --- a/virt/kvm/arm/vgic/vgic-mmio.h
>> +++ b/virt/kvm/arm/vgic/vgic-mmio.h
>> @@ -96,7 +96,7 @@ unsigned long vgic_data_mmio_bus_to_host(const void *val, unsigned int len);
>>  void vgic_data_host_to_mmio_bus(void *buf, unsigned int len,
>>  				unsigned long data);
>>  
>> -unsigned long extract_bytes(unsigned long data, unsigned int offset,
>> +unsigned long extract_bytes(u64 data, unsigned int offset,
>>  			    unsigned int num);
>>  
>>  u64 update_64bit_reg(u64 reg, unsigned int offset, unsigned int len,
>>
> 
> My personal preference would be to split this in two patches. One that
> changes extract_bytes to work on 64bit quantities, and another one that
> addresses the 64bit issues. Not a big deal though.
> 

Anyway I have to resend, so I'll split it per your preference ;)

> Acked-by: Marc Zyngier <marc.zyngier@arm.com>
> 

Thanks!

Vladimir

> Thanks,
> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 5/8] KVM: arm: vgic: Support 64-bit data manipulation on 32-bit host systems
@ 2016-09-12  9:25       ` Vladimir Murzin
  0 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-12  9:25 UTC (permalink / raw)
  To: linux-arm-kernel

On 09/09/16 17:55, Marc Zyngier wrote:
> On 08/09/16 17:06, Vladimir Murzin wrote:
>> We have couple of 64-bit registers defined in GICv3 architecture, so
>> unsigned long accesses to these registers will only access a single
>> 32-bit part of that regitser. On the other hand these registers can't
>> be accessed as 64-bit with a single instruction like ldrd/strd or
>> ldmia/stmia if we run a 32-bit host because KVM does not support
>> access to MMIO space done by these instructions.
>>
>> It means that a 32-bit guest accesses these registers in 32-bit
>> chunks, so the only thing we need to do is to ensure that
>> extract_bytes() always takes 64-bit data.
>>
>> Since we are here fix couple of other width related issues catched by
>> gcc
>>
>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>> ---
>>  virt/kvm/arm/vgic/vgic-mmio-v3.c |    6 +++---
>>  virt/kvm/arm/vgic/vgic-mmio.h    |    2 +-
>>  2 files changed, 4 insertions(+), 4 deletions(-)
>>
>> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> index acbe691..0d3c76a 100644
>> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> @@ -23,7 +23,7 @@
>>  #include "vgic-mmio.h"
>>  
>>  /* extract @num bytes at @offset bytes offset in data */
>> -unsigned long extract_bytes(unsigned long data, unsigned int offset,
>> +unsigned long extract_bytes(u64 data, unsigned int offset,
>>  			    unsigned int num)
>>  {
>>  	return (data >> (offset * 8)) & GENMASK_ULL(num * 8 - 1, 0);
>> @@ -181,7 +181,7 @@ static unsigned long vgic_mmio_read_v3r_typer(struct kvm_vcpu *vcpu,
>>  	int target_vcpu_id = vcpu->vcpu_id;
>>  	u64 value;
>>  
>> -	value = (mpidr & GENMASK(23, 0)) << 32;
>> +	value = (u64)(mpidr & GENMASK(23, 0)) << 32;
>>  	value |= ((target_vcpu_id & 0xffff) << 8);
>>  	if (target_vcpu_id == atomic_read(&vcpu->kvm->online_vcpus) - 1)
>>  		value |= GICR_TYPER_LAST;
>> @@ -611,7 +611,7 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)
>>  	bool broadcast;
>>  
>>  	sgi = (reg & ICC_SGI1R_SGI_ID_MASK) >> ICC_SGI1R_SGI_ID_SHIFT;
>> -	broadcast = reg & BIT(ICC_SGI1R_IRQ_ROUTING_MODE_BIT);
>> +	broadcast = reg & BIT_ULL(ICC_SGI1R_IRQ_ROUTING_MODE_BIT);
>>  	target_cpus = (reg & ICC_SGI1R_TARGET_LIST_MASK) >> ICC_SGI1R_TARGET_LIST_SHIFT;
>>  	mpidr = SGI_AFFINITY_LEVEL(reg, 3);
>>  	mpidr |= SGI_AFFINITY_LEVEL(reg, 2);
>> diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
>> index 0b3ecf9..80f92ce 100644
>> --- a/virt/kvm/arm/vgic/vgic-mmio.h
>> +++ b/virt/kvm/arm/vgic/vgic-mmio.h
>> @@ -96,7 +96,7 @@ unsigned long vgic_data_mmio_bus_to_host(const void *val, unsigned int len);
>>  void vgic_data_host_to_mmio_bus(void *buf, unsigned int len,
>>  				unsigned long data);
>>  
>> -unsigned long extract_bytes(unsigned long data, unsigned int offset,
>> +unsigned long extract_bytes(u64 data, unsigned int offset,
>>  			    unsigned int num);
>>  
>>  u64 update_64bit_reg(u64 reg, unsigned int offset, unsigned int len,
>>
> 
> My personal preference would be to split this in two patches. One that
> changes extract_bytes to work on 64bit quantities, and another one that
> addresses the 64bit issues. Not a big deal though.
> 

Anyway I have to resend, so I'll split it per your preference ;)

> Acked-by: Marc Zyngier <marc.zyngier@arm.com>
> 

Thanks!

Vladimir

> Thanks,
> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 6/8] ARM: Change MPIDR_AFFINITY_LEVEL to ignore Aff3
  2016-09-09 16:59     ` Marc Zyngier
@ 2016-09-12  9:39       ` Vladimir Murzin
  -1 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-12  9:39 UTC (permalink / raw)
  To: Marc Zyngier, kvmarm; +Cc: andre.przywara, linux-arm-kernel

On 09/09/16 17:59, Marc Zyngier wrote:
> On 08/09/16 17:06, Vladimir Murzin wrote:
>> vgic-v3 driver queries CPU affinity level up to Aff3, which is valid
>> for arm64.  However, for arm up to Aff2 levels are supported, so
>> querying for third level ends with upper bits of MPIDR are treated as
>> valid affinity level which is not true. Make sure we report zero for
>> any affinity level above two.
>>
>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>> ---
>>  arch/arm/include/asm/cputype.h |    3 ++-
>>  1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/arm/include/asm/cputype.h b/arch/arm/include/asm/cputype.h
>> index 1ee94c7..f08fac4 100644
>> --- a/arch/arm/include/asm/cputype.h
>> +++ b/arch/arm/include/asm/cputype.h
>> @@ -55,9 +55,10 @@
>>  
>>  #define MPIDR_LEVEL_BITS 8
>>  #define MPIDR_LEVEL_MASK ((1 << MPIDR_LEVEL_BITS) - 1)
>> +#define MPIDR_LEVEL_SHIFT(level) (MPIDR_LEVEL_BITS * level)
>>  
>>  #define MPIDR_AFFINITY_LEVEL(mpidr, level) \
>> -	((mpidr >> (MPIDR_LEVEL_BITS * level)) & MPIDR_LEVEL_MASK)
>> +	(((mpidr & MPIDR_HWID_BITMASK) >> MPIDR_LEVEL_SHIFT(level)) & MPIDR_LEVEL_MASK)
>>  
>>  #define ARM_CPU_IMP_ARM			0x41
>>  #define ARM_CPU_IMP_INTEL		0x69
>>
> 
> There is something I don't quite get. Is this patch really necessary?
> Are there cases where we construct a MPIDR that can have Aff3 set on 32bit?

I've just checked and it seems that I've been keeping the change for
MPIDR_AFFINITY_LEVEL() since vgic-old era where it was used with

static u32 compress_mpidr(unsigned long mpidr)
{
        u32 ret;

        ret = MPIDR_AFFINITY_LEVEL(mpidr, 0);
        ret |= MPIDR_AFFINITY_LEVEL(mpidr, 1) << 8;
        ret |= MPIDR_AFFINITY_LEVEL(mpidr, 2) << 16;
        ret |= MPIDR_AFFINITY_LEVEL(mpidr, 3) << 24;

        return ret;
}

and now that code gone and nobody passes level 3 into this macro, so I
can drop this change.

However, I do need MPIDR_LEVEL_SHIFT() macro since it is used in vgic-v3.

Should I leave patch as is or you have something in mind?

Cheers
Vladimir

> 
> Thanks,
> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 6/8] ARM: Change MPIDR_AFFINITY_LEVEL to ignore Aff3
@ 2016-09-12  9:39       ` Vladimir Murzin
  0 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-12  9:39 UTC (permalink / raw)
  To: linux-arm-kernel

On 09/09/16 17:59, Marc Zyngier wrote:
> On 08/09/16 17:06, Vladimir Murzin wrote:
>> vgic-v3 driver queries CPU affinity level up to Aff3, which is valid
>> for arm64.  However, for arm up to Aff2 levels are supported, so
>> querying for third level ends with upper bits of MPIDR are treated as
>> valid affinity level which is not true. Make sure we report zero for
>> any affinity level above two.
>>
>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>> ---
>>  arch/arm/include/asm/cputype.h |    3 ++-
>>  1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/arm/include/asm/cputype.h b/arch/arm/include/asm/cputype.h
>> index 1ee94c7..f08fac4 100644
>> --- a/arch/arm/include/asm/cputype.h
>> +++ b/arch/arm/include/asm/cputype.h
>> @@ -55,9 +55,10 @@
>>  
>>  #define MPIDR_LEVEL_BITS 8
>>  #define MPIDR_LEVEL_MASK ((1 << MPIDR_LEVEL_BITS) - 1)
>> +#define MPIDR_LEVEL_SHIFT(level) (MPIDR_LEVEL_BITS * level)
>>  
>>  #define MPIDR_AFFINITY_LEVEL(mpidr, level) \
>> -	((mpidr >> (MPIDR_LEVEL_BITS * level)) & MPIDR_LEVEL_MASK)
>> +	(((mpidr & MPIDR_HWID_BITMASK) >> MPIDR_LEVEL_SHIFT(level)) & MPIDR_LEVEL_MASK)
>>  
>>  #define ARM_CPU_IMP_ARM			0x41
>>  #define ARM_CPU_IMP_INTEL		0x69
>>
> 
> There is something I don't quite get. Is this patch really necessary?
> Are there cases where we construct a MPIDR that can have Aff3 set on 32bit?

I've just checked and it seems that I've been keeping the change for
MPIDR_AFFINITY_LEVEL() since vgic-old era where it was used with

static u32 compress_mpidr(unsigned long mpidr)
{
        u32 ret;

        ret = MPIDR_AFFINITY_LEVEL(mpidr, 0);
        ret |= MPIDR_AFFINITY_LEVEL(mpidr, 1) << 8;
        ret |= MPIDR_AFFINITY_LEVEL(mpidr, 2) << 16;
        ret |= MPIDR_AFFINITY_LEVEL(mpidr, 3) << 24;

        return ret;
}

and now that code gone and nobody passes level 3 into this macro, so I
can drop this change.

However, I do need MPIDR_LEVEL_SHIFT() macro since it is used in vgic-v3.

Should I leave patch as is or you have something in mind?

Cheers
Vladimir

> 
> Thanks,
> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 7/8] ARM: Move system register accessors to asm/cp15.h
  2016-09-09 17:05     ` Marc Zyngier
@ 2016-09-12  9:42       ` Vladimir Murzin
  -1 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-12  9:42 UTC (permalink / raw)
  To: Marc Zyngier, kvmarm; +Cc: andre.przywara, linux-arm-kernel

On 09/09/16 18:05, Marc Zyngier wrote:
> On 08/09/16 17:06, Vladimir Murzin wrote:
>> Headers linux/irqchip/arm-gic.v3.h and arch/arm/include/asm/kvm_hyp.h
>> are included in virt/kvm/arm/hyp/vgic-v3-sr.c and both define macros
>> called __ACCESS_CP15 and __ACCESS_CP15_64 which obviously creates a
>> conflict. These macros were introduced independently for GIC and KVM
>> and, in fact, do the same thing.
>>
>> As an option we could add prefixes to KVM and GIC version of macros so
>> they won't clash, but it'd introduce code duplication.  Alternatively,
>> we could keep macro in, say, GIC header and include it in KVM one (or
>> vice versa), but such dependency would not look nicer.
>>
>> So we follow arm64 way (it handles this via sysreg.h) and move only
>> single set of macros to asm/cp15.h
>>
>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>> ---
>>  arch/arm/include/asm/arch_gicv3.h |   27 +++++++++++----------------
>>  arch/arm/include/asm/cp15.h       |   15 +++++++++++++++
>>  arch/arm/include/asm/kvm_hyp.h    |   15 +--------------
>>  3 files changed, 27 insertions(+), 30 deletions(-)
>>
>> diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
>> index e08d151..af25c32 100644
>> --- a/arch/arm/include/asm/arch_gicv3.h
>> +++ b/arch/arm/include/asm/arch_gicv3.h
>> @@ -22,9 +22,7 @@
>>  
>>  #include <linux/io.h>
>>  #include <asm/barrier.h>
>> -
>> -#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
>> -#define __ACCESS_CP15_64(Op1, CRm)		p15, Op1, %Q0, %R0, CRm
>> +#include <asm/cp15.h>
>>  
>>  #define ICC_EOIR1			__ACCESS_CP15(c12, 0, c12, 1)
>>  #define ICC_DIR				__ACCESS_CP15(c12, 0, c11, 1)
>> @@ -102,58 +100,55 @@
>>  
>>  static inline void gic_write_eoir(u32 irq)
>>  {
>> -	asm volatile("mcr " __stringify(ICC_EOIR1) : : "r" (irq));
>> +	write_sysreg(irq, ICC_EOIR1);
>>  	isb();
>>  }
>>  
>>  static inline void gic_write_dir(u32 val)
>>  {
>> -	asm volatile("mcr " __stringify(ICC_DIR) : : "r" (val));
>> +	write_sysreg(val, ICC_DIR);
>>  	isb();
>>  }
>>  
>>  static inline u32 gic_read_iar(void)
>>  {
>> -	u32 irqstat;
>> +	u32 irqstat = read_sysreg(ICC_IAR1);
>>  
>> -	asm volatile("mrc " __stringify(ICC_IAR1) : "=r" (irqstat));
>>  	dsb(sy);
>> +
>>  	return irqstat;
>>  }
>>  
>>  static inline void gic_write_pmr(u32 val)
>>  {
>> -	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
>> +	write_sysreg(val, ICC_PMR);
>>  }
>>  
>>  static inline void gic_write_ctlr(u32 val)
>>  {
>> -	asm volatile("mcr " __stringify(ICC_CTLR) : : "r" (val));
>> +	write_sysreg(val, ICC_CTLR);
>>  	isb();
>>  }
>>  
>>  static inline void gic_write_grpen1(u32 val)
>>  {
>> -	asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
>> +	write_sysreg(val, ICC_IGRPEN1);
>>  	isb();
>>  }
>>  
>>  static inline void gic_write_sgi1r(u64 val)
>>  {
>> -	asm volatile("mcrr " __stringify(ICC_SGI1R) : : "r" (val));
>> +	write_sysreg(val, ICC_SGI1R);
>>  }
>>  
>>  static inline u32 gic_read_sre(void)
>>  {
>> -	u32 val;
>> -
>> -	asm volatile("mrc " __stringify(ICC_SRE) : "=r" (val));
>> -	return val;
>> +	return read_sysreg(ICC_SRE);
>>  }
>>  
>>  static inline void gic_write_sre(u32 val)
>>  {
>> -	asm volatile("mcr " __stringify(ICC_SRE) : : "r" (val));
>> +	write_sysreg(val, ICC_SRE);
>>  	isb();
>>  }
>>  
>> diff --git a/arch/arm/include/asm/cp15.h b/arch/arm/include/asm/cp15.h
>> index c3f1152..f661732 100644
>> --- a/arch/arm/include/asm/cp15.h
>> +++ b/arch/arm/include/asm/cp15.h
>> @@ -47,6 +47,21 @@
>>  #define vectors_high()	(0)
>>  #endif
>>  
>> +#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
>> +	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
>> +#define __ACCESS_CP15_64(Op1, CRm)		\
>> +	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
>> +
>> +#define __read_sysreg(r, w, c, t) ({				\
>> +	t __val;						\
>> +	asm volatile(r " " c : "=r" (__val));			\
>> +	__val;							\
>> +})
>> +#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
>> +
>> +#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
>> +#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
>> +
> 
> Shouldn't that be placed after the #ifdef below?
> 
>>  #ifdef CONFIG_CPU_CP15
>>  
>>  extern unsigned long cr_alignment;	/* defined in entry-armv.S */
>> diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
>> index bd9434e..0b475d2 100644
>> --- a/arch/arm/include/asm/kvm_hyp.h
>> +++ b/arch/arm/include/asm/kvm_hyp.h
>> @@ -20,26 +20,13 @@
>>  
>>  #include <linux/compiler.h>
>>  #include <linux/kvm_host.h>
>> +#include <asm/cp15.h>
>>  #include <asm/kvm_mmu.h>
>>  #include <asm/vfp.h>
>>  
>> -#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
>> -	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
>> -#define __ACCESS_CP15_64(Op1, CRm)		\
>> -	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
>>  #define __ACCESS_VFP(CRn)			\
>>  	"mrc", "mcr", __stringify(p10, 7, %0, CRn, cr0, 0), u32
>>  
>> -#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
>> -#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
>> -
>> -#define __read_sysreg(r, w, c, t) ({				\
>> -	t __val;						\
>> -	asm volatile(r " " c : "=r" (__val));			\
>> -	__val;							\
>> -})
>> -#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
>> -
>>  #define write_special(v, r)					\
>>  	asm volatile("msr " __stringify(r) ", %0" : : "r" (v))
>>  #define read_special(r) ({					\
>>
> 
> Could you please cc RMK on this, given that this touches a core arch/arm
> file?

Ok. I'll cc/to him with the next re-spin.

Cheers
Vladimir

> 
> Thanks,
> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 7/8] ARM: Move system register accessors to asm/cp15.h
@ 2016-09-12  9:42       ` Vladimir Murzin
  0 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-12  9:42 UTC (permalink / raw)
  To: linux-arm-kernel

On 09/09/16 18:05, Marc Zyngier wrote:
> On 08/09/16 17:06, Vladimir Murzin wrote:
>> Headers linux/irqchip/arm-gic.v3.h and arch/arm/include/asm/kvm_hyp.h
>> are included in virt/kvm/arm/hyp/vgic-v3-sr.c and both define macros
>> called __ACCESS_CP15 and __ACCESS_CP15_64 which obviously creates a
>> conflict. These macros were introduced independently for GIC and KVM
>> and, in fact, do the same thing.
>>
>> As an option we could add prefixes to KVM and GIC version of macros so
>> they won't clash, but it'd introduce code duplication.  Alternatively,
>> we could keep macro in, say, GIC header and include it in KVM one (or
>> vice versa), but such dependency would not look nicer.
>>
>> So we follow arm64 way (it handles this via sysreg.h) and move only
>> single set of macros to asm/cp15.h
>>
>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>> ---
>>  arch/arm/include/asm/arch_gicv3.h |   27 +++++++++++----------------
>>  arch/arm/include/asm/cp15.h       |   15 +++++++++++++++
>>  arch/arm/include/asm/kvm_hyp.h    |   15 +--------------
>>  3 files changed, 27 insertions(+), 30 deletions(-)
>>
>> diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
>> index e08d151..af25c32 100644
>> --- a/arch/arm/include/asm/arch_gicv3.h
>> +++ b/arch/arm/include/asm/arch_gicv3.h
>> @@ -22,9 +22,7 @@
>>  
>>  #include <linux/io.h>
>>  #include <asm/barrier.h>
>> -
>> -#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
>> -#define __ACCESS_CP15_64(Op1, CRm)		p15, Op1, %Q0, %R0, CRm
>> +#include <asm/cp15.h>
>>  
>>  #define ICC_EOIR1			__ACCESS_CP15(c12, 0, c12, 1)
>>  #define ICC_DIR				__ACCESS_CP15(c12, 0, c11, 1)
>> @@ -102,58 +100,55 @@
>>  
>>  static inline void gic_write_eoir(u32 irq)
>>  {
>> -	asm volatile("mcr " __stringify(ICC_EOIR1) : : "r" (irq));
>> +	write_sysreg(irq, ICC_EOIR1);
>>  	isb();
>>  }
>>  
>>  static inline void gic_write_dir(u32 val)
>>  {
>> -	asm volatile("mcr " __stringify(ICC_DIR) : : "r" (val));
>> +	write_sysreg(val, ICC_DIR);
>>  	isb();
>>  }
>>  
>>  static inline u32 gic_read_iar(void)
>>  {
>> -	u32 irqstat;
>> +	u32 irqstat = read_sysreg(ICC_IAR1);
>>  
>> -	asm volatile("mrc " __stringify(ICC_IAR1) : "=r" (irqstat));
>>  	dsb(sy);
>> +
>>  	return irqstat;
>>  }
>>  
>>  static inline void gic_write_pmr(u32 val)
>>  {
>> -	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
>> +	write_sysreg(val, ICC_PMR);
>>  }
>>  
>>  static inline void gic_write_ctlr(u32 val)
>>  {
>> -	asm volatile("mcr " __stringify(ICC_CTLR) : : "r" (val));
>> +	write_sysreg(val, ICC_CTLR);
>>  	isb();
>>  }
>>  
>>  static inline void gic_write_grpen1(u32 val)
>>  {
>> -	asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
>> +	write_sysreg(val, ICC_IGRPEN1);
>>  	isb();
>>  }
>>  
>>  static inline void gic_write_sgi1r(u64 val)
>>  {
>> -	asm volatile("mcrr " __stringify(ICC_SGI1R) : : "r" (val));
>> +	write_sysreg(val, ICC_SGI1R);
>>  }
>>  
>>  static inline u32 gic_read_sre(void)
>>  {
>> -	u32 val;
>> -
>> -	asm volatile("mrc " __stringify(ICC_SRE) : "=r" (val));
>> -	return val;
>> +	return read_sysreg(ICC_SRE);
>>  }
>>  
>>  static inline void gic_write_sre(u32 val)
>>  {
>> -	asm volatile("mcr " __stringify(ICC_SRE) : : "r" (val));
>> +	write_sysreg(val, ICC_SRE);
>>  	isb();
>>  }
>>  
>> diff --git a/arch/arm/include/asm/cp15.h b/arch/arm/include/asm/cp15.h
>> index c3f1152..f661732 100644
>> --- a/arch/arm/include/asm/cp15.h
>> +++ b/arch/arm/include/asm/cp15.h
>> @@ -47,6 +47,21 @@
>>  #define vectors_high()	(0)
>>  #endif
>>  
>> +#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
>> +	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
>> +#define __ACCESS_CP15_64(Op1, CRm)		\
>> +	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
>> +
>> +#define __read_sysreg(r, w, c, t) ({				\
>> +	t __val;						\
>> +	asm volatile(r " " c : "=r" (__val));			\
>> +	__val;							\
>> +})
>> +#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
>> +
>> +#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
>> +#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
>> +
> 
> Shouldn't that be placed after the #ifdef below?
> 
>>  #ifdef CONFIG_CPU_CP15
>>  
>>  extern unsigned long cr_alignment;	/* defined in entry-armv.S */
>> diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
>> index bd9434e..0b475d2 100644
>> --- a/arch/arm/include/asm/kvm_hyp.h
>> +++ b/arch/arm/include/asm/kvm_hyp.h
>> @@ -20,26 +20,13 @@
>>  
>>  #include <linux/compiler.h>
>>  #include <linux/kvm_host.h>
>> +#include <asm/cp15.h>
>>  #include <asm/kvm_mmu.h>
>>  #include <asm/vfp.h>
>>  
>> -#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
>> -	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
>> -#define __ACCESS_CP15_64(Op1, CRm)		\
>> -	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
>>  #define __ACCESS_VFP(CRn)			\
>>  	"mrc", "mcr", __stringify(p10, 7, %0, CRn, cr0, 0), u32
>>  
>> -#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
>> -#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
>> -
>> -#define __read_sysreg(r, w, c, t) ({				\
>> -	t __val;						\
>> -	asm volatile(r " " c : "=r" (__val));			\
>> -	__val;							\
>> -})
>> -#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
>> -
>>  #define write_special(v, r)					\
>>  	asm volatile("msr " __stringify(r) ", %0" : : "r" (v))
>>  #define read_special(r) ({					\
>>
> 
> Could you please cc RMK on this, given that this touches a core arch/arm
> file?

Ok. I'll cc/to him with the next re-spin.

Cheers
Vladimir

> 
> Thanks,
> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 7/8] ARM: Move system register accessors to asm/cp15.h
  2016-09-09 17:05     ` Marc Zyngier
@ 2016-09-12  9:44       ` Vladimir Murzin
  -1 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-12  9:44 UTC (permalink / raw)
  To: Marc Zyngier, kvmarm; +Cc: andre.przywara, linux-arm-kernel

Sorry, missed this part...

On 09/09/16 18:05, Marc Zyngier wrote:
>>  
>> > +#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
>> > +	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
>> > +#define __ACCESS_CP15_64(Op1, CRm)		\
>> > +	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
>> > +
>> > +#define __read_sysreg(r, w, c, t) ({				\
>> > +	t __val;						\
>> > +	asm volatile(r " " c : "=r" (__val));			\
>> > +	__val;							\
>> > +})
>> > +#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
>> > +
>> > +#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
>> > +#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
>> > +
> Shouldn't that be placed after the #ifdef below?
> 

Yes, I'll move it under #ifdef.

Thanks
Vladimir

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 7/8] ARM: Move system register accessors to asm/cp15.h
@ 2016-09-12  9:44       ` Vladimir Murzin
  0 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-12  9:44 UTC (permalink / raw)
  To: linux-arm-kernel

Sorry, missed this part...

On 09/09/16 18:05, Marc Zyngier wrote:
>>  
>> > +#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
>> > +	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
>> > +#define __ACCESS_CP15_64(Op1, CRm)		\
>> > +	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
>> > +
>> > +#define __read_sysreg(r, w, c, t) ({				\
>> > +	t __val;						\
>> > +	asm volatile(r " " c : "=r" (__val));			\
>> > +	__val;							\
>> > +})
>> > +#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
>> > +
>> > +#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
>> > +#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
>> > +
> Shouldn't that be placed after the #ifdef below?
> 

Yes, I'll move it under #ifdef.

Thanks
Vladimir

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 6/8] ARM: Change MPIDR_AFFINITY_LEVEL to ignore Aff3
  2016-09-12  9:39       ` Vladimir Murzin
@ 2016-09-12  9:48         ` Marc Zyngier
  -1 siblings, 0 replies; 72+ messages in thread
From: Marc Zyngier @ 2016-09-12  9:48 UTC (permalink / raw)
  To: Vladimir Murzin, kvmarm; +Cc: andre.przywara, linux-arm-kernel

On 12/09/16 10:39, Vladimir Murzin wrote:
> On 09/09/16 17:59, Marc Zyngier wrote:
>> On 08/09/16 17:06, Vladimir Murzin wrote:
>>> vgic-v3 driver queries CPU affinity level up to Aff3, which is valid
>>> for arm64.  However, for arm up to Aff2 levels are supported, so
>>> querying for third level ends with upper bits of MPIDR are treated as
>>> valid affinity level which is not true. Make sure we report zero for
>>> any affinity level above two.
>>>
>>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>>> ---
>>>  arch/arm/include/asm/cputype.h |    3 ++-
>>>  1 file changed, 2 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/arch/arm/include/asm/cputype.h b/arch/arm/include/asm/cputype.h
>>> index 1ee94c7..f08fac4 100644
>>> --- a/arch/arm/include/asm/cputype.h
>>> +++ b/arch/arm/include/asm/cputype.h
>>> @@ -55,9 +55,10 @@
>>>  
>>>  #define MPIDR_LEVEL_BITS 8
>>>  #define MPIDR_LEVEL_MASK ((1 << MPIDR_LEVEL_BITS) - 1)
>>> +#define MPIDR_LEVEL_SHIFT(level) (MPIDR_LEVEL_BITS * level)
>>>  
>>>  #define MPIDR_AFFINITY_LEVEL(mpidr, level) \
>>> -	((mpidr >> (MPIDR_LEVEL_BITS * level)) & MPIDR_LEVEL_MASK)
>>> +	(((mpidr & MPIDR_HWID_BITMASK) >> MPIDR_LEVEL_SHIFT(level)) & MPIDR_LEVEL_MASK)
>>>  
>>>  #define ARM_CPU_IMP_ARM			0x41
>>>  #define ARM_CPU_IMP_INTEL		0x69
>>>
>>
>> There is something I don't quite get. Is this patch really necessary?
>> Are there cases where we construct a MPIDR that can have Aff3 set on 32bit?
> 
> I've just checked and it seems that I've been keeping the change for
> MPIDR_AFFINITY_LEVEL() since vgic-old era where it was used with
> 
> static u32 compress_mpidr(unsigned long mpidr)
> {
>         u32 ret;
> 
>         ret = MPIDR_AFFINITY_LEVEL(mpidr, 0);
>         ret |= MPIDR_AFFINITY_LEVEL(mpidr, 1) << 8;
>         ret |= MPIDR_AFFINITY_LEVEL(mpidr, 2) << 16;
>         ret |= MPIDR_AFFINITY_LEVEL(mpidr, 3) << 24;
> 
>         return ret;
> }
> 
> and now that code gone and nobody passes level 3 into this macro, so I
> can drop this change.
> 
> However, I do need MPIDR_LEVEL_SHIFT() macro since it is used in vgic-v3.
> 
> Should I leave patch as is or you have something in mind?

I think that for the sake of keeping the change minimal, it'd be better
to just add the MPIDR_LEVEL_SHIFT macro, and drop the other changes.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 6/8] ARM: Change MPIDR_AFFINITY_LEVEL to ignore Aff3
@ 2016-09-12  9:48         ` Marc Zyngier
  0 siblings, 0 replies; 72+ messages in thread
From: Marc Zyngier @ 2016-09-12  9:48 UTC (permalink / raw)
  To: linux-arm-kernel

On 12/09/16 10:39, Vladimir Murzin wrote:
> On 09/09/16 17:59, Marc Zyngier wrote:
>> On 08/09/16 17:06, Vladimir Murzin wrote:
>>> vgic-v3 driver queries CPU affinity level up to Aff3, which is valid
>>> for arm64.  However, for arm up to Aff2 levels are supported, so
>>> querying for third level ends with upper bits of MPIDR are treated as
>>> valid affinity level which is not true. Make sure we report zero for
>>> any affinity level above two.
>>>
>>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>>> ---
>>>  arch/arm/include/asm/cputype.h |    3 ++-
>>>  1 file changed, 2 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/arch/arm/include/asm/cputype.h b/arch/arm/include/asm/cputype.h
>>> index 1ee94c7..f08fac4 100644
>>> --- a/arch/arm/include/asm/cputype.h
>>> +++ b/arch/arm/include/asm/cputype.h
>>> @@ -55,9 +55,10 @@
>>>  
>>>  #define MPIDR_LEVEL_BITS 8
>>>  #define MPIDR_LEVEL_MASK ((1 << MPIDR_LEVEL_BITS) - 1)
>>> +#define MPIDR_LEVEL_SHIFT(level) (MPIDR_LEVEL_BITS * level)
>>>  
>>>  #define MPIDR_AFFINITY_LEVEL(mpidr, level) \
>>> -	((mpidr >> (MPIDR_LEVEL_BITS * level)) & MPIDR_LEVEL_MASK)
>>> +	(((mpidr & MPIDR_HWID_BITMASK) >> MPIDR_LEVEL_SHIFT(level)) & MPIDR_LEVEL_MASK)
>>>  
>>>  #define ARM_CPU_IMP_ARM			0x41
>>>  #define ARM_CPU_IMP_INTEL		0x69
>>>
>>
>> There is something I don't quite get. Is this patch really necessary?
>> Are there cases where we construct a MPIDR that can have Aff3 set on 32bit?
> 
> I've just checked and it seems that I've been keeping the change for
> MPIDR_AFFINITY_LEVEL() since vgic-old era where it was used with
> 
> static u32 compress_mpidr(unsigned long mpidr)
> {
>         u32 ret;
> 
>         ret = MPIDR_AFFINITY_LEVEL(mpidr, 0);
>         ret |= MPIDR_AFFINITY_LEVEL(mpidr, 1) << 8;
>         ret |= MPIDR_AFFINITY_LEVEL(mpidr, 2) << 16;
>         ret |= MPIDR_AFFINITY_LEVEL(mpidr, 3) << 24;
> 
>         return ret;
> }
> 
> and now that code gone and nobody passes level 3 into this macro, so I
> can drop this change.
> 
> However, I do need MPIDR_LEVEL_SHIFT() macro since it is used in vgic-v3.
> 
> Should I leave patch as is or you have something in mind?

I think that for the sake of keeping the change minimal, it'd be better
to just add the MPIDR_LEVEL_SHIFT macro, and drop the other changes.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 72+ messages in thread
* Re: [PATCH v3 6/8] ARM: Change MPIDR_AFFINITY_LEVEL to ignore Aff3
  2016-09-12  9:48         ` Marc Zyngier
@ 2016-09-12  9:51           ` Vladimir Murzin
  -1 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-12  9:51 UTC (permalink / raw)
  To: Marc Zyngier, kvmarm; +Cc: andre.przywara, linux-arm-kernel

On 12/09/16 10:48, Marc Zyngier wrote:
> On 12/09/16 10:39, Vladimir Murzin wrote:
>> On 09/09/16 17:59, Marc Zyngier wrote:
>>> On 08/09/16 17:06, Vladimir Murzin wrote:
>>>> vgic-v3 driver queries CPU affinity level up to Aff3, which is valid
>>>> for arm64.  However, for arm up to Aff2 levels are supported, so
>>>> querying for third level ends with upper bits of MPIDR are treated as
>>>> valid affinity level which is not true. Make sure we report zero for
>>>> any affinity level above two.
>>>>
>>>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>>>> ---
>>>>  arch/arm/include/asm/cputype.h |    3 ++-
>>>>  1 file changed, 2 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/arch/arm/include/asm/cputype.h b/arch/arm/include/asm/cputype.h
>>>> index 1ee94c7..f08fac4 100644
>>>> --- a/arch/arm/include/asm/cputype.h
>>>> +++ b/arch/arm/include/asm/cputype.h
>>>> @@ -55,9 +55,10 @@
>>>>  
>>>>  #define MPIDR_LEVEL_BITS 8
>>>>  #define MPIDR_LEVEL_MASK ((1 << MPIDR_LEVEL_BITS) - 1)
>>>> +#define MPIDR_LEVEL_SHIFT(level) (MPIDR_LEVEL_BITS * level)
>>>>  
>>>>  #define MPIDR_AFFINITY_LEVEL(mpidr, level) \
>>>> -	((mpidr >> (MPIDR_LEVEL_BITS * level)) & MPIDR_LEVEL_MASK)
>>>> +	(((mpidr & MPIDR_HWID_BITMASK) >> MPIDR_LEVEL_SHIFT(level)) & MPIDR_LEVEL_MASK)
>>>>  
>>>>  #define ARM_CPU_IMP_ARM			0x41
>>>>  #define ARM_CPU_IMP_INTEL		0x69
>>>>
>>>
>>> There is something I don't quite get. Is this patch really necessary?
>>> Are there cases where we construct a MPIDR that can have Aff3 set on 32bit?
>>
>> I've just checked and it seems that I've been keeping the change for
>> MPIDR_AFFINITY_LEVEL() since vgic-old era where it was used with
>>
>> static u32 compress_mpidr(unsigned long mpidr)
>> {
>>         u32 ret;
>>
>>         ret = MPIDR_AFFINITY_LEVEL(mpidr, 0);
>>         ret |= MPIDR_AFFINITY_LEVEL(mpidr, 1) << 8;
>>         ret |= MPIDR_AFFINITY_LEVEL(mpidr, 2) << 16;
>>         ret |= MPIDR_AFFINITY_LEVEL(mpidr, 3) << 24;
>>
>>         return ret;
>> }
>>
>> and now that code gone and nobody passes level 3 into this macro, so I
>> can drop this change.
>>
>> However, I do need MPIDR_LEVEL_SHIFT() macro since it is used in vgic-v3.
>>
>> Should I leave patch as is or you have something in mind?
> 
> I think that for the sake of keeping the change minimal, it'd be better
> to just add the MPIDR_LEVEL_SHIFT macro, and drop the other changes.

Noted!

Thanks
Vladimir

> 
> Thanks,
> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread
* [PATCH v3 6/8] ARM: Change MPIDR_AFFINITY_LEVEL to ignore Aff3
@ 2016-09-12  9:51           ` Vladimir Murzin
  0 siblings, 0 replies; 72+ messages in thread
From: Vladimir Murzin @ 2016-09-12  9:51 UTC (permalink / raw)
  To: linux-arm-kernel

On 12/09/16 10:48, Marc Zyngier wrote:
> On 12/09/16 10:39, Vladimir Murzin wrote:
>> On 09/09/16 17:59, Marc Zyngier wrote:
>>> On 08/09/16 17:06, Vladimir Murzin wrote:
>>>> vgic-v3 driver queries CPU affinity level up to Aff3, which is valid
>>>> for arm64.  However, for arm up to Aff2 levels are supported, so
>>>> querying for third level ends with upper bits of MPIDR are treated as
>>>> valid affinity level which is not true. Make sure we report zero for
>>>> any affinity level above two.
>>>>
>>>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>>>> ---
>>>>  arch/arm/include/asm/cputype.h |    3 ++-
>>>>  1 file changed, 2 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/arch/arm/include/asm/cputype.h b/arch/arm/include/asm/cputype.h
>>>> index 1ee94c7..f08fac4 100644
>>>> --- a/arch/arm/include/asm/cputype.h
>>>> +++ b/arch/arm/include/asm/cputype.h
>>>> @@ -55,9 +55,10 @@
>>>>  
>>>>  #define MPIDR_LEVEL_BITS 8
>>>>  #define MPIDR_LEVEL_MASK ((1 << MPIDR_LEVEL_BITS) - 1)
>>>> +#define MPIDR_LEVEL_SHIFT(level) (MPIDR_LEVEL_BITS * level)
>>>>  
>>>>  #define MPIDR_AFFINITY_LEVEL(mpidr, level) \
>>>> -	((mpidr >> (MPIDR_LEVEL_BITS * level)) & MPIDR_LEVEL_MASK)
>>>> +	(((mpidr & MPIDR_HWID_BITMASK) >> MPIDR_LEVEL_SHIFT(level)) & MPIDR_LEVEL_MASK)
>>>>  
>>>>  #define ARM_CPU_IMP_ARM			0x41
>>>>  #define ARM_CPU_IMP_INTEL		0x69
>>>>
>>>
>>> There is something I don't quite get. Is this patch really necessary?
>>> Are there cases where we construct a MPIDR that can have Aff3 set on 32bit?
>>
>> I've just checked and it seems that I've been keeping the change for
>> MPIDR_AFFINITY_LEVEL() since vgic-old era where it was used with
>>
>> static u32 compress_mpidr(unsigned long mpidr)
>> {
>>         u32 ret;
>>
>>         ret = MPIDR_AFFINITY_LEVEL(mpidr, 0);
>>         ret |= MPIDR_AFFINITY_LEVEL(mpidr, 1) << 8;
>>         ret |= MPIDR_AFFINITY_LEVEL(mpidr, 2) << 16;
>>         ret |= MPIDR_AFFINITY_LEVEL(mpidr, 3) << 24;
>>
>>         return ret;
>> }
>>
>> and now that code gone and nobody passes level 3 into this macro, so I
>> can drop this change.
>>
>> However, I do need MPIDR_LEVEL_SHIFT() macro since it is used in vgic-v3.
>>
>> Should I leave patch as is or you have something in mind?
> 
> I think that for the sake of keeping the change minimal, it'd be better
> to just add the MPIDR_LEVEL_SHIFT macro, and drop the other changes.

Noted!

Thanks
Vladimir

> 
> Thanks,
> 
> 	M.
> 

^ permalink raw reply	[flat|nested] 72+ messages in thread
end of thread, other threads:[~2016-09-12  9:51 UTC | newest]

Thread overview: 72+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-09-08 16:06 [PATCH v3 0/8] ARM: KVM: Support for vgic-v3 Vladimir Murzin
2016-09-08 16:06 ` Vladimir Murzin
2016-09-08 16:06 ` [PATCH v3 1/8] arm64: KVM: Use static keys for selecting the GIC backend Vladimir Murzin
2016-09-08 16:06   ` Vladimir Murzin
2016-09-09  9:19   ` Marc Zyngier
2016-09-09  9:19     ` Marc Zyngier
2016-09-09  9:33     ` Vladimir Murzin
2016-09-09  9:33       ` Vladimir Murzin
2016-09-09 13:45       ` Vladimir Murzin
2016-09-09 13:45         ` Vladimir Murzin
2016-09-09 14:17         ` Marc Zyngier
2016-09-09 14:17           ` Marc Zyngier
2016-09-09 15:14           ` Vladimir Murzin
2016-09-09 15:14             ` Vladimir Murzin
2016-09-09 15:25             ` Marc Zyngier
2016-09-09 15:25               ` Marc Zyngier
2016-09-09 16:18               ` Vladimir Murzin
2016-09-09 16:18                 ` Vladimir Murzin
2016-09-09 17:06                 ` Marc Zyngier
2016-09-09 17:06                   ` Marc Zyngier
2016-09-08 16:06 ` [PATCH v3 2/8] arm64: KVM: Move GIC accessors to arch_gicv3.h Vladimir Murzin
2016-09-08 16:06   ` Vladimir Murzin
2016-09-09 16:32   ` Marc Zyngier
2016-09-09 16:32     ` Marc Zyngier
2016-09-12  9:18     ` Vladimir Murzin
2016-09-12  9:18       ` Vladimir Murzin
2016-09-08 16:06 ` [PATCH v3 3/8] arm64: KVM: Move vgic-v3 save/restore to virt/kvm/arm/hyp Vladimir Murzin
2016-09-08 16:06   ` Vladimir Murzin
2016-09-09 16:33   ` Marc Zyngier
2016-09-09 16:33     ` Marc Zyngier
2016-09-12  9:18     ` Vladimir Murzin
2016-09-12  9:18       ` Vladimir Murzin
2016-09-08 16:06 ` [PATCH v3 4/8] KVM: arm64: vgic-its: Introduce config option to guard ITS specific code Vladimir Murzin
2016-09-08 16:06   ` Vladimir Murzin
2016-09-09 16:46   ` Marc Zyngier
2016-09-09 16:46     ` Marc Zyngier
2016-09-12  9:23     ` Vladimir Murzin
2016-09-12  9:23       ` Vladimir Murzin
2016-09-08 16:06 ` [PATCH v3 5/8] KVM: arm: vgic: Support 64-bit data manipulation on 32-bit host systems Vladimir Murzin
2016-09-08 16:06   ` Vladimir Murzin
2016-09-09 16:55   ` Marc Zyngier
2016-09-09 16:55     ` Marc Zyngier
2016-09-12  9:25     ` Vladimir Murzin
2016-09-12  9:25       ` Vladimir Murzin
2016-09-08 16:06 ` [PATCH v3 6/8] ARM: Change MPIDR_AFFINITY_LEVEL to ignore Aff3 Vladimir Murzin
2016-09-08 16:06   ` Vladimir Murzin
2016-09-09 16:59   ` Marc Zyngier
2016-09-09 16:59     ` Marc Zyngier
2016-09-12  9:39     ` Vladimir Murzin
2016-09-12  9:39       ` Vladimir Murzin
2016-09-12  9:48       ` Marc Zyngier
2016-09-12  9:48         ` Marc Zyngier
2016-09-12  9:51         ` Vladimir Murzin
2016-09-12  9:51           ` Vladimir Murzin
2016-09-08 16:06 ` [PATCH v3 7/8] ARM: Move system register accessors to asm/cp15.h Vladimir Murzin
2016-09-08 16:06   ` Vladimir Murzin
2016-09-09 17:05   ` Marc Zyngier
2016-09-09 17:05     ` Marc Zyngier
2016-09-12  9:42     ` Vladimir Murzin
2016-09-12  9:42       ` Vladimir Murzin
2016-09-12  9:44     ` Vladimir Murzin
2016-09-12  9:44       ` Vladimir Murzin
2016-09-08 16:06 ` [PATCH v3 8/8] ARM: KVM: Support vgic-v3 Vladimir Murzin
2016-09-08 16:06   ` Vladimir Murzin
2016-09-09 16:45   ` Marc Zyngier
2016-09-09 16:45     ` Marc Zyngier
2016-09-12  9:23     ` Vladimir Murzin
2016-09-12  9:23       ` Vladimir Murzin
2016-09-09  7:58 ` [PATCH v3 0/8] ARM: KVM: Support for vgic-v3 Vladimir Murzin
2016-09-09  7:58   ` Vladimir Murzin
2016-09-09 11:26   ` Christoffer Dall
2016-09-09 11:26     ` Christoffer Dall

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.