All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v2 0/7] ARM: KVM: Support for vgic-v3
@ 2016-08-16 10:46 ` Vladimir Murzin
  0 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-08-16 10:46 UTC (permalink / raw)
  To: kvmarm, linux-arm-kernel; +Cc: marc.zyngier, andre.przywara

Hi,

This is an attempt to make use vgic-v3 under arch/arm since save-restore
functionality got re-written in C and can be shared between arm/arm64
like it has already been done for vgic-v2 and timer.

With this patches I'm able to get 32 core ARMv7 guest boot:

...
GICv3: CPU31: found redistributor 703 region 0:0x000000003ffd0000
CPU31: thread -1, cpu 3, socket 7, mpidr 80000703
Brought up 32 CPUs
SMP: Total of 32 processors activated (768.00 BogoMIPS).
CPU: All CPU(s) started in SVC mode.
...

Additionally, quite lightweight test based on Self IPI guest test[1]
has been run with up to 255 cpus.

[1] http://www.spinics.net/lists/kvm/msg128974.html

Changelog:

    v1 -> v2
       - rebased on v4.8-rc2
       - introduced guard for ITS code

Thanks!

Vladimir Murzin (7):
  arm64: KVM: Move GIC accessors to arch_gicv3.h
  arm64: KVM: Move vgic-v3 save/restore to virt/kvm/arm/hyp
  KVM: arm: vgic-new: improve compatibility with 32-bit
  ARM: update MPIDR accessors macro
  ARM: move system register accessors to asm/cp15.h
  ARM: KVM: Get ready to use vgic-v3
  ARM: KVM: Unlock vgic-v3 support

 arch/arm/include/asm/arch_gicv3.h                 |   91 +++++++++++++++++----
 arch/arm/include/asm/cp15.h                       |   15 ++++
 arch/arm/include/asm/cputype.h                    |    3 +-
 arch/arm/include/asm/kvm_asm.h                    |    3 +
 arch/arm/include/asm/kvm_host.h                   |    4 +
 arch/arm/include/asm/kvm_hyp.h                    |   20 ++---
 arch/arm/include/uapi/asm/kvm.h                   |    7 ++
 arch/arm/kvm/Makefile                             |    2 +
 arch/arm/kvm/coproc.c                             |   36 ++++++++
 arch/arm/kvm/hyp/Makefile                         |    1 +
 arch/arm/kvm/hyp/switch.c                         |   20 ++++-
 arch/arm64/include/asm/arch_gicv3.h               |   13 +++
 arch/arm64/kvm/Kconfig                            |    4 +-
 arch/arm64/kvm/hyp/Makefile                       |    2 +-
 include/kvm/arm_vgic.h                            |    8 --
 {arch/arm64/kvm => virt/kvm/arm}/hyp/vgic-v3-sr.c |   13 ---
 virt/kvm/arm/vgic/vgic-kvm-device.c               |   10 +--
 virt/kvm/arm/vgic/vgic-mmio-v3.c                  |    8 +-
 virt/kvm/arm/vgic/vgic-mmio.c                     |    2 -
 virt/kvm/arm/vgic/vgic-mmio.h                     |    5 +-
 virt/kvm/arm/vgic/vgic.h                          |   54 +-----------
 21 files changed, 195 insertions(+), 126 deletions(-)
 rename {arch/arm64/kvm => virt/kvm/arm}/hyp/vgic-v3-sr.c (96%)

-- 
1.7.9.5

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 0/7] ARM: KVM: Support for vgic-v3
@ 2016-08-16 10:46 ` Vladimir Murzin
  0 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-08-16 10:46 UTC (permalink / raw)
  To: linux-arm-kernel

Hi,

This is an attempt to make use vgic-v3 under arch/arm since save-restore
functionality got re-written in C and can be shared between arm/arm64
like it has already been done for vgic-v2 and timer.

With this patches I'm able to get 32 core ARMv7 guest boot:

...
GICv3: CPU31: found redistributor 703 region 0:0x000000003ffd0000
CPU31: thread -1, cpu 3, socket 7, mpidr 80000703
Brought up 32 CPUs
SMP: Total of 32 processors activated (768.00 BogoMIPS).
CPU: All CPU(s) started in SVC mode.
...

Additionally, quite lightweight test based on Self IPI guest test[1]
has been run with up to 255 cpus.

[1] http://www.spinics.net/lists/kvm/msg128974.html

Changelog:

    v1 -> v2
       - rebased on v4.8-rc2
       - introduced guard for ITS code

Thanks!

Vladimir Murzin (7):
  arm64: KVM: Move GIC accessors to arch_gicv3.h
  arm64: KVM: Move vgic-v3 save/restore to virt/kvm/arm/hyp
  KVM: arm: vgic-new: improve compatibility with 32-bit
  ARM: update MPIDR accessors macro
  ARM: move system register accessors to asm/cp15.h
  ARM: KVM: Get ready to use vgic-v3
  ARM: KVM: Unlock vgic-v3 support

 arch/arm/include/asm/arch_gicv3.h                 |   91 +++++++++++++++++----
 arch/arm/include/asm/cp15.h                       |   15 ++++
 arch/arm/include/asm/cputype.h                    |    3 +-
 arch/arm/include/asm/kvm_asm.h                    |    3 +
 arch/arm/include/asm/kvm_host.h                   |    4 +
 arch/arm/include/asm/kvm_hyp.h                    |   20 ++---
 arch/arm/include/uapi/asm/kvm.h                   |    7 ++
 arch/arm/kvm/Makefile                             |    2 +
 arch/arm/kvm/coproc.c                             |   36 ++++++++
 arch/arm/kvm/hyp/Makefile                         |    1 +
 arch/arm/kvm/hyp/switch.c                         |   20 ++++-
 arch/arm64/include/asm/arch_gicv3.h               |   13 +++
 arch/arm64/kvm/Kconfig                            |    4 +-
 arch/arm64/kvm/hyp/Makefile                       |    2 +-
 include/kvm/arm_vgic.h                            |    8 --
 {arch/arm64/kvm => virt/kvm/arm}/hyp/vgic-v3-sr.c |   13 ---
 virt/kvm/arm/vgic/vgic-kvm-device.c               |   10 +--
 virt/kvm/arm/vgic/vgic-mmio-v3.c                  |    8 +-
 virt/kvm/arm/vgic/vgic-mmio.c                     |    2 -
 virt/kvm/arm/vgic/vgic-mmio.h                     |    5 +-
 virt/kvm/arm/vgic/vgic.h                          |   54 +-----------
 21 files changed, 195 insertions(+), 126 deletions(-)
 rename {arch/arm64/kvm => virt/kvm/arm}/hyp/vgic-v3-sr.c (96%)

-- 
1.7.9.5

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 1/7] arm64: KVM: Move GIC accessors to arch_gicv3.h
  2016-08-16 10:46 ` Vladimir Murzin
@ 2016-08-16 10:46   ` Vladimir Murzin
  -1 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-08-16 10:46 UTC (permalink / raw)
  To: kvmarm, linux-arm-kernel; +Cc: marc.zyngier, andre.przywara

Since we are going to share vgic-v3 save/restore code with ARM
keep arch specific accessors separately.

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 arch/arm64/include/asm/arch_gicv3.h |   13 +++++++++++++
 arch/arm64/kvm/hyp/vgic-v3-sr.c     |   13 -------------
 2 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h
index 8ec88e5..ae7dbd7 100644
--- a/arch/arm64/include/asm/arch_gicv3.h
+++ b/arch/arm64/include/asm/arch_gicv3.h
@@ -79,6 +79,19 @@
 #include <linux/stringify.h>
 #include <asm/barrier.h>
 
+#define read_gicreg(r)							\
+	({								\
+		u64 reg;						\
+		asm volatile("mrs_s %0, " __stringify(r) : "=r" (reg));	\
+		reg;							\
+	})
+
+#define write_gicreg(v,r)						\
+	do {								\
+		u64 __val = (v);					\
+		asm volatile("msr_s " __stringify(r) ", %0" : : "r" (__val));\
+	} while (0)
+
 /*
  * Low-level accessors
  *
diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
index 5f8f80b..f2dbd2e 100644
--- a/arch/arm64/kvm/hyp/vgic-v3-sr.c
+++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
@@ -24,19 +24,6 @@
 #define vtr_to_max_lr_idx(v)		((v) & 0xf)
 #define vtr_to_nr_pri_bits(v)		(((u32)(v) >> 29) + 1)
 
-#define read_gicreg(r)							\
-	({								\
-		u64 reg;						\
-		asm volatile("mrs_s %0, " __stringify(r) : "=r" (reg));	\
-		reg;							\
-	})
-
-#define write_gicreg(v,r)						\
-	do {								\
-		u64 __val = (v);					\
-		asm volatile("msr_s " __stringify(r) ", %0" : : "r" (__val));\
-	} while (0)
-
 static u64 __hyp_text __gic_v3_get_lr(unsigned int lr)
 {
 	switch (lr & 0xf) {
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 74+ messages in thread
* [PATCH v2 1/7] arm64: KVM: Move GIC accessors to arch_gicv3.h
@ 2016-08-16 10:46   ` Vladimir Murzin
  0 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-08-16 10:46 UTC (permalink / raw)
  To: linux-arm-kernel

Since we are going to share vgic-v3 save/restore code with ARM
keep arch specific accessors separately.

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 arch/arm64/include/asm/arch_gicv3.h |   13 +++++++++++++
 arch/arm64/kvm/hyp/vgic-v3-sr.c     |   13 -------------
 2 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h
index 8ec88e5..ae7dbd7 100644
--- a/arch/arm64/include/asm/arch_gicv3.h
+++ b/arch/arm64/include/asm/arch_gicv3.h
@@ -79,6 +79,19 @@
 #include <linux/stringify.h>
 #include <asm/barrier.h>
 
+#define read_gicreg(r)							\
+	({								\
+		u64 reg;						\
+		asm volatile("mrs_s %0, " __stringify(r) : "=r" (reg));	\
+		reg;							\
+	})
+
+#define write_gicreg(v,r)						\
+	do {								\
+		u64 __val = (v);					\
+		asm volatile("msr_s " __stringify(r) ", %0" : : "r" (__val));\
+	} while (0)
+
 /*
  * Low-level accessors
  *
diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
index 5f8f80b..f2dbd2e 100644
--- a/arch/arm64/kvm/hyp/vgic-v3-sr.c
+++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
@@ -24,19 +24,6 @@
 #define vtr_to_max_lr_idx(v)		((v) & 0xf)
 #define vtr_to_nr_pri_bits(v)		(((u32)(v) >> 29) + 1)
 
-#define read_gicreg(r)							\
-	({								\
-		u64 reg;						\
-		asm volatile("mrs_s %0, " __stringify(r) : "=r" (reg));	\
-		reg;							\
-	})
-
-#define write_gicreg(v,r)						\
-	do {								\
-		u64 __val = (v);					\
-		asm volatile("msr_s " __stringify(r) ", %0" : : "r" (__val));\
-	} while (0)
-
 static u64 __hyp_text __gic_v3_get_lr(unsigned int lr)
 {
 	switch (lr & 0xf) {
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 74+ messages in thread
* [PATCH v2 2/7] arm64: KVM: Move vgic-v3 save/restore to virt/kvm/arm/hyp
  2016-08-16 10:46 ` Vladimir Murzin
@ 2016-08-16 10:46   ` Vladimir Murzin
  -1 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-08-16 10:46 UTC (permalink / raw)
  To: kvmarm, linux-arm-kernel; +Cc: marc.zyngier, andre.przywara

So we can reuse the code under arch/arm

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 arch/arm64/kvm/hyp/Makefile                       |    2 +-
 {arch/arm64/kvm => virt/kvm/arm}/hyp/vgic-v3-sr.c |    0
 2 files changed, 1 insertion(+), 1 deletion(-)
 rename {arch/arm64/kvm => virt/kvm/arm}/hyp/vgic-v3-sr.c (100%)

diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
index 0c85feb..aaf42ae 100644
--- a/arch/arm64/kvm/hyp/Makefile
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -5,9 +5,9 @@
 KVM=../../../../virt/kvm
 
 obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
+obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
 
-obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += entry.o
diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/virt/kvm/arm/hyp/vgic-v3-sr.c
similarity index 100%
rename from arch/arm64/kvm/hyp/vgic-v3-sr.c
rename to virt/kvm/arm/hyp/vgic-v3-sr.c
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 74+ messages in thread
* [PATCH v2 2/7] arm64: KVM: Move vgic-v3 save/restore to virt/kvm/arm/hyp
@ 2016-08-16 10:46   ` Vladimir Murzin
  0 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-08-16 10:46 UTC (permalink / raw)
  To: linux-arm-kernel

So we can reuse the code under arch/arm

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 arch/arm64/kvm/hyp/Makefile                       |    2 +-
 {arch/arm64/kvm => virt/kvm/arm}/hyp/vgic-v3-sr.c |    0
 2 files changed, 1 insertion(+), 1 deletion(-)
 rename {arch/arm64/kvm => virt/kvm/arm}/hyp/vgic-v3-sr.c (100%)

diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
index 0c85feb..aaf42ae 100644
--- a/arch/arm64/kvm/hyp/Makefile
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -5,9 +5,9 @@
 KVM=../../../../virt/kvm
 
 obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
+obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
 
-obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += entry.o
diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/virt/kvm/arm/hyp/vgic-v3-sr.c
similarity index 100%
rename from arch/arm64/kvm/hyp/vgic-v3-sr.c
rename to virt/kvm/arm/hyp/vgic-v3-sr.c
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 74+ messages in thread
* [PATCH v2 3/7] KVM: arm: vgic-new: improve compatibility with 32-bit
  2016-08-16 10:46 ` Vladimir Murzin
@ 2016-08-16 10:46   ` Vladimir Murzin
  -1 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-08-16 10:46 UTC (permalink / raw)
  To: kvmarm, linux-arm-kernel; +Cc: marc.zyngier, andre.przywara

We have couple of 64-bit register defined in GICv3 architecture, so
"unsigned long" kind of accessors wouldn't work for 32-bit. However,
these registers can't be access as 64-bit in a one go if we run 32-bit
host simply because KVM doesn't support multiple load/store on MMIO
space.

It means that 32-bit guest access these registers in 32-bit chunks, so
the only thing we need to do is to ensure that extract_bytes() always
takes 64-bit data.

Since we are here fix couple of other width related issues by using
ULL variants over UL.

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 virt/kvm/arm/vgic/vgic-mmio-v3.c |    6 +++---
 virt/kvm/arm/vgic/vgic-mmio.h    |    2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index ff668e0..cc20b60 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -23,7 +23,7 @@
 #include "vgic-mmio.h"
 
 /* extract @num bytes at @offset bytes offset in data */
-unsigned long extract_bytes(unsigned long data, unsigned int offset,
+unsigned long extract_bytes(u64 data, unsigned int offset,
 			    unsigned int num)
 {
 	return (data >> (offset * 8)) & GENMASK_ULL(num * 8 - 1, 0);
@@ -179,7 +179,7 @@ static unsigned long vgic_mmio_read_v3r_typer(struct kvm_vcpu *vcpu,
 	int target_vcpu_id = vcpu->vcpu_id;
 	u64 value;
 
-	value = (mpidr & GENMASK(23, 0)) << 32;
+	value = (mpidr & GENMASK_ULL(23, 0)) << 32;
 	value |= ((target_vcpu_id & 0xffff) << 8);
 	if (target_vcpu_id == atomic_read(&vcpu->kvm->online_vcpus) - 1)
 		value |= GICR_TYPER_LAST;
@@ -603,7 +603,7 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)
 	bool broadcast;
 
 	sgi = (reg & ICC_SGI1R_SGI_ID_MASK) >> ICC_SGI1R_SGI_ID_SHIFT;
-	broadcast = reg & BIT(ICC_SGI1R_IRQ_ROUTING_MODE_BIT);
+	broadcast = reg & BIT_ULL(ICC_SGI1R_IRQ_ROUTING_MODE_BIT);
 	target_cpus = (reg & ICC_SGI1R_TARGET_LIST_MASK) >> ICC_SGI1R_TARGET_LIST_SHIFT;
 	mpidr = SGI_AFFINITY_LEVEL(reg, 3);
 	mpidr |= SGI_AFFINITY_LEVEL(reg, 2);
diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
index 0b3ecf9..80f92ce 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.h
+++ b/virt/kvm/arm/vgic/vgic-mmio.h
@@ -96,7 +96,7 @@ unsigned long vgic_data_mmio_bus_to_host(const void *val, unsigned int len);
 void vgic_data_host_to_mmio_bus(void *buf, unsigned int len,
 				unsigned long data);
 
-unsigned long extract_bytes(unsigned long data, unsigned int offset,
+unsigned long extract_bytes(u64 data, unsigned int offset,
 			    unsigned int num);
 
 u64 update_64bit_reg(u64 reg, unsigned int offset, unsigned int len,
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 74+ messages in thread
* [PATCH v2 3/7] KVM: arm: vgic-new: improve compatibility with 32-bit
@ 2016-08-16 10:46   ` Vladimir Murzin
  0 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-08-16 10:46 UTC (permalink / raw)
  To: linux-arm-kernel

We have couple of 64-bit register defined in GICv3 architecture, so
"unsigned long" kind of accessors wouldn't work for 32-bit. However,
these registers can't be access as 64-bit in a one go if we run 32-bit
host simply because KVM doesn't support multiple load/store on MMIO
space.

It means that 32-bit guest access these registers in 32-bit chunks, so
the only thing we need to do is to ensure that extract_bytes() always
takes 64-bit data.

Since we are here fix couple of other width related issues by using
ULL variants over UL.

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 virt/kvm/arm/vgic/vgic-mmio-v3.c |    6 +++---
 virt/kvm/arm/vgic/vgic-mmio.h    |    2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index ff668e0..cc20b60 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -23,7 +23,7 @@
 #include "vgic-mmio.h"
 
 /* extract @num bytes at @offset bytes offset in data */
-unsigned long extract_bytes(unsigned long data, unsigned int offset,
+unsigned long extract_bytes(u64 data, unsigned int offset,
 			    unsigned int num)
 {
 	return (data >> (offset * 8)) & GENMASK_ULL(num * 8 - 1, 0);
@@ -179,7 +179,7 @@ static unsigned long vgic_mmio_read_v3r_typer(struct kvm_vcpu *vcpu,
 	int target_vcpu_id = vcpu->vcpu_id;
 	u64 value;
 
-	value = (mpidr & GENMASK(23, 0)) << 32;
+	value = (mpidr & GENMASK_ULL(23, 0)) << 32;
 	value |= ((target_vcpu_id & 0xffff) << 8);
 	if (target_vcpu_id == atomic_read(&vcpu->kvm->online_vcpus) - 1)
 		value |= GICR_TYPER_LAST;
@@ -603,7 +603,7 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)
 	bool broadcast;
 
 	sgi = (reg & ICC_SGI1R_SGI_ID_MASK) >> ICC_SGI1R_SGI_ID_SHIFT;
-	broadcast = reg & BIT(ICC_SGI1R_IRQ_ROUTING_MODE_BIT);
+	broadcast = reg & BIT_ULL(ICC_SGI1R_IRQ_ROUTING_MODE_BIT);
 	target_cpus = (reg & ICC_SGI1R_TARGET_LIST_MASK) >> ICC_SGI1R_TARGET_LIST_SHIFT;
 	mpidr = SGI_AFFINITY_LEVEL(reg, 3);
 	mpidr |= SGI_AFFINITY_LEVEL(reg, 2);
diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
index 0b3ecf9..80f92ce 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.h
+++ b/virt/kvm/arm/vgic/vgic-mmio.h
@@ -96,7 +96,7 @@ unsigned long vgic_data_mmio_bus_to_host(const void *val, unsigned int len);
 void vgic_data_host_to_mmio_bus(void *buf, unsigned int len,
 				unsigned long data);
 
-unsigned long extract_bytes(unsigned long data, unsigned int offset,
+unsigned long extract_bytes(u64 data, unsigned int offset,
 			    unsigned int num);
 
 u64 update_64bit_reg(u64 reg, unsigned int offset, unsigned int len,
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 74+ messages in thread
* [PATCH v2 4/7] ARM: update MPIDR accessors macro
  2016-08-16 10:46 ` Vladimir Murzin
@ 2016-08-16 10:46   ` Vladimir Murzin
  -1 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-08-16 10:46 UTC (permalink / raw)
  To: kvmarm, linux-arm-kernel; +Cc: marc.zyngier, andre.przywara

vgic-v3 driver queries CPU affinity level up to Aff3, which is valid for arm64.
However, for arm up to Aff2 levels are supported, so querying for 3rd level
ends with upper bits of MPIDR are treated as valid affinity level which
is not true. So, report zero for any affinity level above 2.

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 arch/arm/include/asm/cputype.h |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/arm/include/asm/cputype.h b/arch/arm/include/asm/cputype.h
index 1ee94c7..96cef49 100644
--- a/arch/arm/include/asm/cputype.h
+++ b/arch/arm/include/asm/cputype.h
@@ -55,9 +55,10 @@
 
 #define MPIDR_LEVEL_BITS 8
 #define MPIDR_LEVEL_MASK ((1 << MPIDR_LEVEL_BITS) - 1)
+#define MPIDR_LEVEL_SHIFT(level) (MPIDR_LEVEL_BITS * level)
 
 #define MPIDR_AFFINITY_LEVEL(mpidr, level) \
-	((mpidr >> (MPIDR_LEVEL_BITS * level)) & MPIDR_LEVEL_MASK)
+	((level < 3) ? ((mpidr >> MPIDR_LEVEL_SHIFT(level)) & MPIDR_LEVEL_MASK) : 0)
 
 #define ARM_CPU_IMP_ARM			0x41
 #define ARM_CPU_IMP_INTEL		0x69
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 74+ messages in thread
* [PATCH v2 4/7] ARM: update MPIDR accessors macro
@ 2016-08-16 10:46   ` Vladimir Murzin
  0 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-08-16 10:46 UTC (permalink / raw)
  To: linux-arm-kernel

vgic-v3 driver queries CPU affinity level up to Aff3, which is valid for arm64.
However, for arm up to Aff2 levels are supported, so querying for 3rd level
ends with upper bits of MPIDR are treated as valid affinity level which
is not true. So, report zero for any affinity level above 2.

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 arch/arm/include/asm/cputype.h |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/arm/include/asm/cputype.h b/arch/arm/include/asm/cputype.h
index 1ee94c7..96cef49 100644
--- a/arch/arm/include/asm/cputype.h
+++ b/arch/arm/include/asm/cputype.h
@@ -55,9 +55,10 @@
 
 #define MPIDR_LEVEL_BITS 8
 #define MPIDR_LEVEL_MASK ((1 << MPIDR_LEVEL_BITS) - 1)
+#define MPIDR_LEVEL_SHIFT(level) (MPIDR_LEVEL_BITS * level)
 
 #define MPIDR_AFFINITY_LEVEL(mpidr, level) \
-	((mpidr >> (MPIDR_LEVEL_BITS * level)) & MPIDR_LEVEL_MASK)
+	((level < 3) ? ((mpidr >> MPIDR_LEVEL_SHIFT(level)) & MPIDR_LEVEL_MASK) : 0)
 
 #define ARM_CPU_IMP_ARM			0x41
 #define ARM_CPU_IMP_INTEL		0x69
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 74+ messages in thread
* [PATCH v2 5/7] ARM: move system register accessors to asm/cp15.h
  2016-08-16 10:46 ` Vladimir Murzin
@ 2016-08-16 10:46   ` Vladimir Murzin
  -1 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-08-16 10:46 UTC (permalink / raw)
  To: kvmarm, linux-arm-kernel; +Cc: marc.zyngier, andre.przywara

Macro __ACCESS_CP15{_64} is defined in two headers (arch_gicv3.h and
kvm_hyp.h) which are going to be requested by vgic-v3 altogether.
GCC would not like it because it'd see that macro is redefined and (hey!)
they are different.  So, let's put only single macro version under common
place and use it everywhere.

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 arch/arm/include/asm/arch_gicv3.h |   27 +++++++++++----------------
 arch/arm/include/asm/cp15.h       |   15 +++++++++++++++
 arch/arm/include/asm/kvm_hyp.h    |   15 +--------------
 3 files changed, 27 insertions(+), 30 deletions(-)

diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
index e08d151..af25c32 100644
--- a/arch/arm/include/asm/arch_gicv3.h
+++ b/arch/arm/include/asm/arch_gicv3.h
@@ -22,9 +22,7 @@
 
 #include <linux/io.h>
 #include <asm/barrier.h>
-
-#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
-#define __ACCESS_CP15_64(Op1, CRm)		p15, Op1, %Q0, %R0, CRm
+#include <asm/cp15.h>
 
 #define ICC_EOIR1			__ACCESS_CP15(c12, 0, c12, 1)
 #define ICC_DIR				__ACCESS_CP15(c12, 0, c11, 1)
@@ -102,58 +100,55 @@
 
 static inline void gic_write_eoir(u32 irq)
 {
-	asm volatile("mcr " __stringify(ICC_EOIR1) : : "r" (irq));
+	write_sysreg(irq, ICC_EOIR1);
 	isb();
 }
 
 static inline void gic_write_dir(u32 val)
 {
-	asm volatile("mcr " __stringify(ICC_DIR) : : "r" (val));
+	write_sysreg(val, ICC_DIR);
 	isb();
 }
 
 static inline u32 gic_read_iar(void)
 {
-	u32 irqstat;
+	u32 irqstat = read_sysreg(ICC_IAR1);
 
-	asm volatile("mrc " __stringify(ICC_IAR1) : "=r" (irqstat));
 	dsb(sy);
+
 	return irqstat;
 }
 
 static inline void gic_write_pmr(u32 val)
 {
-	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
+	write_sysreg(val, ICC_PMR);
 }
 
 static inline void gic_write_ctlr(u32 val)
 {
-	asm volatile("mcr " __stringify(ICC_CTLR) : : "r" (val));
+	write_sysreg(val, ICC_CTLR);
 	isb();
 }
 
 static inline void gic_write_grpen1(u32 val)
 {
-	asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
+	write_sysreg(val, ICC_IGRPEN1);
 	isb();
 }
 
 static inline void gic_write_sgi1r(u64 val)
 {
-	asm volatile("mcrr " __stringify(ICC_SGI1R) : : "r" (val));
+	write_sysreg(val, ICC_SGI1R);
 }
 
 static inline u32 gic_read_sre(void)
 {
-	u32 val;
-
-	asm volatile("mrc " __stringify(ICC_SRE) : "=r" (val));
-	return val;
+	return read_sysreg(ICC_SRE);
 }
 
 static inline void gic_write_sre(u32 val)
 {
-	asm volatile("mcr " __stringify(ICC_SRE) : : "r" (val));
+	write_sysreg(val, ICC_SRE);
 	isb();
 }
 
diff --git a/arch/arm/include/asm/cp15.h b/arch/arm/include/asm/cp15.h
index c3f1152..f661732 100644
--- a/arch/arm/include/asm/cp15.h
+++ b/arch/arm/include/asm/cp15.h
@@ -47,6 +47,21 @@
 #define vectors_high()	(0)
 #endif
 
+#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
+	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
+#define __ACCESS_CP15_64(Op1, CRm)		\
+	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
+
+#define __read_sysreg(r, w, c, t) ({				\
+	t __val;						\
+	asm volatile(r " " c : "=r" (__val));			\
+	__val;							\
+})
+#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
+
+#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
+#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
+
 #ifdef CONFIG_CPU_CP15
 
 extern unsigned long cr_alignment;	/* defined in entry-armv.S */
diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
index 6eaff28..e604ad68 100644
--- a/arch/arm/include/asm/kvm_hyp.h
+++ b/arch/arm/include/asm/kvm_hyp.h
@@ -20,28 +20,15 @@
 
 #include <linux/compiler.h>
 #include <linux/kvm_host.h>
+#include <asm/cp15.h>
 #include <asm/kvm_mmu.h>
 #include <asm/vfp.h>
 
 #define __hyp_text __section(.hyp.text) notrace
 
-#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
-	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
-#define __ACCESS_CP15_64(Op1, CRm)		\
-	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
 #define __ACCESS_VFP(CRn)			\
 	"mrc", "mcr", __stringify(p10, 7, %0, CRn, cr0, 0), u32
 
-#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
-#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
-
-#define __read_sysreg(r, w, c, t) ({				\
-	t __val;						\
-	asm volatile(r " " c : "=r" (__val));			\
-	__val;							\
-})
-#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
-
 #define write_special(v, r)					\
 	asm volatile("msr " __stringify(r) ", %0" : : "r" (v))
 #define read_special(r) ({					\
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 74+ messages in thread
* [PATCH v2 5/7] ARM: move system register accessors to asm/cp15.h
@ 2016-08-16 10:46   ` Vladimir Murzin
  0 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-08-16 10:46 UTC (permalink / raw)
  To: linux-arm-kernel

Macro __ACCESS_CP15{_64} is defined in two headers (arch_gicv3.h and
kvm_hyp.h) which are going to be requested by vgic-v3 altogether.
GCC would not like it because it'd see that macro is redefined and (hey!)
they are different.  So, let's put only single macro version under common
place and use it everywhere.

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 arch/arm/include/asm/arch_gicv3.h |   27 +++++++++++----------------
 arch/arm/include/asm/cp15.h       |   15 +++++++++++++++
 arch/arm/include/asm/kvm_hyp.h    |   15 +--------------
 3 files changed, 27 insertions(+), 30 deletions(-)

diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
index e08d151..af25c32 100644
--- a/arch/arm/include/asm/arch_gicv3.h
+++ b/arch/arm/include/asm/arch_gicv3.h
@@ -22,9 +22,7 @@
 
 #include <linux/io.h>
 #include <asm/barrier.h>
-
-#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
-#define __ACCESS_CP15_64(Op1, CRm)		p15, Op1, %Q0, %R0, CRm
+#include <asm/cp15.h>
 
 #define ICC_EOIR1			__ACCESS_CP15(c12, 0, c12, 1)
 #define ICC_DIR				__ACCESS_CP15(c12, 0, c11, 1)
@@ -102,58 +100,55 @@
 
 static inline void gic_write_eoir(u32 irq)
 {
-	asm volatile("mcr " __stringify(ICC_EOIR1) : : "r" (irq));
+	write_sysreg(irq, ICC_EOIR1);
 	isb();
 }
 
 static inline void gic_write_dir(u32 val)
 {
-	asm volatile("mcr " __stringify(ICC_DIR) : : "r" (val));
+	write_sysreg(val, ICC_DIR);
 	isb();
 }
 
 static inline u32 gic_read_iar(void)
 {
-	u32 irqstat;
+	u32 irqstat = read_sysreg(ICC_IAR1);
 
-	asm volatile("mrc " __stringify(ICC_IAR1) : "=r" (irqstat));
 	dsb(sy);
+
 	return irqstat;
 }
 
 static inline void gic_write_pmr(u32 val)
 {
-	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
+	write_sysreg(val, ICC_PMR);
 }
 
 static inline void gic_write_ctlr(u32 val)
 {
-	asm volatile("mcr " __stringify(ICC_CTLR) : : "r" (val));
+	write_sysreg(val, ICC_CTLR);
 	isb();
 }
 
 static inline void gic_write_grpen1(u32 val)
 {
-	asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
+	write_sysreg(val, ICC_IGRPEN1);
 	isb();
 }
 
 static inline void gic_write_sgi1r(u64 val)
 {
-	asm volatile("mcrr " __stringify(ICC_SGI1R) : : "r" (val));
+	write_sysreg(val, ICC_SGI1R);
 }
 
 static inline u32 gic_read_sre(void)
 {
-	u32 val;
-
-	asm volatile("mrc " __stringify(ICC_SRE) : "=r" (val));
-	return val;
+	return read_sysreg(ICC_SRE);
 }
 
 static inline void gic_write_sre(u32 val)
 {
-	asm volatile("mcr " __stringify(ICC_SRE) : : "r" (val));
+	write_sysreg(val, ICC_SRE);
 	isb();
 }
 
diff --git a/arch/arm/include/asm/cp15.h b/arch/arm/include/asm/cp15.h
index c3f1152..f661732 100644
--- a/arch/arm/include/asm/cp15.h
+++ b/arch/arm/include/asm/cp15.h
@@ -47,6 +47,21 @@
 #define vectors_high()	(0)
 #endif
 
+#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
+	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
+#define __ACCESS_CP15_64(Op1, CRm)		\
+	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
+
+#define __read_sysreg(r, w, c, t) ({				\
+	t __val;						\
+	asm volatile(r " " c : "=r" (__val));			\
+	__val;							\
+})
+#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
+
+#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
+#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
+
 #ifdef CONFIG_CPU_CP15
 
 extern unsigned long cr_alignment;	/* defined in entry-armv.S */
diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
index 6eaff28..e604ad68 100644
--- a/arch/arm/include/asm/kvm_hyp.h
+++ b/arch/arm/include/asm/kvm_hyp.h
@@ -20,28 +20,15 @@
 
 #include <linux/compiler.h>
 #include <linux/kvm_host.h>
+#include <asm/cp15.h>
 #include <asm/kvm_mmu.h>
 #include <asm/vfp.h>
 
 #define __hyp_text __section(.hyp.text) notrace
 
-#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
-	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
-#define __ACCESS_CP15_64(Op1, CRm)		\
-	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
 #define __ACCESS_VFP(CRn)			\
 	"mrc", "mcr", __stringify(p10, 7, %0, CRn, cr0, 0), u32
 
-#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
-#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
-
-#define __read_sysreg(r, w, c, t) ({				\
-	t __val;						\
-	asm volatile(r " " c : "=r" (__val));			\
-	__val;							\
-})
-#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
-
 #define write_special(v, r)					\
 	asm volatile("msr " __stringify(r) ", %0" : : "r" (v))
 #define read_special(r) ({					\
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 74+ messages in thread
* [PATCH v2 6/7] ARM: KVM: Get ready to use vgic-v3
  2016-08-16 10:46 ` Vladimir Murzin
@ 2016-08-16 10:46   ` Vladimir Murzin
  -1 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-08-16 10:46 UTC (permalink / raw)
  To: kvmarm, linux-arm-kernel; +Cc: marc.zyngier, andre.przywara

We need to take care we have everything vgic-v3 expects from us before
a quantum leap:
- provide required macros via uapi.h
- handle access to GICv3 cpu interface from the guest
- provide mapping between arm64 version of GICv3 cpu registers and arm's

The later is handled via redirection of read{write}_gicreg() and
required mainly because 64-bit wide ICH_LR is split in two 32-bit
halves (ICH_LR and ICH_LRC) accessed independently.

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 arch/arm/include/asm/arch_gicv3.h |   64 +++++++++++++++++++++++++++++++++++++
 arch/arm/include/asm/kvm_asm.h    |    3 ++
 arch/arm/include/uapi/asm/kvm.h   |    7 ++++
 arch/arm/kvm/coproc.c             |   36 +++++++++++++++++++++
 4 files changed, 110 insertions(+)

diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
index af25c32..f93f6bd 100644
--- a/arch/arm/include/asm/arch_gicv3.h
+++ b/arch/arm/include/asm/arch_gicv3.h
@@ -96,6 +96,70 @@
 #define ICH_AP1R2			__AP1Rx(2)
 #define ICH_AP1R3			__AP1Rx(3)
 
+/* A32-to-A64 mappings used by VGIC save/restore */
+
+#define CPUIF_MAP(a32, a64)			\
+static inline void write_ ## a64(u32 val)	\
+{						\
+	write_sysreg(val, a32);			\
+}						\
+static inline u32 read_ ## a64(void)		\
+{						\
+	return read_sysreg(a32); 		\
+}						\
+
+#define CPUIF_MAP_LO_HI(a32lo, a32hi, a64)	\
+static inline void write_ ## a64(u64 val)	\
+{						\
+	write_sysreg((u32)val, a32lo);		\
+	write_sysreg((u32)(val >> 32), a32hi);	\
+}						\
+static inline u64 read_ ## a64(void)		\
+{						\
+	u64 val = read_sysreg(a32lo);		\
+						\
+	val |=	(u64)read_sysreg(a32hi) << 32;	\
+						\
+	return val; 				\
+}
+
+CPUIF_MAP(ICH_HCR, ICH_HCR_EL2)
+CPUIF_MAP(ICH_VTR, ICH_VTR_EL2)
+CPUIF_MAP(ICH_MISR, ICH_MISR_EL2)
+CPUIF_MAP(ICH_EISR, ICH_EISR_EL2)
+CPUIF_MAP(ICH_ELSR, ICH_ELSR_EL2)
+CPUIF_MAP(ICH_VMCR, ICH_VMCR_EL2)
+CPUIF_MAP(ICH_AP0R3, ICH_AP0R3_EL2)
+CPUIF_MAP(ICH_AP0R2, ICH_AP0R2_EL2)
+CPUIF_MAP(ICH_AP0R1, ICH_AP0R1_EL2)
+CPUIF_MAP(ICH_AP0R0, ICH_AP0R0_EL2)
+CPUIF_MAP(ICH_AP1R3, ICH_AP1R3_EL2)
+CPUIF_MAP(ICH_AP1R2, ICH_AP1R2_EL2)
+CPUIF_MAP(ICH_AP1R1, ICH_AP1R1_EL2)
+CPUIF_MAP(ICH_AP1R0, ICH_AP1R0_EL2)
+CPUIF_MAP(ICC_HSRE, ICC_SRE_EL2)
+CPUIF_MAP(ICC_SRE, ICC_SRE_EL1)
+
+CPUIF_MAP_LO_HI(ICH_LR15, ICH_LRC15, ICH_LR15_EL2)
+CPUIF_MAP_LO_HI(ICH_LR14, ICH_LRC14, ICH_LR14_EL2)
+CPUIF_MAP_LO_HI(ICH_LR13, ICH_LRC13, ICH_LR13_EL2)
+CPUIF_MAP_LO_HI(ICH_LR12, ICH_LRC12, ICH_LR12_EL2)
+CPUIF_MAP_LO_HI(ICH_LR11, ICH_LRC11, ICH_LR11_EL2)
+CPUIF_MAP_LO_HI(ICH_LR10, ICH_LRC10, ICH_LR10_EL2)
+CPUIF_MAP_LO_HI(ICH_LR9, ICH_LRC9, ICH_LR9_EL2)
+CPUIF_MAP_LO_HI(ICH_LR8, ICH_LRC8, ICH_LR8_EL2)
+CPUIF_MAP_LO_HI(ICH_LR7, ICH_LRC7, ICH_LR7_EL2)
+CPUIF_MAP_LO_HI(ICH_LR6, ICH_LRC6, ICH_LR6_EL2)
+CPUIF_MAP_LO_HI(ICH_LR5, ICH_LRC5, ICH_LR5_EL2)
+CPUIF_MAP_LO_HI(ICH_LR4, ICH_LRC4, ICH_LR4_EL2)
+CPUIF_MAP_LO_HI(ICH_LR3, ICH_LRC3, ICH_LR3_EL2)
+CPUIF_MAP_LO_HI(ICH_LR2, ICH_LRC2, ICH_LR2_EL2)
+CPUIF_MAP_LO_HI(ICH_LR1, ICH_LRC1, ICH_LR1_EL2)
+CPUIF_MAP_LO_HI(ICH_LR0, ICH_LRC0, ICH_LR0_EL2)
+
+#define read_gicreg(r)                 read_##r()
+#define write_gicreg(v, r)             write_##r(v)
+
 /* Low-level accessors */
 
 static inline void gic_write_eoir(u32 irq)
diff --git a/arch/arm/include/asm/kvm_asm.h b/arch/arm/include/asm/kvm_asm.h
index 58faff5..dfccf94 100644
--- a/arch/arm/include/asm/kvm_asm.h
+++ b/arch/arm/include/asm/kvm_asm.h
@@ -68,6 +68,9 @@ extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
 extern void __init_stage2_translation(void);
 
 extern void __kvm_hyp_reset(unsigned long);
+
+extern u64 __vgic_v3_get_ich_vtr_el2(void);
+extern void __vgic_v3_init_lrs(void);
 #endif
 
 #endif /* __ARM_KVM_ASM_H__ */
diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h
index a2b3eb3..b38c10c 100644
--- a/arch/arm/include/uapi/asm/kvm.h
+++ b/arch/arm/include/uapi/asm/kvm.h
@@ -84,6 +84,13 @@ struct kvm_regs {
 #define KVM_VGIC_V2_DIST_SIZE		0x1000
 #define KVM_VGIC_V2_CPU_SIZE		0x2000
 
+/* Supported VGICv3 address types  */
+#define KVM_VGIC_V3_ADDR_TYPE_DIST	2
+#define KVM_VGIC_V3_ADDR_TYPE_REDIST	3
+
+#define KVM_VGIC_V3_DIST_SIZE		SZ_64K
+#define KVM_VGIC_V3_REDIST_SIZE		(2 * SZ_64K)
+
 #define KVM_ARM_VCPU_POWER_OFF		0 /* CPU is started in OFF state */
 #define KVM_ARM_VCPU_PSCI_0_2		1 /* CPU uses PSCI v0.2 */
 
diff --git a/arch/arm/kvm/coproc.c b/arch/arm/kvm/coproc.c
index 1bb2b79..10c0244 100644
--- a/arch/arm/kvm/coproc.c
+++ b/arch/arm/kvm/coproc.c
@@ -228,6 +228,36 @@ bool access_vm_reg(struct kvm_vcpu *vcpu,
 	return true;
 }
 
+static bool access_gic_sgi(struct kvm_vcpu *vcpu,
+			   const struct coproc_params *p,
+			   const struct coproc_reg *r)
+{
+	u64 reg;
+
+	if (!p->is_write)
+		return read_from_write_only(vcpu, p);
+
+	reg = *vcpu_reg(vcpu, p->Rt2);
+	reg <<= 32;
+	reg |= *vcpu_reg(vcpu, p->Rt1) ;
+
+	vgic_v3_dispatch_sgi(vcpu, reg);
+
+	return true;
+}
+
+static bool access_gic_sre(struct kvm_vcpu *vcpu,
+			   const struct coproc_params *p,
+			   const struct coproc_reg *r)
+{
+	if (p->is_write)
+		return ignore_write(vcpu, p);
+
+	*vcpu_reg(vcpu, p->Rt1) = vcpu->arch.vgic_cpu.vgic_v3.vgic_sre;
+
+	return true;
+}
+
 /*
  * We could trap ID_DFR0 and tell the guest we don't support performance
  * monitoring.  Unfortunately the patch to make the kernel check ID_DFR0 was
@@ -361,10 +391,16 @@ static const struct coproc_reg cp15_regs[] = {
 	{ CRn(10), CRm( 3), Op1( 0), Op2( 1), is32,
 			access_vm_reg, reset_unknown, c10_AMAIR1},
 
+	/* ICC_SGI1R */
+	{ CRm64(12), Op1( 0), is64, access_gic_sgi},
+
 	/* VBAR: swapped by interrupt.S. */
 	{ CRn(12), CRm( 0), Op1( 0), Op2( 0), is32,
 			NULL, reset_val, c12_VBAR, 0x00000000 },
 
+	/* ICC_SRE */
+	{ CRn(12), CRm(12), Op1( 0), Op2(5), is32, access_gic_sre },
+
 	/* CONTEXTIDR/TPIDRURW/TPIDRURO/TPIDRPRW: swapped by interrupt.S. */
 	{ CRn(13), CRm( 0), Op1( 0), Op2( 1), is32,
 			access_vm_reg, reset_val, c13_CID, 0x00000000 },
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 74+ messages in thread
* [PATCH v2 6/7] ARM: KVM: Get ready to use vgic-v3
@ 2016-08-16 10:46   ` Vladimir Murzin
  0 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-08-16 10:46 UTC (permalink / raw)
  To: linux-arm-kernel

We need to take care we have everything vgic-v3 expects from us before
a quantum leap:
- provide required macros via uapi.h
- handle access to GICv3 cpu interface from the guest
- provide mapping between arm64 version of GICv3 cpu registers and arm's

The later is handled via redirection of read{write}_gicreg() and
required mainly because 64-bit wide ICH_LR is split in two 32-bit
halves (ICH_LR and ICH_LRC) accessed independently.

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 arch/arm/include/asm/arch_gicv3.h |   64 +++++++++++++++++++++++++++++++++++++
 arch/arm/include/asm/kvm_asm.h    |    3 ++
 arch/arm/include/uapi/asm/kvm.h   |    7 ++++
 arch/arm/kvm/coproc.c             |   36 +++++++++++++++++++++
 4 files changed, 110 insertions(+)

diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
index af25c32..f93f6bd 100644
--- a/arch/arm/include/asm/arch_gicv3.h
+++ b/arch/arm/include/asm/arch_gicv3.h
@@ -96,6 +96,70 @@
 #define ICH_AP1R2			__AP1Rx(2)
 #define ICH_AP1R3			__AP1Rx(3)
 
+/* A32-to-A64 mappings used by VGIC save/restore */
+
+#define CPUIF_MAP(a32, a64)			\
+static inline void write_ ## a64(u32 val)	\
+{						\
+	write_sysreg(val, a32);			\
+}						\
+static inline u32 read_ ## a64(void)		\
+{						\
+	return read_sysreg(a32); 		\
+}						\
+
+#define CPUIF_MAP_LO_HI(a32lo, a32hi, a64)	\
+static inline void write_ ## a64(u64 val)	\
+{						\
+	write_sysreg((u32)val, a32lo);		\
+	write_sysreg((u32)(val >> 32), a32hi);	\
+}						\
+static inline u64 read_ ## a64(void)		\
+{						\
+	u64 val = read_sysreg(a32lo);		\
+						\
+	val |=	(u64)read_sysreg(a32hi) << 32;	\
+						\
+	return val; 				\
+}
+
+CPUIF_MAP(ICH_HCR, ICH_HCR_EL2)
+CPUIF_MAP(ICH_VTR, ICH_VTR_EL2)
+CPUIF_MAP(ICH_MISR, ICH_MISR_EL2)
+CPUIF_MAP(ICH_EISR, ICH_EISR_EL2)
+CPUIF_MAP(ICH_ELSR, ICH_ELSR_EL2)
+CPUIF_MAP(ICH_VMCR, ICH_VMCR_EL2)
+CPUIF_MAP(ICH_AP0R3, ICH_AP0R3_EL2)
+CPUIF_MAP(ICH_AP0R2, ICH_AP0R2_EL2)
+CPUIF_MAP(ICH_AP0R1, ICH_AP0R1_EL2)
+CPUIF_MAP(ICH_AP0R0, ICH_AP0R0_EL2)
+CPUIF_MAP(ICH_AP1R3, ICH_AP1R3_EL2)
+CPUIF_MAP(ICH_AP1R2, ICH_AP1R2_EL2)
+CPUIF_MAP(ICH_AP1R1, ICH_AP1R1_EL2)
+CPUIF_MAP(ICH_AP1R0, ICH_AP1R0_EL2)
+CPUIF_MAP(ICC_HSRE, ICC_SRE_EL2)
+CPUIF_MAP(ICC_SRE, ICC_SRE_EL1)
+
+CPUIF_MAP_LO_HI(ICH_LR15, ICH_LRC15, ICH_LR15_EL2)
+CPUIF_MAP_LO_HI(ICH_LR14, ICH_LRC14, ICH_LR14_EL2)
+CPUIF_MAP_LO_HI(ICH_LR13, ICH_LRC13, ICH_LR13_EL2)
+CPUIF_MAP_LO_HI(ICH_LR12, ICH_LRC12, ICH_LR12_EL2)
+CPUIF_MAP_LO_HI(ICH_LR11, ICH_LRC11, ICH_LR11_EL2)
+CPUIF_MAP_LO_HI(ICH_LR10, ICH_LRC10, ICH_LR10_EL2)
+CPUIF_MAP_LO_HI(ICH_LR9, ICH_LRC9, ICH_LR9_EL2)
+CPUIF_MAP_LO_HI(ICH_LR8, ICH_LRC8, ICH_LR8_EL2)
+CPUIF_MAP_LO_HI(ICH_LR7, ICH_LRC7, ICH_LR7_EL2)
+CPUIF_MAP_LO_HI(ICH_LR6, ICH_LRC6, ICH_LR6_EL2)
+CPUIF_MAP_LO_HI(ICH_LR5, ICH_LRC5, ICH_LR5_EL2)
+CPUIF_MAP_LO_HI(ICH_LR4, ICH_LRC4, ICH_LR4_EL2)
+CPUIF_MAP_LO_HI(ICH_LR3, ICH_LRC3, ICH_LR3_EL2)
+CPUIF_MAP_LO_HI(ICH_LR2, ICH_LRC2, ICH_LR2_EL2)
+CPUIF_MAP_LO_HI(ICH_LR1, ICH_LRC1, ICH_LR1_EL2)
+CPUIF_MAP_LO_HI(ICH_LR0, ICH_LRC0, ICH_LR0_EL2)
+
+#define read_gicreg(r)                 read_##r()
+#define write_gicreg(v, r)             write_##r(v)
+
 /* Low-level accessors */
 
 static inline void gic_write_eoir(u32 irq)
diff --git a/arch/arm/include/asm/kvm_asm.h b/arch/arm/include/asm/kvm_asm.h
index 58faff5..dfccf94 100644
--- a/arch/arm/include/asm/kvm_asm.h
+++ b/arch/arm/include/asm/kvm_asm.h
@@ -68,6 +68,9 @@ extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
 extern void __init_stage2_translation(void);
 
 extern void __kvm_hyp_reset(unsigned long);
+
+extern u64 __vgic_v3_get_ich_vtr_el2(void);
+extern void __vgic_v3_init_lrs(void);
 #endif
 
 #endif /* __ARM_KVM_ASM_H__ */
diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h
index a2b3eb3..b38c10c 100644
--- a/arch/arm/include/uapi/asm/kvm.h
+++ b/arch/arm/include/uapi/asm/kvm.h
@@ -84,6 +84,13 @@ struct kvm_regs {
 #define KVM_VGIC_V2_DIST_SIZE		0x1000
 #define KVM_VGIC_V2_CPU_SIZE		0x2000
 
+/* Supported VGICv3 address types  */
+#define KVM_VGIC_V3_ADDR_TYPE_DIST	2
+#define KVM_VGIC_V3_ADDR_TYPE_REDIST	3
+
+#define KVM_VGIC_V3_DIST_SIZE		SZ_64K
+#define KVM_VGIC_V3_REDIST_SIZE		(2 * SZ_64K)
+
 #define KVM_ARM_VCPU_POWER_OFF		0 /* CPU is started in OFF state */
 #define KVM_ARM_VCPU_PSCI_0_2		1 /* CPU uses PSCI v0.2 */
 
diff --git a/arch/arm/kvm/coproc.c b/arch/arm/kvm/coproc.c
index 1bb2b79..10c0244 100644
--- a/arch/arm/kvm/coproc.c
+++ b/arch/arm/kvm/coproc.c
@@ -228,6 +228,36 @@ bool access_vm_reg(struct kvm_vcpu *vcpu,
 	return true;
 }
 
+static bool access_gic_sgi(struct kvm_vcpu *vcpu,
+			   const struct coproc_params *p,
+			   const struct coproc_reg *r)
+{
+	u64 reg;
+
+	if (!p->is_write)
+		return read_from_write_only(vcpu, p);
+
+	reg = *vcpu_reg(vcpu, p->Rt2);
+	reg <<= 32;
+	reg |= *vcpu_reg(vcpu, p->Rt1) ;
+
+	vgic_v3_dispatch_sgi(vcpu, reg);
+
+	return true;
+}
+
+static bool access_gic_sre(struct kvm_vcpu *vcpu,
+			   const struct coproc_params *p,
+			   const struct coproc_reg *r)
+{
+	if (p->is_write)
+		return ignore_write(vcpu, p);
+
+	*vcpu_reg(vcpu, p->Rt1) = vcpu->arch.vgic_cpu.vgic_v3.vgic_sre;
+
+	return true;
+}
+
 /*
  * We could trap ID_DFR0 and tell the guest we don't support performance
  * monitoring.  Unfortunately the patch to make the kernel check ID_DFR0 was
@@ -361,10 +391,16 @@ static const struct coproc_reg cp15_regs[] = {
 	{ CRn(10), CRm( 3), Op1( 0), Op2( 1), is32,
 			access_vm_reg, reset_unknown, c10_AMAIR1},
 
+	/* ICC_SGI1R */
+	{ CRm64(12), Op1( 0), is64, access_gic_sgi},
+
 	/* VBAR: swapped by interrupt.S. */
 	{ CRn(12), CRm( 0), Op1( 0), Op2( 0), is32,
 			NULL, reset_val, c12_VBAR, 0x00000000 },
 
+	/* ICC_SRE */
+	{ CRn(12), CRm(12), Op1( 0), Op2(5), is32, access_gic_sre },
+
 	/* CONTEXTIDR/TPIDRURW/TPIDRURO/TPIDRPRW: swapped by interrupt.S. */
 	{ CRn(13), CRm( 0), Op1( 0), Op2( 1), is32,
 			access_vm_reg, reset_val, c13_CID, 0x00000000 },
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 74+ messages in thread
* [PATCH v2 7/7] ARM: KVM: Unlock vgic-v3 support
  2016-08-16 10:46 ` Vladimir Murzin
@ 2016-08-16 10:46   ` Vladimir Murzin
  -1 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-08-16 10:46 UTC (permalink / raw)
  To: kvmarm, linux-arm-kernel; +Cc: marc.zyngier, andre.przywara

It is time to get access to common version of vgic-v3.

We basically would need to tell build system how to pick it up and
undo KVM_ARM_VGIC_V3 guarding introduced in 4f64cb6 ("arm/arm64: KVM:
Only allow 64bit hosts to build VGICv3") and remove stubs. However,
since vgic got ITS support KVM_ARM_VGIC_V3 tend to protect a little
bit more than just plain vgic-v3 - this guard is used for ITS too
which is not supported in 32-bit world yet.  So, along with removal of
KVM_ARM_VGIC_V3 guard introduce the new one - KVM_ARM_VGIC_V3_ITS to
protect ITS related code.

The only unpleasant part is how we decide which save/restore sequence to
use under __vgic_save_state() - we don't have patching framework in hand
like arm64, so have to check runtime on every invocation.

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 arch/arm/include/asm/kvm_host.h     |    4 +++
 arch/arm/include/asm/kvm_hyp.h      |    5 ++++
 arch/arm/kvm/Makefile               |    2 ++
 arch/arm/kvm/hyp/Makefile           |    1 +
 arch/arm/kvm/hyp/switch.c           |   20 +++++++++++--
 arch/arm64/kvm/Kconfig              |    4 +--
 include/kvm/arm_vgic.h              |    8 ------
 virt/kvm/arm/vgic/vgic-kvm-device.c |   10 ++-----
 virt/kvm/arm/vgic/vgic-mmio-v3.c    |    2 ++
 virt/kvm/arm/vgic/vgic-mmio.c       |    2 --
 virt/kvm/arm/vgic/vgic-mmio.h       |    3 --
 virt/kvm/arm/vgic/vgic.h            |   54 ++---------------------------------
 12 files changed, 38 insertions(+), 77 deletions(-)

diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
index de338d9..1312597 100644
--- a/arch/arm/include/asm/kvm_host.h
+++ b/arch/arm/include/asm/kvm_host.h
@@ -39,7 +39,11 @@
 
 #include <kvm/arm_vgic.h>
 
+#ifdef CONFIG_ARM_GIC_V3
+#define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS
+#else
 #define KVM_MAX_VCPUS VGIC_V2_MAX_CPUS
+#endif
 
 #define KVM_REQ_VCPU_EXIT	8
 
diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
index e604ad68..95669b3 100644
--- a/arch/arm/include/asm/kvm_hyp.h
+++ b/arch/arm/include/asm/kvm_hyp.h
@@ -88,6 +88,8 @@
 
 #define VFP_FPEXC	__ACCESS_VFP(FPEXC)
 
+#define ID_PFR1		__ACCESS_CP15(c0, 0, c1, 1)
+
 /* AArch64 compatibility macros, only for the timer so far */
 #define read_sysreg_el0(r)		read_sysreg(r##_el0)
 #define write_sysreg_el0(v, r)		write_sysreg(v, r##_el0)
@@ -103,6 +105,9 @@ void __timer_restore_state(struct kvm_vcpu *vcpu);
 void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
 void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
 
+void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
+void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
+
 void __sysreg_save_state(struct kvm_cpu_context *ctxt);
 void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
 
diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
index 10d77a6..043d817f 100644
--- a/arch/arm/kvm/Makefile
+++ b/arch/arm/kvm/Makefile
@@ -26,8 +26,10 @@ obj-y += $(KVM)/arm/vgic/vgic.o
 obj-y += $(KVM)/arm/vgic/vgic-init.o
 obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
 obj-y += $(KVM)/arm/vgic/vgic-v2.o
+obj-y += $(KVM)/arm/vgic/vgic-v3.o
 obj-y += $(KVM)/arm/vgic/vgic-mmio.o
 obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
+obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
 obj-y += $(KVM)/arm/vgic/vgic-kvm-device.o
 obj-y += $(KVM)/irqchip.o
 obj-y += $(KVM)/arm/arch_timer.o
diff --git a/arch/arm/kvm/hyp/Makefile b/arch/arm/kvm/hyp/Makefile
index 8dfa5f7..3023bb5 100644
--- a/arch/arm/kvm/hyp/Makefile
+++ b/arch/arm/kvm/hyp/Makefile
@@ -5,6 +5,7 @@
 KVM=../../../../virt/kvm
 
 obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
+obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
 
 obj-$(CONFIG_KVM_ARM_HOST) += tlb.o
diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
index b13caa9..9666bae 100644
--- a/arch/arm/kvm/hyp/switch.c
+++ b/arch/arm/kvm/hyp/switch.c
@@ -15,6 +15,8 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
+#include <linux/irqchip/arm-gic-v3.h>
+
 #include <asm/kvm_asm.h>
 #include <asm/kvm_hyp.h>
 
@@ -74,14 +76,28 @@ static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
 	write_sysreg(read_sysreg(MIDR), VPIDR);
 }
 
+static bool __hyp_text __has_useable_gicv3_cpuif(void)
+{
+	if (IS_ENABLED(CONFIG_ARM_GIC_V3) && (read_sysreg(ID_PFR1) >> 28))
+		return !!(read_sysreg(ICC_HSRE) & ICC_SRE_EL2_SRE);
+	else
+		return false;
+}
+
 static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
 {
-	__vgic_v2_save_state(vcpu);
+	if (__has_useable_gicv3_cpuif())
+		__vgic_v3_save_state(vcpu);
+	else
+		__vgic_v2_save_state(vcpu);
 }
 
 static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
 {
-	__vgic_v2_restore_state(vcpu);
+	if (__has_useable_gicv3_cpuif())
+		__vgic_v3_restore_state(vcpu);
+	else
+		__vgic_v2_restore_state(vcpu);
 }
 
 static bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu)
diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index 9c9edc9..6eaf12c 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -16,7 +16,7 @@ menuconfig VIRTUALIZATION
 
 if VIRTUALIZATION
 
-config KVM_ARM_VGIC_V3
+config KVM_ARM_VGIC_V3_ITS
 	bool
 
 config KVM
@@ -34,7 +34,7 @@ config KVM
 	select KVM_VFIO
 	select HAVE_KVM_EVENTFD
 	select HAVE_KVM_IRQFD
-	select KVM_ARM_VGIC_V3
+	select KVM_ARM_VGIC_V3_ITS
 	select KVM_ARM_PMU if HW_PERF_EVENTS
 	select HAVE_KVM_MSI
 	select HAVE_KVM_IRQCHIP
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 19b698e..7462138 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -217,7 +217,6 @@ struct vgic_v2_cpu_if {
 };
 
 struct vgic_v3_cpu_if {
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 	u32		vgic_hcr;
 	u32		vgic_vmcr;
 	u32		vgic_sre;	/* Restored only, change ignored */
@@ -227,7 +226,6 @@ struct vgic_v3_cpu_if {
 	u32		vgic_ap0r[4];
 	u32		vgic_ap1r[4];
 	u64		vgic_lr[VGIC_V3_MAX_LRS];
-#endif
 };
 
 struct vgic_cpu {
@@ -294,13 +292,7 @@ bool kvm_vcpu_has_pending_irqs(struct kvm_vcpu *vcpu);
 void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu);
 void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu);
 
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg);
-#else
-static inline void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)
-{
-}
-#endif
 
 /**
  * kvm_vgic_get_max_vcpus - Get the maximum number of VCPUs allowed by HW
diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
index 1813f93..e4f0c33 100644
--- a/virt/kvm/arm/vgic/vgic-kvm-device.c
+++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
@@ -71,7 +71,6 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write)
 		addr_ptr = &vgic->vgic_cpu_base;
 		alignment = SZ_4K;
 		break;
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 	case KVM_VGIC_V3_ADDR_TYPE_DIST:
 		type_needed = KVM_DEV_TYPE_ARM_VGIC_V3;
 		addr_ptr = &vgic->vgic_dist_base;
@@ -82,7 +81,6 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write)
 		addr_ptr = &vgic->vgic_redist_base;
 		alignment = SZ_64K;
 		break;
-#endif
 	default:
 		r = -ENODEV;
 		goto out;
@@ -219,15 +217,15 @@ int kvm_register_vgic_device(unsigned long type)
 		ret = kvm_register_device_ops(&kvm_arm_vgic_v2_ops,
 					      KVM_DEV_TYPE_ARM_VGIC_V2);
 		break;
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 	case KVM_DEV_TYPE_ARM_VGIC_V3:
 		ret = kvm_register_device_ops(&kvm_arm_vgic_v3_ops,
 					      KVM_DEV_TYPE_ARM_VGIC_V3);
+#ifdef CONFIG_KVM_ARM_VGIC_V3_ITS
 		if (ret)
 			break;
 		ret = kvm_vgic_register_its_device();
-		break;
 #endif
+		break;
 	}
 
 	return ret;
@@ -389,8 +387,6 @@ struct kvm_device_ops kvm_arm_vgic_v2_ops = {
 
 /* V3 ops */
 
-#ifdef CONFIG_KVM_ARM_VGIC_V3
-
 static int vgic_v3_set_attr(struct kvm_device *dev,
 			    struct kvm_device_attr *attr)
 {
@@ -433,5 +429,3 @@ struct kvm_device_ops kvm_arm_vgic_v3_ops = {
 	.get_attr = vgic_v3_get_attr,
 	.has_attr = vgic_v3_has_attr,
 };
-
-#endif /* CONFIG_KVM_ARM_VGIC_V3 */
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index cc20b60..4709dd65 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -42,6 +42,7 @@ u64 update_64bit_reg(u64 reg, unsigned int offset, unsigned int len,
 	return reg | ((u64)val << lower);
 }
 
+#ifdef CONFIG_KVM_ARM_VGIC_V3_ITS
 bool vgic_has_its(struct kvm *kvm)
 {
 	struct vgic_dist *dist = &kvm->arch.vgic;
@@ -51,6 +52,7 @@ bool vgic_has_its(struct kvm *kvm)
 
 	return dist->has_its;
 }
+#endif
 
 static unsigned long vgic_mmio_read_v3_misc(struct kvm_vcpu *vcpu,
 					    gpa_t addr, unsigned int len)
diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
index 3bad3c5..e18b30d 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.c
+++ b/virt/kvm/arm/vgic/vgic-mmio.c
@@ -550,11 +550,9 @@ int vgic_register_dist_iodev(struct kvm *kvm, gpa_t dist_base_address,
 	case VGIC_V2:
 		len = vgic_v2_init_dist_iodev(io_device);
 		break;
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 	case VGIC_V3:
 		len = vgic_v3_init_dist_iodev(io_device);
 		break;
-#endif
 	default:
 		BUG_ON(1);
 	}
diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
index 80f92ce..9098aca 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.h
+++ b/virt/kvm/arm/vgic/vgic-mmio.h
@@ -162,12 +162,9 @@ unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
 
 unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
 
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 u64 vgic_sanitise_outer_cacheability(u64 reg);
 u64 vgic_sanitise_inner_cacheability(u64 reg);
 u64 vgic_sanitise_shareability(u64 reg);
 u64 vgic_sanitise_field(u64 reg, u64 field_mask, int field_shift,
 			u64 (*sanitise_fn)(u64));
 #endif
-
-#endif
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index 1d8e21d..633c512 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -72,7 +72,6 @@ static inline void vgic_get_irq_kref(struct vgic_irq *irq)
 	kref_get(&irq->refcount);
 }
 
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu);
 void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu);
 void vgic_v3_populate_lr(struct kvm_vcpu *vcpu, struct vgic_irq *irq, int lr);
@@ -84,62 +83,13 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu);
 int vgic_v3_probe(const struct gic_kvm_info *info);
 int vgic_v3_map_resources(struct kvm *kvm);
 int vgic_register_redist_iodevs(struct kvm *kvm, gpa_t dist_base_address);
+
+#ifdef CONFIG_KVM_ARM_VGIC_V3_ITS
 bool vgic_has_its(struct kvm *kvm);
 int kvm_vgic_register_its_device(void);
 void vgic_enable_lpis(struct kvm_vcpu *vcpu);
 int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi);
 #else
-static inline void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu)
-{
-}
-
-static inline void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu)
-{
-}
-
-static inline void vgic_v3_populate_lr(struct kvm_vcpu *vcpu,
-				       struct vgic_irq *irq, int lr)
-{
-}
-
-static inline void vgic_v3_clear_lr(struct kvm_vcpu *vcpu, int lr)
-{
-}
-
-static inline void vgic_v3_set_underflow(struct kvm_vcpu *vcpu)
-{
-}
-
-static inline
-void vgic_v3_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
-{
-}
-
-static inline
-void vgic_v3_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
-{
-}
-
-static inline void vgic_v3_enable(struct kvm_vcpu *vcpu)
-{
-}
-
-static inline int vgic_v3_probe(const struct gic_kvm_info *info)
-{
-	return -ENODEV;
-}
-
-static inline int vgic_v3_map_resources(struct kvm *kvm)
-{
-	return -ENODEV;
-}
-
-static inline int vgic_register_redist_iodevs(struct kvm *kvm,
-					      gpa_t dist_base_address)
-{
-	return -ENODEV;
-}
-
 static inline bool vgic_has_its(struct kvm *kvm)
 {
 	return false;
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 74+ messages in thread
* [PATCH v2 7/7] ARM: KVM: Unlock vgic-v3 support
@ 2016-08-16 10:46   ` Vladimir Murzin
  0 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-08-16 10:46 UTC (permalink / raw)
  To: linux-arm-kernel

It is time to get access to common version of vgic-v3.

We basically would need to tell build system how to pick it up and
undo KVM_ARM_VGIC_V3 guarding introduced in 4f64cb6 ("arm/arm64: KVM:
Only allow 64bit hosts to build VGICv3") and remove stubs. However,
since vgic got ITS support KVM_ARM_VGIC_V3 tend to protect a little
bit more than just plain vgic-v3 - this guard is used for ITS too
which is not supported in 32-bit world yet.  So, along with removal of
KVM_ARM_VGIC_V3 guard introduce the new one - KVM_ARM_VGIC_V3_ITS to
protect ITS related code.

The only unpleasant part is how we decide which save/restore sequence to
use under __vgic_save_state() - we don't have patching framework in hand
like arm64, so have to check runtime on every invocation.

Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
---
 arch/arm/include/asm/kvm_host.h     |    4 +++
 arch/arm/include/asm/kvm_hyp.h      |    5 ++++
 arch/arm/kvm/Makefile               |    2 ++
 arch/arm/kvm/hyp/Makefile           |    1 +
 arch/arm/kvm/hyp/switch.c           |   20 +++++++++++--
 arch/arm64/kvm/Kconfig              |    4 +--
 include/kvm/arm_vgic.h              |    8 ------
 virt/kvm/arm/vgic/vgic-kvm-device.c |   10 ++-----
 virt/kvm/arm/vgic/vgic-mmio-v3.c    |    2 ++
 virt/kvm/arm/vgic/vgic-mmio.c       |    2 --
 virt/kvm/arm/vgic/vgic-mmio.h       |    3 --
 virt/kvm/arm/vgic/vgic.h            |   54 ++---------------------------------
 12 files changed, 38 insertions(+), 77 deletions(-)

diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
index de338d9..1312597 100644
--- a/arch/arm/include/asm/kvm_host.h
+++ b/arch/arm/include/asm/kvm_host.h
@@ -39,7 +39,11 @@
 
 #include <kvm/arm_vgic.h>
 
+#ifdef CONFIG_ARM_GIC_V3
+#define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS
+#else
 #define KVM_MAX_VCPUS VGIC_V2_MAX_CPUS
+#endif
 
 #define KVM_REQ_VCPU_EXIT	8
 
diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
index e604ad68..95669b3 100644
--- a/arch/arm/include/asm/kvm_hyp.h
+++ b/arch/arm/include/asm/kvm_hyp.h
@@ -88,6 +88,8 @@
 
 #define VFP_FPEXC	__ACCESS_VFP(FPEXC)
 
+#define ID_PFR1		__ACCESS_CP15(c0, 0, c1, 1)
+
 /* AArch64 compatibility macros, only for the timer so far */
 #define read_sysreg_el0(r)		read_sysreg(r##_el0)
 #define write_sysreg_el0(v, r)		write_sysreg(v, r##_el0)
@@ -103,6 +105,9 @@ void __timer_restore_state(struct kvm_vcpu *vcpu);
 void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
 void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
 
+void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
+void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
+
 void __sysreg_save_state(struct kvm_cpu_context *ctxt);
 void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
 
diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
index 10d77a6..043d817f 100644
--- a/arch/arm/kvm/Makefile
+++ b/arch/arm/kvm/Makefile
@@ -26,8 +26,10 @@ obj-y += $(KVM)/arm/vgic/vgic.o
 obj-y += $(KVM)/arm/vgic/vgic-init.o
 obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
 obj-y += $(KVM)/arm/vgic/vgic-v2.o
+obj-y += $(KVM)/arm/vgic/vgic-v3.o
 obj-y += $(KVM)/arm/vgic/vgic-mmio.o
 obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
+obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
 obj-y += $(KVM)/arm/vgic/vgic-kvm-device.o
 obj-y += $(KVM)/irqchip.o
 obj-y += $(KVM)/arm/arch_timer.o
diff --git a/arch/arm/kvm/hyp/Makefile b/arch/arm/kvm/hyp/Makefile
index 8dfa5f7..3023bb5 100644
--- a/arch/arm/kvm/hyp/Makefile
+++ b/arch/arm/kvm/hyp/Makefile
@@ -5,6 +5,7 @@
 KVM=../../../../virt/kvm
 
 obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
+obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
 obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
 
 obj-$(CONFIG_KVM_ARM_HOST) += tlb.o
diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
index b13caa9..9666bae 100644
--- a/arch/arm/kvm/hyp/switch.c
+++ b/arch/arm/kvm/hyp/switch.c
@@ -15,6 +15,8 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
+#include <linux/irqchip/arm-gic-v3.h>
+
 #include <asm/kvm_asm.h>
 #include <asm/kvm_hyp.h>
 
@@ -74,14 +76,28 @@ static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
 	write_sysreg(read_sysreg(MIDR), VPIDR);
 }
 
+static bool __hyp_text __has_useable_gicv3_cpuif(void)
+{
+	if (IS_ENABLED(CONFIG_ARM_GIC_V3) && (read_sysreg(ID_PFR1) >> 28))
+		return !!(read_sysreg(ICC_HSRE) & ICC_SRE_EL2_SRE);
+	else
+		return false;
+}
+
 static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
 {
-	__vgic_v2_save_state(vcpu);
+	if (__has_useable_gicv3_cpuif())
+		__vgic_v3_save_state(vcpu);
+	else
+		__vgic_v2_save_state(vcpu);
 }
 
 static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
 {
-	__vgic_v2_restore_state(vcpu);
+	if (__has_useable_gicv3_cpuif())
+		__vgic_v3_restore_state(vcpu);
+	else
+		__vgic_v2_restore_state(vcpu);
 }
 
 static bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu)
diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index 9c9edc9..6eaf12c 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -16,7 +16,7 @@ menuconfig VIRTUALIZATION
 
 if VIRTUALIZATION
 
-config KVM_ARM_VGIC_V3
+config KVM_ARM_VGIC_V3_ITS
 	bool
 
 config KVM
@@ -34,7 +34,7 @@ config KVM
 	select KVM_VFIO
 	select HAVE_KVM_EVENTFD
 	select HAVE_KVM_IRQFD
-	select KVM_ARM_VGIC_V3
+	select KVM_ARM_VGIC_V3_ITS
 	select KVM_ARM_PMU if HW_PERF_EVENTS
 	select HAVE_KVM_MSI
 	select HAVE_KVM_IRQCHIP
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 19b698e..7462138 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -217,7 +217,6 @@ struct vgic_v2_cpu_if {
 };
 
 struct vgic_v3_cpu_if {
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 	u32		vgic_hcr;
 	u32		vgic_vmcr;
 	u32		vgic_sre;	/* Restored only, change ignored */
@@ -227,7 +226,6 @@ struct vgic_v3_cpu_if {
 	u32		vgic_ap0r[4];
 	u32		vgic_ap1r[4];
 	u64		vgic_lr[VGIC_V3_MAX_LRS];
-#endif
 };
 
 struct vgic_cpu {
@@ -294,13 +292,7 @@ bool kvm_vcpu_has_pending_irqs(struct kvm_vcpu *vcpu);
 void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu);
 void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu);
 
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg);
-#else
-static inline void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)
-{
-}
-#endif
 
 /**
  * kvm_vgic_get_max_vcpus - Get the maximum number of VCPUs allowed by HW
diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
index 1813f93..e4f0c33 100644
--- a/virt/kvm/arm/vgic/vgic-kvm-device.c
+++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
@@ -71,7 +71,6 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write)
 		addr_ptr = &vgic->vgic_cpu_base;
 		alignment = SZ_4K;
 		break;
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 	case KVM_VGIC_V3_ADDR_TYPE_DIST:
 		type_needed = KVM_DEV_TYPE_ARM_VGIC_V3;
 		addr_ptr = &vgic->vgic_dist_base;
@@ -82,7 +81,6 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write)
 		addr_ptr = &vgic->vgic_redist_base;
 		alignment = SZ_64K;
 		break;
-#endif
 	default:
 		r = -ENODEV;
 		goto out;
@@ -219,15 +217,15 @@ int kvm_register_vgic_device(unsigned long type)
 		ret = kvm_register_device_ops(&kvm_arm_vgic_v2_ops,
 					      KVM_DEV_TYPE_ARM_VGIC_V2);
 		break;
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 	case KVM_DEV_TYPE_ARM_VGIC_V3:
 		ret = kvm_register_device_ops(&kvm_arm_vgic_v3_ops,
 					      KVM_DEV_TYPE_ARM_VGIC_V3);
+#ifdef CONFIG_KVM_ARM_VGIC_V3_ITS
 		if (ret)
 			break;
 		ret = kvm_vgic_register_its_device();
-		break;
 #endif
+		break;
 	}
 
 	return ret;
@@ -389,8 +387,6 @@ struct kvm_device_ops kvm_arm_vgic_v2_ops = {
 
 /* V3 ops */
 
-#ifdef CONFIG_KVM_ARM_VGIC_V3
-
 static int vgic_v3_set_attr(struct kvm_device *dev,
 			    struct kvm_device_attr *attr)
 {
@@ -433,5 +429,3 @@ struct kvm_device_ops kvm_arm_vgic_v3_ops = {
 	.get_attr = vgic_v3_get_attr,
 	.has_attr = vgic_v3_has_attr,
 };
-
-#endif /* CONFIG_KVM_ARM_VGIC_V3 */
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index cc20b60..4709dd65 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -42,6 +42,7 @@ u64 update_64bit_reg(u64 reg, unsigned int offset, unsigned int len,
 	return reg | ((u64)val << lower);
 }
 
+#ifdef CONFIG_KVM_ARM_VGIC_V3_ITS
 bool vgic_has_its(struct kvm *kvm)
 {
 	struct vgic_dist *dist = &kvm->arch.vgic;
@@ -51,6 +52,7 @@ bool vgic_has_its(struct kvm *kvm)
 
 	return dist->has_its;
 }
+#endif
 
 static unsigned long vgic_mmio_read_v3_misc(struct kvm_vcpu *vcpu,
 					    gpa_t addr, unsigned int len)
diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
index 3bad3c5..e18b30d 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.c
+++ b/virt/kvm/arm/vgic/vgic-mmio.c
@@ -550,11 +550,9 @@ int vgic_register_dist_iodev(struct kvm *kvm, gpa_t dist_base_address,
 	case VGIC_V2:
 		len = vgic_v2_init_dist_iodev(io_device);
 		break;
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 	case VGIC_V3:
 		len = vgic_v3_init_dist_iodev(io_device);
 		break;
-#endif
 	default:
 		BUG_ON(1);
 	}
diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
index 80f92ce..9098aca 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.h
+++ b/virt/kvm/arm/vgic/vgic-mmio.h
@@ -162,12 +162,9 @@ unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
 
 unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
 
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 u64 vgic_sanitise_outer_cacheability(u64 reg);
 u64 vgic_sanitise_inner_cacheability(u64 reg);
 u64 vgic_sanitise_shareability(u64 reg);
 u64 vgic_sanitise_field(u64 reg, u64 field_mask, int field_shift,
 			u64 (*sanitise_fn)(u64));
 #endif
-
-#endif
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index 1d8e21d..633c512 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -72,7 +72,6 @@ static inline void vgic_get_irq_kref(struct vgic_irq *irq)
 	kref_get(&irq->refcount);
 }
 
-#ifdef CONFIG_KVM_ARM_VGIC_V3
 void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu);
 void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu);
 void vgic_v3_populate_lr(struct kvm_vcpu *vcpu, struct vgic_irq *irq, int lr);
@@ -84,62 +83,13 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu);
 int vgic_v3_probe(const struct gic_kvm_info *info);
 int vgic_v3_map_resources(struct kvm *kvm);
 int vgic_register_redist_iodevs(struct kvm *kvm, gpa_t dist_base_address);
+
+#ifdef CONFIG_KVM_ARM_VGIC_V3_ITS
 bool vgic_has_its(struct kvm *kvm);
 int kvm_vgic_register_its_device(void);
 void vgic_enable_lpis(struct kvm_vcpu *vcpu);
 int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi);
 #else
-static inline void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu)
-{
-}
-
-static inline void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu)
-{
-}
-
-static inline void vgic_v3_populate_lr(struct kvm_vcpu *vcpu,
-				       struct vgic_irq *irq, int lr)
-{
-}
-
-static inline void vgic_v3_clear_lr(struct kvm_vcpu *vcpu, int lr)
-{
-}
-
-static inline void vgic_v3_set_underflow(struct kvm_vcpu *vcpu)
-{
-}
-
-static inline
-void vgic_v3_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
-{
-}
-
-static inline
-void vgic_v3_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
-{
-}
-
-static inline void vgic_v3_enable(struct kvm_vcpu *vcpu)
-{
-}
-
-static inline int vgic_v3_probe(const struct gic_kvm_info *info)
-{
-	return -ENODEV;
-}
-
-static inline int vgic_v3_map_resources(struct kvm *kvm)
-{
-	return -ENODEV;
-}
-
-static inline int vgic_register_redist_iodevs(struct kvm *kvm,
-					      gpa_t dist_base_address)
-{
-	return -ENODEV;
-}
-
 static inline bool vgic_has_its(struct kvm *kvm)
 {
 	return false;
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 0/7] ARM: KVM: Support for vgic-v3
  2016-08-16 10:46 ` Vladimir Murzin
@ 2016-09-05 11:28   ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-05 11:28 UTC (permalink / raw)
  To: Vladimir Murzin; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

On Tue, Aug 16, 2016 at 11:46:51AM +0100, Vladimir Murzin wrote:
> Hi,
> 
> This is an attempt to make use vgic-v3 under arch/arm since save-restore
> functionality got re-written in C and can be shared between arm/arm64
> like it has already been done for vgic-v2 and timer.
> 
> With this patches I'm able to get 32 core ARMv7 guest boot:

Is it really an ARMv7 guest or an AArch32 ARMv8 guest?

> 
> ...
> GICv3: CPU31: found redistributor 703 region 0:0x000000003ffd0000
> CPU31: thread -1, cpu 3, socket 7, mpidr 80000703
> Brought up 32 CPUs
> SMP: Total of 32 processors activated (768.00 BogoMIPS).
> CPU: All CPU(s) started in SVC mode.
> ...
> 
> Additionally, quite lightweight test based on Self IPI guest test[1]
> has been run with up to 255 cpus.
> 
> [1] http://www.spinics.net/lists/kvm/msg128974.html
> 
> Changelog:
> 
>     v1 -> v2
>        - rebased on v4.8-rc2
>        - introduced guard for ITS code
> 
> Thanks!
> 
> Vladimir Murzin (7):
>   arm64: KVM: Move GIC accessors to arch_gicv3.h
>   arm64: KVM: Move vgic-v3 save/restore to virt/kvm/arm/hyp
>   KVM: arm: vgic-new: improve compatibility with 32-bit
>   ARM: update MPIDR accessors macro
>   ARM: move system register accessors to asm/cp15.h
>   ARM: KVM: Get ready to use vgic-v3
>   ARM: KVM: Unlock vgic-v3 support
> 
>  arch/arm/include/asm/arch_gicv3.h                 |   91 +++++++++++++++++----
>  arch/arm/include/asm/cp15.h                       |   15 ++++
>  arch/arm/include/asm/cputype.h                    |    3 +-
>  arch/arm/include/asm/kvm_asm.h                    |    3 +
>  arch/arm/include/asm/kvm_host.h                   |    4 +
>  arch/arm/include/asm/kvm_hyp.h                    |   20 ++---
>  arch/arm/include/uapi/asm/kvm.h                   |    7 ++
>  arch/arm/kvm/Makefile                             |    2 +
>  arch/arm/kvm/coproc.c                             |   36 ++++++++
>  arch/arm/kvm/hyp/Makefile                         |    1 +
>  arch/arm/kvm/hyp/switch.c                         |   20 ++++-
>  arch/arm64/include/asm/arch_gicv3.h               |   13 +++
>  arch/arm64/kvm/Kconfig                            |    4 +-
>  arch/arm64/kvm/hyp/Makefile                       |    2 +-
>  include/kvm/arm_vgic.h                            |    8 --
>  {arch/arm64/kvm => virt/kvm/arm}/hyp/vgic-v3-sr.c |   13 ---
>  virt/kvm/arm/vgic/vgic-kvm-device.c               |   10 +--
>  virt/kvm/arm/vgic/vgic-mmio-v3.c                  |    8 +-
>  virt/kvm/arm/vgic/vgic-mmio.c                     |    2 -
>  virt/kvm/arm/vgic/vgic-mmio.h                     |    5 +-
>  virt/kvm/arm/vgic/vgic.h                          |   54 +-----------
>  21 files changed, 195 insertions(+), 126 deletions(-)
>  rename {arch/arm64/kvm => virt/kvm/arm}/hyp/vgic-v3-sr.c (96%)
> 
> -- 
> 1.7.9.5
> 

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 0/7] ARM: KVM: Support for vgic-v3
@ 2016-09-05 11:28   ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-05 11:28 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Aug 16, 2016 at 11:46:51AM +0100, Vladimir Murzin wrote:
> Hi,
> 
> This is an attempt to make use vgic-v3 under arch/arm since save-restore
> functionality got re-written in C and can be shared between arm/arm64
> like it has already been done for vgic-v2 and timer.
> 
> With this patches I'm able to get 32 core ARMv7 guest boot:

Is it really an ARMv7 guest or an AArch32 ARMv8 guest?

> 
> ...
> GICv3: CPU31: found redistributor 703 region 0:0x000000003ffd0000
> CPU31: thread -1, cpu 3, socket 7, mpidr 80000703
> Brought up 32 CPUs
> SMP: Total of 32 processors activated (768.00 BogoMIPS).
> CPU: All CPU(s) started in SVC mode.
> ...
> 
> Additionally, quite lightweight test based on Self IPI guest test[1]
> has been run with up to 255 cpus.
> 
> [1] http://www.spinics.net/lists/kvm/msg128974.html
> 
> Changelog:
> 
>     v1 -> v2
>        - rebased on v4.8-rc2
>        - introduced guard for ITS code
> 
> Thanks!
> 
> Vladimir Murzin (7):
>   arm64: KVM: Move GIC accessors to arch_gicv3.h
>   arm64: KVM: Move vgic-v3 save/restore to virt/kvm/arm/hyp
>   KVM: arm: vgic-new: improve compatibility with 32-bit
>   ARM: update MPIDR accessors macro
>   ARM: move system register accessors to asm/cp15.h
>   ARM: KVM: Get ready to use vgic-v3
>   ARM: KVM: Unlock vgic-v3 support
> 
>  arch/arm/include/asm/arch_gicv3.h                 |   91 +++++++++++++++++----
>  arch/arm/include/asm/cp15.h                       |   15 ++++
>  arch/arm/include/asm/cputype.h                    |    3 +-
>  arch/arm/include/asm/kvm_asm.h                    |    3 +
>  arch/arm/include/asm/kvm_host.h                   |    4 +
>  arch/arm/include/asm/kvm_hyp.h                    |   20 ++---
>  arch/arm/include/uapi/asm/kvm.h                   |    7 ++
>  arch/arm/kvm/Makefile                             |    2 +
>  arch/arm/kvm/coproc.c                             |   36 ++++++++
>  arch/arm/kvm/hyp/Makefile                         |    1 +
>  arch/arm/kvm/hyp/switch.c                         |   20 ++++-
>  arch/arm64/include/asm/arch_gicv3.h               |   13 +++
>  arch/arm64/kvm/Kconfig                            |    4 +-
>  arch/arm64/kvm/hyp/Makefile                       |    2 +-
>  include/kvm/arm_vgic.h                            |    8 --
>  {arch/arm64/kvm => virt/kvm/arm}/hyp/vgic-v3-sr.c |   13 ---
>  virt/kvm/arm/vgic/vgic-kvm-device.c               |   10 +--
>  virt/kvm/arm/vgic/vgic-mmio-v3.c                  |    8 +-
>  virt/kvm/arm/vgic/vgic-mmio.c                     |    2 -
>  virt/kvm/arm/vgic/vgic-mmio.h                     |    5 +-
>  virt/kvm/arm/vgic/vgic.h                          |   54 +-----------
>  21 files changed, 195 insertions(+), 126 deletions(-)
>  rename {arch/arm64/kvm => virt/kvm/arm}/hyp/vgic-v3-sr.c (96%)
> 
> -- 
> 1.7.9.5
> 

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 1/7] arm64: KVM: Move GIC accessors to arch_gicv3.h
  2016-08-16 10:46   ` Vladimir Murzin
@ 2016-09-05 11:28     ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-05 11:28 UTC (permalink / raw)
  To: Vladimir Murzin; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

On Tue, Aug 16, 2016 at 11:46:52AM +0100, Vladimir Murzin wrote:
> Since we are going to share vgic-v3 save/restore code with ARM
> keep arch specific accessors separately.
> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 1/7] arm64: KVM: Move GIC accessors to arch_gicv3.h
@ 2016-09-05 11:28     ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-05 11:28 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Aug 16, 2016 at 11:46:52AM +0100, Vladimir Murzin wrote:
> Since we are going to share vgic-v3 save/restore code with ARM
> keep arch specific accessors separately.
> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 3/7] KVM: arm: vgic-new: improve compatibility with 32-bit
  2016-08-16 10:46   ` Vladimir Murzin
@ 2016-09-05 11:29     ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-05 11:29 UTC (permalink / raw)
  To: Vladimir Murzin; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

Hi Vladimir,

I think commit title is too vague, can you be more specific?

On Tue, Aug 16, 2016 at 11:46:54AM +0100, Vladimir Murzin wrote:
> We have couple of 64-bit register defined in GICv3 architecture, so

'a couple',  'registers' (plural)

> "unsigned long" kind of accessors wouldn't work for 32-bit. However,

'wouldn't work for 32-bit' is kind of generic as well.  Perhaps you mean
that unsigned long accesses to these registers will only access a single
32-bit work of that register.

> these registers can't be access as 64-bit in a one go if we run 32-bit

'accessed', 's/in one go/with a single instruction/' ?

'a 32-bit host'

> host simply because KVM doesn't support multiple load/store on MMIO

by 'multiple load/store' you mean the 'load/store multiple' instructions
specifically, right?  Not a sequence of multiple loads and stores.  I
think you should be more specific here as well, for example, I think
ldrd and strd are problematic as well.

> space.
> 
> It means that 32-bit guest access these registers in 32-bit chunks, so

'a 32-bit guest', 'accesses'

> the only thing we need to do is to ensure that extract_bytes() always
> takes 64-bit data.
> 
> Since we are here fix couple of other width related issues by using
> ULL variants over UL.
> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-mmio-v3.c |    6 +++---
>  virt/kvm/arm/vgic/vgic-mmio.h    |    2 +-
>  2 files changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> index ff668e0..cc20b60 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> @@ -23,7 +23,7 @@
>  #include "vgic-mmio.h"
>  
>  /* extract @num bytes at @offset bytes offset in data */
> -unsigned long extract_bytes(unsigned long data, unsigned int offset,
> +unsigned long extract_bytes(u64 data, unsigned int offset,
>  			    unsigned int num)
>  {
>  	return (data >> (offset * 8)) & GENMASK_ULL(num * 8 - 1, 0);
> @@ -179,7 +179,7 @@ static unsigned long vgic_mmio_read_v3r_typer(struct kvm_vcpu *vcpu,
>  	int target_vcpu_id = vcpu->vcpu_id;
>  	u64 value;
>  
> -	value = (mpidr & GENMASK(23, 0)) << 32;
> +	value = (mpidr & GENMASK_ULL(23, 0)) << 32;

why does this make a difference when mpidr is an unsigned long?

>  	value |= ((target_vcpu_id & 0xffff) << 8);
>  	if (target_vcpu_id == atomic_read(&vcpu->kvm->online_vcpus) - 1)
>  		value |= GICR_TYPER_LAST;
> @@ -603,7 +603,7 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)
>  	bool broadcast;
>  
>  	sgi = (reg & ICC_SGI1R_SGI_ID_MASK) >> ICC_SGI1R_SGI_ID_SHIFT;
> -	broadcast = reg & BIT(ICC_SGI1R_IRQ_ROUTING_MODE_BIT);
> +	broadcast = reg & BIT_ULL(ICC_SGI1R_IRQ_ROUTING_MODE_BIT);
>  	target_cpus = (reg & ICC_SGI1R_TARGET_LIST_MASK) >> ICC_SGI1R_TARGET_LIST_SHIFT;
>  	mpidr = SGI_AFFINITY_LEVEL(reg, 3);
>  	mpidr |= SGI_AFFINITY_LEVEL(reg, 2);
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
> index 0b3ecf9..80f92ce 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.h
> +++ b/virt/kvm/arm/vgic/vgic-mmio.h
> @@ -96,7 +96,7 @@ unsigned long vgic_data_mmio_bus_to_host(const void *val, unsigned int len);
>  void vgic_data_host_to_mmio_bus(void *buf, unsigned int len,
>  				unsigned long data);
>  
> -unsigned long extract_bytes(unsigned long data, unsigned int offset,
> +unsigned long extract_bytes(u64 data, unsigned int offset,
>  			    unsigned int num);
>  
>  u64 update_64bit_reg(u64 reg, unsigned int offset, unsigned int len,
> -- 
> 1.7.9.5
> 

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 3/7] KVM: arm: vgic-new: improve compatibility with 32-bit
@ 2016-09-05 11:29     ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-05 11:29 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Vladimir,

I think commit title is too vague, can you be more specific?

On Tue, Aug 16, 2016 at 11:46:54AM +0100, Vladimir Murzin wrote:
> We have couple of 64-bit register defined in GICv3 architecture, so

'a couple',  'registers' (plural)

> "unsigned long" kind of accessors wouldn't work for 32-bit. However,

'wouldn't work for 32-bit' is kind of generic as well.  Perhaps you mean
that unsigned long accesses to these registers will only access a single
32-bit work of that register.

> these registers can't be access as 64-bit in a one go if we run 32-bit

'accessed', 's/in one go/with a single instruction/' ?

'a 32-bit host'

> host simply because KVM doesn't support multiple load/store on MMIO

by 'multiple load/store' you mean the 'load/store multiple' instructions
specifically, right?  Not a sequence of multiple loads and stores.  I
think you should be more specific here as well, for example, I think
ldrd and strd are problematic as well.

> space.
> 
> It means that 32-bit guest access these registers in 32-bit chunks, so

'a 32-bit guest', 'accesses'

> the only thing we need to do is to ensure that extract_bytes() always
> takes 64-bit data.
> 
> Since we are here fix couple of other width related issues by using
> ULL variants over UL.
> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> ---
>  virt/kvm/arm/vgic/vgic-mmio-v3.c |    6 +++---
>  virt/kvm/arm/vgic/vgic-mmio.h    |    2 +-
>  2 files changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> index ff668e0..cc20b60 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> @@ -23,7 +23,7 @@
>  #include "vgic-mmio.h"
>  
>  /* extract @num bytes at @offset bytes offset in data */
> -unsigned long extract_bytes(unsigned long data, unsigned int offset,
> +unsigned long extract_bytes(u64 data, unsigned int offset,
>  			    unsigned int num)
>  {
>  	return (data >> (offset * 8)) & GENMASK_ULL(num * 8 - 1, 0);
> @@ -179,7 +179,7 @@ static unsigned long vgic_mmio_read_v3r_typer(struct kvm_vcpu *vcpu,
>  	int target_vcpu_id = vcpu->vcpu_id;
>  	u64 value;
>  
> -	value = (mpidr & GENMASK(23, 0)) << 32;
> +	value = (mpidr & GENMASK_ULL(23, 0)) << 32;

why does this make a difference when mpidr is an unsigned long?

>  	value |= ((target_vcpu_id & 0xffff) << 8);
>  	if (target_vcpu_id == atomic_read(&vcpu->kvm->online_vcpus) - 1)
>  		value |= GICR_TYPER_LAST;
> @@ -603,7 +603,7 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)
>  	bool broadcast;
>  
>  	sgi = (reg & ICC_SGI1R_SGI_ID_MASK) >> ICC_SGI1R_SGI_ID_SHIFT;
> -	broadcast = reg & BIT(ICC_SGI1R_IRQ_ROUTING_MODE_BIT);
> +	broadcast = reg & BIT_ULL(ICC_SGI1R_IRQ_ROUTING_MODE_BIT);
>  	target_cpus = (reg & ICC_SGI1R_TARGET_LIST_MASK) >> ICC_SGI1R_TARGET_LIST_SHIFT;
>  	mpidr = SGI_AFFINITY_LEVEL(reg, 3);
>  	mpidr |= SGI_AFFINITY_LEVEL(reg, 2);
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
> index 0b3ecf9..80f92ce 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.h
> +++ b/virt/kvm/arm/vgic/vgic-mmio.h
> @@ -96,7 +96,7 @@ unsigned long vgic_data_mmio_bus_to_host(const void *val, unsigned int len);
>  void vgic_data_host_to_mmio_bus(void *buf, unsigned int len,
>  				unsigned long data);
>  
> -unsigned long extract_bytes(unsigned long data, unsigned int offset,
> +unsigned long extract_bytes(u64 data, unsigned int offset,
>  			    unsigned int num);
>  
>  u64 update_64bit_reg(u64 reg, unsigned int offset, unsigned int len,
> -- 
> 1.7.9.5
> 

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 4/7] ARM: update MPIDR accessors macro
  2016-08-16 10:46   ` Vladimir Murzin
@ 2016-09-05 11:29     ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-05 11:29 UTC (permalink / raw)
  To: Vladimir Murzin; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

The title of this patch is quite generic, especially the 'update'.
Perhaps say: 

"ARM: Change MPIDR_AFFINITY_LEVEL to ignore Aff3"

On Tue, Aug 16, 2016 at 11:46:55AM +0100, Vladimir Murzin wrote:
> vgic-v3 driver queries CPU affinity level up to Aff3, which is valid for arm64.
> However, for arm up to Aff2 levels are supported, so querying for 3rd level
> ends with upper bits of MPIDR are treated as valid affinity level which
> is not true. So, report zero for any affinity level above 2.
> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> ---
>  arch/arm/include/asm/cputype.h |    3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/arm/include/asm/cputype.h b/arch/arm/include/asm/cputype.h
> index 1ee94c7..96cef49 100644
> --- a/arch/arm/include/asm/cputype.h
> +++ b/arch/arm/include/asm/cputype.h
> @@ -55,9 +55,10 @@
>  
>  #define MPIDR_LEVEL_BITS 8
>  #define MPIDR_LEVEL_MASK ((1 << MPIDR_LEVEL_BITS) - 1)
> +#define MPIDR_LEVEL_SHIFT(level) (MPIDR_LEVEL_BITS * level)
>  
>  #define MPIDR_AFFINITY_LEVEL(mpidr, level) \
> -	((mpidr >> (MPIDR_LEVEL_BITS * level)) & MPIDR_LEVEL_MASK)
> +	((level < 3) ? ((mpidr >> MPIDR_LEVEL_SHIFT(level)) & MPIDR_LEVEL_MASK) : 0)

Instead of using a conditional here, you could just apply the
MPIDR_HWID_BITMASK to the mpidr argument instead.

>  
>  #define ARM_CPU_IMP_ARM			0x41
>  #define ARM_CPU_IMP_INTEL		0x69
> -- 
> 1.7.9.5
> 

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 4/7] ARM: update MPIDR accessors macro
@ 2016-09-05 11:29     ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-05 11:29 UTC (permalink / raw)
  To: linux-arm-kernel

The title of this patch is quite generic, especially the 'update'.
Perhaps say: 

"ARM: Change MPIDR_AFFINITY_LEVEL to ignore Aff3"

On Tue, Aug 16, 2016 at 11:46:55AM +0100, Vladimir Murzin wrote:
> vgic-v3 driver queries CPU affinity level up to Aff3, which is valid for arm64.
> However, for arm up to Aff2 levels are supported, so querying for 3rd level
> ends with upper bits of MPIDR are treated as valid affinity level which
> is not true. So, report zero for any affinity level above 2.
> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> ---
>  arch/arm/include/asm/cputype.h |    3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/arm/include/asm/cputype.h b/arch/arm/include/asm/cputype.h
> index 1ee94c7..96cef49 100644
> --- a/arch/arm/include/asm/cputype.h
> +++ b/arch/arm/include/asm/cputype.h
> @@ -55,9 +55,10 @@
>  
>  #define MPIDR_LEVEL_BITS 8
>  #define MPIDR_LEVEL_MASK ((1 << MPIDR_LEVEL_BITS) - 1)
> +#define MPIDR_LEVEL_SHIFT(level) (MPIDR_LEVEL_BITS * level)
>  
>  #define MPIDR_AFFINITY_LEVEL(mpidr, level) \
> -	((mpidr >> (MPIDR_LEVEL_BITS * level)) & MPIDR_LEVEL_MASK)
> +	((level < 3) ? ((mpidr >> MPIDR_LEVEL_SHIFT(level)) & MPIDR_LEVEL_MASK) : 0)

Instead of using a conditional here, you could just apply the
MPIDR_HWID_BITMASK to the mpidr argument instead.

>  
>  #define ARM_CPU_IMP_ARM			0x41
>  #define ARM_CPU_IMP_INTEL		0x69
> -- 
> 1.7.9.5
> 

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 5/7] ARM: move system register accessors to asm/cp15.h
  2016-08-16 10:46   ` Vladimir Murzin
@ 2016-09-05 11:29     ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-05 11:29 UTC (permalink / raw)
  To: Vladimir Murzin; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

On Tue, Aug 16, 2016 at 11:46:56AM +0100, Vladimir Murzin wrote:
> Macro __ACCESS_CP15{_64} is defined in two headers (arch_gicv3.h and
> kvm_hyp.h) which are going to be requested by vgic-v3 altogether.
> GCC would not like it because it'd see that macro is redefined and (hey!)
> they are different.  So, let's put only single macro version under common
> place and use it everywhere.

I'm sorry, but I don't understand this commit text.

> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> ---
>  arch/arm/include/asm/arch_gicv3.h |   27 +++++++++++----------------
>  arch/arm/include/asm/cp15.h       |   15 +++++++++++++++
>  arch/arm/include/asm/kvm_hyp.h    |   15 +--------------
>  3 files changed, 27 insertions(+), 30 deletions(-)
> 
> diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
> index e08d151..af25c32 100644
> --- a/arch/arm/include/asm/arch_gicv3.h
> +++ b/arch/arm/include/asm/arch_gicv3.h
> @@ -22,9 +22,7 @@
>  
>  #include <linux/io.h>
>  #include <asm/barrier.h>
> -
> -#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
> -#define __ACCESS_CP15_64(Op1, CRm)		p15, Op1, %Q0, %R0, CRm
> +#include <asm/cp15.h>
>  
>  #define ICC_EOIR1			__ACCESS_CP15(c12, 0, c12, 1)
>  #define ICC_DIR				__ACCESS_CP15(c12, 0, c11, 1)
> @@ -102,58 +100,55 @@
>  
>  static inline void gic_write_eoir(u32 irq)
>  {
> -	asm volatile("mcr " __stringify(ICC_EOIR1) : : "r" (irq));
> +	write_sysreg(irq, ICC_EOIR1);
>  	isb();
>  }
>  
>  static inline void gic_write_dir(u32 val)
>  {
> -	asm volatile("mcr " __stringify(ICC_DIR) : : "r" (val));
> +	write_sysreg(val, ICC_DIR);
>  	isb();
>  }
>  
>  static inline u32 gic_read_iar(void)
>  {
> -	u32 irqstat;
> +	u32 irqstat = read_sysreg(ICC_IAR1);
>  
> -	asm volatile("mrc " __stringify(ICC_IAR1) : "=r" (irqstat));
>  	dsb(sy);
> +
>  	return irqstat;
>  }
>  
>  static inline void gic_write_pmr(u32 val)
>  {
> -	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
> +	write_sysreg(val, ICC_PMR);
>  }
>  
>  static inline void gic_write_ctlr(u32 val)
>  {
> -	asm volatile("mcr " __stringify(ICC_CTLR) : : "r" (val));
> +	write_sysreg(val, ICC_CTLR);
>  	isb();
>  }
>  
>  static inline void gic_write_grpen1(u32 val)
>  {
> -	asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
> +	write_sysreg(val, ICC_IGRPEN1);
>  	isb();
>  }
>  
>  static inline void gic_write_sgi1r(u64 val)
>  {
> -	asm volatile("mcrr " __stringify(ICC_SGI1R) : : "r" (val));
> +	write_sysreg(val, ICC_SGI1R);
>  }
>  
>  static inline u32 gic_read_sre(void)
>  {
> -	u32 val;
> -
> -	asm volatile("mrc " __stringify(ICC_SRE) : "=r" (val));
> -	return val;
> +	return read_sysreg(ICC_SRE);
>  }
>  
>  static inline void gic_write_sre(u32 val)
>  {
> -	asm volatile("mcr " __stringify(ICC_SRE) : : "r" (val));
> +	write_sysreg(val, ICC_SRE);
>  	isb();
>  }
>  
> diff --git a/arch/arm/include/asm/cp15.h b/arch/arm/include/asm/cp15.h
> index c3f1152..f661732 100644
> --- a/arch/arm/include/asm/cp15.h
> +++ b/arch/arm/include/asm/cp15.h
> @@ -47,6 +47,21 @@
>  #define vectors_high()	(0)
>  #endif
>  
> +#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
> +	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
> +#define __ACCESS_CP15_64(Op1, CRm)		\
> +	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
> +
> +#define __read_sysreg(r, w, c, t) ({				\
> +	t __val;						\
> +	asm volatile(r " " c : "=r" (__val));			\
> +	__val;							\
> +})
> +#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
> +
> +#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
> +#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
> +

I feel a bit strange about adding this sort of stuff in a
non-kvm-non-gic-specific ARM header file, without it being used (or
planned to be used) in a broader sense.

Is there not a way to keep the required changes local to KVM and the
gic?

>  #ifdef CONFIG_CPU_CP15
>  
>  extern unsigned long cr_alignment;	/* defined in entry-armv.S */
> diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
> index 6eaff28..e604ad68 100644
> --- a/arch/arm/include/asm/kvm_hyp.h
> +++ b/arch/arm/include/asm/kvm_hyp.h
> @@ -20,28 +20,15 @@
>  
>  #include <linux/compiler.h>
>  #include <linux/kvm_host.h>
> +#include <asm/cp15.h>
>  #include <asm/kvm_mmu.h>
>  #include <asm/vfp.h>
>  
>  #define __hyp_text __section(.hyp.text) notrace
>  
> -#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
> -	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
> -#define __ACCESS_CP15_64(Op1, CRm)		\
> -	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
>  #define __ACCESS_VFP(CRn)			\
>  	"mrc", "mcr", __stringify(p10, 7, %0, CRn, cr0, 0), u32
>  
> -#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
> -#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
> -
> -#define __read_sysreg(r, w, c, t) ({				\
> -	t __val;						\
> -	asm volatile(r " " c : "=r" (__val));			\
> -	__val;							\
> -})
> -#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
> -
>  #define write_special(v, r)					\
>  	asm volatile("msr " __stringify(r) ", %0" : : "r" (v))
>  #define read_special(r) ({					\
> -- 
> 1.7.9.5
> 

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 5/7] ARM: move system register accessors to asm/cp15.h
@ 2016-09-05 11:29     ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-05 11:29 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Aug 16, 2016 at 11:46:56AM +0100, Vladimir Murzin wrote:
> Macro __ACCESS_CP15{_64} is defined in two headers (arch_gicv3.h and
> kvm_hyp.h) which are going to be requested by vgic-v3 altogether.
> GCC would not like it because it'd see that macro is redefined and (hey!)
> they are different.  So, let's put only single macro version under common
> place and use it everywhere.

I'm sorry, but I don't understand this commit text.

> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> ---
>  arch/arm/include/asm/arch_gicv3.h |   27 +++++++++++----------------
>  arch/arm/include/asm/cp15.h       |   15 +++++++++++++++
>  arch/arm/include/asm/kvm_hyp.h    |   15 +--------------
>  3 files changed, 27 insertions(+), 30 deletions(-)
> 
> diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
> index e08d151..af25c32 100644
> --- a/arch/arm/include/asm/arch_gicv3.h
> +++ b/arch/arm/include/asm/arch_gicv3.h
> @@ -22,9 +22,7 @@
>  
>  #include <linux/io.h>
>  #include <asm/barrier.h>
> -
> -#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
> -#define __ACCESS_CP15_64(Op1, CRm)		p15, Op1, %Q0, %R0, CRm
> +#include <asm/cp15.h>
>  
>  #define ICC_EOIR1			__ACCESS_CP15(c12, 0, c12, 1)
>  #define ICC_DIR				__ACCESS_CP15(c12, 0, c11, 1)
> @@ -102,58 +100,55 @@
>  
>  static inline void gic_write_eoir(u32 irq)
>  {
> -	asm volatile("mcr " __stringify(ICC_EOIR1) : : "r" (irq));
> +	write_sysreg(irq, ICC_EOIR1);
>  	isb();
>  }
>  
>  static inline void gic_write_dir(u32 val)
>  {
> -	asm volatile("mcr " __stringify(ICC_DIR) : : "r" (val));
> +	write_sysreg(val, ICC_DIR);
>  	isb();
>  }
>  
>  static inline u32 gic_read_iar(void)
>  {
> -	u32 irqstat;
> +	u32 irqstat = read_sysreg(ICC_IAR1);
>  
> -	asm volatile("mrc " __stringify(ICC_IAR1) : "=r" (irqstat));
>  	dsb(sy);
> +
>  	return irqstat;
>  }
>  
>  static inline void gic_write_pmr(u32 val)
>  {
> -	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
> +	write_sysreg(val, ICC_PMR);
>  }
>  
>  static inline void gic_write_ctlr(u32 val)
>  {
> -	asm volatile("mcr " __stringify(ICC_CTLR) : : "r" (val));
> +	write_sysreg(val, ICC_CTLR);
>  	isb();
>  }
>  
>  static inline void gic_write_grpen1(u32 val)
>  {
> -	asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
> +	write_sysreg(val, ICC_IGRPEN1);
>  	isb();
>  }
>  
>  static inline void gic_write_sgi1r(u64 val)
>  {
> -	asm volatile("mcrr " __stringify(ICC_SGI1R) : : "r" (val));
> +	write_sysreg(val, ICC_SGI1R);
>  }
>  
>  static inline u32 gic_read_sre(void)
>  {
> -	u32 val;
> -
> -	asm volatile("mrc " __stringify(ICC_SRE) : "=r" (val));
> -	return val;
> +	return read_sysreg(ICC_SRE);
>  }
>  
>  static inline void gic_write_sre(u32 val)
>  {
> -	asm volatile("mcr " __stringify(ICC_SRE) : : "r" (val));
> +	write_sysreg(val, ICC_SRE);
>  	isb();
>  }
>  
> diff --git a/arch/arm/include/asm/cp15.h b/arch/arm/include/asm/cp15.h
> index c3f1152..f661732 100644
> --- a/arch/arm/include/asm/cp15.h
> +++ b/arch/arm/include/asm/cp15.h
> @@ -47,6 +47,21 @@
>  #define vectors_high()	(0)
>  #endif
>  
> +#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
> +	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
> +#define __ACCESS_CP15_64(Op1, CRm)		\
> +	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
> +
> +#define __read_sysreg(r, w, c, t) ({				\
> +	t __val;						\
> +	asm volatile(r " " c : "=r" (__val));			\
> +	__val;							\
> +})
> +#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
> +
> +#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
> +#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
> +

I feel a bit strange about adding this sort of stuff in a
non-kvm-non-gic-specific ARM header file, without it being used (or
planned to be used) in a broader sense.

Is there not a way to keep the required changes local to KVM and the
gic?

>  #ifdef CONFIG_CPU_CP15
>  
>  extern unsigned long cr_alignment;	/* defined in entry-armv.S */
> diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
> index 6eaff28..e604ad68 100644
> --- a/arch/arm/include/asm/kvm_hyp.h
> +++ b/arch/arm/include/asm/kvm_hyp.h
> @@ -20,28 +20,15 @@
>  
>  #include <linux/compiler.h>
>  #include <linux/kvm_host.h>
> +#include <asm/cp15.h>
>  #include <asm/kvm_mmu.h>
>  #include <asm/vfp.h>
>  
>  #define __hyp_text __section(.hyp.text) notrace
>  
> -#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
> -	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
> -#define __ACCESS_CP15_64(Op1, CRm)		\
> -	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
>  #define __ACCESS_VFP(CRn)			\
>  	"mrc", "mcr", __stringify(p10, 7, %0, CRn, cr0, 0), u32
>  
> -#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
> -#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
> -
> -#define __read_sysreg(r, w, c, t) ({				\
> -	t __val;						\
> -	asm volatile(r " " c : "=r" (__val));			\
> -	__val;							\
> -})
> -#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
> -
>  #define write_special(v, r)					\
>  	asm volatile("msr " __stringify(r) ", %0" : : "r" (v))
>  #define read_special(r) ({					\
> -- 
> 1.7.9.5
> 

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 6/7] ARM: KVM: Get ready to use vgic-v3
  2016-08-16 10:46   ` Vladimir Murzin
@ 2016-09-05 11:29     ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-05 11:29 UTC (permalink / raw)
  To: Vladimir Murzin; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

On Tue, Aug 16, 2016 at 11:46:57AM +0100, Vladimir Murzin wrote:
> We need to take care we have everything vgic-v3 expects from us before
> a quantum leap:
> - provide required macros via uapi.h
> - handle access to GICv3 cpu interface from the guest
> - provide mapping between arm64 version of GICv3 cpu registers and arm's
> 
> The later is handled via redirection of read{write}_gicreg() and

'The latter'

> required mainly because 64-bit wide ICH_LR is split in two 32-bit
> halves (ICH_LR and ICH_LRC) accessed independently.
> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> ---
>  arch/arm/include/asm/arch_gicv3.h |   64 +++++++++++++++++++++++++++++++++++++
>  arch/arm/include/asm/kvm_asm.h    |    3 ++
>  arch/arm/include/uapi/asm/kvm.h   |    7 ++++
>  arch/arm/kvm/coproc.c             |   36 +++++++++++++++++++++
>  4 files changed, 110 insertions(+)
> 
> diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
> index af25c32..f93f6bd 100644
> --- a/arch/arm/include/asm/arch_gicv3.h
> +++ b/arch/arm/include/asm/arch_gicv3.h
> @@ -96,6 +96,70 @@
>  #define ICH_AP1R2			__AP1Rx(2)
>  #define ICH_AP1R3			__AP1Rx(3)
>  
> +/* A32-to-A64 mappings used by VGIC save/restore */
> +
> +#define CPUIF_MAP(a32, a64)			\
> +static inline void write_ ## a64(u32 val)	\
> +{						\
> +	write_sysreg(val, a32);			\
> +}						\
> +static inline u32 read_ ## a64(void)		\
> +{						\
> +	return read_sysreg(a32); 		\
> +}						\
> +
> +#define CPUIF_MAP_LO_HI(a32lo, a32hi, a64)	\
> +static inline void write_ ## a64(u64 val)	\
> +{						\
> +	write_sysreg((u32)val, a32lo);		\
> +	write_sysreg((u32)(val >> 32), a32hi);	\
> +}						\
> +static inline u64 read_ ## a64(void)		\
> +{						\
> +	u64 val = read_sysreg(a32lo);		\
> +						\
> +	val |=	(u64)read_sysreg(a32hi) << 32;	\
> +						\
> +	return val; 				\
> +}

I really don't like that we're defining new functions using a macro.

Why can't you just do whatever series of actions and/or type
conversions using familiar tricks such as 'do { foo; } while (0);'
and so on ?

> +
> +CPUIF_MAP(ICH_HCR, ICH_HCR_EL2)
> +CPUIF_MAP(ICH_VTR, ICH_VTR_EL2)
> +CPUIF_MAP(ICH_MISR, ICH_MISR_EL2)
> +CPUIF_MAP(ICH_EISR, ICH_EISR_EL2)
> +CPUIF_MAP(ICH_ELSR, ICH_ELSR_EL2)
> +CPUIF_MAP(ICH_VMCR, ICH_VMCR_EL2)
> +CPUIF_MAP(ICH_AP0R3, ICH_AP0R3_EL2)
> +CPUIF_MAP(ICH_AP0R2, ICH_AP0R2_EL2)
> +CPUIF_MAP(ICH_AP0R1, ICH_AP0R1_EL2)
> +CPUIF_MAP(ICH_AP0R0, ICH_AP0R0_EL2)
> +CPUIF_MAP(ICH_AP1R3, ICH_AP1R3_EL2)
> +CPUIF_MAP(ICH_AP1R2, ICH_AP1R2_EL2)
> +CPUIF_MAP(ICH_AP1R1, ICH_AP1R1_EL2)
> +CPUIF_MAP(ICH_AP1R0, ICH_AP1R0_EL2)
> +CPUIF_MAP(ICC_HSRE, ICC_SRE_EL2)
> +CPUIF_MAP(ICC_SRE, ICC_SRE_EL1)
> +
> +CPUIF_MAP_LO_HI(ICH_LR15, ICH_LRC15, ICH_LR15_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR14, ICH_LRC14, ICH_LR14_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR13, ICH_LRC13, ICH_LR13_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR12, ICH_LRC12, ICH_LR12_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR11, ICH_LRC11, ICH_LR11_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR10, ICH_LRC10, ICH_LR10_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR9, ICH_LRC9, ICH_LR9_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR8, ICH_LRC8, ICH_LR8_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR7, ICH_LRC7, ICH_LR7_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR6, ICH_LRC6, ICH_LR6_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR5, ICH_LRC5, ICH_LR5_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR4, ICH_LRC4, ICH_LR4_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR3, ICH_LRC3, ICH_LR3_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR2, ICH_LRC2, ICH_LR2_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR1, ICH_LRC1, ICH_LR1_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR0, ICH_LRC0, ICH_LR0_EL2)
> +
> +#define read_gicreg(r)                 read_##r()
> +#define write_gicreg(v, r)             write_##r(v)
> +
>  /* Low-level accessors */
>  
>  static inline void gic_write_eoir(u32 irq)
> diff --git a/arch/arm/include/asm/kvm_asm.h b/arch/arm/include/asm/kvm_asm.h
> index 58faff5..dfccf94 100644
> --- a/arch/arm/include/asm/kvm_asm.h
> +++ b/arch/arm/include/asm/kvm_asm.h
> @@ -68,6 +68,9 @@ extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
>  extern void __init_stage2_translation(void);
>  
>  extern void __kvm_hyp_reset(unsigned long);
> +
> +extern u64 __vgic_v3_get_ich_vtr_el2(void);
> +extern void __vgic_v3_init_lrs(void);
>  #endif
>  
>  #endif /* __ARM_KVM_ASM_H__ */
> diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h
> index a2b3eb3..b38c10c 100644
> --- a/arch/arm/include/uapi/asm/kvm.h
> +++ b/arch/arm/include/uapi/asm/kvm.h
> @@ -84,6 +84,13 @@ struct kvm_regs {
>  #define KVM_VGIC_V2_DIST_SIZE		0x1000
>  #define KVM_VGIC_V2_CPU_SIZE		0x2000
>  
> +/* Supported VGICv3 address types  */
> +#define KVM_VGIC_V3_ADDR_TYPE_DIST	2
> +#define KVM_VGIC_V3_ADDR_TYPE_REDIST	3
> +
> +#define KVM_VGIC_V3_DIST_SIZE		SZ_64K
> +#define KVM_VGIC_V3_REDIST_SIZE		(2 * SZ_64K)
> +
>  #define KVM_ARM_VCPU_POWER_OFF		0 /* CPU is started in OFF state */
>  #define KVM_ARM_VCPU_PSCI_0_2		1 /* CPU uses PSCI v0.2 */
>  
> diff --git a/arch/arm/kvm/coproc.c b/arch/arm/kvm/coproc.c
> index 1bb2b79..10c0244 100644
> --- a/arch/arm/kvm/coproc.c
> +++ b/arch/arm/kvm/coproc.c
> @@ -228,6 +228,36 @@ bool access_vm_reg(struct kvm_vcpu *vcpu,
>  	return true;
>  }
>  
> +static bool access_gic_sgi(struct kvm_vcpu *vcpu,
> +			   const struct coproc_params *p,
> +			   const struct coproc_reg *r)
> +{
> +	u64 reg;
> +
> +	if (!p->is_write)
> +		return read_from_write_only(vcpu, p);
> +
> +	reg = *vcpu_reg(vcpu, p->Rt2);
> +	reg <<= 32;
> +	reg |= *vcpu_reg(vcpu, p->Rt1) ;
> +
> +	vgic_v3_dispatch_sgi(vcpu, reg);
> +
> +	return true;
> +}
> +
> +static bool access_gic_sre(struct kvm_vcpu *vcpu,
> +			   const struct coproc_params *p,
> +			   const struct coproc_reg *r)
> +{
> +	if (p->is_write)
> +		return ignore_write(vcpu, p);
> +
> +	*vcpu_reg(vcpu, p->Rt1) = vcpu->arch.vgic_cpu.vgic_v3.vgic_sre;
> +
> +	return true;
> +}
> +
>  /*
>   * We could trap ID_DFR0 and tell the guest we don't support performance
>   * monitoring.  Unfortunately the patch to make the kernel check ID_DFR0 was
> @@ -361,10 +391,16 @@ static const struct coproc_reg cp15_regs[] = {
>  	{ CRn(10), CRm( 3), Op1( 0), Op2( 1), is32,
>  			access_vm_reg, reset_unknown, c10_AMAIR1},
>  
> +	/* ICC_SGI1R */
> +	{ CRm64(12), Op1( 0), is64, access_gic_sgi},
> +
>  	/* VBAR: swapped by interrupt.S. */
>  	{ CRn(12), CRm( 0), Op1( 0), Op2( 0), is32,
>  			NULL, reset_val, c12_VBAR, 0x00000000 },
>  
> +	/* ICC_SRE */
> +	{ CRn(12), CRm(12), Op1( 0), Op2(5), is32, access_gic_sre },
> +
>  	/* CONTEXTIDR/TPIDRURW/TPIDRURO/TPIDRPRW: swapped by interrupt.S. */
>  	{ CRn(13), CRm( 0), Op1( 0), Op2( 1), is32,
>  			access_vm_reg, reset_val, c13_CID, 0x00000000 },
> -- 
> 1.7.9.5
> 

Otherwise this looks correct.

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 6/7] ARM: KVM: Get ready to use vgic-v3
@ 2016-09-05 11:29     ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-05 11:29 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Aug 16, 2016 at 11:46:57AM +0100, Vladimir Murzin wrote:
> We need to take care we have everything vgic-v3 expects from us before
> a quantum leap:
> - provide required macros via uapi.h
> - handle access to GICv3 cpu interface from the guest
> - provide mapping between arm64 version of GICv3 cpu registers and arm's
> 
> The later is handled via redirection of read{write}_gicreg() and

'The latter'

> required mainly because 64-bit wide ICH_LR is split in two 32-bit
> halves (ICH_LR and ICH_LRC) accessed independently.
> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> ---
>  arch/arm/include/asm/arch_gicv3.h |   64 +++++++++++++++++++++++++++++++++++++
>  arch/arm/include/asm/kvm_asm.h    |    3 ++
>  arch/arm/include/uapi/asm/kvm.h   |    7 ++++
>  arch/arm/kvm/coproc.c             |   36 +++++++++++++++++++++
>  4 files changed, 110 insertions(+)
> 
> diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
> index af25c32..f93f6bd 100644
> --- a/arch/arm/include/asm/arch_gicv3.h
> +++ b/arch/arm/include/asm/arch_gicv3.h
> @@ -96,6 +96,70 @@
>  #define ICH_AP1R2			__AP1Rx(2)
>  #define ICH_AP1R3			__AP1Rx(3)
>  
> +/* A32-to-A64 mappings used by VGIC save/restore */
> +
> +#define CPUIF_MAP(a32, a64)			\
> +static inline void write_ ## a64(u32 val)	\
> +{						\
> +	write_sysreg(val, a32);			\
> +}						\
> +static inline u32 read_ ## a64(void)		\
> +{						\
> +	return read_sysreg(a32); 		\
> +}						\
> +
> +#define CPUIF_MAP_LO_HI(a32lo, a32hi, a64)	\
> +static inline void write_ ## a64(u64 val)	\
> +{						\
> +	write_sysreg((u32)val, a32lo);		\
> +	write_sysreg((u32)(val >> 32), a32hi);	\
> +}						\
> +static inline u64 read_ ## a64(void)		\
> +{						\
> +	u64 val = read_sysreg(a32lo);		\
> +						\
> +	val |=	(u64)read_sysreg(a32hi) << 32;	\
> +						\
> +	return val; 				\
> +}

I really don't like that we're defining new functions using a macro.

Why can't you just do whatever series of actions and/or type
conversions using familiar tricks such as 'do { foo; } while (0);'
and so on ?

> +
> +CPUIF_MAP(ICH_HCR, ICH_HCR_EL2)
> +CPUIF_MAP(ICH_VTR, ICH_VTR_EL2)
> +CPUIF_MAP(ICH_MISR, ICH_MISR_EL2)
> +CPUIF_MAP(ICH_EISR, ICH_EISR_EL2)
> +CPUIF_MAP(ICH_ELSR, ICH_ELSR_EL2)
> +CPUIF_MAP(ICH_VMCR, ICH_VMCR_EL2)
> +CPUIF_MAP(ICH_AP0R3, ICH_AP0R3_EL2)
> +CPUIF_MAP(ICH_AP0R2, ICH_AP0R2_EL2)
> +CPUIF_MAP(ICH_AP0R1, ICH_AP0R1_EL2)
> +CPUIF_MAP(ICH_AP0R0, ICH_AP0R0_EL2)
> +CPUIF_MAP(ICH_AP1R3, ICH_AP1R3_EL2)
> +CPUIF_MAP(ICH_AP1R2, ICH_AP1R2_EL2)
> +CPUIF_MAP(ICH_AP1R1, ICH_AP1R1_EL2)
> +CPUIF_MAP(ICH_AP1R0, ICH_AP1R0_EL2)
> +CPUIF_MAP(ICC_HSRE, ICC_SRE_EL2)
> +CPUIF_MAP(ICC_SRE, ICC_SRE_EL1)
> +
> +CPUIF_MAP_LO_HI(ICH_LR15, ICH_LRC15, ICH_LR15_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR14, ICH_LRC14, ICH_LR14_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR13, ICH_LRC13, ICH_LR13_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR12, ICH_LRC12, ICH_LR12_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR11, ICH_LRC11, ICH_LR11_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR10, ICH_LRC10, ICH_LR10_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR9, ICH_LRC9, ICH_LR9_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR8, ICH_LRC8, ICH_LR8_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR7, ICH_LRC7, ICH_LR7_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR6, ICH_LRC6, ICH_LR6_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR5, ICH_LRC5, ICH_LR5_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR4, ICH_LRC4, ICH_LR4_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR3, ICH_LRC3, ICH_LR3_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR2, ICH_LRC2, ICH_LR2_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR1, ICH_LRC1, ICH_LR1_EL2)
> +CPUIF_MAP_LO_HI(ICH_LR0, ICH_LRC0, ICH_LR0_EL2)
> +
> +#define read_gicreg(r)                 read_##r()
> +#define write_gicreg(v, r)             write_##r(v)
> +
>  /* Low-level accessors */
>  
>  static inline void gic_write_eoir(u32 irq)
> diff --git a/arch/arm/include/asm/kvm_asm.h b/arch/arm/include/asm/kvm_asm.h
> index 58faff5..dfccf94 100644
> --- a/arch/arm/include/asm/kvm_asm.h
> +++ b/arch/arm/include/asm/kvm_asm.h
> @@ -68,6 +68,9 @@ extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
>  extern void __init_stage2_translation(void);
>  
>  extern void __kvm_hyp_reset(unsigned long);
> +
> +extern u64 __vgic_v3_get_ich_vtr_el2(void);
> +extern void __vgic_v3_init_lrs(void);
>  #endif
>  
>  #endif /* __ARM_KVM_ASM_H__ */
> diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h
> index a2b3eb3..b38c10c 100644
> --- a/arch/arm/include/uapi/asm/kvm.h
> +++ b/arch/arm/include/uapi/asm/kvm.h
> @@ -84,6 +84,13 @@ struct kvm_regs {
>  #define KVM_VGIC_V2_DIST_SIZE		0x1000
>  #define KVM_VGIC_V2_CPU_SIZE		0x2000
>  
> +/* Supported VGICv3 address types  */
> +#define KVM_VGIC_V3_ADDR_TYPE_DIST	2
> +#define KVM_VGIC_V3_ADDR_TYPE_REDIST	3
> +
> +#define KVM_VGIC_V3_DIST_SIZE		SZ_64K
> +#define KVM_VGIC_V3_REDIST_SIZE		(2 * SZ_64K)
> +
>  #define KVM_ARM_VCPU_POWER_OFF		0 /* CPU is started in OFF state */
>  #define KVM_ARM_VCPU_PSCI_0_2		1 /* CPU uses PSCI v0.2 */
>  
> diff --git a/arch/arm/kvm/coproc.c b/arch/arm/kvm/coproc.c
> index 1bb2b79..10c0244 100644
> --- a/arch/arm/kvm/coproc.c
> +++ b/arch/arm/kvm/coproc.c
> @@ -228,6 +228,36 @@ bool access_vm_reg(struct kvm_vcpu *vcpu,
>  	return true;
>  }
>  
> +static bool access_gic_sgi(struct kvm_vcpu *vcpu,
> +			   const struct coproc_params *p,
> +			   const struct coproc_reg *r)
> +{
> +	u64 reg;
> +
> +	if (!p->is_write)
> +		return read_from_write_only(vcpu, p);
> +
> +	reg = *vcpu_reg(vcpu, p->Rt2);
> +	reg <<= 32;
> +	reg |= *vcpu_reg(vcpu, p->Rt1) ;
> +
> +	vgic_v3_dispatch_sgi(vcpu, reg);
> +
> +	return true;
> +}
> +
> +static bool access_gic_sre(struct kvm_vcpu *vcpu,
> +			   const struct coproc_params *p,
> +			   const struct coproc_reg *r)
> +{
> +	if (p->is_write)
> +		return ignore_write(vcpu, p);
> +
> +	*vcpu_reg(vcpu, p->Rt1) = vcpu->arch.vgic_cpu.vgic_v3.vgic_sre;
> +
> +	return true;
> +}
> +
>  /*
>   * We could trap ID_DFR0 and tell the guest we don't support performance
>   * monitoring.  Unfortunately the patch to make the kernel check ID_DFR0 was
> @@ -361,10 +391,16 @@ static const struct coproc_reg cp15_regs[] = {
>  	{ CRn(10), CRm( 3), Op1( 0), Op2( 1), is32,
>  			access_vm_reg, reset_unknown, c10_AMAIR1},
>  
> +	/* ICC_SGI1R */
> +	{ CRm64(12), Op1( 0), is64, access_gic_sgi},
> +
>  	/* VBAR: swapped by interrupt.S. */
>  	{ CRn(12), CRm( 0), Op1( 0), Op2( 0), is32,
>  			NULL, reset_val, c12_VBAR, 0x00000000 },
>  
> +	/* ICC_SRE */
> +	{ CRn(12), CRm(12), Op1( 0), Op2(5), is32, access_gic_sre },
> +
>  	/* CONTEXTIDR/TPIDRURW/TPIDRURO/TPIDRPRW: swapped by interrupt.S. */
>  	{ CRn(13), CRm( 0), Op1( 0), Op2( 1), is32,
>  			access_vm_reg, reset_val, c13_CID, 0x00000000 },
> -- 
> 1.7.9.5
> 

Otherwise this looks correct.

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 7/7] ARM: KVM: Unlock vgic-v3 support
  2016-08-16 10:46   ` Vladimir Murzin
@ 2016-09-05 11:29     ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-05 11:29 UTC (permalink / raw)
  To: Vladimir Murzin; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

On Tue, Aug 16, 2016 at 11:46:58AM +0100, Vladimir Murzin wrote:
> It is time to get access to common version of vgic-v3.

common version?

> 
> We basically would need to tell build system how to pick it up and
> undo KVM_ARM_VGIC_V3 guarding introduced in 4f64cb6 ("arm/arm64: KVM:
> Only allow 64bit hosts to build VGICv3") and remove stubs. However,
> since vgic got ITS support KVM_ARM_VGIC_V3 tend to protect a little
> bit more than just plain vgic-v3 - this guard is used for ITS too
> which is not supported in 32-bit world yet.  So, along with removal of
> KVM_ARM_VGIC_V3 guard introduce the new one - KVM_ARM_VGIC_V3_ITS to
> protect ITS related code.

I don't find this paragraph particularly helpful, I'm afraid.

> 
> The only unpleasant part is how we decide which save/restore sequence to
> use under __vgic_save_state() - we don't have patching framework in hand
> like arm64, so have to check runtime on every invocation.

Do static keys work on 32-bit arm?

> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> ---
>  arch/arm/include/asm/kvm_host.h     |    4 +++
>  arch/arm/include/asm/kvm_hyp.h      |    5 ++++
>  arch/arm/kvm/Makefile               |    2 ++
>  arch/arm/kvm/hyp/Makefile           |    1 +
>  arch/arm/kvm/hyp/switch.c           |   20 +++++++++++--
>  arch/arm64/kvm/Kconfig              |    4 +--
>  include/kvm/arm_vgic.h              |    8 ------
>  virt/kvm/arm/vgic/vgic-kvm-device.c |   10 ++-----
>  virt/kvm/arm/vgic/vgic-mmio-v3.c    |    2 ++
>  virt/kvm/arm/vgic/vgic-mmio.c       |    2 --
>  virt/kvm/arm/vgic/vgic-mmio.h       |    3 --
>  virt/kvm/arm/vgic/vgic.h            |   54 ++---------------------------------
>  12 files changed, 38 insertions(+), 77 deletions(-)
> 
> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> index de338d9..1312597 100644
> --- a/arch/arm/include/asm/kvm_host.h
> +++ b/arch/arm/include/asm/kvm_host.h
> @@ -39,7 +39,11 @@
>  
>  #include <kvm/arm_vgic.h>
>  
> +#ifdef CONFIG_ARM_GIC_V3
> +#define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS
> +#else
>  #define KVM_MAX_VCPUS VGIC_V2_MAX_CPUS
> +#endif
>  
>  #define KVM_REQ_VCPU_EXIT	8
>  
> diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
> index e604ad68..95669b3 100644
> --- a/arch/arm/include/asm/kvm_hyp.h
> +++ b/arch/arm/include/asm/kvm_hyp.h
> @@ -88,6 +88,8 @@
>  
>  #define VFP_FPEXC	__ACCESS_VFP(FPEXC)
>  
> +#define ID_PFR1		__ACCESS_CP15(c0, 0, c1, 1)
> +
>  /* AArch64 compatibility macros, only for the timer so far */
>  #define read_sysreg_el0(r)		read_sysreg(r##_el0)
>  #define write_sysreg_el0(v, r)		write_sysreg(v, r##_el0)
> @@ -103,6 +105,9 @@ void __timer_restore_state(struct kvm_vcpu *vcpu);
>  void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
>  void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
>  
> +void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
> +void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
> +
>  void __sysreg_save_state(struct kvm_cpu_context *ctxt);
>  void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
>  
> diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
> index 10d77a6..043d817f 100644
> --- a/arch/arm/kvm/Makefile
> +++ b/arch/arm/kvm/Makefile
> @@ -26,8 +26,10 @@ obj-y += $(KVM)/arm/vgic/vgic.o
>  obj-y += $(KVM)/arm/vgic/vgic-init.o
>  obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
>  obj-y += $(KVM)/arm/vgic/vgic-v2.o
> +obj-y += $(KVM)/arm/vgic/vgic-v3.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
> +obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
>  obj-y += $(KVM)/arm/vgic/vgic-kvm-device.o
>  obj-y += $(KVM)/irqchip.o
>  obj-y += $(KVM)/arm/arch_timer.o
> diff --git a/arch/arm/kvm/hyp/Makefile b/arch/arm/kvm/hyp/Makefile
> index 8dfa5f7..3023bb5 100644
> --- a/arch/arm/kvm/hyp/Makefile
> +++ b/arch/arm/kvm/hyp/Makefile
> @@ -5,6 +5,7 @@
>  KVM=../../../../virt/kvm
>  
>  obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
> +obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
>  obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
>  
>  obj-$(CONFIG_KVM_ARM_HOST) += tlb.o
> diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
> index b13caa9..9666bae 100644
> --- a/arch/arm/kvm/hyp/switch.c
> +++ b/arch/arm/kvm/hyp/switch.c
> @@ -15,6 +15,8 @@
>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>   */
>  
> +#include <linux/irqchip/arm-gic-v3.h>
> +
>  #include <asm/kvm_asm.h>
>  #include <asm/kvm_hyp.h>
>  
> @@ -74,14 +76,28 @@ static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
>  	write_sysreg(read_sysreg(MIDR), VPIDR);
>  }
>  
> +static bool __hyp_text __has_useable_gicv3_cpuif(void)
> +{
> +	if (IS_ENABLED(CONFIG_ARM_GIC_V3) && (read_sysreg(ID_PFR1) >> 28))

Do we have a define for bit 28 we could use?

Does this actually work on all v7 boards?  The v7 ARM ARM seems to state
that this bitfield is Reserved, UNK.  Does that somehow mean 'is going
to be zero'?

> +		return !!(read_sysreg(ICC_HSRE) & ICC_SRE_EL2_SRE);
> +	else
> +		return false;
> +}
> +
>  static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
>  {
> -	__vgic_v2_save_state(vcpu);
> +	if (__has_useable_gicv3_cpuif())
> +		__vgic_v3_save_state(vcpu);
> +	else
> +		__vgic_v2_save_state(vcpu);
>  }
>  
>  static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
>  {
> -	__vgic_v2_restore_state(vcpu);
> +	if (__has_useable_gicv3_cpuif())
> +		__vgic_v3_restore_state(vcpu);
> +	else
> +		__vgic_v2_restore_state(vcpu);
>  }
>  
>  static bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu)
> diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
> index 9c9edc9..6eaf12c 100644
> --- a/arch/arm64/kvm/Kconfig
> +++ b/arch/arm64/kvm/Kconfig
> @@ -16,7 +16,7 @@ menuconfig VIRTUALIZATION
>  
>  if VIRTUALIZATION
>  
> -config KVM_ARM_VGIC_V3
> +config KVM_ARM_VGIC_V3_ITS

I feel like this could have been simplified with adding the new ITS
guard in a separate patch.

In fact, you could then have a separate patch that enables compilation
of the gicv3 code and removes the static inlines etc., and then finally
a patch that adds in the logic for the world switch.

>  	bool
>  
>  config KVM
> @@ -34,7 +34,7 @@ config KVM
>  	select KVM_VFIO
>  	select HAVE_KVM_EVENTFD
>  	select HAVE_KVM_IRQFD
> -	select KVM_ARM_VGIC_V3
> +	select KVM_ARM_VGIC_V3_ITS
>  	select KVM_ARM_PMU if HW_PERF_EVENTS
>  	select HAVE_KVM_MSI
>  	select HAVE_KVM_IRQCHIP
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index 19b698e..7462138 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -217,7 +217,6 @@ struct vgic_v2_cpu_if {
>  };
>  
>  struct vgic_v3_cpu_if {
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  	u32		vgic_hcr;
>  	u32		vgic_vmcr;
>  	u32		vgic_sre;	/* Restored only, change ignored */
> @@ -227,7 +226,6 @@ struct vgic_v3_cpu_if {
>  	u32		vgic_ap0r[4];
>  	u32		vgic_ap1r[4];
>  	u64		vgic_lr[VGIC_V3_MAX_LRS];
> -#endif
>  };
>  
>  struct vgic_cpu {
> @@ -294,13 +292,7 @@ bool kvm_vcpu_has_pending_irqs(struct kvm_vcpu *vcpu);
>  void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu);
>  void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu);
>  
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg);
> -#else
> -static inline void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)
> -{
> -}
> -#endif
>  
>  /**
>   * kvm_vgic_get_max_vcpus - Get the maximum number of VCPUs allowed by HW
> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
> index 1813f93..e4f0c33 100644
> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
> @@ -71,7 +71,6 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write)
>  		addr_ptr = &vgic->vgic_cpu_base;
>  		alignment = SZ_4K;
>  		break;
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  	case KVM_VGIC_V3_ADDR_TYPE_DIST:
>  		type_needed = KVM_DEV_TYPE_ARM_VGIC_V3;
>  		addr_ptr = &vgic->vgic_dist_base;
> @@ -82,7 +81,6 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write)
>  		addr_ptr = &vgic->vgic_redist_base;
>  		alignment = SZ_64K;
>  		break;
> -#endif
>  	default:
>  		r = -ENODEV;
>  		goto out;
> @@ -219,15 +217,15 @@ int kvm_register_vgic_device(unsigned long type)
>  		ret = kvm_register_device_ops(&kvm_arm_vgic_v2_ops,
>  					      KVM_DEV_TYPE_ARM_VGIC_V2);
>  		break;
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  	case KVM_DEV_TYPE_ARM_VGIC_V3:
>  		ret = kvm_register_device_ops(&kvm_arm_vgic_v3_ops,
>  					      KVM_DEV_TYPE_ARM_VGIC_V3);
> +#ifdef CONFIG_KVM_ARM_VGIC_V3_ITS
>  		if (ret)
>  			break;
>  		ret = kvm_vgic_register_its_device();
> -		break;
>  #endif
> +		break;
>  	}
>  
>  	return ret;
> @@ -389,8 +387,6 @@ struct kvm_device_ops kvm_arm_vgic_v2_ops = {
>  
>  /* V3 ops */
>  
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
> -
>  static int vgic_v3_set_attr(struct kvm_device *dev,
>  			    struct kvm_device_attr *attr)
>  {
> @@ -433,5 +429,3 @@ struct kvm_device_ops kvm_arm_vgic_v3_ops = {
>  	.get_attr = vgic_v3_get_attr,
>  	.has_attr = vgic_v3_has_attr,
>  };
> -
> -#endif /* CONFIG_KVM_ARM_VGIC_V3 */
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> index cc20b60..4709dd65 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> @@ -42,6 +42,7 @@ u64 update_64bit_reg(u64 reg, unsigned int offset, unsigned int len,
>  	return reg | ((u64)val << lower);
>  }
>  
> +#ifdef CONFIG_KVM_ARM_VGIC_V3_ITS
>  bool vgic_has_its(struct kvm *kvm)
>  {
>  	struct vgic_dist *dist = &kvm->arch.vgic;
> @@ -51,6 +52,7 @@ bool vgic_has_its(struct kvm *kvm)
>  
>  	return dist->has_its;
>  }
> +#endif
>  
>  static unsigned long vgic_mmio_read_v3_misc(struct kvm_vcpu *vcpu,
>  					    gpa_t addr, unsigned int len)
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
> index 3bad3c5..e18b30d 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio.c
> @@ -550,11 +550,9 @@ int vgic_register_dist_iodev(struct kvm *kvm, gpa_t dist_base_address,
>  	case VGIC_V2:
>  		len = vgic_v2_init_dist_iodev(io_device);
>  		break;
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  	case VGIC_V3:
>  		len = vgic_v3_init_dist_iodev(io_device);
>  		break;
> -#endif
>  	default:
>  		BUG_ON(1);
>  	}
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
> index 80f92ce..9098aca 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.h
> +++ b/virt/kvm/arm/vgic/vgic-mmio.h
> @@ -162,12 +162,9 @@ unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
>  
>  unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
>  
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  u64 vgic_sanitise_outer_cacheability(u64 reg);
>  u64 vgic_sanitise_inner_cacheability(u64 reg);
>  u64 vgic_sanitise_shareability(u64 reg);
>  u64 vgic_sanitise_field(u64 reg, u64 field_mask, int field_shift,
>  			u64 (*sanitise_fn)(u64));
>  #endif
> -
> -#endif
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index 1d8e21d..633c512 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -72,7 +72,6 @@ static inline void vgic_get_irq_kref(struct vgic_irq *irq)
>  	kref_get(&irq->refcount);
>  }
>  
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu);
>  void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu);
>  void vgic_v3_populate_lr(struct kvm_vcpu *vcpu, struct vgic_irq *irq, int lr);
> @@ -84,62 +83,13 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu);
>  int vgic_v3_probe(const struct gic_kvm_info *info);
>  int vgic_v3_map_resources(struct kvm *kvm);
>  int vgic_register_redist_iodevs(struct kvm *kvm, gpa_t dist_base_address);
> +
> +#ifdef CONFIG_KVM_ARM_VGIC_V3_ITS
>  bool vgic_has_its(struct kvm *kvm);
>  int kvm_vgic_register_its_device(void);
>  void vgic_enable_lpis(struct kvm_vcpu *vcpu);
>  int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi);
>  #else
> -static inline void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu)
> -{
> -}
> -
> -static inline void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu)
> -{
> -}
> -
> -static inline void vgic_v3_populate_lr(struct kvm_vcpu *vcpu,
> -				       struct vgic_irq *irq, int lr)
> -{
> -}
> -
> -static inline void vgic_v3_clear_lr(struct kvm_vcpu *vcpu, int lr)
> -{
> -}
> -
> -static inline void vgic_v3_set_underflow(struct kvm_vcpu *vcpu)
> -{
> -}
> -
> -static inline
> -void vgic_v3_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
> -{
> -}
> -
> -static inline
> -void vgic_v3_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
> -{
> -}
> -
> -static inline void vgic_v3_enable(struct kvm_vcpu *vcpu)
> -{
> -}
> -
> -static inline int vgic_v3_probe(const struct gic_kvm_info *info)
> -{
> -	return -ENODEV;
> -}
> -
> -static inline int vgic_v3_map_resources(struct kvm *kvm)
> -{
> -	return -ENODEV;
> -}
> -
> -static inline int vgic_register_redist_iodevs(struct kvm *kvm,
> -					      gpa_t dist_base_address)
> -{
> -	return -ENODEV;
> -}
> -
>  static inline bool vgic_has_its(struct kvm *kvm)
>  {
>  	return false;
> -- 
> 1.7.9.5
> 

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 7/7] ARM: KVM: Unlock vgic-v3 support
@ 2016-09-05 11:29     ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-05 11:29 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Aug 16, 2016 at 11:46:58AM +0100, Vladimir Murzin wrote:
> It is time to get access to common version of vgic-v3.

common version?

> 
> We basically would need to tell build system how to pick it up and
> undo KVM_ARM_VGIC_V3 guarding introduced in 4f64cb6 ("arm/arm64: KVM:
> Only allow 64bit hosts to build VGICv3") and remove stubs. However,
> since vgic got ITS support KVM_ARM_VGIC_V3 tend to protect a little
> bit more than just plain vgic-v3 - this guard is used for ITS too
> which is not supported in 32-bit world yet.  So, along with removal of
> KVM_ARM_VGIC_V3 guard introduce the new one - KVM_ARM_VGIC_V3_ITS to
> protect ITS related code.

I don't find this paragraph particularly helpful, I'm afraid.

> 
> The only unpleasant part is how we decide which save/restore sequence to
> use under __vgic_save_state() - we don't have patching framework in hand
> like arm64, so have to check runtime on every invocation.

Do static keys work on 32-bit arm?

> 
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> ---
>  arch/arm/include/asm/kvm_host.h     |    4 +++
>  arch/arm/include/asm/kvm_hyp.h      |    5 ++++
>  arch/arm/kvm/Makefile               |    2 ++
>  arch/arm/kvm/hyp/Makefile           |    1 +
>  arch/arm/kvm/hyp/switch.c           |   20 +++++++++++--
>  arch/arm64/kvm/Kconfig              |    4 +--
>  include/kvm/arm_vgic.h              |    8 ------
>  virt/kvm/arm/vgic/vgic-kvm-device.c |   10 ++-----
>  virt/kvm/arm/vgic/vgic-mmio-v3.c    |    2 ++
>  virt/kvm/arm/vgic/vgic-mmio.c       |    2 --
>  virt/kvm/arm/vgic/vgic-mmio.h       |    3 --
>  virt/kvm/arm/vgic/vgic.h            |   54 ++---------------------------------
>  12 files changed, 38 insertions(+), 77 deletions(-)
> 
> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> index de338d9..1312597 100644
> --- a/arch/arm/include/asm/kvm_host.h
> +++ b/arch/arm/include/asm/kvm_host.h
> @@ -39,7 +39,11 @@
>  
>  #include <kvm/arm_vgic.h>
>  
> +#ifdef CONFIG_ARM_GIC_V3
> +#define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS
> +#else
>  #define KVM_MAX_VCPUS VGIC_V2_MAX_CPUS
> +#endif
>  
>  #define KVM_REQ_VCPU_EXIT	8
>  
> diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
> index e604ad68..95669b3 100644
> --- a/arch/arm/include/asm/kvm_hyp.h
> +++ b/arch/arm/include/asm/kvm_hyp.h
> @@ -88,6 +88,8 @@
>  
>  #define VFP_FPEXC	__ACCESS_VFP(FPEXC)
>  
> +#define ID_PFR1		__ACCESS_CP15(c0, 0, c1, 1)
> +
>  /* AArch64 compatibility macros, only for the timer so far */
>  #define read_sysreg_el0(r)		read_sysreg(r##_el0)
>  #define write_sysreg_el0(v, r)		write_sysreg(v, r##_el0)
> @@ -103,6 +105,9 @@ void __timer_restore_state(struct kvm_vcpu *vcpu);
>  void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
>  void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
>  
> +void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
> +void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
> +
>  void __sysreg_save_state(struct kvm_cpu_context *ctxt);
>  void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
>  
> diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
> index 10d77a6..043d817f 100644
> --- a/arch/arm/kvm/Makefile
> +++ b/arch/arm/kvm/Makefile
> @@ -26,8 +26,10 @@ obj-y += $(KVM)/arm/vgic/vgic.o
>  obj-y += $(KVM)/arm/vgic/vgic-init.o
>  obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
>  obj-y += $(KVM)/arm/vgic/vgic-v2.o
> +obj-y += $(KVM)/arm/vgic/vgic-v3.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio.o
>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
> +obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
>  obj-y += $(KVM)/arm/vgic/vgic-kvm-device.o
>  obj-y += $(KVM)/irqchip.o
>  obj-y += $(KVM)/arm/arch_timer.o
> diff --git a/arch/arm/kvm/hyp/Makefile b/arch/arm/kvm/hyp/Makefile
> index 8dfa5f7..3023bb5 100644
> --- a/arch/arm/kvm/hyp/Makefile
> +++ b/arch/arm/kvm/hyp/Makefile
> @@ -5,6 +5,7 @@
>  KVM=../../../../virt/kvm
>  
>  obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
> +obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
>  obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
>  
>  obj-$(CONFIG_KVM_ARM_HOST) += tlb.o
> diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
> index b13caa9..9666bae 100644
> --- a/arch/arm/kvm/hyp/switch.c
> +++ b/arch/arm/kvm/hyp/switch.c
> @@ -15,6 +15,8 @@
>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>   */
>  
> +#include <linux/irqchip/arm-gic-v3.h>
> +
>  #include <asm/kvm_asm.h>
>  #include <asm/kvm_hyp.h>
>  
> @@ -74,14 +76,28 @@ static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
>  	write_sysreg(read_sysreg(MIDR), VPIDR);
>  }
>  
> +static bool __hyp_text __has_useable_gicv3_cpuif(void)
> +{
> +	if (IS_ENABLED(CONFIG_ARM_GIC_V3) && (read_sysreg(ID_PFR1) >> 28))

Do we have a define for bit 28 we could use?

Does this actually work on all v7 boards?  The v7 ARM ARM seems to state
that this bitfield is Reserved, UNK.  Does that somehow mean 'is going
to be zero'?

> +		return !!(read_sysreg(ICC_HSRE) & ICC_SRE_EL2_SRE);
> +	else
> +		return false;
> +}
> +
>  static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
>  {
> -	__vgic_v2_save_state(vcpu);
> +	if (__has_useable_gicv3_cpuif())
> +		__vgic_v3_save_state(vcpu);
> +	else
> +		__vgic_v2_save_state(vcpu);
>  }
>  
>  static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
>  {
> -	__vgic_v2_restore_state(vcpu);
> +	if (__has_useable_gicv3_cpuif())
> +		__vgic_v3_restore_state(vcpu);
> +	else
> +		__vgic_v2_restore_state(vcpu);
>  }
>  
>  static bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu)
> diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
> index 9c9edc9..6eaf12c 100644
> --- a/arch/arm64/kvm/Kconfig
> +++ b/arch/arm64/kvm/Kconfig
> @@ -16,7 +16,7 @@ menuconfig VIRTUALIZATION
>  
>  if VIRTUALIZATION
>  
> -config KVM_ARM_VGIC_V3
> +config KVM_ARM_VGIC_V3_ITS

I feel like this could have been simplified with adding the new ITS
guard in a separate patch.

In fact, you could then have a separate patch that enables compilation
of the gicv3 code and removes the static inlines etc., and then finally
a patch that adds in the logic for the world switch.

>  	bool
>  
>  config KVM
> @@ -34,7 +34,7 @@ config KVM
>  	select KVM_VFIO
>  	select HAVE_KVM_EVENTFD
>  	select HAVE_KVM_IRQFD
> -	select KVM_ARM_VGIC_V3
> +	select KVM_ARM_VGIC_V3_ITS
>  	select KVM_ARM_PMU if HW_PERF_EVENTS
>  	select HAVE_KVM_MSI
>  	select HAVE_KVM_IRQCHIP
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index 19b698e..7462138 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -217,7 +217,6 @@ struct vgic_v2_cpu_if {
>  };
>  
>  struct vgic_v3_cpu_if {
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  	u32		vgic_hcr;
>  	u32		vgic_vmcr;
>  	u32		vgic_sre;	/* Restored only, change ignored */
> @@ -227,7 +226,6 @@ struct vgic_v3_cpu_if {
>  	u32		vgic_ap0r[4];
>  	u32		vgic_ap1r[4];
>  	u64		vgic_lr[VGIC_V3_MAX_LRS];
> -#endif
>  };
>  
>  struct vgic_cpu {
> @@ -294,13 +292,7 @@ bool kvm_vcpu_has_pending_irqs(struct kvm_vcpu *vcpu);
>  void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu);
>  void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu);
>  
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg);
> -#else
> -static inline void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)
> -{
> -}
> -#endif
>  
>  /**
>   * kvm_vgic_get_max_vcpus - Get the maximum number of VCPUs allowed by HW
> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
> index 1813f93..e4f0c33 100644
> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
> @@ -71,7 +71,6 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write)
>  		addr_ptr = &vgic->vgic_cpu_base;
>  		alignment = SZ_4K;
>  		break;
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  	case KVM_VGIC_V3_ADDR_TYPE_DIST:
>  		type_needed = KVM_DEV_TYPE_ARM_VGIC_V3;
>  		addr_ptr = &vgic->vgic_dist_base;
> @@ -82,7 +81,6 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write)
>  		addr_ptr = &vgic->vgic_redist_base;
>  		alignment = SZ_64K;
>  		break;
> -#endif
>  	default:
>  		r = -ENODEV;
>  		goto out;
> @@ -219,15 +217,15 @@ int kvm_register_vgic_device(unsigned long type)
>  		ret = kvm_register_device_ops(&kvm_arm_vgic_v2_ops,
>  					      KVM_DEV_TYPE_ARM_VGIC_V2);
>  		break;
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  	case KVM_DEV_TYPE_ARM_VGIC_V3:
>  		ret = kvm_register_device_ops(&kvm_arm_vgic_v3_ops,
>  					      KVM_DEV_TYPE_ARM_VGIC_V3);
> +#ifdef CONFIG_KVM_ARM_VGIC_V3_ITS
>  		if (ret)
>  			break;
>  		ret = kvm_vgic_register_its_device();
> -		break;
>  #endif
> +		break;
>  	}
>  
>  	return ret;
> @@ -389,8 +387,6 @@ struct kvm_device_ops kvm_arm_vgic_v2_ops = {
>  
>  /* V3 ops */
>  
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
> -
>  static int vgic_v3_set_attr(struct kvm_device *dev,
>  			    struct kvm_device_attr *attr)
>  {
> @@ -433,5 +429,3 @@ struct kvm_device_ops kvm_arm_vgic_v3_ops = {
>  	.get_attr = vgic_v3_get_attr,
>  	.has_attr = vgic_v3_has_attr,
>  };
> -
> -#endif /* CONFIG_KVM_ARM_VGIC_V3 */
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> index cc20b60..4709dd65 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> @@ -42,6 +42,7 @@ u64 update_64bit_reg(u64 reg, unsigned int offset, unsigned int len,
>  	return reg | ((u64)val << lower);
>  }
>  
> +#ifdef CONFIG_KVM_ARM_VGIC_V3_ITS
>  bool vgic_has_its(struct kvm *kvm)
>  {
>  	struct vgic_dist *dist = &kvm->arch.vgic;
> @@ -51,6 +52,7 @@ bool vgic_has_its(struct kvm *kvm)
>  
>  	return dist->has_its;
>  }
> +#endif
>  
>  static unsigned long vgic_mmio_read_v3_misc(struct kvm_vcpu *vcpu,
>  					    gpa_t addr, unsigned int len)
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
> index 3bad3c5..e18b30d 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio.c
> @@ -550,11 +550,9 @@ int vgic_register_dist_iodev(struct kvm *kvm, gpa_t dist_base_address,
>  	case VGIC_V2:
>  		len = vgic_v2_init_dist_iodev(io_device);
>  		break;
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  	case VGIC_V3:
>  		len = vgic_v3_init_dist_iodev(io_device);
>  		break;
> -#endif
>  	default:
>  		BUG_ON(1);
>  	}
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
> index 80f92ce..9098aca 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.h
> +++ b/virt/kvm/arm/vgic/vgic-mmio.h
> @@ -162,12 +162,9 @@ unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
>  
>  unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
>  
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  u64 vgic_sanitise_outer_cacheability(u64 reg);
>  u64 vgic_sanitise_inner_cacheability(u64 reg);
>  u64 vgic_sanitise_shareability(u64 reg);
>  u64 vgic_sanitise_field(u64 reg, u64 field_mask, int field_shift,
>  			u64 (*sanitise_fn)(u64));
>  #endif
> -
> -#endif
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index 1d8e21d..633c512 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -72,7 +72,6 @@ static inline void vgic_get_irq_kref(struct vgic_irq *irq)
>  	kref_get(&irq->refcount);
>  }
>  
> -#ifdef CONFIG_KVM_ARM_VGIC_V3
>  void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu);
>  void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu);
>  void vgic_v3_populate_lr(struct kvm_vcpu *vcpu, struct vgic_irq *irq, int lr);
> @@ -84,62 +83,13 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu);
>  int vgic_v3_probe(const struct gic_kvm_info *info);
>  int vgic_v3_map_resources(struct kvm *kvm);
>  int vgic_register_redist_iodevs(struct kvm *kvm, gpa_t dist_base_address);
> +
> +#ifdef CONFIG_KVM_ARM_VGIC_V3_ITS
>  bool vgic_has_its(struct kvm *kvm);
>  int kvm_vgic_register_its_device(void);
>  void vgic_enable_lpis(struct kvm_vcpu *vcpu);
>  int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi);
>  #else
> -static inline void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu)
> -{
> -}
> -
> -static inline void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu)
> -{
> -}
> -
> -static inline void vgic_v3_populate_lr(struct kvm_vcpu *vcpu,
> -				       struct vgic_irq *irq, int lr)
> -{
> -}
> -
> -static inline void vgic_v3_clear_lr(struct kvm_vcpu *vcpu, int lr)
> -{
> -}
> -
> -static inline void vgic_v3_set_underflow(struct kvm_vcpu *vcpu)
> -{
> -}
> -
> -static inline
> -void vgic_v3_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
> -{
> -}
> -
> -static inline
> -void vgic_v3_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
> -{
> -}
> -
> -static inline void vgic_v3_enable(struct kvm_vcpu *vcpu)
> -{
> -}
> -
> -static inline int vgic_v3_probe(const struct gic_kvm_info *info)
> -{
> -	return -ENODEV;
> -}
> -
> -static inline int vgic_v3_map_resources(struct kvm *kvm)
> -{
> -	return -ENODEV;
> -}
> -
> -static inline int vgic_register_redist_iodevs(struct kvm *kvm,
> -					      gpa_t dist_base_address)
> -{
> -	return -ENODEV;
> -}
> -
>  static inline bool vgic_has_its(struct kvm *kvm)
>  {
>  	return false;
> -- 
> 1.7.9.5
> 

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 0/7] ARM: KVM: Support for vgic-v3
  2016-09-05 11:28   ` Christoffer Dall
@ 2016-09-06 12:32     ` Vladimir Murzin
  -1 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-09-06 12:32 UTC (permalink / raw)
  To: Christoffer Dall; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

On 05/09/16 12:28, Christoffer Dall wrote:
> On Tue, Aug 16, 2016 at 11:46:51AM +0100, Vladimir Murzin wrote:
>> Hi,
>>
>> This is an attempt to make use vgic-v3 under arch/arm since save-restore
>> functionality got re-written in C and can be shared between arm/arm64
>> like it has already been done for vgic-v2 and timer.
>>
>> With this patches I'm able to get 32 core ARMv7 guest boot:
> 
> Is it really an ARMv7 guest or an AArch32 ARMv8 guest?
> 

It is an AArch32 ARMv8 guest.

Vladimir

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 0/7] ARM: KVM: Support for vgic-v3
@ 2016-09-06 12:32     ` Vladimir Murzin
  0 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-09-06 12:32 UTC (permalink / raw)
  To: linux-arm-kernel

On 05/09/16 12:28, Christoffer Dall wrote:
> On Tue, Aug 16, 2016 at 11:46:51AM +0100, Vladimir Murzin wrote:
>> Hi,
>>
>> This is an attempt to make use vgic-v3 under arch/arm since save-restore
>> functionality got re-written in C and can be shared between arm/arm64
>> like it has already been done for vgic-v2 and timer.
>>
>> With this patches I'm able to get 32 core ARMv7 guest boot:
> 
> Is it really an ARMv7 guest or an AArch32 ARMv8 guest?
> 

It is an AArch32 ARMv8 guest.

Vladimir

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 1/7] arm64: KVM: Move GIC accessors to arch_gicv3.h
  2016-09-05 11:28     ` Christoffer Dall
@ 2016-09-06 12:33       ` Vladimir Murzin
  -1 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-09-06 12:33 UTC (permalink / raw)
  To: Christoffer Dall; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

On 05/09/16 12:28, Christoffer Dall wrote:
> On Tue, Aug 16, 2016 at 11:46:52AM +0100, Vladimir Murzin wrote:
>> Since we are going to share vgic-v3 save/restore code with ARM
>> keep arch specific accessors separately.
>>
>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> 
> Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
> 
> 

Thanks!

Vladimir

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 1/7] arm64: KVM: Move GIC accessors to arch_gicv3.h
@ 2016-09-06 12:33       ` Vladimir Murzin
  0 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-09-06 12:33 UTC (permalink / raw)
  To: linux-arm-kernel

On 05/09/16 12:28, Christoffer Dall wrote:
> On Tue, Aug 16, 2016 at 11:46:52AM +0100, Vladimir Murzin wrote:
>> Since we are going to share vgic-v3 save/restore code with ARM
>> keep arch specific accessors separately.
>>
>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> 
> Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
> 
> 

Thanks!

Vladimir

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 3/7] KVM: arm: vgic-new: improve compatibility with 32-bit
  2016-09-05 11:29     ` Christoffer Dall
@ 2016-09-06 12:41       ` Vladimir Murzin
  -1 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-09-06 12:41 UTC (permalink / raw)
  To: Christoffer Dall; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

Hi Christoffer,

On 05/09/16 12:29, Christoffer Dall wrote:
> Hi Vladimir,
> 
> I think commit title is too vague, can you be more specific?
> 

KVM: arm: vgic-new: make extract_bytes to always work on 64-bit data

is it better?

> On Tue, Aug 16, 2016 at 11:46:54AM +0100, Vladimir Murzin wrote:
>> We have couple of 64-bit register defined in GICv3 architecture, so
> 
> 'a couple',  'registers' (plural)
> 
>> "unsigned long" kind of accessors wouldn't work for 32-bit. However,
> 
> 'wouldn't work for 32-bit' is kind of generic as well.  Perhaps you mean
> that unsigned long accesses to these registers will only access a single
> 32-bit work of that register.
> 
>> these registers can't be access as 64-bit in a one go if we run 32-bit
> 
> 'accessed', 's/in one go/with a single instruction/' ?
> 
> 'a 32-bit host'
> 
>> host simply because KVM doesn't support multiple load/store on MMIO
> 
> by 'multiple load/store' you mean the 'load/store multiple' instructions
> specifically, right?  Not a sequence of multiple loads and stores.  I
> think you should be more specific here as well, for example, I think
> ldrd and strd are problematic as well.
> 
>> space.
>>
>> It means that 32-bit guest access these registers in 32-bit chunks, so
> 
> 'a 32-bit guest', 'accesses'
> 

all suggestions you've made above are true. I'll rework commit message
to be more precise.

>> the only thing we need to do is to ensure that extract_bytes() always
>> takes 64-bit data.
>>
>> Since we are here fix couple of other width related issues by using
>> ULL variants over UL.
>>
>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>> ---
>>  virt/kvm/arm/vgic/vgic-mmio-v3.c |    6 +++---
>>  virt/kvm/arm/vgic/vgic-mmio.h    |    2 +-
>>  2 files changed, 4 insertions(+), 4 deletions(-)
>>
>> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> index ff668e0..cc20b60 100644
>> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> @@ -23,7 +23,7 @@
>>  #include "vgic-mmio.h"
>>  
>>  /* extract @num bytes at @offset bytes offset in data */
>> -unsigned long extract_bytes(unsigned long data, unsigned int offset,
>> +unsigned long extract_bytes(u64 data, unsigned int offset,
>>  			    unsigned int num)
>>  {
>>  	return (data >> (offset * 8)) & GENMASK_ULL(num * 8 - 1, 0);
>> @@ -179,7 +179,7 @@ static unsigned long vgic_mmio_read_v3r_typer(struct kvm_vcpu *vcpu,
>>  	int target_vcpu_id = vcpu->vcpu_id;
>>  	u64 value;
>>  
>> -	value = (mpidr & GENMASK(23, 0)) << 32;
>> +	value = (mpidr & GENMASK_ULL(23, 0)) << 32;
> 
> why does this make a difference when mpidr is an unsigned long?

because we access a little bit further than unsigned long can accommodate

  CC      arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.o
arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c: In function
'vgic_mmio_read_v3r_typer':
arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c:184:35: warning:
left shift count >= width of type [-Wshift-count-overflow]
  value = (mpidr & GENMASK(23, 0)) << 32;
                                   ^

I can include this warning in commit message or maybe you want a
separate patch?

Vladimir

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 3/7] KVM: arm: vgic-new: improve compatibility with 32-bit
@ 2016-09-06 12:41       ` Vladimir Murzin
  0 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-09-06 12:41 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Christoffer,

On 05/09/16 12:29, Christoffer Dall wrote:
> Hi Vladimir,
> 
> I think commit title is too vague, can you be more specific?
> 

KVM: arm: vgic-new: make extract_bytes to always work on 64-bit data

is it better?

> On Tue, Aug 16, 2016 at 11:46:54AM +0100, Vladimir Murzin wrote:
>> We have couple of 64-bit register defined in GICv3 architecture, so
> 
> 'a couple',  'registers' (plural)
> 
>> "unsigned long" kind of accessors wouldn't work for 32-bit. However,
> 
> 'wouldn't work for 32-bit' is kind of generic as well.  Perhaps you mean
> that unsigned long accesses to these registers will only access a single
> 32-bit work of that register.
> 
>> these registers can't be access as 64-bit in a one go if we run 32-bit
> 
> 'accessed', 's/in one go/with a single instruction/' ?
> 
> 'a 32-bit host'
> 
>> host simply because KVM doesn't support multiple load/store on MMIO
> 
> by 'multiple load/store' you mean the 'load/store multiple' instructions
> specifically, right?  Not a sequence of multiple loads and stores.  I
> think you should be more specific here as well, for example, I think
> ldrd and strd are problematic as well.
> 
>> space.
>>
>> It means that 32-bit guest access these registers in 32-bit chunks, so
> 
> 'a 32-bit guest', 'accesses'
> 

all suggestions you've made above are true. I'll rework commit message
to be more precise.

>> the only thing we need to do is to ensure that extract_bytes() always
>> takes 64-bit data.
>>
>> Since we are here fix couple of other width related issues by using
>> ULL variants over UL.
>>
>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>> ---
>>  virt/kvm/arm/vgic/vgic-mmio-v3.c |    6 +++---
>>  virt/kvm/arm/vgic/vgic-mmio.h    |    2 +-
>>  2 files changed, 4 insertions(+), 4 deletions(-)
>>
>> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> index ff668e0..cc20b60 100644
>> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> @@ -23,7 +23,7 @@
>>  #include "vgic-mmio.h"
>>  
>>  /* extract @num bytes at @offset bytes offset in data */
>> -unsigned long extract_bytes(unsigned long data, unsigned int offset,
>> +unsigned long extract_bytes(u64 data, unsigned int offset,
>>  			    unsigned int num)
>>  {
>>  	return (data >> (offset * 8)) & GENMASK_ULL(num * 8 - 1, 0);
>> @@ -179,7 +179,7 @@ static unsigned long vgic_mmio_read_v3r_typer(struct kvm_vcpu *vcpu,
>>  	int target_vcpu_id = vcpu->vcpu_id;
>>  	u64 value;
>>  
>> -	value = (mpidr & GENMASK(23, 0)) << 32;
>> +	value = (mpidr & GENMASK_ULL(23, 0)) << 32;
> 
> why does this make a difference when mpidr is an unsigned long?

because we access a little bit further than unsigned long can accommodate

  CC      arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.o
arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c: In function
'vgic_mmio_read_v3r_typer':
arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c:184:35: warning:
left shift count >= width of type [-Wshift-count-overflow]
  value = (mpidr & GENMASK(23, 0)) << 32;
                                   ^

I can include this warning in commit message or maybe you want a
separate patch?

Vladimir

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 4/7] ARM: update MPIDR accessors macro
  2016-09-05 11:29     ` Christoffer Dall
@ 2016-09-06 12:42       ` Vladimir Murzin
  -1 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-09-06 12:42 UTC (permalink / raw)
  To: Christoffer Dall; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

On 05/09/16 12:29, Christoffer Dall wrote:
> The title of this patch is quite generic, especially the 'update'.
> Perhaps say: 
> 
> "ARM: Change MPIDR_AFFINITY_LEVEL to ignore Aff3"
> 

Fixed.

> On Tue, Aug 16, 2016 at 11:46:55AM +0100, Vladimir Murzin wrote:
>> vgic-v3 driver queries CPU affinity level up to Aff3, which is valid for arm64.
>> However, for arm up to Aff2 levels are supported, so querying for 3rd level
>> ends with upper bits of MPIDR are treated as valid affinity level which
>> is not true. So, report zero for any affinity level above 2.
>>
>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>> ---
>>  arch/arm/include/asm/cputype.h |    3 ++-
>>  1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/arm/include/asm/cputype.h b/arch/arm/include/asm/cputype.h
>> index 1ee94c7..96cef49 100644
>> --- a/arch/arm/include/asm/cputype.h
>> +++ b/arch/arm/include/asm/cputype.h
>> @@ -55,9 +55,10 @@
>>  
>>  #define MPIDR_LEVEL_BITS 8
>>  #define MPIDR_LEVEL_MASK ((1 << MPIDR_LEVEL_BITS) - 1)
>> +#define MPIDR_LEVEL_SHIFT(level) (MPIDR_LEVEL_BITS * level)
>>  
>>  #define MPIDR_AFFINITY_LEVEL(mpidr, level) \
>> -	((mpidr >> (MPIDR_LEVEL_BITS * level)) & MPIDR_LEVEL_MASK)
>> +	((level < 3) ? ((mpidr >> MPIDR_LEVEL_SHIFT(level)) & MPIDR_LEVEL_MASK) : 0)
> 
> Instead of using a conditional here, you could just apply the
> MPIDR_HWID_BITMASK to the mpidr argument instead.
> 

Fixed.

Thanks
Vladimir

>>  
>>  #define ARM_CPU_IMP_ARM			0x41
>>  #define ARM_CPU_IMP_INTEL		0x69
>> -- 
>> 1.7.9.5
>>
> 
> 

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 4/7] ARM: update MPIDR accessors macro
@ 2016-09-06 12:42       ` Vladimir Murzin
  0 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-09-06 12:42 UTC (permalink / raw)
  To: linux-arm-kernel

On 05/09/16 12:29, Christoffer Dall wrote:
> The title of this patch is quite generic, especially the 'update'.
> Perhaps say: 
> 
> "ARM: Change MPIDR_AFFINITY_LEVEL to ignore Aff3"
> 

Fixed.

> On Tue, Aug 16, 2016 at 11:46:55AM +0100, Vladimir Murzin wrote:
>> vgic-v3 driver queries CPU affinity level up to Aff3, which is valid for arm64.
>> However, for arm up to Aff2 levels are supported, so querying for 3rd level
>> ends with upper bits of MPIDR are treated as valid affinity level which
>> is not true. So, report zero for any affinity level above 2.
>>
>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>> ---
>>  arch/arm/include/asm/cputype.h |    3 ++-
>>  1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/arm/include/asm/cputype.h b/arch/arm/include/asm/cputype.h
>> index 1ee94c7..96cef49 100644
>> --- a/arch/arm/include/asm/cputype.h
>> +++ b/arch/arm/include/asm/cputype.h
>> @@ -55,9 +55,10 @@
>>  
>>  #define MPIDR_LEVEL_BITS 8
>>  #define MPIDR_LEVEL_MASK ((1 << MPIDR_LEVEL_BITS) - 1)
>> +#define MPIDR_LEVEL_SHIFT(level) (MPIDR_LEVEL_BITS * level)
>>  
>>  #define MPIDR_AFFINITY_LEVEL(mpidr, level) \
>> -	((mpidr >> (MPIDR_LEVEL_BITS * level)) & MPIDR_LEVEL_MASK)
>> +	((level < 3) ? ((mpidr >> MPIDR_LEVEL_SHIFT(level)) & MPIDR_LEVEL_MASK) : 0)
> 
> Instead of using a conditional here, you could just apply the
> MPIDR_HWID_BITMASK to the mpidr argument instead.
> 

Fixed.

Thanks
Vladimir

>>  
>>  #define ARM_CPU_IMP_ARM			0x41
>>  #define ARM_CPU_IMP_INTEL		0x69
>> -- 
>> 1.7.9.5
>>
> 
> 

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 5/7] ARM: move system register accessors to asm/cp15.h
  2016-09-05 11:29     ` Christoffer Dall
@ 2016-09-06 13:05       ` Vladimir Murzin
  -1 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-09-06 13:05 UTC (permalink / raw)
  To: Christoffer Dall; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

On 05/09/16 12:29, Christoffer Dall wrote:
> On Tue, Aug 16, 2016 at 11:46:56AM +0100, Vladimir Murzin wrote:
>> Macro __ACCESS_CP15{_64} is defined in two headers (arch_gicv3.h and
>> kvm_hyp.h) which are going to be requested by vgic-v3 altogether.
>> GCC would not like it because it'd see that macro is redefined and (hey!)
>> they are different.  So, let's put only single macro version under common
>> place and use it everywhere.
> 
> I'm sorry, but I don't understand this commit text.
> 

Sorry for that :(

The issue the patch is trying to solve happens because
virt/kvm/arm/hyp/vgic-v3-sr.c has

#include <linux/irqchip/arm-gic-v3.h>
...
#include <asm/kvm_hyp.h>

each of these headers defines it's own __ACCESS_CP15 macro:

asm/kvm_hyp.h

#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32

and linux/irqchip/arm-gic-v3.h via asm/arch_gicv3.h

#define __ACCESS_CP15(CRn, Op1, CRm, Op2) p15, Op1, %0, CRn, CRm, Op2

When these headers are used together conflict happens. The same applies
to __ACCESS_CP15_64 macro.

To address that only single set of macros is used and call sites updated.

>>
>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>> ---
>>  arch/arm/include/asm/arch_gicv3.h |   27 +++++++++++----------------
>>  arch/arm/include/asm/cp15.h       |   15 +++++++++++++++
>>  arch/arm/include/asm/kvm_hyp.h    |   15 +--------------
>>  3 files changed, 27 insertions(+), 30 deletions(-)
>>
>> diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
>> index e08d151..af25c32 100644
>> --- a/arch/arm/include/asm/arch_gicv3.h
>> +++ b/arch/arm/include/asm/arch_gicv3.h
>> @@ -22,9 +22,7 @@
>>  
>>  #include <linux/io.h>
>>  #include <asm/barrier.h>
>> -
>> -#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
>> -#define __ACCESS_CP15_64(Op1, CRm)		p15, Op1, %Q0, %R0, CRm
>> +#include <asm/cp15.h>
>>  
>>  #define ICC_EOIR1			__ACCESS_CP15(c12, 0, c12, 1)
>>  #define ICC_DIR				__ACCESS_CP15(c12, 0, c11, 1)
>> @@ -102,58 +100,55 @@
>>  
>>  static inline void gic_write_eoir(u32 irq)
>>  {
>> -	asm volatile("mcr " __stringify(ICC_EOIR1) : : "r" (irq));
>> +	write_sysreg(irq, ICC_EOIR1);
>>  	isb();
>>  }
>>  
>>  static inline void gic_write_dir(u32 val)
>>  {
>> -	asm volatile("mcr " __stringify(ICC_DIR) : : "r" (val));
>> +	write_sysreg(val, ICC_DIR);
>>  	isb();
>>  }
>>  
>>  static inline u32 gic_read_iar(void)
>>  {
>> -	u32 irqstat;
>> +	u32 irqstat = read_sysreg(ICC_IAR1);
>>  
>> -	asm volatile("mrc " __stringify(ICC_IAR1) : "=r" (irqstat));
>>  	dsb(sy);
>> +
>>  	return irqstat;
>>  }
>>  
>>  static inline void gic_write_pmr(u32 val)
>>  {
>> -	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
>> +	write_sysreg(val, ICC_PMR);
>>  }
>>  
>>  static inline void gic_write_ctlr(u32 val)
>>  {
>> -	asm volatile("mcr " __stringify(ICC_CTLR) : : "r" (val));
>> +	write_sysreg(val, ICC_CTLR);
>>  	isb();
>>  }
>>  
>>  static inline void gic_write_grpen1(u32 val)
>>  {
>> -	asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
>> +	write_sysreg(val, ICC_IGRPEN1);
>>  	isb();
>>  }
>>  
>>  static inline void gic_write_sgi1r(u64 val)
>>  {
>> -	asm volatile("mcrr " __stringify(ICC_SGI1R) : : "r" (val));
>> +	write_sysreg(val, ICC_SGI1R);
>>  }
>>  
>>  static inline u32 gic_read_sre(void)
>>  {
>> -	u32 val;
>> -
>> -	asm volatile("mrc " __stringify(ICC_SRE) : "=r" (val));
>> -	return val;
>> +	return read_sysreg(ICC_SRE);
>>  }
>>  
>>  static inline void gic_write_sre(u32 val)
>>  {
>> -	asm volatile("mcr " __stringify(ICC_SRE) : : "r" (val));
>> +	write_sysreg(val, ICC_SRE);
>>  	isb();
>>  }
>>  
>> diff --git a/arch/arm/include/asm/cp15.h b/arch/arm/include/asm/cp15.h
>> index c3f1152..f661732 100644
>> --- a/arch/arm/include/asm/cp15.h
>> +++ b/arch/arm/include/asm/cp15.h
>> @@ -47,6 +47,21 @@
>>  #define vectors_high()	(0)
>>  #endif
>>  
>> +#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
>> +	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
>> +#define __ACCESS_CP15_64(Op1, CRm)		\
>> +	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
>> +
>> +#define __read_sysreg(r, w, c, t) ({				\
>> +	t __val;						\
>> +	asm volatile(r " " c : "=r" (__val));			\
>> +	__val;							\
>> +})
>> +#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
>> +
>> +#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
>> +#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
>> +
> 
> I feel a bit strange about adding this sort of stuff in a
> non-kvm-non-gic-specific ARM header file, without it being used (or
> planned to be used) in a broader sense.
> 
> Is there not a way to keep the required changes local to KVM and the
> gic?
> 

We could add prefixes to KVM and GIC version of macros so they won't
clash, but it'd introduce code duplication.
We could keep macro in, say, GIC header and include it in KVM one (or
vice versa), but such dependency would not look nicer, IMO.

The way arm64 handles this is via sysreg.h and the closest counterpart
under arch/arm is cp15.h

I'm open to suggestions.

Thanks
Vladimir

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 5/7] ARM: move system register accessors to asm/cp15.h
@ 2016-09-06 13:05       ` Vladimir Murzin
  0 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-09-06 13:05 UTC (permalink / raw)
  To: linux-arm-kernel

On 05/09/16 12:29, Christoffer Dall wrote:
> On Tue, Aug 16, 2016 at 11:46:56AM +0100, Vladimir Murzin wrote:
>> Macro __ACCESS_CP15{_64} is defined in two headers (arch_gicv3.h and
>> kvm_hyp.h) which are going to be requested by vgic-v3 altogether.
>> GCC would not like it because it'd see that macro is redefined and (hey!)
>> they are different.  So, let's put only single macro version under common
>> place and use it everywhere.
> 
> I'm sorry, but I don't understand this commit text.
> 

Sorry for that :(

The issue the patch is trying to solve happens because
virt/kvm/arm/hyp/vgic-v3-sr.c has

#include <linux/irqchip/arm-gic-v3.h>
...
#include <asm/kvm_hyp.h>

each of these headers defines it's own __ACCESS_CP15 macro:

asm/kvm_hyp.h

#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32

and linux/irqchip/arm-gic-v3.h via asm/arch_gicv3.h

#define __ACCESS_CP15(CRn, Op1, CRm, Op2) p15, Op1, %0, CRn, CRm, Op2

When these headers are used together conflict happens. The same applies
to __ACCESS_CP15_64 macro.

To address that only single set of macros is used and call sites updated.

>>
>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>> ---
>>  arch/arm/include/asm/arch_gicv3.h |   27 +++++++++++----------------
>>  arch/arm/include/asm/cp15.h       |   15 +++++++++++++++
>>  arch/arm/include/asm/kvm_hyp.h    |   15 +--------------
>>  3 files changed, 27 insertions(+), 30 deletions(-)
>>
>> diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
>> index e08d151..af25c32 100644
>> --- a/arch/arm/include/asm/arch_gicv3.h
>> +++ b/arch/arm/include/asm/arch_gicv3.h
>> @@ -22,9 +22,7 @@
>>  
>>  #include <linux/io.h>
>>  #include <asm/barrier.h>
>> -
>> -#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
>> -#define __ACCESS_CP15_64(Op1, CRm)		p15, Op1, %Q0, %R0, CRm
>> +#include <asm/cp15.h>
>>  
>>  #define ICC_EOIR1			__ACCESS_CP15(c12, 0, c12, 1)
>>  #define ICC_DIR				__ACCESS_CP15(c12, 0, c11, 1)
>> @@ -102,58 +100,55 @@
>>  
>>  static inline void gic_write_eoir(u32 irq)
>>  {
>> -	asm volatile("mcr " __stringify(ICC_EOIR1) : : "r" (irq));
>> +	write_sysreg(irq, ICC_EOIR1);
>>  	isb();
>>  }
>>  
>>  static inline void gic_write_dir(u32 val)
>>  {
>> -	asm volatile("mcr " __stringify(ICC_DIR) : : "r" (val));
>> +	write_sysreg(val, ICC_DIR);
>>  	isb();
>>  }
>>  
>>  static inline u32 gic_read_iar(void)
>>  {
>> -	u32 irqstat;
>> +	u32 irqstat = read_sysreg(ICC_IAR1);
>>  
>> -	asm volatile("mrc " __stringify(ICC_IAR1) : "=r" (irqstat));
>>  	dsb(sy);
>> +
>>  	return irqstat;
>>  }
>>  
>>  static inline void gic_write_pmr(u32 val)
>>  {
>> -	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
>> +	write_sysreg(val, ICC_PMR);
>>  }
>>  
>>  static inline void gic_write_ctlr(u32 val)
>>  {
>> -	asm volatile("mcr " __stringify(ICC_CTLR) : : "r" (val));
>> +	write_sysreg(val, ICC_CTLR);
>>  	isb();
>>  }
>>  
>>  static inline void gic_write_grpen1(u32 val)
>>  {
>> -	asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
>> +	write_sysreg(val, ICC_IGRPEN1);
>>  	isb();
>>  }
>>  
>>  static inline void gic_write_sgi1r(u64 val)
>>  {
>> -	asm volatile("mcrr " __stringify(ICC_SGI1R) : : "r" (val));
>> +	write_sysreg(val, ICC_SGI1R);
>>  }
>>  
>>  static inline u32 gic_read_sre(void)
>>  {
>> -	u32 val;
>> -
>> -	asm volatile("mrc " __stringify(ICC_SRE) : "=r" (val));
>> -	return val;
>> +	return read_sysreg(ICC_SRE);
>>  }
>>  
>>  static inline void gic_write_sre(u32 val)
>>  {
>> -	asm volatile("mcr " __stringify(ICC_SRE) : : "r" (val));
>> +	write_sysreg(val, ICC_SRE);
>>  	isb();
>>  }
>>  
>> diff --git a/arch/arm/include/asm/cp15.h b/arch/arm/include/asm/cp15.h
>> index c3f1152..f661732 100644
>> --- a/arch/arm/include/asm/cp15.h
>> +++ b/arch/arm/include/asm/cp15.h
>> @@ -47,6 +47,21 @@
>>  #define vectors_high()	(0)
>>  #endif
>>  
>> +#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
>> +	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
>> +#define __ACCESS_CP15_64(Op1, CRm)		\
>> +	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
>> +
>> +#define __read_sysreg(r, w, c, t) ({				\
>> +	t __val;						\
>> +	asm volatile(r " " c : "=r" (__val));			\
>> +	__val;							\
>> +})
>> +#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
>> +
>> +#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
>> +#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
>> +
> 
> I feel a bit strange about adding this sort of stuff in a
> non-kvm-non-gic-specific ARM header file, without it being used (or
> planned to be used) in a broader sense.
> 
> Is there not a way to keep the required changes local to KVM and the
> gic?
> 

We could add prefixes to KVM and GIC version of macros so they won't
clash, but it'd introduce code duplication.
We could keep macro in, say, GIC header and include it in KVM one (or
vice versa), but such dependency would not look nicer, IMO.

The way arm64 handles this is via sysreg.h and the closest counterpart
under arch/arm is cp15.h

I'm open to suggestions.

Thanks
Vladimir

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 7/7] ARM: KVM: Unlock vgic-v3 support
  2016-09-05 11:29     ` Christoffer Dall
@ 2016-09-06 13:08       ` Marc Zyngier
  -1 siblings, 0 replies; 74+ messages in thread
From: Marc Zyngier @ 2016-09-06 13:08 UTC (permalink / raw)
  To: Christoffer Dall, Vladimir Murzin
  Cc: andre.przywara, kvmarm, linux-arm-kernel

On 05/09/16 12:29, Christoffer Dall wrote:
> On Tue, Aug 16, 2016 at 11:46:58AM +0100, Vladimir Murzin wrote:
>> It is time to get access to common version of vgic-v3.
> 
> common version?
> 
>>
>> We basically would need to tell build system how to pick it up and
>> undo KVM_ARM_VGIC_V3 guarding introduced in 4f64cb6 ("arm/arm64: KVM:
>> Only allow 64bit hosts to build VGICv3") and remove stubs. However,
>> since vgic got ITS support KVM_ARM_VGIC_V3 tend to protect a little
>> bit more than just plain vgic-v3 - this guard is used for ITS too
>> which is not supported in 32-bit world yet.  So, along with removal of
>> KVM_ARM_VGIC_V3 guard introduce the new one - KVM_ARM_VGIC_V3_ITS to
>> protect ITS related code.
> 
> I don't find this paragraph particularly helpful, I'm afraid.
> 
>>
>> The only unpleasant part is how we decide which save/restore sequence to
>> use under __vgic_save_state() - we don't have patching framework in hand
>> like arm64, so have to check runtime on every invocation.
> 
> Do static keys work on 32-bit arm?

They do. It'd be interesting to see if we could move both architectures
to use static keys for selecting the GIC backend.

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 7/7] ARM: KVM: Unlock vgic-v3 support
@ 2016-09-06 13:08       ` Marc Zyngier
  0 siblings, 0 replies; 74+ messages in thread
From: Marc Zyngier @ 2016-09-06 13:08 UTC (permalink / raw)
  To: linux-arm-kernel

On 05/09/16 12:29, Christoffer Dall wrote:
> On Tue, Aug 16, 2016 at 11:46:58AM +0100, Vladimir Murzin wrote:
>> It is time to get access to common version of vgic-v3.
> 
> common version?
> 
>>
>> We basically would need to tell build system how to pick it up and
>> undo KVM_ARM_VGIC_V3 guarding introduced in 4f64cb6 ("arm/arm64: KVM:
>> Only allow 64bit hosts to build VGICv3") and remove stubs. However,
>> since vgic got ITS support KVM_ARM_VGIC_V3 tend to protect a little
>> bit more than just plain vgic-v3 - this guard is used for ITS too
>> which is not supported in 32-bit world yet.  So, along with removal of
>> KVM_ARM_VGIC_V3 guard introduce the new one - KVM_ARM_VGIC_V3_ITS to
>> protect ITS related code.
> 
> I don't find this paragraph particularly helpful, I'm afraid.
> 
>>
>> The only unpleasant part is how we decide which save/restore sequence to
>> use under __vgic_save_state() - we don't have patching framework in hand
>> like arm64, so have to check runtime on every invocation.
> 
> Do static keys work on 32-bit arm?

They do. It'd be interesting to see if we could move both architectures
to use static keys for selecting the GIC backend.

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 6/7] ARM: KVM: Get ready to use vgic-v3
  2016-09-05 11:29     ` Christoffer Dall
@ 2016-09-06 13:12       ` Vladimir Murzin
  -1 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-09-06 13:12 UTC (permalink / raw)
  To: Christoffer Dall; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

On 05/09/16 12:29, Christoffer Dall wrote:
> On Tue, Aug 16, 2016 at 11:46:57AM +0100, Vladimir Murzin wrote:
>> We need to take care we have everything vgic-v3 expects from us before
>> a quantum leap:
>> - provide required macros via uapi.h
>> - handle access to GICv3 cpu interface from the guest
>> - provide mapping between arm64 version of GICv3 cpu registers and arm's
>>
>> The later is handled via redirection of read{write}_gicreg() and
> 
> 'The latter'
> 

Fixed.

>> required mainly because 64-bit wide ICH_LR is split in two 32-bit
>> halves (ICH_LR and ICH_LRC) accessed independently.
>>
>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>> ---
>>  arch/arm/include/asm/arch_gicv3.h |   64 +++++++++++++++++++++++++++++++++++++
>>  arch/arm/include/asm/kvm_asm.h    |    3 ++
>>  arch/arm/include/uapi/asm/kvm.h   |    7 ++++
>>  arch/arm/kvm/coproc.c             |   36 +++++++++++++++++++++
>>  4 files changed, 110 insertions(+)
>>
>> diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
>> index af25c32..f93f6bd 100644
>> --- a/arch/arm/include/asm/arch_gicv3.h
>> +++ b/arch/arm/include/asm/arch_gicv3.h
>> @@ -96,6 +96,70 @@
>>  #define ICH_AP1R2			__AP1Rx(2)
>>  #define ICH_AP1R3			__AP1Rx(3)
>>  
>> +/* A32-to-A64 mappings used by VGIC save/restore */
>> +
>> +#define CPUIF_MAP(a32, a64)			\
>> +static inline void write_ ## a64(u32 val)	\
>> +{						\
>> +	write_sysreg(val, a32);			\
>> +}						\
>> +static inline u32 read_ ## a64(void)		\
>> +{						\
>> +	return read_sysreg(a32); 		\
>> +}						\
>> +
>> +#define CPUIF_MAP_LO_HI(a32lo, a32hi, a64)	\
>> +static inline void write_ ## a64(u64 val)	\
>> +{						\
>> +	write_sysreg((u32)val, a32lo);		\
>> +	write_sysreg((u32)(val >> 32), a32hi);	\
>> +}						\
>> +static inline u64 read_ ## a64(void)		\
>> +{						\
>> +	u64 val = read_sysreg(a32lo);		\
>> +						\
>> +	val |=	(u64)read_sysreg(a32hi) << 32;	\
>> +						\
>> +	return val; 				\
>> +}
> 
> I really don't like that we're defining new functions using a macro.

I can expand it manually if think it'd look better. I can also suggest
to prefix generated functions with underscore.

> 
> Why can't you just do whatever series of actions and/or type
> conversions using familiar tricks such as 'do { foo; } while (0);'
> and so on ?

Probably, because I don't how to apply these tricks here to keep
virt/kvm/arm/hyp/vgic-v3-sr.c unchanged. If you have something
particular in mind, please, suggest!

> 
>> +
>> +CPUIF_MAP(ICH_HCR, ICH_HCR_EL2)
>> +CPUIF_MAP(ICH_VTR, ICH_VTR_EL2)
>> +CPUIF_MAP(ICH_MISR, ICH_MISR_EL2)
>> +CPUIF_MAP(ICH_EISR, ICH_EISR_EL2)
>> +CPUIF_MAP(ICH_ELSR, ICH_ELSR_EL2)
>> +CPUIF_MAP(ICH_VMCR, ICH_VMCR_EL2)
>> +CPUIF_MAP(ICH_AP0R3, ICH_AP0R3_EL2)
>> +CPUIF_MAP(ICH_AP0R2, ICH_AP0R2_EL2)
>> +CPUIF_MAP(ICH_AP0R1, ICH_AP0R1_EL2)
>> +CPUIF_MAP(ICH_AP0R0, ICH_AP0R0_EL2)
>> +CPUIF_MAP(ICH_AP1R3, ICH_AP1R3_EL2)
>> +CPUIF_MAP(ICH_AP1R2, ICH_AP1R2_EL2)
>> +CPUIF_MAP(ICH_AP1R1, ICH_AP1R1_EL2)
>> +CPUIF_MAP(ICH_AP1R0, ICH_AP1R0_EL2)
>> +CPUIF_MAP(ICC_HSRE, ICC_SRE_EL2)
>> +CPUIF_MAP(ICC_SRE, ICC_SRE_EL1)
>> +
>> +CPUIF_MAP_LO_HI(ICH_LR15, ICH_LRC15, ICH_LR15_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR14, ICH_LRC14, ICH_LR14_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR13, ICH_LRC13, ICH_LR13_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR12, ICH_LRC12, ICH_LR12_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR11, ICH_LRC11, ICH_LR11_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR10, ICH_LRC10, ICH_LR10_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR9, ICH_LRC9, ICH_LR9_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR8, ICH_LRC8, ICH_LR8_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR7, ICH_LRC7, ICH_LR7_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR6, ICH_LRC6, ICH_LR6_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR5, ICH_LRC5, ICH_LR5_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR4, ICH_LRC4, ICH_LR4_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR3, ICH_LRC3, ICH_LR3_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR2, ICH_LRC2, ICH_LR2_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR1, ICH_LRC1, ICH_LR1_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR0, ICH_LRC0, ICH_LR0_EL2)
>> +
>> +#define read_gicreg(r)                 read_##r()
>> +#define write_gicreg(v, r)             write_##r(v)
>> +
>>  /* Low-level accessors */
>>  
>>  static inline void gic_write_eoir(u32 irq)
>> diff --git a/arch/arm/include/asm/kvm_asm.h b/arch/arm/include/asm/kvm_asm.h
>> index 58faff5..dfccf94 100644
>> --- a/arch/arm/include/asm/kvm_asm.h
>> +++ b/arch/arm/include/asm/kvm_asm.h
>> @@ -68,6 +68,9 @@ extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
>>  extern void __init_stage2_translation(void);
>>  
>>  extern void __kvm_hyp_reset(unsigned long);
>> +
>> +extern u64 __vgic_v3_get_ich_vtr_el2(void);
>> +extern void __vgic_v3_init_lrs(void);
>>  #endif
>>  
>>  #endif /* __ARM_KVM_ASM_H__ */
>> diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h
>> index a2b3eb3..b38c10c 100644
>> --- a/arch/arm/include/uapi/asm/kvm.h
>> +++ b/arch/arm/include/uapi/asm/kvm.h
>> @@ -84,6 +84,13 @@ struct kvm_regs {
>>  #define KVM_VGIC_V2_DIST_SIZE		0x1000
>>  #define KVM_VGIC_V2_CPU_SIZE		0x2000
>>  
>> +/* Supported VGICv3 address types  */
>> +#define KVM_VGIC_V3_ADDR_TYPE_DIST	2
>> +#define KVM_VGIC_V3_ADDR_TYPE_REDIST	3
>> +
>> +#define KVM_VGIC_V3_DIST_SIZE		SZ_64K
>> +#define KVM_VGIC_V3_REDIST_SIZE		(2 * SZ_64K)
>> +
>>  #define KVM_ARM_VCPU_POWER_OFF		0 /* CPU is started in OFF state */
>>  #define KVM_ARM_VCPU_PSCI_0_2		1 /* CPU uses PSCI v0.2 */
>>  
>> diff --git a/arch/arm/kvm/coproc.c b/arch/arm/kvm/coproc.c
>> index 1bb2b79..10c0244 100644
>> --- a/arch/arm/kvm/coproc.c
>> +++ b/arch/arm/kvm/coproc.c
>> @@ -228,6 +228,36 @@ bool access_vm_reg(struct kvm_vcpu *vcpu,
>>  	return true;
>>  }
>>  
>> +static bool access_gic_sgi(struct kvm_vcpu *vcpu,
>> +			   const struct coproc_params *p,
>> +			   const struct coproc_reg *r)
>> +{
>> +	u64 reg;
>> +
>> +	if (!p->is_write)
>> +		return read_from_write_only(vcpu, p);
>> +
>> +	reg = *vcpu_reg(vcpu, p->Rt2);
>> +	reg <<= 32;
>> +	reg |= *vcpu_reg(vcpu, p->Rt1) ;
>> +
>> +	vgic_v3_dispatch_sgi(vcpu, reg);
>> +
>> +	return true;
>> +}
>> +
>> +static bool access_gic_sre(struct kvm_vcpu *vcpu,
>> +			   const struct coproc_params *p,
>> +			   const struct coproc_reg *r)
>> +{
>> +	if (p->is_write)
>> +		return ignore_write(vcpu, p);
>> +
>> +	*vcpu_reg(vcpu, p->Rt1) = vcpu->arch.vgic_cpu.vgic_v3.vgic_sre;
>> +
>> +	return true;
>> +}
>> +
>>  /*
>>   * We could trap ID_DFR0 and tell the guest we don't support performance
>>   * monitoring.  Unfortunately the patch to make the kernel check ID_DFR0 was
>> @@ -361,10 +391,16 @@ static const struct coproc_reg cp15_regs[] = {
>>  	{ CRn(10), CRm( 3), Op1( 0), Op2( 1), is32,
>>  			access_vm_reg, reset_unknown, c10_AMAIR1},
>>  
>> +	/* ICC_SGI1R */
>> +	{ CRm64(12), Op1( 0), is64, access_gic_sgi},
>> +
>>  	/* VBAR: swapped by interrupt.S. */
>>  	{ CRn(12), CRm( 0), Op1( 0), Op2( 0), is32,
>>  			NULL, reset_val, c12_VBAR, 0x00000000 },
>>  
>> +	/* ICC_SRE */
>> +	{ CRn(12), CRm(12), Op1( 0), Op2(5), is32, access_gic_sre },
>> +
>>  	/* CONTEXTIDR/TPIDRURW/TPIDRURO/TPIDRPRW: swapped by interrupt.S. */
>>  	{ CRn(13), CRm( 0), Op1( 0), Op2( 1), is32,
>>  			access_vm_reg, reset_val, c13_CID, 0x00000000 },
>> -- 
>> 1.7.9.5
>>
> 
> Otherwise this looks correct.

Thanks
Vladimir

> 
> Thanks,
> -Christoffer
> 
> 

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 6/7] ARM: KVM: Get ready to use vgic-v3
@ 2016-09-06 13:12       ` Vladimir Murzin
  0 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-09-06 13:12 UTC (permalink / raw)
  To: linux-arm-kernel

On 05/09/16 12:29, Christoffer Dall wrote:
> On Tue, Aug 16, 2016 at 11:46:57AM +0100, Vladimir Murzin wrote:
>> We need to take care we have everything vgic-v3 expects from us before
>> a quantum leap:
>> - provide required macros via uapi.h
>> - handle access to GICv3 cpu interface from the guest
>> - provide mapping between arm64 version of GICv3 cpu registers and arm's
>>
>> The later is handled via redirection of read{write}_gicreg() and
> 
> 'The latter'
> 

Fixed.

>> required mainly because 64-bit wide ICH_LR is split in two 32-bit
>> halves (ICH_LR and ICH_LRC) accessed independently.
>>
>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>> ---
>>  arch/arm/include/asm/arch_gicv3.h |   64 +++++++++++++++++++++++++++++++++++++
>>  arch/arm/include/asm/kvm_asm.h    |    3 ++
>>  arch/arm/include/uapi/asm/kvm.h   |    7 ++++
>>  arch/arm/kvm/coproc.c             |   36 +++++++++++++++++++++
>>  4 files changed, 110 insertions(+)
>>
>> diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
>> index af25c32..f93f6bd 100644
>> --- a/arch/arm/include/asm/arch_gicv3.h
>> +++ b/arch/arm/include/asm/arch_gicv3.h
>> @@ -96,6 +96,70 @@
>>  #define ICH_AP1R2			__AP1Rx(2)
>>  #define ICH_AP1R3			__AP1Rx(3)
>>  
>> +/* A32-to-A64 mappings used by VGIC save/restore */
>> +
>> +#define CPUIF_MAP(a32, a64)			\
>> +static inline void write_ ## a64(u32 val)	\
>> +{						\
>> +	write_sysreg(val, a32);			\
>> +}						\
>> +static inline u32 read_ ## a64(void)		\
>> +{						\
>> +	return read_sysreg(a32); 		\
>> +}						\
>> +
>> +#define CPUIF_MAP_LO_HI(a32lo, a32hi, a64)	\
>> +static inline void write_ ## a64(u64 val)	\
>> +{						\
>> +	write_sysreg((u32)val, a32lo);		\
>> +	write_sysreg((u32)(val >> 32), a32hi);	\
>> +}						\
>> +static inline u64 read_ ## a64(void)		\
>> +{						\
>> +	u64 val = read_sysreg(a32lo);		\
>> +						\
>> +	val |=	(u64)read_sysreg(a32hi) << 32;	\
>> +						\
>> +	return val; 				\
>> +}
> 
> I really don't like that we're defining new functions using a macro.

I can expand it manually if think it'd look better. I can also suggest
to prefix generated functions with underscore.

> 
> Why can't you just do whatever series of actions and/or type
> conversions using familiar tricks such as 'do { foo; } while (0);'
> and so on ?

Probably, because I don't how to apply these tricks here to keep
virt/kvm/arm/hyp/vgic-v3-sr.c unchanged. If you have something
particular in mind, please, suggest!

> 
>> +
>> +CPUIF_MAP(ICH_HCR, ICH_HCR_EL2)
>> +CPUIF_MAP(ICH_VTR, ICH_VTR_EL2)
>> +CPUIF_MAP(ICH_MISR, ICH_MISR_EL2)
>> +CPUIF_MAP(ICH_EISR, ICH_EISR_EL2)
>> +CPUIF_MAP(ICH_ELSR, ICH_ELSR_EL2)
>> +CPUIF_MAP(ICH_VMCR, ICH_VMCR_EL2)
>> +CPUIF_MAP(ICH_AP0R3, ICH_AP0R3_EL2)
>> +CPUIF_MAP(ICH_AP0R2, ICH_AP0R2_EL2)
>> +CPUIF_MAP(ICH_AP0R1, ICH_AP0R1_EL2)
>> +CPUIF_MAP(ICH_AP0R0, ICH_AP0R0_EL2)
>> +CPUIF_MAP(ICH_AP1R3, ICH_AP1R3_EL2)
>> +CPUIF_MAP(ICH_AP1R2, ICH_AP1R2_EL2)
>> +CPUIF_MAP(ICH_AP1R1, ICH_AP1R1_EL2)
>> +CPUIF_MAP(ICH_AP1R0, ICH_AP1R0_EL2)
>> +CPUIF_MAP(ICC_HSRE, ICC_SRE_EL2)
>> +CPUIF_MAP(ICC_SRE, ICC_SRE_EL1)
>> +
>> +CPUIF_MAP_LO_HI(ICH_LR15, ICH_LRC15, ICH_LR15_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR14, ICH_LRC14, ICH_LR14_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR13, ICH_LRC13, ICH_LR13_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR12, ICH_LRC12, ICH_LR12_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR11, ICH_LRC11, ICH_LR11_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR10, ICH_LRC10, ICH_LR10_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR9, ICH_LRC9, ICH_LR9_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR8, ICH_LRC8, ICH_LR8_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR7, ICH_LRC7, ICH_LR7_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR6, ICH_LRC6, ICH_LR6_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR5, ICH_LRC5, ICH_LR5_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR4, ICH_LRC4, ICH_LR4_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR3, ICH_LRC3, ICH_LR3_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR2, ICH_LRC2, ICH_LR2_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR1, ICH_LRC1, ICH_LR1_EL2)
>> +CPUIF_MAP_LO_HI(ICH_LR0, ICH_LRC0, ICH_LR0_EL2)
>> +
>> +#define read_gicreg(r)                 read_##r()
>> +#define write_gicreg(v, r)             write_##r(v)
>> +
>>  /* Low-level accessors */
>>  
>>  static inline void gic_write_eoir(u32 irq)
>> diff --git a/arch/arm/include/asm/kvm_asm.h b/arch/arm/include/asm/kvm_asm.h
>> index 58faff5..dfccf94 100644
>> --- a/arch/arm/include/asm/kvm_asm.h
>> +++ b/arch/arm/include/asm/kvm_asm.h
>> @@ -68,6 +68,9 @@ extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
>>  extern void __init_stage2_translation(void);
>>  
>>  extern void __kvm_hyp_reset(unsigned long);
>> +
>> +extern u64 __vgic_v3_get_ich_vtr_el2(void);
>> +extern void __vgic_v3_init_lrs(void);
>>  #endif
>>  
>>  #endif /* __ARM_KVM_ASM_H__ */
>> diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h
>> index a2b3eb3..b38c10c 100644
>> --- a/arch/arm/include/uapi/asm/kvm.h
>> +++ b/arch/arm/include/uapi/asm/kvm.h
>> @@ -84,6 +84,13 @@ struct kvm_regs {
>>  #define KVM_VGIC_V2_DIST_SIZE		0x1000
>>  #define KVM_VGIC_V2_CPU_SIZE		0x2000
>>  
>> +/* Supported VGICv3 address types  */
>> +#define KVM_VGIC_V3_ADDR_TYPE_DIST	2
>> +#define KVM_VGIC_V3_ADDR_TYPE_REDIST	3
>> +
>> +#define KVM_VGIC_V3_DIST_SIZE		SZ_64K
>> +#define KVM_VGIC_V3_REDIST_SIZE		(2 * SZ_64K)
>> +
>>  #define KVM_ARM_VCPU_POWER_OFF		0 /* CPU is started in OFF state */
>>  #define KVM_ARM_VCPU_PSCI_0_2		1 /* CPU uses PSCI v0.2 */
>>  
>> diff --git a/arch/arm/kvm/coproc.c b/arch/arm/kvm/coproc.c
>> index 1bb2b79..10c0244 100644
>> --- a/arch/arm/kvm/coproc.c
>> +++ b/arch/arm/kvm/coproc.c
>> @@ -228,6 +228,36 @@ bool access_vm_reg(struct kvm_vcpu *vcpu,
>>  	return true;
>>  }
>>  
>> +static bool access_gic_sgi(struct kvm_vcpu *vcpu,
>> +			   const struct coproc_params *p,
>> +			   const struct coproc_reg *r)
>> +{
>> +	u64 reg;
>> +
>> +	if (!p->is_write)
>> +		return read_from_write_only(vcpu, p);
>> +
>> +	reg = *vcpu_reg(vcpu, p->Rt2);
>> +	reg <<= 32;
>> +	reg |= *vcpu_reg(vcpu, p->Rt1) ;
>> +
>> +	vgic_v3_dispatch_sgi(vcpu, reg);
>> +
>> +	return true;
>> +}
>> +
>> +static bool access_gic_sre(struct kvm_vcpu *vcpu,
>> +			   const struct coproc_params *p,
>> +			   const struct coproc_reg *r)
>> +{
>> +	if (p->is_write)
>> +		return ignore_write(vcpu, p);
>> +
>> +	*vcpu_reg(vcpu, p->Rt1) = vcpu->arch.vgic_cpu.vgic_v3.vgic_sre;
>> +
>> +	return true;
>> +}
>> +
>>  /*
>>   * We could trap ID_DFR0 and tell the guest we don't support performance
>>   * monitoring.  Unfortunately the patch to make the kernel check ID_DFR0 was
>> @@ -361,10 +391,16 @@ static const struct coproc_reg cp15_regs[] = {
>>  	{ CRn(10), CRm( 3), Op1( 0), Op2( 1), is32,
>>  			access_vm_reg, reset_unknown, c10_AMAIR1},
>>  
>> +	/* ICC_SGI1R */
>> +	{ CRm64(12), Op1( 0), is64, access_gic_sgi},
>> +
>>  	/* VBAR: swapped by interrupt.S. */
>>  	{ CRn(12), CRm( 0), Op1( 0), Op2( 0), is32,
>>  			NULL, reset_val, c12_VBAR, 0x00000000 },
>>  
>> +	/* ICC_SRE */
>> +	{ CRn(12), CRm(12), Op1( 0), Op2(5), is32, access_gic_sre },
>> +
>>  	/* CONTEXTIDR/TPIDRURW/TPIDRURO/TPIDRPRW: swapped by interrupt.S. */
>>  	{ CRn(13), CRm( 0), Op1( 0), Op2( 1), is32,
>>  			access_vm_reg, reset_val, c13_CID, 0x00000000 },
>> -- 
>> 1.7.9.5
>>
> 
> Otherwise this looks correct.

Thanks
Vladimir

> 
> Thanks,
> -Christoffer
> 
> 

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 7/7] ARM: KVM: Unlock vgic-v3 support
  2016-09-05 11:29     ` Christoffer Dall
@ 2016-09-06 13:18       ` Vladimir Murzin
  -1 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-09-06 13:18 UTC (permalink / raw)
  To: Christoffer Dall; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

On 05/09/16 12:29, Christoffer Dall wrote:
> On Tue, Aug 16, 2016 at 11:46:58AM +0100, Vladimir Murzin wrote:
>> It is time to get access to common version of vgic-v3.
> 
> common version?
> 

Since patch#2 it not private to arm64 or I should rephrase this?

>>
>> We basically would need to tell build system how to pick it up and
>> undo KVM_ARM_VGIC_V3 guarding introduced in 4f64cb6 ("arm/arm64: KVM:
>> Only allow 64bit hosts to build VGICv3") and remove stubs. However,
>> since vgic got ITS support KVM_ARM_VGIC_V3 tend to protect a little
>> bit more than just plain vgic-v3 - this guard is used for ITS too
>> which is not supported in 32-bit world yet.  So, along with removal of
>> KVM_ARM_VGIC_V3 guard introduce the new one - KVM_ARM_VGIC_V3_ITS to
>> protect ITS related code.
> 
> I don't find this paragraph particularly helpful, I'm afraid.
> 

Sorry for that. It seems to much for one patch, I'll split it per your
suggestion bellow.

>>
>> The only unpleasant part is how we decide which save/restore sequence to
>> use under __vgic_save_state() - we don't have patching framework in hand
>> like arm64, so have to check runtime on every invocation.
> 
> Do static keys work on 32-bit arm?
> 

I'll try to use them.

>>
>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>> ---
>>  arch/arm/include/asm/kvm_host.h     |    4 +++
>>  arch/arm/include/asm/kvm_hyp.h      |    5 ++++
>>  arch/arm/kvm/Makefile               |    2 ++
>>  arch/arm/kvm/hyp/Makefile           |    1 +
>>  arch/arm/kvm/hyp/switch.c           |   20 +++++++++++--
>>  arch/arm64/kvm/Kconfig              |    4 +--
>>  include/kvm/arm_vgic.h              |    8 ------
>>  virt/kvm/arm/vgic/vgic-kvm-device.c |   10 ++-----
>>  virt/kvm/arm/vgic/vgic-mmio-v3.c    |    2 ++
>>  virt/kvm/arm/vgic/vgic-mmio.c       |    2 --
>>  virt/kvm/arm/vgic/vgic-mmio.h       |    3 --
>>  virt/kvm/arm/vgic/vgic.h            |   54 ++---------------------------------
>>  12 files changed, 38 insertions(+), 77 deletions(-)
>>
>> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
>> index de338d9..1312597 100644
>> --- a/arch/arm/include/asm/kvm_host.h
>> +++ b/arch/arm/include/asm/kvm_host.h
>> @@ -39,7 +39,11 @@
>>  
>>  #include <kvm/arm_vgic.h>
>>  
>> +#ifdef CONFIG_ARM_GIC_V3
>> +#define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS
>> +#else
>>  #define KVM_MAX_VCPUS VGIC_V2_MAX_CPUS
>> +#endif
>>  
>>  #define KVM_REQ_VCPU_EXIT	8
>>  
>> diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
>> index e604ad68..95669b3 100644
>> --- a/arch/arm/include/asm/kvm_hyp.h
>> +++ b/arch/arm/include/asm/kvm_hyp.h
>> @@ -88,6 +88,8 @@
>>  
>>  #define VFP_FPEXC	__ACCESS_VFP(FPEXC)
>>  
>> +#define ID_PFR1		__ACCESS_CP15(c0, 0, c1, 1)
>> +
>>  /* AArch64 compatibility macros, only for the timer so far */
>>  #define read_sysreg_el0(r)		read_sysreg(r##_el0)
>>  #define write_sysreg_el0(v, r)		write_sysreg(v, r##_el0)
>> @@ -103,6 +105,9 @@ void __timer_restore_state(struct kvm_vcpu *vcpu);
>>  void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
>>  void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
>>  
>> +void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
>> +void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
>> +
>>  void __sysreg_save_state(struct kvm_cpu_context *ctxt);
>>  void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
>>  
>> diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
>> index 10d77a6..043d817f 100644
>> --- a/arch/arm/kvm/Makefile
>> +++ b/arch/arm/kvm/Makefile
>> @@ -26,8 +26,10 @@ obj-y += $(KVM)/arm/vgic/vgic.o
>>  obj-y += $(KVM)/arm/vgic/vgic-init.o
>>  obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
>>  obj-y += $(KVM)/arm/vgic/vgic-v2.o
>> +obj-y += $(KVM)/arm/vgic/vgic-v3.o
>>  obj-y += $(KVM)/arm/vgic/vgic-mmio.o
>>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
>> +obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
>>  obj-y += $(KVM)/arm/vgic/vgic-kvm-device.o
>>  obj-y += $(KVM)/irqchip.o
>>  obj-y += $(KVM)/arm/arch_timer.o
>> diff --git a/arch/arm/kvm/hyp/Makefile b/arch/arm/kvm/hyp/Makefile
>> index 8dfa5f7..3023bb5 100644
>> --- a/arch/arm/kvm/hyp/Makefile
>> +++ b/arch/arm/kvm/hyp/Makefile
>> @@ -5,6 +5,7 @@
>>  KVM=../../../../virt/kvm
>>  
>>  obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
>> +obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
>>  obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
>>  
>>  obj-$(CONFIG_KVM_ARM_HOST) += tlb.o
>> diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
>> index b13caa9..9666bae 100644
>> --- a/arch/arm/kvm/hyp/switch.c
>> +++ b/arch/arm/kvm/hyp/switch.c
>> @@ -15,6 +15,8 @@
>>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>>   */
>>  
>> +#include <linux/irqchip/arm-gic-v3.h>
>> +
>>  #include <asm/kvm_asm.h>
>>  #include <asm/kvm_hyp.h>
>>  
>> @@ -74,14 +76,28 @@ static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
>>  	write_sysreg(read_sysreg(MIDR), VPIDR);
>>  }
>>  
>> +static bool __hyp_text __has_useable_gicv3_cpuif(void)
>> +{
>> +	if (IS_ENABLED(CONFIG_ARM_GIC_V3) && (read_sysreg(ID_PFR1) >> 28))
> 
> Do we have a define for bit 28 we could use?
> 
> Does this actually work on all v7 boards?  The v7 ARM ARM seems to state
> that this bitfield is Reserved, UNK.  Does that somehow mean 'is going
> to be zero'?
> 
>> +		return !!(read_sysreg(ICC_HSRE) & ICC_SRE_EL2_SRE);
>> +	else
>> +		return false;
>> +}
>> +
>>  static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
>>  {
>> -	__vgic_v2_save_state(vcpu);
>> +	if (__has_useable_gicv3_cpuif())
>> +		__vgic_v3_save_state(vcpu);
>> +	else
>> +		__vgic_v2_save_state(vcpu);
>>  }
>>  
>>  static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
>>  {
>> -	__vgic_v2_restore_state(vcpu);
>> +	if (__has_useable_gicv3_cpuif())
>> +		__vgic_v3_restore_state(vcpu);
>> +	else
>> +		__vgic_v2_restore_state(vcpu);
>>  }
>>  
>>  static bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu)
>> diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
>> index 9c9edc9..6eaf12c 100644
>> --- a/arch/arm64/kvm/Kconfig
>> +++ b/arch/arm64/kvm/Kconfig
>> @@ -16,7 +16,7 @@ menuconfig VIRTUALIZATION
>>  
>>  if VIRTUALIZATION
>>  
>> -config KVM_ARM_VGIC_V3
>> +config KVM_ARM_VGIC_V3_ITS
> 
> I feel like this could have been simplified with adding the new ITS
> guard in a separate patch.
> 
> In fact, you could then have a separate patch that enables compilation
> of the gicv3 code and removes the static inlines etc., and then finally
> a patch that adds in the logic for the world switch.

Agree. Will rework this patch.

Thanks
Vladimir

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 7/7] ARM: KVM: Unlock vgic-v3 support
@ 2016-09-06 13:18       ` Vladimir Murzin
  0 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-09-06 13:18 UTC (permalink / raw)
  To: linux-arm-kernel

On 05/09/16 12:29, Christoffer Dall wrote:
> On Tue, Aug 16, 2016 at 11:46:58AM +0100, Vladimir Murzin wrote:
>> It is time to get access to common version of vgic-v3.
> 
> common version?
> 

Since patch#2 it not private to arm64 or I should rephrase this?

>>
>> We basically would need to tell build system how to pick it up and
>> undo KVM_ARM_VGIC_V3 guarding introduced in 4f64cb6 ("arm/arm64: KVM:
>> Only allow 64bit hosts to build VGICv3") and remove stubs. However,
>> since vgic got ITS support KVM_ARM_VGIC_V3 tend to protect a little
>> bit more than just plain vgic-v3 - this guard is used for ITS too
>> which is not supported in 32-bit world yet.  So, along with removal of
>> KVM_ARM_VGIC_V3 guard introduce the new one - KVM_ARM_VGIC_V3_ITS to
>> protect ITS related code.
> 
> I don't find this paragraph particularly helpful, I'm afraid.
> 

Sorry for that. It seems to much for one patch, I'll split it per your
suggestion bellow.

>>
>> The only unpleasant part is how we decide which save/restore sequence to
>> use under __vgic_save_state() - we don't have patching framework in hand
>> like arm64, so have to check runtime on every invocation.
> 
> Do static keys work on 32-bit arm?
> 

I'll try to use them.

>>
>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>> ---
>>  arch/arm/include/asm/kvm_host.h     |    4 +++
>>  arch/arm/include/asm/kvm_hyp.h      |    5 ++++
>>  arch/arm/kvm/Makefile               |    2 ++
>>  arch/arm/kvm/hyp/Makefile           |    1 +
>>  arch/arm/kvm/hyp/switch.c           |   20 +++++++++++--
>>  arch/arm64/kvm/Kconfig              |    4 +--
>>  include/kvm/arm_vgic.h              |    8 ------
>>  virt/kvm/arm/vgic/vgic-kvm-device.c |   10 ++-----
>>  virt/kvm/arm/vgic/vgic-mmio-v3.c    |    2 ++
>>  virt/kvm/arm/vgic/vgic-mmio.c       |    2 --
>>  virt/kvm/arm/vgic/vgic-mmio.h       |    3 --
>>  virt/kvm/arm/vgic/vgic.h            |   54 ++---------------------------------
>>  12 files changed, 38 insertions(+), 77 deletions(-)
>>
>> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
>> index de338d9..1312597 100644
>> --- a/arch/arm/include/asm/kvm_host.h
>> +++ b/arch/arm/include/asm/kvm_host.h
>> @@ -39,7 +39,11 @@
>>  
>>  #include <kvm/arm_vgic.h>
>>  
>> +#ifdef CONFIG_ARM_GIC_V3
>> +#define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS
>> +#else
>>  #define KVM_MAX_VCPUS VGIC_V2_MAX_CPUS
>> +#endif
>>  
>>  #define KVM_REQ_VCPU_EXIT	8
>>  
>> diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
>> index e604ad68..95669b3 100644
>> --- a/arch/arm/include/asm/kvm_hyp.h
>> +++ b/arch/arm/include/asm/kvm_hyp.h
>> @@ -88,6 +88,8 @@
>>  
>>  #define VFP_FPEXC	__ACCESS_VFP(FPEXC)
>>  
>> +#define ID_PFR1		__ACCESS_CP15(c0, 0, c1, 1)
>> +
>>  /* AArch64 compatibility macros, only for the timer so far */
>>  #define read_sysreg_el0(r)		read_sysreg(r##_el0)
>>  #define write_sysreg_el0(v, r)		write_sysreg(v, r##_el0)
>> @@ -103,6 +105,9 @@ void __timer_restore_state(struct kvm_vcpu *vcpu);
>>  void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
>>  void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
>>  
>> +void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
>> +void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
>> +
>>  void __sysreg_save_state(struct kvm_cpu_context *ctxt);
>>  void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
>>  
>> diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
>> index 10d77a6..043d817f 100644
>> --- a/arch/arm/kvm/Makefile
>> +++ b/arch/arm/kvm/Makefile
>> @@ -26,8 +26,10 @@ obj-y += $(KVM)/arm/vgic/vgic.o
>>  obj-y += $(KVM)/arm/vgic/vgic-init.o
>>  obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
>>  obj-y += $(KVM)/arm/vgic/vgic-v2.o
>> +obj-y += $(KVM)/arm/vgic/vgic-v3.o
>>  obj-y += $(KVM)/arm/vgic/vgic-mmio.o
>>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
>> +obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
>>  obj-y += $(KVM)/arm/vgic/vgic-kvm-device.o
>>  obj-y += $(KVM)/irqchip.o
>>  obj-y += $(KVM)/arm/arch_timer.o
>> diff --git a/arch/arm/kvm/hyp/Makefile b/arch/arm/kvm/hyp/Makefile
>> index 8dfa5f7..3023bb5 100644
>> --- a/arch/arm/kvm/hyp/Makefile
>> +++ b/arch/arm/kvm/hyp/Makefile
>> @@ -5,6 +5,7 @@
>>  KVM=../../../../virt/kvm
>>  
>>  obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
>> +obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
>>  obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
>>  
>>  obj-$(CONFIG_KVM_ARM_HOST) += tlb.o
>> diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
>> index b13caa9..9666bae 100644
>> --- a/arch/arm/kvm/hyp/switch.c
>> +++ b/arch/arm/kvm/hyp/switch.c
>> @@ -15,6 +15,8 @@
>>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>>   */
>>  
>> +#include <linux/irqchip/arm-gic-v3.h>
>> +
>>  #include <asm/kvm_asm.h>
>>  #include <asm/kvm_hyp.h>
>>  
>> @@ -74,14 +76,28 @@ static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
>>  	write_sysreg(read_sysreg(MIDR), VPIDR);
>>  }
>>  
>> +static bool __hyp_text __has_useable_gicv3_cpuif(void)
>> +{
>> +	if (IS_ENABLED(CONFIG_ARM_GIC_V3) && (read_sysreg(ID_PFR1) >> 28))
> 
> Do we have a define for bit 28 we could use?
> 
> Does this actually work on all v7 boards?  The v7 ARM ARM seems to state
> that this bitfield is Reserved, UNK.  Does that somehow mean 'is going
> to be zero'?
> 
>> +		return !!(read_sysreg(ICC_HSRE) & ICC_SRE_EL2_SRE);
>> +	else
>> +		return false;
>> +}
>> +
>>  static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
>>  {
>> -	__vgic_v2_save_state(vcpu);
>> +	if (__has_useable_gicv3_cpuif())
>> +		__vgic_v3_save_state(vcpu);
>> +	else
>> +		__vgic_v2_save_state(vcpu);
>>  }
>>  
>>  static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
>>  {
>> -	__vgic_v2_restore_state(vcpu);
>> +	if (__has_useable_gicv3_cpuif())
>> +		__vgic_v3_restore_state(vcpu);
>> +	else
>> +		__vgic_v2_restore_state(vcpu);
>>  }
>>  
>>  static bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu)
>> diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
>> index 9c9edc9..6eaf12c 100644
>> --- a/arch/arm64/kvm/Kconfig
>> +++ b/arch/arm64/kvm/Kconfig
>> @@ -16,7 +16,7 @@ menuconfig VIRTUALIZATION
>>  
>>  if VIRTUALIZATION
>>  
>> -config KVM_ARM_VGIC_V3
>> +config KVM_ARM_VGIC_V3_ITS
> 
> I feel like this could have been simplified with adding the new ITS
> guard in a separate patch.
> 
> In fact, you could then have a separate patch that enables compilation
> of the gicv3 code and removes the static inlines etc., and then finally
> a patch that adds in the logic for the world switch.

Agree. Will rework this patch.

Thanks
Vladimir

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 3/7] KVM: arm: vgic-new: improve compatibility with 32-bit
  2016-09-06 12:41       ` Vladimir Murzin
@ 2016-09-06 13:22         ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-06 13:22 UTC (permalink / raw)
  To: Vladimir Murzin; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

On Tue, Sep 06, 2016 at 01:41:37PM +0100, Vladimir Murzin wrote:
> Hi Christoffer,
> 
> On 05/09/16 12:29, Christoffer Dall wrote:
> > Hi Vladimir,
> > 
> > I think commit title is too vague, can you be more specific?
> > 
> 
> KVM: arm: vgic-new: make extract_bytes to always work on 64-bit data
> 
> is it better?

I would suggest:

KVM: arm: vgic: Support 64-bit data manipulation on 32-bit host systems

> 
> > On Tue, Aug 16, 2016 at 11:46:54AM +0100, Vladimir Murzin wrote:
> >> We have couple of 64-bit register defined in GICv3 architecture, so
> > 
> > 'a couple',  'registers' (plural)
> > 
> >> "unsigned long" kind of accessors wouldn't work for 32-bit. However,
> > 
> > 'wouldn't work for 32-bit' is kind of generic as well.  Perhaps you mean
> > that unsigned long accesses to these registers will only access a single
> > 32-bit work of that register.
> > 
> >> these registers can't be access as 64-bit in a one go if we run 32-bit
> > 
> > 'accessed', 's/in one go/with a single instruction/' ?
> > 
> > 'a 32-bit host'
> > 
> >> host simply because KVM doesn't support multiple load/store on MMIO
> > 
> > by 'multiple load/store' you mean the 'load/store multiple' instructions
> > specifically, right?  Not a sequence of multiple loads and stores.  I
> > think you should be more specific here as well, for example, I think
> > ldrd and strd are problematic as well.
> > 
> >> space.
> >>
> >> It means that 32-bit guest access these registers in 32-bit chunks, so
> > 
> > 'a 32-bit guest', 'accesses'
> > 
> 
> all suggestions you've made above are true. I'll rework commit message
> to be more precise.
> 

Thanks!

> >> the only thing we need to do is to ensure that extract_bytes() always
> >> takes 64-bit data.
> >>
> >> Since we are here fix couple of other width related issues by using
> >> ULL variants over UL.
> >>
> >> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> >> ---
> >>  virt/kvm/arm/vgic/vgic-mmio-v3.c |    6 +++---
> >>  virt/kvm/arm/vgic/vgic-mmio.h    |    2 +-
> >>  2 files changed, 4 insertions(+), 4 deletions(-)
> >>
> >> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> index ff668e0..cc20b60 100644
> >> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> @@ -23,7 +23,7 @@
> >>  #include "vgic-mmio.h"
> >>  
> >>  /* extract @num bytes at @offset bytes offset in data */
> >> -unsigned long extract_bytes(unsigned long data, unsigned int offset,
> >> +unsigned long extract_bytes(u64 data, unsigned int offset,
> >>  			    unsigned int num)
> >>  {
> >>  	return (data >> (offset * 8)) & GENMASK_ULL(num * 8 - 1, 0);
> >> @@ -179,7 +179,7 @@ static unsigned long vgic_mmio_read_v3r_typer(struct kvm_vcpu *vcpu,
> >>  	int target_vcpu_id = vcpu->vcpu_id;
> >>  	u64 value;
> >>  
> >> -	value = (mpidr & GENMASK(23, 0)) << 32;
> >> +	value = (mpidr & GENMASK_ULL(23, 0)) << 32;
> > 
> > why does this make a difference when mpidr is an unsigned long?
> 
> because we access a little bit further than unsigned long can accommodate
> 
>   CC      arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.o
> arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c: In function
> 'vgic_mmio_read_v3r_typer':
> arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c:184:35: warning:
> left shift count >= width of type [-Wshift-count-overflow]
>   value = (mpidr & GENMASK(23, 0)) << 32;
>                                    ^
> 
> I can include this warning in commit message or maybe you want a
> separate patch?
> 
My point was that the code doesn't really make sense when compiled on a
32-bit platform without also modifing the type for the mpidr variable.
Am I missing something?

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 3/7] KVM: arm: vgic-new: improve compatibility with 32-bit
@ 2016-09-06 13:22         ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-06 13:22 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Sep 06, 2016 at 01:41:37PM +0100, Vladimir Murzin wrote:
> Hi Christoffer,
> 
> On 05/09/16 12:29, Christoffer Dall wrote:
> > Hi Vladimir,
> > 
> > I think commit title is too vague, can you be more specific?
> > 
> 
> KVM: arm: vgic-new: make extract_bytes to always work on 64-bit data
> 
> is it better?

I would suggest:

KVM: arm: vgic: Support 64-bit data manipulation on 32-bit host systems

> 
> > On Tue, Aug 16, 2016 at 11:46:54AM +0100, Vladimir Murzin wrote:
> >> We have couple of 64-bit register defined in GICv3 architecture, so
> > 
> > 'a couple',  'registers' (plural)
> > 
> >> "unsigned long" kind of accessors wouldn't work for 32-bit. However,
> > 
> > 'wouldn't work for 32-bit' is kind of generic as well.  Perhaps you mean
> > that unsigned long accesses to these registers will only access a single
> > 32-bit work of that register.
> > 
> >> these registers can't be access as 64-bit in a one go if we run 32-bit
> > 
> > 'accessed', 's/in one go/with a single instruction/' ?
> > 
> > 'a 32-bit host'
> > 
> >> host simply because KVM doesn't support multiple load/store on MMIO
> > 
> > by 'multiple load/store' you mean the 'load/store multiple' instructions
> > specifically, right?  Not a sequence of multiple loads and stores.  I
> > think you should be more specific here as well, for example, I think
> > ldrd and strd are problematic as well.
> > 
> >> space.
> >>
> >> It means that 32-bit guest access these registers in 32-bit chunks, so
> > 
> > 'a 32-bit guest', 'accesses'
> > 
> 
> all suggestions you've made above are true. I'll rework commit message
> to be more precise.
> 

Thanks!

> >> the only thing we need to do is to ensure that extract_bytes() always
> >> takes 64-bit data.
> >>
> >> Since we are here fix couple of other width related issues by using
> >> ULL variants over UL.
> >>
> >> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> >> ---
> >>  virt/kvm/arm/vgic/vgic-mmio-v3.c |    6 +++---
> >>  virt/kvm/arm/vgic/vgic-mmio.h    |    2 +-
> >>  2 files changed, 4 insertions(+), 4 deletions(-)
> >>
> >> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> index ff668e0..cc20b60 100644
> >> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> @@ -23,7 +23,7 @@
> >>  #include "vgic-mmio.h"
> >>  
> >>  /* extract @num bytes at @offset bytes offset in data */
> >> -unsigned long extract_bytes(unsigned long data, unsigned int offset,
> >> +unsigned long extract_bytes(u64 data, unsigned int offset,
> >>  			    unsigned int num)
> >>  {
> >>  	return (data >> (offset * 8)) & GENMASK_ULL(num * 8 - 1, 0);
> >> @@ -179,7 +179,7 @@ static unsigned long vgic_mmio_read_v3r_typer(struct kvm_vcpu *vcpu,
> >>  	int target_vcpu_id = vcpu->vcpu_id;
> >>  	u64 value;
> >>  
> >> -	value = (mpidr & GENMASK(23, 0)) << 32;
> >> +	value = (mpidr & GENMASK_ULL(23, 0)) << 32;
> > 
> > why does this make a difference when mpidr is an unsigned long?
> 
> because we access a little bit further than unsigned long can accommodate
> 
>   CC      arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.o
> arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c: In function
> 'vgic_mmio_read_v3r_typer':
> arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c:184:35: warning:
> left shift count >= width of type [-Wshift-count-overflow]
>   value = (mpidr & GENMASK(23, 0)) << 32;
>                                    ^
> 
> I can include this warning in commit message or maybe you want a
> separate patch?
> 
My point was that the code doesn't really make sense when compiled on a
32-bit platform without also modifing the type for the mpidr variable.
Am I missing something?

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 7/7] ARM: KVM: Unlock vgic-v3 support
  2016-09-05 11:29     ` Christoffer Dall
@ 2016-09-06 13:23       ` Vladimir Murzin
  -1 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-09-06 13:23 UTC (permalink / raw)
  To: Christoffer Dall; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel


Sorry, missed this one

On 05/09/16 12:29, Christoffer Dall wrote:
>>  
>> > +static bool __hyp_text __has_useable_gicv3_cpuif(void)
>> > +{
>> > +	if (IS_ENABLED(CONFIG_ARM_GIC_V3) && (read_sysreg(ID_PFR1) >> 28))
> Do we have a define for bit 28 we could use?

I'll check it.

> 
> Does this actually work on all v7 boards?  The v7 ARM ARM seems to state
> that this bitfield is Reserved, UNK.  Does that somehow mean 'is going
> to be zero'?

It is how v7ARM ARM I have defines UNK

An abbreviation indicating that software must treat a field as
containing an UNKNOWN value. Hardware must implement the bit as read as
0, or all 0s for a bit field. Software must not rely on the field
reading as zero.

It seems goes under 'is going to be zero' case, no?

Cheers
Vladimir

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 7/7] ARM: KVM: Unlock vgic-v3 support
@ 2016-09-06 13:23       ` Vladimir Murzin
  0 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-09-06 13:23 UTC (permalink / raw)
  To: linux-arm-kernel


Sorry, missed this one

On 05/09/16 12:29, Christoffer Dall wrote:
>>  
>> > +static bool __hyp_text __has_useable_gicv3_cpuif(void)
>> > +{
>> > +	if (IS_ENABLED(CONFIG_ARM_GIC_V3) && (read_sysreg(ID_PFR1) >> 28))
> Do we have a define for bit 28 we could use?

I'll check it.

> 
> Does this actually work on all v7 boards?  The v7 ARM ARM seems to state
> that this bitfield is Reserved, UNK.  Does that somehow mean 'is going
> to be zero'?

It is how v7ARM ARM I have defines UNK

An abbreviation indicating that software must treat a field as
containing an UNKNOWN value. Hardware must implement the bit as read as
0, or all 0s for a bit field. Software must not rely on the field
reading as zero.

It seems goes under 'is going to be zero' case, no?

Cheers
Vladimir

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 3/7] KVM: arm: vgic-new: improve compatibility with 32-bit
  2016-09-06 13:22         ` Christoffer Dall
@ 2016-09-06 13:54           ` Vladimir Murzin
  -1 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-09-06 13:54 UTC (permalink / raw)
  To: Christoffer Dall; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

On 06/09/16 14:22, Christoffer Dall wrote:
> On Tue, Sep 06, 2016 at 01:41:37PM +0100, Vladimir Murzin wrote:
>> Hi Christoffer,
>>
>> On 05/09/16 12:29, Christoffer Dall wrote:
>>> Hi Vladimir,
>>>
>>> I think commit title is too vague, can you be more specific?
>>>
>>
>> KVM: arm: vgic-new: make extract_bytes to always work on 64-bit data
>>
>> is it better?
> 
> I would suggest:
> 
> KVM: arm: vgic: Support 64-bit data manipulation on 32-bit host systems
> 
>>
>>> On Tue, Aug 16, 2016 at 11:46:54AM +0100, Vladimir Murzin wrote:
>>>> We have couple of 64-bit register defined in GICv3 architecture, so
>>>
>>> 'a couple',  'registers' (plural)
>>>
>>>> "unsigned long" kind of accessors wouldn't work for 32-bit. However,
>>>
>>> 'wouldn't work for 32-bit' is kind of generic as well.  Perhaps you mean
>>> that unsigned long accesses to these registers will only access a single
>>> 32-bit work of that register.
>>>
>>>> these registers can't be access as 64-bit in a one go if we run 32-bit
>>>
>>> 'accessed', 's/in one go/with a single instruction/' ?
>>>
>>> 'a 32-bit host'
>>>
>>>> host simply because KVM doesn't support multiple load/store on MMIO
>>>
>>> by 'multiple load/store' you mean the 'load/store multiple' instructions
>>> specifically, right?  Not a sequence of multiple loads and stores.  I
>>> think you should be more specific here as well, for example, I think
>>> ldrd and strd are problematic as well.
>>>
>>>> space.
>>>>
>>>> It means that 32-bit guest access these registers in 32-bit chunks, so
>>>
>>> 'a 32-bit guest', 'accesses'
>>>
>>
>> all suggestions you've made above are true. I'll rework commit message
>> to be more precise.
>>
> 
> Thanks!
> 
>>>> the only thing we need to do is to ensure that extract_bytes() always
>>>> takes 64-bit data.
>>>>
>>>> Since we are here fix couple of other width related issues by using
>>>> ULL variants over UL.
>>>>
>>>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>>>> ---
>>>>  virt/kvm/arm/vgic/vgic-mmio-v3.c |    6 +++---
>>>>  virt/kvm/arm/vgic/vgic-mmio.h    |    2 +-
>>>>  2 files changed, 4 insertions(+), 4 deletions(-)
>>>>
>>>> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>>>> index ff668e0..cc20b60 100644
>>>> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
>>>> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>>>> @@ -23,7 +23,7 @@
>>>>  #include "vgic-mmio.h"
>>>>  
>>>>  /* extract @num bytes at @offset bytes offset in data */
>>>> -unsigned long extract_bytes(unsigned long data, unsigned int offset,
>>>> +unsigned long extract_bytes(u64 data, unsigned int offset,
>>>>  			    unsigned int num)
>>>>  {
>>>>  	return (data >> (offset * 8)) & GENMASK_ULL(num * 8 - 1, 0);
>>>> @@ -179,7 +179,7 @@ static unsigned long vgic_mmio_read_v3r_typer(struct kvm_vcpu *vcpu,
>>>>  	int target_vcpu_id = vcpu->vcpu_id;
>>>>  	u64 value;
>>>>  
>>>> -	value = (mpidr & GENMASK(23, 0)) << 32;
>>>> +	value = (mpidr & GENMASK_ULL(23, 0)) << 32;
>>>
>>> why does this make a difference when mpidr is an unsigned long?
>>
>> because we access a little bit further than unsigned long can accommodate
>>
>>   CC      arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.o
>> arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c: In function
>> 'vgic_mmio_read_v3r_typer':
>> arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c:184:35: warning:
>> left shift count >= width of type [-Wshift-count-overflow]
>>   value = (mpidr & GENMASK(23, 0)) << 32;
>>                                    ^
>>
>> I can include this warning in commit message or maybe you want a
>> separate patch?
>>
> My point was that the code doesn't really make sense when compiled on a
> 32-bit platform without also modifing the type for the mpidr variable.
> Am I missing something?

I've not seen any difference in generated code, but for consistency I'll
update mpidr variable to u64.

Thanks
Vladimir

> 
> Thanks,
> -Christoffer
> 
> 

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 3/7] KVM: arm: vgic-new: improve compatibility with 32-bit
@ 2016-09-06 13:54           ` Vladimir Murzin
  0 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-09-06 13:54 UTC (permalink / raw)
  To: linux-arm-kernel

On 06/09/16 14:22, Christoffer Dall wrote:
> On Tue, Sep 06, 2016 at 01:41:37PM +0100, Vladimir Murzin wrote:
>> Hi Christoffer,
>>
>> On 05/09/16 12:29, Christoffer Dall wrote:
>>> Hi Vladimir,
>>>
>>> I think commit title is too vague, can you be more specific?
>>>
>>
>> KVM: arm: vgic-new: make extract_bytes to always work on 64-bit data
>>
>> is it better?
> 
> I would suggest:
> 
> KVM: arm: vgic: Support 64-bit data manipulation on 32-bit host systems
> 
>>
>>> On Tue, Aug 16, 2016 at 11:46:54AM +0100, Vladimir Murzin wrote:
>>>> We have couple of 64-bit register defined in GICv3 architecture, so
>>>
>>> 'a couple',  'registers' (plural)
>>>
>>>> "unsigned long" kind of accessors wouldn't work for 32-bit. However,
>>>
>>> 'wouldn't work for 32-bit' is kind of generic as well.  Perhaps you mean
>>> that unsigned long accesses to these registers will only access a single
>>> 32-bit work of that register.
>>>
>>>> these registers can't be access as 64-bit in a one go if we run 32-bit
>>>
>>> 'accessed', 's/in one go/with a single instruction/' ?
>>>
>>> 'a 32-bit host'
>>>
>>>> host simply because KVM doesn't support multiple load/store on MMIO
>>>
>>> by 'multiple load/store' you mean the 'load/store multiple' instructions
>>> specifically, right?  Not a sequence of multiple loads and stores.  I
>>> think you should be more specific here as well, for example, I think
>>> ldrd and strd are problematic as well.
>>>
>>>> space.
>>>>
>>>> It means that 32-bit guest access these registers in 32-bit chunks, so
>>>
>>> 'a 32-bit guest', 'accesses'
>>>
>>
>> all suggestions you've made above are true. I'll rework commit message
>> to be more precise.
>>
> 
> Thanks!
> 
>>>> the only thing we need to do is to ensure that extract_bytes() always
>>>> takes 64-bit data.
>>>>
>>>> Since we are here fix couple of other width related issues by using
>>>> ULL variants over UL.
>>>>
>>>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>>>> ---
>>>>  virt/kvm/arm/vgic/vgic-mmio-v3.c |    6 +++---
>>>>  virt/kvm/arm/vgic/vgic-mmio.h    |    2 +-
>>>>  2 files changed, 4 insertions(+), 4 deletions(-)
>>>>
>>>> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>>>> index ff668e0..cc20b60 100644
>>>> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
>>>> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>>>> @@ -23,7 +23,7 @@
>>>>  #include "vgic-mmio.h"
>>>>  
>>>>  /* extract @num bytes at @offset bytes offset in data */
>>>> -unsigned long extract_bytes(unsigned long data, unsigned int offset,
>>>> +unsigned long extract_bytes(u64 data, unsigned int offset,
>>>>  			    unsigned int num)
>>>>  {
>>>>  	return (data >> (offset * 8)) & GENMASK_ULL(num * 8 - 1, 0);
>>>> @@ -179,7 +179,7 @@ static unsigned long vgic_mmio_read_v3r_typer(struct kvm_vcpu *vcpu,
>>>>  	int target_vcpu_id = vcpu->vcpu_id;
>>>>  	u64 value;
>>>>  
>>>> -	value = (mpidr & GENMASK(23, 0)) << 32;
>>>> +	value = (mpidr & GENMASK_ULL(23, 0)) << 32;
>>>
>>> why does this make a difference when mpidr is an unsigned long?
>>
>> because we access a little bit further than unsigned long can accommodate
>>
>>   CC      arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.o
>> arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c: In function
>> 'vgic_mmio_read_v3r_typer':
>> arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c:184:35: warning:
>> left shift count >= width of type [-Wshift-count-overflow]
>>   value = (mpidr & GENMASK(23, 0)) << 32;
>>                                    ^
>>
>> I can include this warning in commit message or maybe you want a
>> separate patch?
>>
> My point was that the code doesn't really make sense when compiled on a
> 32-bit platform without also modifing the type for the mpidr variable.
> Am I missing something?

I've not seen any difference in generated code, but for consistency I'll
update mpidr variable to u64.

Thanks
Vladimir

> 
> Thanks,
> -Christoffer
> 
> 

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 3/7] KVM: arm: vgic-new: improve compatibility with 32-bit
  2016-09-06 13:54           ` Vladimir Murzin
@ 2016-09-06 16:31             ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-06 16:31 UTC (permalink / raw)
  To: Vladimir Murzin; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

On Tue, Sep 06, 2016 at 02:54:01PM +0100, Vladimir Murzin wrote:
> On 06/09/16 14:22, Christoffer Dall wrote:
> > On Tue, Sep 06, 2016 at 01:41:37PM +0100, Vladimir Murzin wrote:
> >> Hi Christoffer,
> >>
> >> On 05/09/16 12:29, Christoffer Dall wrote:
> >>> Hi Vladimir,
> >>>
> >>> I think commit title is too vague, can you be more specific?
> >>>
> >>
> >> KVM: arm: vgic-new: make extract_bytes to always work on 64-bit data
> >>
> >> is it better?
> > 
> > I would suggest:
> > 
> > KVM: arm: vgic: Support 64-bit data manipulation on 32-bit host systems
> > 
> >>
> >>> On Tue, Aug 16, 2016 at 11:46:54AM +0100, Vladimir Murzin wrote:
> >>>> We have couple of 64-bit register defined in GICv3 architecture, so
> >>>
> >>> 'a couple',  'registers' (plural)
> >>>
> >>>> "unsigned long" kind of accessors wouldn't work for 32-bit. However,
> >>>
> >>> 'wouldn't work for 32-bit' is kind of generic as well.  Perhaps you mean
> >>> that unsigned long accesses to these registers will only access a single
> >>> 32-bit work of that register.
> >>>
> >>>> these registers can't be access as 64-bit in a one go if we run 32-bit
> >>>
> >>> 'accessed', 's/in one go/with a single instruction/' ?
> >>>
> >>> 'a 32-bit host'
> >>>
> >>>> host simply because KVM doesn't support multiple load/store on MMIO
> >>>
> >>> by 'multiple load/store' you mean the 'load/store multiple' instructions
> >>> specifically, right?  Not a sequence of multiple loads and stores.  I
> >>> think you should be more specific here as well, for example, I think
> >>> ldrd and strd are problematic as well.
> >>>
> >>>> space.
> >>>>
> >>>> It means that 32-bit guest access these registers in 32-bit chunks, so
> >>>
> >>> 'a 32-bit guest', 'accesses'
> >>>
> >>
> >> all suggestions you've made above are true. I'll rework commit message
> >> to be more precise.
> >>
> > 
> > Thanks!
> > 
> >>>> the only thing we need to do is to ensure that extract_bytes() always
> >>>> takes 64-bit data.
> >>>>
> >>>> Since we are here fix couple of other width related issues by using
> >>>> ULL variants over UL.
> >>>>
> >>>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> >>>> ---
> >>>>  virt/kvm/arm/vgic/vgic-mmio-v3.c |    6 +++---
> >>>>  virt/kvm/arm/vgic/vgic-mmio.h    |    2 +-
> >>>>  2 files changed, 4 insertions(+), 4 deletions(-)
> >>>>
> >>>> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >>>> index ff668e0..cc20b60 100644
> >>>> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >>>> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >>>> @@ -23,7 +23,7 @@
> >>>>  #include "vgic-mmio.h"
> >>>>  
> >>>>  /* extract @num bytes at @offset bytes offset in data */
> >>>> -unsigned long extract_bytes(unsigned long data, unsigned int offset,
> >>>> +unsigned long extract_bytes(u64 data, unsigned int offset,
> >>>>  			    unsigned int num)
> >>>>  {
> >>>>  	return (data >> (offset * 8)) & GENMASK_ULL(num * 8 - 1, 0);
> >>>> @@ -179,7 +179,7 @@ static unsigned long vgic_mmio_read_v3r_typer(struct kvm_vcpu *vcpu,
> >>>>  	int target_vcpu_id = vcpu->vcpu_id;
> >>>>  	u64 value;
> >>>>  
> >>>> -	value = (mpidr & GENMASK(23, 0)) << 32;
> >>>> +	value = (mpidr & GENMASK_ULL(23, 0)) << 32;
> >>>
> >>> why does this make a difference when mpidr is an unsigned long?
> >>
> >> because we access a little bit further than unsigned long can accommodate
> >>
> >>   CC      arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.o
> >> arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c: In function
> >> 'vgic_mmio_read_v3r_typer':
> >> arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c:184:35: warning:
> >> left shift count >= width of type [-Wshift-count-overflow]
> >>   value = (mpidr & GENMASK(23, 0)) << 32;
> >>                                    ^
> >>
> >> I can include this warning in commit message or maybe you want a
> >> separate patch?
> >>
> > My point was that the code doesn't really make sense when compiled on a
> > 32-bit platform without also modifing the type for the mpidr variable.
> > Am I missing something?
> 
> I've not seen any difference in generated code, but for consistency I'll
> update mpidr variable to u64.
> 

That could be because you need to update kvm_vcpu_get_mpidr_aff() to
return a u64 as well.

-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 3/7] KVM: arm: vgic-new: improve compatibility with 32-bit
@ 2016-09-06 16:31             ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-06 16:31 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Sep 06, 2016 at 02:54:01PM +0100, Vladimir Murzin wrote:
> On 06/09/16 14:22, Christoffer Dall wrote:
> > On Tue, Sep 06, 2016 at 01:41:37PM +0100, Vladimir Murzin wrote:
> >> Hi Christoffer,
> >>
> >> On 05/09/16 12:29, Christoffer Dall wrote:
> >>> Hi Vladimir,
> >>>
> >>> I think commit title is too vague, can you be more specific?
> >>>
> >>
> >> KVM: arm: vgic-new: make extract_bytes to always work on 64-bit data
> >>
> >> is it better?
> > 
> > I would suggest:
> > 
> > KVM: arm: vgic: Support 64-bit data manipulation on 32-bit host systems
> > 
> >>
> >>> On Tue, Aug 16, 2016 at 11:46:54AM +0100, Vladimir Murzin wrote:
> >>>> We have couple of 64-bit register defined in GICv3 architecture, so
> >>>
> >>> 'a couple',  'registers' (plural)
> >>>
> >>>> "unsigned long" kind of accessors wouldn't work for 32-bit. However,
> >>>
> >>> 'wouldn't work for 32-bit' is kind of generic as well.  Perhaps you mean
> >>> that unsigned long accesses to these registers will only access a single
> >>> 32-bit work of that register.
> >>>
> >>>> these registers can't be access as 64-bit in a one go if we run 32-bit
> >>>
> >>> 'accessed', 's/in one go/with a single instruction/' ?
> >>>
> >>> 'a 32-bit host'
> >>>
> >>>> host simply because KVM doesn't support multiple load/store on MMIO
> >>>
> >>> by 'multiple load/store' you mean the 'load/store multiple' instructions
> >>> specifically, right?  Not a sequence of multiple loads and stores.  I
> >>> think you should be more specific here as well, for example, I think
> >>> ldrd and strd are problematic as well.
> >>>
> >>>> space.
> >>>>
> >>>> It means that 32-bit guest access these registers in 32-bit chunks, so
> >>>
> >>> 'a 32-bit guest', 'accesses'
> >>>
> >>
> >> all suggestions you've made above are true. I'll rework commit message
> >> to be more precise.
> >>
> > 
> > Thanks!
> > 
> >>>> the only thing we need to do is to ensure that extract_bytes() always
> >>>> takes 64-bit data.
> >>>>
> >>>> Since we are here fix couple of other width related issues by using
> >>>> ULL variants over UL.
> >>>>
> >>>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> >>>> ---
> >>>>  virt/kvm/arm/vgic/vgic-mmio-v3.c |    6 +++---
> >>>>  virt/kvm/arm/vgic/vgic-mmio.h    |    2 +-
> >>>>  2 files changed, 4 insertions(+), 4 deletions(-)
> >>>>
> >>>> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >>>> index ff668e0..cc20b60 100644
> >>>> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >>>> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >>>> @@ -23,7 +23,7 @@
> >>>>  #include "vgic-mmio.h"
> >>>>  
> >>>>  /* extract @num bytes at @offset bytes offset in data */
> >>>> -unsigned long extract_bytes(unsigned long data, unsigned int offset,
> >>>> +unsigned long extract_bytes(u64 data, unsigned int offset,
> >>>>  			    unsigned int num)
> >>>>  {
> >>>>  	return (data >> (offset * 8)) & GENMASK_ULL(num * 8 - 1, 0);
> >>>> @@ -179,7 +179,7 @@ static unsigned long vgic_mmio_read_v3r_typer(struct kvm_vcpu *vcpu,
> >>>>  	int target_vcpu_id = vcpu->vcpu_id;
> >>>>  	u64 value;
> >>>>  
> >>>> -	value = (mpidr & GENMASK(23, 0)) << 32;
> >>>> +	value = (mpidr & GENMASK_ULL(23, 0)) << 32;
> >>>
> >>> why does this make a difference when mpidr is an unsigned long?
> >>
> >> because we access a little bit further than unsigned long can accommodate
> >>
> >>   CC      arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.o
> >> arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c: In function
> >> 'vgic_mmio_read_v3r_typer':
> >> arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c:184:35: warning:
> >> left shift count >= width of type [-Wshift-count-overflow]
> >>   value = (mpidr & GENMASK(23, 0)) << 32;
> >>                                    ^
> >>
> >> I can include this warning in commit message or maybe you want a
> >> separate patch?
> >>
> > My point was that the code doesn't really make sense when compiled on a
> > 32-bit platform without also modifing the type for the mpidr variable.
> > Am I missing something?
> 
> I've not seen any difference in generated code, but for consistency I'll
> update mpidr variable to u64.
> 

That could be because you need to update kvm_vcpu_get_mpidr_aff() to
return a u64 as well.

-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 5/7] ARM: move system register accessors to asm/cp15.h
  2016-09-06 13:05       ` Vladimir Murzin
@ 2016-09-06 16:34         ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-06 16:34 UTC (permalink / raw)
  To: Vladimir Murzin; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

On Tue, Sep 06, 2016 at 02:05:30PM +0100, Vladimir Murzin wrote:
> On 05/09/16 12:29, Christoffer Dall wrote:
> > On Tue, Aug 16, 2016 at 11:46:56AM +0100, Vladimir Murzin wrote:
> >> Macro __ACCESS_CP15{_64} is defined in two headers (arch_gicv3.h and
> >> kvm_hyp.h) which are going to be requested by vgic-v3 altogether.
> >> GCC would not like it because it'd see that macro is redefined and (hey!)
> >> they are different.  So, let's put only single macro version under common
> >> place and use it everywhere.
> > 
> > I'm sorry, but I don't understand this commit text.
> > 
> 
> Sorry for that :(
> 
> The issue the patch is trying to solve happens because
> virt/kvm/arm/hyp/vgic-v3-sr.c has
> 
> #include <linux/irqchip/arm-gic-v3.h>
> ...
> #include <asm/kvm_hyp.h>
> 
> each of these headers defines it's own __ACCESS_CP15 macro:
> 
> asm/kvm_hyp.h
> 
> #define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
> 	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
> 
> and linux/irqchip/arm-gic-v3.h via asm/arch_gicv3.h
> 
> #define __ACCESS_CP15(CRn, Op1, CRm, Op2) p15, Op1, %0, CRn, CRm, Op2
> 
> When these headers are used together conflict happens. The same applies
> to __ACCESS_CP15_64 macro.
> 
> To address that only single set of macros is used and call sites updated.

Thanks for the explanation!

So a simpler way, IMHO, to explain this is to simply say that both
linux/irqchip/arm-gic.v3.h and arch/arm/include/asm/kvm_hyp.h define a
macro called __ACCESS_CP15, which obviously creates a conflict.

Then you could explain if these are just different implementations of
the same thing, or if they are semantically different, and finally how
your patch solves this problem.

> 
> >>
> >> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> >> ---
> >>  arch/arm/include/asm/arch_gicv3.h |   27 +++++++++++----------------
> >>  arch/arm/include/asm/cp15.h       |   15 +++++++++++++++
> >>  arch/arm/include/asm/kvm_hyp.h    |   15 +--------------
> >>  3 files changed, 27 insertions(+), 30 deletions(-)
> >>
> >> diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
> >> index e08d151..af25c32 100644
> >> --- a/arch/arm/include/asm/arch_gicv3.h
> >> +++ b/arch/arm/include/asm/arch_gicv3.h
> >> @@ -22,9 +22,7 @@
> >>  
> >>  #include <linux/io.h>
> >>  #include <asm/barrier.h>
> >> -
> >> -#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
> >> -#define __ACCESS_CP15_64(Op1, CRm)		p15, Op1, %Q0, %R0, CRm
> >> +#include <asm/cp15.h>
> >>  
> >>  #define ICC_EOIR1			__ACCESS_CP15(c12, 0, c12, 1)
> >>  #define ICC_DIR				__ACCESS_CP15(c12, 0, c11, 1)
> >> @@ -102,58 +100,55 @@
> >>  
> >>  static inline void gic_write_eoir(u32 irq)
> >>  {
> >> -	asm volatile("mcr " __stringify(ICC_EOIR1) : : "r" (irq));
> >> +	write_sysreg(irq, ICC_EOIR1);
> >>  	isb();
> >>  }
> >>  
> >>  static inline void gic_write_dir(u32 val)
> >>  {
> >> -	asm volatile("mcr " __stringify(ICC_DIR) : : "r" (val));
> >> +	write_sysreg(val, ICC_DIR);
> >>  	isb();
> >>  }
> >>  
> >>  static inline u32 gic_read_iar(void)
> >>  {
> >> -	u32 irqstat;
> >> +	u32 irqstat = read_sysreg(ICC_IAR1);
> >>  
> >> -	asm volatile("mrc " __stringify(ICC_IAR1) : "=r" (irqstat));
> >>  	dsb(sy);
> >> +
> >>  	return irqstat;
> >>  }
> >>  
> >>  static inline void gic_write_pmr(u32 val)
> >>  {
> >> -	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
> >> +	write_sysreg(val, ICC_PMR);
> >>  }
> >>  
> >>  static inline void gic_write_ctlr(u32 val)
> >>  {
> >> -	asm volatile("mcr " __stringify(ICC_CTLR) : : "r" (val));
> >> +	write_sysreg(val, ICC_CTLR);
> >>  	isb();
> >>  }
> >>  
> >>  static inline void gic_write_grpen1(u32 val)
> >>  {
> >> -	asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
> >> +	write_sysreg(val, ICC_IGRPEN1);
> >>  	isb();
> >>  }
> >>  
> >>  static inline void gic_write_sgi1r(u64 val)
> >>  {
> >> -	asm volatile("mcrr " __stringify(ICC_SGI1R) : : "r" (val));
> >> +	write_sysreg(val, ICC_SGI1R);
> >>  }
> >>  
> >>  static inline u32 gic_read_sre(void)
> >>  {
> >> -	u32 val;
> >> -
> >> -	asm volatile("mrc " __stringify(ICC_SRE) : "=r" (val));
> >> -	return val;
> >> +	return read_sysreg(ICC_SRE);
> >>  }
> >>  
> >>  static inline void gic_write_sre(u32 val)
> >>  {
> >> -	asm volatile("mcr " __stringify(ICC_SRE) : : "r" (val));
> >> +	write_sysreg(val, ICC_SRE);
> >>  	isb();
> >>  }
> >>  
> >> diff --git a/arch/arm/include/asm/cp15.h b/arch/arm/include/asm/cp15.h
> >> index c3f1152..f661732 100644
> >> --- a/arch/arm/include/asm/cp15.h
> >> +++ b/arch/arm/include/asm/cp15.h
> >> @@ -47,6 +47,21 @@
> >>  #define vectors_high()	(0)
> >>  #endif
> >>  
> >> +#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
> >> +	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
> >> +#define __ACCESS_CP15_64(Op1, CRm)		\
> >> +	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
> >> +
> >> +#define __read_sysreg(r, w, c, t) ({				\
> >> +	t __val;						\
> >> +	asm volatile(r " " c : "=r" (__val));			\
> >> +	__val;							\
> >> +})
> >> +#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
> >> +
> >> +#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
> >> +#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
> >> +
> > 
> > I feel a bit strange about adding this sort of stuff in a
> > non-kvm-non-gic-specific ARM header file, without it being used (or
> > planned to be used) in a broader sense.
> > 
> > Is there not a way to keep the required changes local to KVM and the
> > gic?
> > 
> 
> We could add prefixes to KVM and GIC version of macros so they won't
> clash, but it'd introduce code duplication.
> We could keep macro in, say, GIC header and include it in KVM one (or
> vice versa), but such dependency would not look nicer, IMO.
> 
> The way arm64 handles this is via sysreg.h and the closest counterpart
> under arch/arm is cp15.h
> 
> I'm open to suggestions.
> 

Hmm, I guess you're right, your approach does make sense.

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 5/7] ARM: move system register accessors to asm/cp15.h
@ 2016-09-06 16:34         ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-06 16:34 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Sep 06, 2016 at 02:05:30PM +0100, Vladimir Murzin wrote:
> On 05/09/16 12:29, Christoffer Dall wrote:
> > On Tue, Aug 16, 2016 at 11:46:56AM +0100, Vladimir Murzin wrote:
> >> Macro __ACCESS_CP15{_64} is defined in two headers (arch_gicv3.h and
> >> kvm_hyp.h) which are going to be requested by vgic-v3 altogether.
> >> GCC would not like it because it'd see that macro is redefined and (hey!)
> >> they are different.  So, let's put only single macro version under common
> >> place and use it everywhere.
> > 
> > I'm sorry, but I don't understand this commit text.
> > 
> 
> Sorry for that :(
> 
> The issue the patch is trying to solve happens because
> virt/kvm/arm/hyp/vgic-v3-sr.c has
> 
> #include <linux/irqchip/arm-gic-v3.h>
> ...
> #include <asm/kvm_hyp.h>
> 
> each of these headers defines it's own __ACCESS_CP15 macro:
> 
> asm/kvm_hyp.h
> 
> #define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
> 	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
> 
> and linux/irqchip/arm-gic-v3.h via asm/arch_gicv3.h
> 
> #define __ACCESS_CP15(CRn, Op1, CRm, Op2) p15, Op1, %0, CRn, CRm, Op2
> 
> When these headers are used together conflict happens. The same applies
> to __ACCESS_CP15_64 macro.
> 
> To address that only single set of macros is used and call sites updated.

Thanks for the explanation!

So a simpler way, IMHO, to explain this is to simply say that both
linux/irqchip/arm-gic.v3.h and arch/arm/include/asm/kvm_hyp.h define a
macro called __ACCESS_CP15, which obviously creates a conflict.

Then you could explain if these are just different implementations of
the same thing, or if they are semantically different, and finally how
your patch solves this problem.

> 
> >>
> >> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> >> ---
> >>  arch/arm/include/asm/arch_gicv3.h |   27 +++++++++++----------------
> >>  arch/arm/include/asm/cp15.h       |   15 +++++++++++++++
> >>  arch/arm/include/asm/kvm_hyp.h    |   15 +--------------
> >>  3 files changed, 27 insertions(+), 30 deletions(-)
> >>
> >> diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
> >> index e08d151..af25c32 100644
> >> --- a/arch/arm/include/asm/arch_gicv3.h
> >> +++ b/arch/arm/include/asm/arch_gicv3.h
> >> @@ -22,9 +22,7 @@
> >>  
> >>  #include <linux/io.h>
> >>  #include <asm/barrier.h>
> >> -
> >> -#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
> >> -#define __ACCESS_CP15_64(Op1, CRm)		p15, Op1, %Q0, %R0, CRm
> >> +#include <asm/cp15.h>
> >>  
> >>  #define ICC_EOIR1			__ACCESS_CP15(c12, 0, c12, 1)
> >>  #define ICC_DIR				__ACCESS_CP15(c12, 0, c11, 1)
> >> @@ -102,58 +100,55 @@
> >>  
> >>  static inline void gic_write_eoir(u32 irq)
> >>  {
> >> -	asm volatile("mcr " __stringify(ICC_EOIR1) : : "r" (irq));
> >> +	write_sysreg(irq, ICC_EOIR1);
> >>  	isb();
> >>  }
> >>  
> >>  static inline void gic_write_dir(u32 val)
> >>  {
> >> -	asm volatile("mcr " __stringify(ICC_DIR) : : "r" (val));
> >> +	write_sysreg(val, ICC_DIR);
> >>  	isb();
> >>  }
> >>  
> >>  static inline u32 gic_read_iar(void)
> >>  {
> >> -	u32 irqstat;
> >> +	u32 irqstat = read_sysreg(ICC_IAR1);
> >>  
> >> -	asm volatile("mrc " __stringify(ICC_IAR1) : "=r" (irqstat));
> >>  	dsb(sy);
> >> +
> >>  	return irqstat;
> >>  }
> >>  
> >>  static inline void gic_write_pmr(u32 val)
> >>  {
> >> -	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
> >> +	write_sysreg(val, ICC_PMR);
> >>  }
> >>  
> >>  static inline void gic_write_ctlr(u32 val)
> >>  {
> >> -	asm volatile("mcr " __stringify(ICC_CTLR) : : "r" (val));
> >> +	write_sysreg(val, ICC_CTLR);
> >>  	isb();
> >>  }
> >>  
> >>  static inline void gic_write_grpen1(u32 val)
> >>  {
> >> -	asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
> >> +	write_sysreg(val, ICC_IGRPEN1);
> >>  	isb();
> >>  }
> >>  
> >>  static inline void gic_write_sgi1r(u64 val)
> >>  {
> >> -	asm volatile("mcrr " __stringify(ICC_SGI1R) : : "r" (val));
> >> +	write_sysreg(val, ICC_SGI1R);
> >>  }
> >>  
> >>  static inline u32 gic_read_sre(void)
> >>  {
> >> -	u32 val;
> >> -
> >> -	asm volatile("mrc " __stringify(ICC_SRE) : "=r" (val));
> >> -	return val;
> >> +	return read_sysreg(ICC_SRE);
> >>  }
> >>  
> >>  static inline void gic_write_sre(u32 val)
> >>  {
> >> -	asm volatile("mcr " __stringify(ICC_SRE) : : "r" (val));
> >> +	write_sysreg(val, ICC_SRE);
> >>  	isb();
> >>  }
> >>  
> >> diff --git a/arch/arm/include/asm/cp15.h b/arch/arm/include/asm/cp15.h
> >> index c3f1152..f661732 100644
> >> --- a/arch/arm/include/asm/cp15.h
> >> +++ b/arch/arm/include/asm/cp15.h
> >> @@ -47,6 +47,21 @@
> >>  #define vectors_high()	(0)
> >>  #endif
> >>  
> >> +#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	\
> >> +	"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
> >> +#define __ACCESS_CP15_64(Op1, CRm)		\
> >> +	"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
> >> +
> >> +#define __read_sysreg(r, w, c, t) ({				\
> >> +	t __val;						\
> >> +	asm volatile(r " " c : "=r" (__val));			\
> >> +	__val;							\
> >> +})
> >> +#define read_sysreg(...)		__read_sysreg(__VA_ARGS__)
> >> +
> >> +#define __write_sysreg(v, r, w, c, t)	asm volatile(w " " c : : "r" ((t)(v)))
> >> +#define write_sysreg(v, ...)		__write_sysreg(v, __VA_ARGS__)
> >> +
> > 
> > I feel a bit strange about adding this sort of stuff in a
> > non-kvm-non-gic-specific ARM header file, without it being used (or
> > planned to be used) in a broader sense.
> > 
> > Is there not a way to keep the required changes local to KVM and the
> > gic?
> > 
> 
> We could add prefixes to KVM and GIC version of macros so they won't
> clash, but it'd introduce code duplication.
> We could keep macro in, say, GIC header and include it in KVM one (or
> vice versa), but such dependency would not look nicer, IMO.
> 
> The way arm64 handles this is via sysreg.h and the closest counterpart
> under arch/arm is cp15.h
> 
> I'm open to suggestions.
> 

Hmm, I guess you're right, your approach does make sense.

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 6/7] ARM: KVM: Get ready to use vgic-v3
  2016-09-06 13:12       ` Vladimir Murzin
@ 2016-09-06 16:49         ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-06 16:49 UTC (permalink / raw)
  To: Vladimir Murzin; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

On Tue, Sep 06, 2016 at 02:12:39PM +0100, Vladimir Murzin wrote:
> On 05/09/16 12:29, Christoffer Dall wrote:
> > On Tue, Aug 16, 2016 at 11:46:57AM +0100, Vladimir Murzin wrote:
> >> We need to take care we have everything vgic-v3 expects from us before
> >> a quantum leap:
> >> - provide required macros via uapi.h
> >> - handle access to GICv3 cpu interface from the guest
> >> - provide mapping between arm64 version of GICv3 cpu registers and arm's
> >>
> >> The later is handled via redirection of read{write}_gicreg() and
> > 
> > 'The latter'
> > 
> 
> Fixed.
> 
> >> required mainly because 64-bit wide ICH_LR is split in two 32-bit
> >> halves (ICH_LR and ICH_LRC) accessed independently.
> >>
> >> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> >> ---
> >>  arch/arm/include/asm/arch_gicv3.h |   64 +++++++++++++++++++++++++++++++++++++
> >>  arch/arm/include/asm/kvm_asm.h    |    3 ++
> >>  arch/arm/include/uapi/asm/kvm.h   |    7 ++++
> >>  arch/arm/kvm/coproc.c             |   36 +++++++++++++++++++++
> >>  4 files changed, 110 insertions(+)
> >>
> >> diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
> >> index af25c32..f93f6bd 100644
> >> --- a/arch/arm/include/asm/arch_gicv3.h
> >> +++ b/arch/arm/include/asm/arch_gicv3.h
> >> @@ -96,6 +96,70 @@
> >>  #define ICH_AP1R2			__AP1Rx(2)
> >>  #define ICH_AP1R3			__AP1Rx(3)
> >>  
> >> +/* A32-to-A64 mappings used by VGIC save/restore */
> >> +
> >> +#define CPUIF_MAP(a32, a64)			\
> >> +static inline void write_ ## a64(u32 val)	\
> >> +{						\
> >> +	write_sysreg(val, a32);			\
> >> +}						\
> >> +static inline u32 read_ ## a64(void)		\
> >> +{						\
> >> +	return read_sysreg(a32); 		\
> >> +}						\
> >> +
> >> +#define CPUIF_MAP_LO_HI(a32lo, a32hi, a64)	\
> >> +static inline void write_ ## a64(u64 val)	\
> >> +{						\
> >> +	write_sysreg((u32)val, a32lo);		\
> >> +	write_sysreg((u32)(val >> 32), a32hi);	\
> >> +}						\
> >> +static inline u64 read_ ## a64(void)		\
> >> +{						\
> >> +	u64 val = read_sysreg(a32lo);		\
> >> +						\
> >> +	val |=	(u64)read_sysreg(a32hi) << 32;	\
> >> +						\
> >> +	return val; 				\
> >> +}
> > 
> > I really don't like that we're defining new functions using a macro.
> 
> I can expand it manually if think it'd look better. I can also suggest
> to prefix generated functions with underscore.
> 

I understand the desire to avoid manually writing all these static
functions.

> > 
> > Why can't you just do whatever series of actions and/or type
> > conversions using familiar tricks such as 'do { foo; } while (0);'
> > and so on ?
> 
> Probably, because I don't how to apply these tricks here to keep
> virt/kvm/arm/hyp/vgic-v3-sr.c unchanged. If you have something
> particular in mind, please, suggest!
> 

Hmmm, no I see now what you're doing, by defining all those static
inlines, you're effectively creating a mapping with a function for each
register mapping the 32-bit to the 64-bit accessors.

I guess this makes sense and that is probably why you call this _MAP.

I wish there were a good/better/cleaner way to do this, but I can live
with it.

Thanks,
-Christoffer


> > 
> >> +
> >> +CPUIF_MAP(ICH_HCR, ICH_HCR_EL2)
> >> +CPUIF_MAP(ICH_VTR, ICH_VTR_EL2)
> >> +CPUIF_MAP(ICH_MISR, ICH_MISR_EL2)
> >> +CPUIF_MAP(ICH_EISR, ICH_EISR_EL2)
> >> +CPUIF_MAP(ICH_ELSR, ICH_ELSR_EL2)
> >> +CPUIF_MAP(ICH_VMCR, ICH_VMCR_EL2)
> >> +CPUIF_MAP(ICH_AP0R3, ICH_AP0R3_EL2)
> >> +CPUIF_MAP(ICH_AP0R2, ICH_AP0R2_EL2)
> >> +CPUIF_MAP(ICH_AP0R1, ICH_AP0R1_EL2)
> >> +CPUIF_MAP(ICH_AP0R0, ICH_AP0R0_EL2)
> >> +CPUIF_MAP(ICH_AP1R3, ICH_AP1R3_EL2)
> >> +CPUIF_MAP(ICH_AP1R2, ICH_AP1R2_EL2)
> >> +CPUIF_MAP(ICH_AP1R1, ICH_AP1R1_EL2)
> >> +CPUIF_MAP(ICH_AP1R0, ICH_AP1R0_EL2)
> >> +CPUIF_MAP(ICC_HSRE, ICC_SRE_EL2)
> >> +CPUIF_MAP(ICC_SRE, ICC_SRE_EL1)
> >> +
> >> +CPUIF_MAP_LO_HI(ICH_LR15, ICH_LRC15, ICH_LR15_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR14, ICH_LRC14, ICH_LR14_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR13, ICH_LRC13, ICH_LR13_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR12, ICH_LRC12, ICH_LR12_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR11, ICH_LRC11, ICH_LR11_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR10, ICH_LRC10, ICH_LR10_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR9, ICH_LRC9, ICH_LR9_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR8, ICH_LRC8, ICH_LR8_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR7, ICH_LRC7, ICH_LR7_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR6, ICH_LRC6, ICH_LR6_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR5, ICH_LRC5, ICH_LR5_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR4, ICH_LRC4, ICH_LR4_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR3, ICH_LRC3, ICH_LR3_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR2, ICH_LRC2, ICH_LR2_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR1, ICH_LRC1, ICH_LR1_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR0, ICH_LRC0, ICH_LR0_EL2)
> >> +
> >> +#define read_gicreg(r)                 read_##r()
> >> +#define write_gicreg(v, r)             write_##r(v)
> >> +
> >>  /* Low-level accessors */
> >>  
> >>  static inline void gic_write_eoir(u32 irq)
> >> diff --git a/arch/arm/include/asm/kvm_asm.h b/arch/arm/include/asm/kvm_asm.h
> >> index 58faff5..dfccf94 100644
> >> --- a/arch/arm/include/asm/kvm_asm.h
> >> +++ b/arch/arm/include/asm/kvm_asm.h
> >> @@ -68,6 +68,9 @@ extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
> >>  extern void __init_stage2_translation(void);
> >>  
> >>  extern void __kvm_hyp_reset(unsigned long);
> >> +
> >> +extern u64 __vgic_v3_get_ich_vtr_el2(void);
> >> +extern void __vgic_v3_init_lrs(void);
> >>  #endif
> >>  
> >>  #endif /* __ARM_KVM_ASM_H__ */
> >> diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h
> >> index a2b3eb3..b38c10c 100644
> >> --- a/arch/arm/include/uapi/asm/kvm.h
> >> +++ b/arch/arm/include/uapi/asm/kvm.h
> >> @@ -84,6 +84,13 @@ struct kvm_regs {
> >>  #define KVM_VGIC_V2_DIST_SIZE		0x1000
> >>  #define KVM_VGIC_V2_CPU_SIZE		0x2000
> >>  
> >> +/* Supported VGICv3 address types  */
> >> +#define KVM_VGIC_V3_ADDR_TYPE_DIST	2
> >> +#define KVM_VGIC_V3_ADDR_TYPE_REDIST	3
> >> +
> >> +#define KVM_VGIC_V3_DIST_SIZE		SZ_64K
> >> +#define KVM_VGIC_V3_REDIST_SIZE		(2 * SZ_64K)
> >> +
> >>  #define KVM_ARM_VCPU_POWER_OFF		0 /* CPU is started in OFF state */
> >>  #define KVM_ARM_VCPU_PSCI_0_2		1 /* CPU uses PSCI v0.2 */
> >>  
> >> diff --git a/arch/arm/kvm/coproc.c b/arch/arm/kvm/coproc.c
> >> index 1bb2b79..10c0244 100644
> >> --- a/arch/arm/kvm/coproc.c
> >> +++ b/arch/arm/kvm/coproc.c
> >> @@ -228,6 +228,36 @@ bool access_vm_reg(struct kvm_vcpu *vcpu,
> >>  	return true;
> >>  }
> >>  
> >> +static bool access_gic_sgi(struct kvm_vcpu *vcpu,
> >> +			   const struct coproc_params *p,
> >> +			   const struct coproc_reg *r)
> >> +{
> >> +	u64 reg;
> >> +
> >> +	if (!p->is_write)
> >> +		return read_from_write_only(vcpu, p);
> >> +
> >> +	reg = *vcpu_reg(vcpu, p->Rt2);
> >> +	reg <<= 32;
> >> +	reg |= *vcpu_reg(vcpu, p->Rt1) ;
> >> +
> >> +	vgic_v3_dispatch_sgi(vcpu, reg);
> >> +
> >> +	return true;
> >> +}
> >> +
> >> +static bool access_gic_sre(struct kvm_vcpu *vcpu,
> >> +			   const struct coproc_params *p,
> >> +			   const struct coproc_reg *r)
> >> +{
> >> +	if (p->is_write)
> >> +		return ignore_write(vcpu, p);
> >> +
> >> +	*vcpu_reg(vcpu, p->Rt1) = vcpu->arch.vgic_cpu.vgic_v3.vgic_sre;
> >> +
> >> +	return true;
> >> +}
> >> +
> >>  /*
> >>   * We could trap ID_DFR0 and tell the guest we don't support performance
> >>   * monitoring.  Unfortunately the patch to make the kernel check ID_DFR0 was
> >> @@ -361,10 +391,16 @@ static const struct coproc_reg cp15_regs[] = {
> >>  	{ CRn(10), CRm( 3), Op1( 0), Op2( 1), is32,
> >>  			access_vm_reg, reset_unknown, c10_AMAIR1},
> >>  
> >> +	/* ICC_SGI1R */
> >> +	{ CRm64(12), Op1( 0), is64, access_gic_sgi},
> >> +
> >>  	/* VBAR: swapped by interrupt.S. */
> >>  	{ CRn(12), CRm( 0), Op1( 0), Op2( 0), is32,
> >>  			NULL, reset_val, c12_VBAR, 0x00000000 },
> >>  
> >> +	/* ICC_SRE */
> >> +	{ CRn(12), CRm(12), Op1( 0), Op2(5), is32, access_gic_sre },
> >> +
> >>  	/* CONTEXTIDR/TPIDRURW/TPIDRURO/TPIDRPRW: swapped by interrupt.S. */
> >>  	{ CRn(13), CRm( 0), Op1( 0), Op2( 1), is32,
> >>  			access_vm_reg, reset_val, c13_CID, 0x00000000 },
> >> -- 
> >> 1.7.9.5
> >>
> > 
> > Otherwise this looks correct.
> 
> Thanks
> Vladimir
> 
> > 
> > Thanks,
> > -Christoffer
> > 
> > 
> 

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 6/7] ARM: KVM: Get ready to use vgic-v3
@ 2016-09-06 16:49         ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-06 16:49 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Sep 06, 2016 at 02:12:39PM +0100, Vladimir Murzin wrote:
> On 05/09/16 12:29, Christoffer Dall wrote:
> > On Tue, Aug 16, 2016 at 11:46:57AM +0100, Vladimir Murzin wrote:
> >> We need to take care we have everything vgic-v3 expects from us before
> >> a quantum leap:
> >> - provide required macros via uapi.h
> >> - handle access to GICv3 cpu interface from the guest
> >> - provide mapping between arm64 version of GICv3 cpu registers and arm's
> >>
> >> The later is handled via redirection of read{write}_gicreg() and
> > 
> > 'The latter'
> > 
> 
> Fixed.
> 
> >> required mainly because 64-bit wide ICH_LR is split in two 32-bit
> >> halves (ICH_LR and ICH_LRC) accessed independently.
> >>
> >> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> >> ---
> >>  arch/arm/include/asm/arch_gicv3.h |   64 +++++++++++++++++++++++++++++++++++++
> >>  arch/arm/include/asm/kvm_asm.h    |    3 ++
> >>  arch/arm/include/uapi/asm/kvm.h   |    7 ++++
> >>  arch/arm/kvm/coproc.c             |   36 +++++++++++++++++++++
> >>  4 files changed, 110 insertions(+)
> >>
> >> diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
> >> index af25c32..f93f6bd 100644
> >> --- a/arch/arm/include/asm/arch_gicv3.h
> >> +++ b/arch/arm/include/asm/arch_gicv3.h
> >> @@ -96,6 +96,70 @@
> >>  #define ICH_AP1R2			__AP1Rx(2)
> >>  #define ICH_AP1R3			__AP1Rx(3)
> >>  
> >> +/* A32-to-A64 mappings used by VGIC save/restore */
> >> +
> >> +#define CPUIF_MAP(a32, a64)			\
> >> +static inline void write_ ## a64(u32 val)	\
> >> +{						\
> >> +	write_sysreg(val, a32);			\
> >> +}						\
> >> +static inline u32 read_ ## a64(void)		\
> >> +{						\
> >> +	return read_sysreg(a32); 		\
> >> +}						\
> >> +
> >> +#define CPUIF_MAP_LO_HI(a32lo, a32hi, a64)	\
> >> +static inline void write_ ## a64(u64 val)	\
> >> +{						\
> >> +	write_sysreg((u32)val, a32lo);		\
> >> +	write_sysreg((u32)(val >> 32), a32hi);	\
> >> +}						\
> >> +static inline u64 read_ ## a64(void)		\
> >> +{						\
> >> +	u64 val = read_sysreg(a32lo);		\
> >> +						\
> >> +	val |=	(u64)read_sysreg(a32hi) << 32;	\
> >> +						\
> >> +	return val; 				\
> >> +}
> > 
> > I really don't like that we're defining new functions using a macro.
> 
> I can expand it manually if think it'd look better. I can also suggest
> to prefix generated functions with underscore.
> 

I understand the desire to avoid manually writing all these static
functions.

> > 
> > Why can't you just do whatever series of actions and/or type
> > conversions using familiar tricks such as 'do { foo; } while (0);'
> > and so on ?
> 
> Probably, because I don't how to apply these tricks here to keep
> virt/kvm/arm/hyp/vgic-v3-sr.c unchanged. If you have something
> particular in mind, please, suggest!
> 

Hmmm, no I see now what you're doing, by defining all those static
inlines, you're effectively creating a mapping with a function for each
register mapping the 32-bit to the 64-bit accessors.

I guess this makes sense and that is probably why you call this _MAP.

I wish there were a good/better/cleaner way to do this, but I can live
with it.

Thanks,
-Christoffer


> > 
> >> +
> >> +CPUIF_MAP(ICH_HCR, ICH_HCR_EL2)
> >> +CPUIF_MAP(ICH_VTR, ICH_VTR_EL2)
> >> +CPUIF_MAP(ICH_MISR, ICH_MISR_EL2)
> >> +CPUIF_MAP(ICH_EISR, ICH_EISR_EL2)
> >> +CPUIF_MAP(ICH_ELSR, ICH_ELSR_EL2)
> >> +CPUIF_MAP(ICH_VMCR, ICH_VMCR_EL2)
> >> +CPUIF_MAP(ICH_AP0R3, ICH_AP0R3_EL2)
> >> +CPUIF_MAP(ICH_AP0R2, ICH_AP0R2_EL2)
> >> +CPUIF_MAP(ICH_AP0R1, ICH_AP0R1_EL2)
> >> +CPUIF_MAP(ICH_AP0R0, ICH_AP0R0_EL2)
> >> +CPUIF_MAP(ICH_AP1R3, ICH_AP1R3_EL2)
> >> +CPUIF_MAP(ICH_AP1R2, ICH_AP1R2_EL2)
> >> +CPUIF_MAP(ICH_AP1R1, ICH_AP1R1_EL2)
> >> +CPUIF_MAP(ICH_AP1R0, ICH_AP1R0_EL2)
> >> +CPUIF_MAP(ICC_HSRE, ICC_SRE_EL2)
> >> +CPUIF_MAP(ICC_SRE, ICC_SRE_EL1)
> >> +
> >> +CPUIF_MAP_LO_HI(ICH_LR15, ICH_LRC15, ICH_LR15_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR14, ICH_LRC14, ICH_LR14_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR13, ICH_LRC13, ICH_LR13_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR12, ICH_LRC12, ICH_LR12_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR11, ICH_LRC11, ICH_LR11_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR10, ICH_LRC10, ICH_LR10_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR9, ICH_LRC9, ICH_LR9_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR8, ICH_LRC8, ICH_LR8_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR7, ICH_LRC7, ICH_LR7_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR6, ICH_LRC6, ICH_LR6_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR5, ICH_LRC5, ICH_LR5_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR4, ICH_LRC4, ICH_LR4_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR3, ICH_LRC3, ICH_LR3_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR2, ICH_LRC2, ICH_LR2_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR1, ICH_LRC1, ICH_LR1_EL2)
> >> +CPUIF_MAP_LO_HI(ICH_LR0, ICH_LRC0, ICH_LR0_EL2)
> >> +
> >> +#define read_gicreg(r)                 read_##r()
> >> +#define write_gicreg(v, r)             write_##r(v)
> >> +
> >>  /* Low-level accessors */
> >>  
> >>  static inline void gic_write_eoir(u32 irq)
> >> diff --git a/arch/arm/include/asm/kvm_asm.h b/arch/arm/include/asm/kvm_asm.h
> >> index 58faff5..dfccf94 100644
> >> --- a/arch/arm/include/asm/kvm_asm.h
> >> +++ b/arch/arm/include/asm/kvm_asm.h
> >> @@ -68,6 +68,9 @@ extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
> >>  extern void __init_stage2_translation(void);
> >>  
> >>  extern void __kvm_hyp_reset(unsigned long);
> >> +
> >> +extern u64 __vgic_v3_get_ich_vtr_el2(void);
> >> +extern void __vgic_v3_init_lrs(void);
> >>  #endif
> >>  
> >>  #endif /* __ARM_KVM_ASM_H__ */
> >> diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h
> >> index a2b3eb3..b38c10c 100644
> >> --- a/arch/arm/include/uapi/asm/kvm.h
> >> +++ b/arch/arm/include/uapi/asm/kvm.h
> >> @@ -84,6 +84,13 @@ struct kvm_regs {
> >>  #define KVM_VGIC_V2_DIST_SIZE		0x1000
> >>  #define KVM_VGIC_V2_CPU_SIZE		0x2000
> >>  
> >> +/* Supported VGICv3 address types  */
> >> +#define KVM_VGIC_V3_ADDR_TYPE_DIST	2
> >> +#define KVM_VGIC_V3_ADDR_TYPE_REDIST	3
> >> +
> >> +#define KVM_VGIC_V3_DIST_SIZE		SZ_64K
> >> +#define KVM_VGIC_V3_REDIST_SIZE		(2 * SZ_64K)
> >> +
> >>  #define KVM_ARM_VCPU_POWER_OFF		0 /* CPU is started in OFF state */
> >>  #define KVM_ARM_VCPU_PSCI_0_2		1 /* CPU uses PSCI v0.2 */
> >>  
> >> diff --git a/arch/arm/kvm/coproc.c b/arch/arm/kvm/coproc.c
> >> index 1bb2b79..10c0244 100644
> >> --- a/arch/arm/kvm/coproc.c
> >> +++ b/arch/arm/kvm/coproc.c
> >> @@ -228,6 +228,36 @@ bool access_vm_reg(struct kvm_vcpu *vcpu,
> >>  	return true;
> >>  }
> >>  
> >> +static bool access_gic_sgi(struct kvm_vcpu *vcpu,
> >> +			   const struct coproc_params *p,
> >> +			   const struct coproc_reg *r)
> >> +{
> >> +	u64 reg;
> >> +
> >> +	if (!p->is_write)
> >> +		return read_from_write_only(vcpu, p);
> >> +
> >> +	reg = *vcpu_reg(vcpu, p->Rt2);
> >> +	reg <<= 32;
> >> +	reg |= *vcpu_reg(vcpu, p->Rt1) ;
> >> +
> >> +	vgic_v3_dispatch_sgi(vcpu, reg);
> >> +
> >> +	return true;
> >> +}
> >> +
> >> +static bool access_gic_sre(struct kvm_vcpu *vcpu,
> >> +			   const struct coproc_params *p,
> >> +			   const struct coproc_reg *r)
> >> +{
> >> +	if (p->is_write)
> >> +		return ignore_write(vcpu, p);
> >> +
> >> +	*vcpu_reg(vcpu, p->Rt1) = vcpu->arch.vgic_cpu.vgic_v3.vgic_sre;
> >> +
> >> +	return true;
> >> +}
> >> +
> >>  /*
> >>   * We could trap ID_DFR0 and tell the guest we don't support performance
> >>   * monitoring.  Unfortunately the patch to make the kernel check ID_DFR0 was
> >> @@ -361,10 +391,16 @@ static const struct coproc_reg cp15_regs[] = {
> >>  	{ CRn(10), CRm( 3), Op1( 0), Op2( 1), is32,
> >>  			access_vm_reg, reset_unknown, c10_AMAIR1},
> >>  
> >> +	/* ICC_SGI1R */
> >> +	{ CRm64(12), Op1( 0), is64, access_gic_sgi},
> >> +
> >>  	/* VBAR: swapped by interrupt.S. */
> >>  	{ CRn(12), CRm( 0), Op1( 0), Op2( 0), is32,
> >>  			NULL, reset_val, c12_VBAR, 0x00000000 },
> >>  
> >> +	/* ICC_SRE */
> >> +	{ CRn(12), CRm(12), Op1( 0), Op2(5), is32, access_gic_sre },
> >> +
> >>  	/* CONTEXTIDR/TPIDRURW/TPIDRURO/TPIDRPRW: swapped by interrupt.S. */
> >>  	{ CRn(13), CRm( 0), Op1( 0), Op2( 1), is32,
> >>  			access_vm_reg, reset_val, c13_CID, 0x00000000 },
> >> -- 
> >> 1.7.9.5
> >>
> > 
> > Otherwise this looks correct.
> 
> Thanks
> Vladimir
> 
> > 
> > Thanks,
> > -Christoffer
> > 
> > 
> 

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 7/7] ARM: KVM: Unlock vgic-v3 support
  2016-09-06 13:18       ` Vladimir Murzin
@ 2016-09-06 16:52         ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-06 16:52 UTC (permalink / raw)
  To: Vladimir Murzin; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

On Tue, Sep 06, 2016 at 02:18:10PM +0100, Vladimir Murzin wrote:
> On 05/09/16 12:29, Christoffer Dall wrote:
> > On Tue, Aug 16, 2016 at 11:46:58AM +0100, Vladimir Murzin wrote:
> >> It is time to get access to common version of vgic-v3.
> > 
> > common version?
> > 
> 
> Since patch#2 it not private to arm64 or I should rephrase this?
> 

I think you should just use the full path of the file or say something
like:

Now when vgic-v3-sr.c can be shared between arm and arm64, we have to
blah blah blah...

> >>
> >> We basically would need to tell build system how to pick it up and
> >> undo KVM_ARM_VGIC_V3 guarding introduced in 4f64cb6 ("arm/arm64: KVM:
> >> Only allow 64bit hosts to build VGICv3") and remove stubs. However,
> >> since vgic got ITS support KVM_ARM_VGIC_V3 tend to protect a little
> >> bit more than just plain vgic-v3 - this guard is used for ITS too
> >> which is not supported in 32-bit world yet.  So, along with removal of
> >> KVM_ARM_VGIC_V3 guard introduce the new one - KVM_ARM_VGIC_V3_ITS to
> >> protect ITS related code.
> > 
> > I don't find this paragraph particularly helpful, I'm afraid.
> > 
> 
> Sorry for that. It seems to much for one patch, I'll split it per your
> suggestion bellow.
> 
> >>
> >> The only unpleasant part is how we decide which save/restore sequence to
> >> use under __vgic_save_state() - we don't have patching framework in hand
> >> like arm64, so have to check runtime on every invocation.
> > 
> > Do static keys work on 32-bit arm?
> > 
> 
> I'll try to use them.
> 
> >>
> >> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> >> ---
> >>  arch/arm/include/asm/kvm_host.h     |    4 +++
> >>  arch/arm/include/asm/kvm_hyp.h      |    5 ++++
> >>  arch/arm/kvm/Makefile               |    2 ++
> >>  arch/arm/kvm/hyp/Makefile           |    1 +
> >>  arch/arm/kvm/hyp/switch.c           |   20 +++++++++++--
> >>  arch/arm64/kvm/Kconfig              |    4 +--
> >>  include/kvm/arm_vgic.h              |    8 ------
> >>  virt/kvm/arm/vgic/vgic-kvm-device.c |   10 ++-----
> >>  virt/kvm/arm/vgic/vgic-mmio-v3.c    |    2 ++
> >>  virt/kvm/arm/vgic/vgic-mmio.c       |    2 --
> >>  virt/kvm/arm/vgic/vgic-mmio.h       |    3 --
> >>  virt/kvm/arm/vgic/vgic.h            |   54 ++---------------------------------
> >>  12 files changed, 38 insertions(+), 77 deletions(-)
> >>
> >> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> >> index de338d9..1312597 100644
> >> --- a/arch/arm/include/asm/kvm_host.h
> >> +++ b/arch/arm/include/asm/kvm_host.h
> >> @@ -39,7 +39,11 @@
> >>  
> >>  #include <kvm/arm_vgic.h>
> >>  
> >> +#ifdef CONFIG_ARM_GIC_V3
> >> +#define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS
> >> +#else
> >>  #define KVM_MAX_VCPUS VGIC_V2_MAX_CPUS
> >> +#endif
> >>  
> >>  #define KVM_REQ_VCPU_EXIT	8
> >>  
> >> diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
> >> index e604ad68..95669b3 100644
> >> --- a/arch/arm/include/asm/kvm_hyp.h
> >> +++ b/arch/arm/include/asm/kvm_hyp.h
> >> @@ -88,6 +88,8 @@
> >>  
> >>  #define VFP_FPEXC	__ACCESS_VFP(FPEXC)
> >>  
> >> +#define ID_PFR1		__ACCESS_CP15(c0, 0, c1, 1)
> >> +
> >>  /* AArch64 compatibility macros, only for the timer so far */
> >>  #define read_sysreg_el0(r)		read_sysreg(r##_el0)
> >>  #define write_sysreg_el0(v, r)		write_sysreg(v, r##_el0)
> >> @@ -103,6 +105,9 @@ void __timer_restore_state(struct kvm_vcpu *vcpu);
> >>  void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
> >>  void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
> >>  
> >> +void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
> >> +void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
> >> +
> >>  void __sysreg_save_state(struct kvm_cpu_context *ctxt);
> >>  void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
> >>  
> >> diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
> >> index 10d77a6..043d817f 100644
> >> --- a/arch/arm/kvm/Makefile
> >> +++ b/arch/arm/kvm/Makefile
> >> @@ -26,8 +26,10 @@ obj-y += $(KVM)/arm/vgic/vgic.o
> >>  obj-y += $(KVM)/arm/vgic/vgic-init.o
> >>  obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
> >>  obj-y += $(KVM)/arm/vgic/vgic-v2.o
> >> +obj-y += $(KVM)/arm/vgic/vgic-v3.o
> >>  obj-y += $(KVM)/arm/vgic/vgic-mmio.o
> >>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
> >> +obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
> >>  obj-y += $(KVM)/arm/vgic/vgic-kvm-device.o
> >>  obj-y += $(KVM)/irqchip.o
> >>  obj-y += $(KVM)/arm/arch_timer.o
> >> diff --git a/arch/arm/kvm/hyp/Makefile b/arch/arm/kvm/hyp/Makefile
> >> index 8dfa5f7..3023bb5 100644
> >> --- a/arch/arm/kvm/hyp/Makefile
> >> +++ b/arch/arm/kvm/hyp/Makefile
> >> @@ -5,6 +5,7 @@
> >>  KVM=../../../../virt/kvm
> >>  
> >>  obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
> >> +obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
> >>  obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
> >>  
> >>  obj-$(CONFIG_KVM_ARM_HOST) += tlb.o
> >> diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
> >> index b13caa9..9666bae 100644
> >> --- a/arch/arm/kvm/hyp/switch.c
> >> +++ b/arch/arm/kvm/hyp/switch.c
> >> @@ -15,6 +15,8 @@
> >>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> >>   */
> >>  
> >> +#include <linux/irqchip/arm-gic-v3.h>
> >> +
> >>  #include <asm/kvm_asm.h>
> >>  #include <asm/kvm_hyp.h>
> >>  
> >> @@ -74,14 +76,28 @@ static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
> >>  	write_sysreg(read_sysreg(MIDR), VPIDR);
> >>  }
> >>  
> >> +static bool __hyp_text __has_useable_gicv3_cpuif(void)
> >> +{
> >> +	if (IS_ENABLED(CONFIG_ARM_GIC_V3) && (read_sysreg(ID_PFR1) >> 28))
> > 
> > Do we have a define for bit 28 we could use?
> > 
> > Does this actually work on all v7 boards?  The v7 ARM ARM seems to state
> > that this bitfield is Reserved, UNK.  Does that somehow mean 'is going
> > to be zero'?
> > 

Any thoughts on this one?

> >> +		return !!(read_sysreg(ICC_HSRE) & ICC_SRE_EL2_SRE);
> >> +	else
> >> +		return false;
> >> +}
> >> +
> >>  static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
> >>  {
> >> -	__vgic_v2_save_state(vcpu);
> >> +	if (__has_useable_gicv3_cpuif())
> >> +		__vgic_v3_save_state(vcpu);
> >> +	else
> >> +		__vgic_v2_save_state(vcpu);
> >>  }
> >>  
> >>  static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
> >>  {
> >> -	__vgic_v2_restore_state(vcpu);
> >> +	if (__has_useable_gicv3_cpuif())
> >> +		__vgic_v3_restore_state(vcpu);
> >> +	else
> >> +		__vgic_v2_restore_state(vcpu);
> >>  }
> >>  
> >>  static bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu)
> >> diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
> >> index 9c9edc9..6eaf12c 100644
> >> --- a/arch/arm64/kvm/Kconfig
> >> +++ b/arch/arm64/kvm/Kconfig
> >> @@ -16,7 +16,7 @@ menuconfig VIRTUALIZATION
> >>  
> >>  if VIRTUALIZATION
> >>  
> >> -config KVM_ARM_VGIC_V3
> >> +config KVM_ARM_VGIC_V3_ITS
> > 
> > I feel like this could have been simplified with adding the new ITS
> > guard in a separate patch.
> > 
> > In fact, you could then have a separate patch that enables compilation
> > of the gicv3 code and removes the static inlines etc., and then finally
> > a patch that adds in the logic for the world switch.
> 
> Agree. Will rework this patch.
> 

Thanks, sounds good!

-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 7/7] ARM: KVM: Unlock vgic-v3 support
@ 2016-09-06 16:52         ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-06 16:52 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Sep 06, 2016 at 02:18:10PM +0100, Vladimir Murzin wrote:
> On 05/09/16 12:29, Christoffer Dall wrote:
> > On Tue, Aug 16, 2016 at 11:46:58AM +0100, Vladimir Murzin wrote:
> >> It is time to get access to common version of vgic-v3.
> > 
> > common version?
> > 
> 
> Since patch#2 it not private to arm64 or I should rephrase this?
> 

I think you should just use the full path of the file or say something
like:

Now when vgic-v3-sr.c can be shared between arm and arm64, we have to
blah blah blah...

> >>
> >> We basically would need to tell build system how to pick it up and
> >> undo KVM_ARM_VGIC_V3 guarding introduced in 4f64cb6 ("arm/arm64: KVM:
> >> Only allow 64bit hosts to build VGICv3") and remove stubs. However,
> >> since vgic got ITS support KVM_ARM_VGIC_V3 tend to protect a little
> >> bit more than just plain vgic-v3 - this guard is used for ITS too
> >> which is not supported in 32-bit world yet.  So, along with removal of
> >> KVM_ARM_VGIC_V3 guard introduce the new one - KVM_ARM_VGIC_V3_ITS to
> >> protect ITS related code.
> > 
> > I don't find this paragraph particularly helpful, I'm afraid.
> > 
> 
> Sorry for that. It seems to much for one patch, I'll split it per your
> suggestion bellow.
> 
> >>
> >> The only unpleasant part is how we decide which save/restore sequence to
> >> use under __vgic_save_state() - we don't have patching framework in hand
> >> like arm64, so have to check runtime on every invocation.
> > 
> > Do static keys work on 32-bit arm?
> > 
> 
> I'll try to use them.
> 
> >>
> >> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> >> ---
> >>  arch/arm/include/asm/kvm_host.h     |    4 +++
> >>  arch/arm/include/asm/kvm_hyp.h      |    5 ++++
> >>  arch/arm/kvm/Makefile               |    2 ++
> >>  arch/arm/kvm/hyp/Makefile           |    1 +
> >>  arch/arm/kvm/hyp/switch.c           |   20 +++++++++++--
> >>  arch/arm64/kvm/Kconfig              |    4 +--
> >>  include/kvm/arm_vgic.h              |    8 ------
> >>  virt/kvm/arm/vgic/vgic-kvm-device.c |   10 ++-----
> >>  virt/kvm/arm/vgic/vgic-mmio-v3.c    |    2 ++
> >>  virt/kvm/arm/vgic/vgic-mmio.c       |    2 --
> >>  virt/kvm/arm/vgic/vgic-mmio.h       |    3 --
> >>  virt/kvm/arm/vgic/vgic.h            |   54 ++---------------------------------
> >>  12 files changed, 38 insertions(+), 77 deletions(-)
> >>
> >> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> >> index de338d9..1312597 100644
> >> --- a/arch/arm/include/asm/kvm_host.h
> >> +++ b/arch/arm/include/asm/kvm_host.h
> >> @@ -39,7 +39,11 @@
> >>  
> >>  #include <kvm/arm_vgic.h>
> >>  
> >> +#ifdef CONFIG_ARM_GIC_V3
> >> +#define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS
> >> +#else
> >>  #define KVM_MAX_VCPUS VGIC_V2_MAX_CPUS
> >> +#endif
> >>  
> >>  #define KVM_REQ_VCPU_EXIT	8
> >>  
> >> diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
> >> index e604ad68..95669b3 100644
> >> --- a/arch/arm/include/asm/kvm_hyp.h
> >> +++ b/arch/arm/include/asm/kvm_hyp.h
> >> @@ -88,6 +88,8 @@
> >>  
> >>  #define VFP_FPEXC	__ACCESS_VFP(FPEXC)
> >>  
> >> +#define ID_PFR1		__ACCESS_CP15(c0, 0, c1, 1)
> >> +
> >>  /* AArch64 compatibility macros, only for the timer so far */
> >>  #define read_sysreg_el0(r)		read_sysreg(r##_el0)
> >>  #define write_sysreg_el0(v, r)		write_sysreg(v, r##_el0)
> >> @@ -103,6 +105,9 @@ void __timer_restore_state(struct kvm_vcpu *vcpu);
> >>  void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
> >>  void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
> >>  
> >> +void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
> >> +void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
> >> +
> >>  void __sysreg_save_state(struct kvm_cpu_context *ctxt);
> >>  void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
> >>  
> >> diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
> >> index 10d77a6..043d817f 100644
> >> --- a/arch/arm/kvm/Makefile
> >> +++ b/arch/arm/kvm/Makefile
> >> @@ -26,8 +26,10 @@ obj-y += $(KVM)/arm/vgic/vgic.o
> >>  obj-y += $(KVM)/arm/vgic/vgic-init.o
> >>  obj-y += $(KVM)/arm/vgic/vgic-irqfd.o
> >>  obj-y += $(KVM)/arm/vgic/vgic-v2.o
> >> +obj-y += $(KVM)/arm/vgic/vgic-v3.o
> >>  obj-y += $(KVM)/arm/vgic/vgic-mmio.o
> >>  obj-y += $(KVM)/arm/vgic/vgic-mmio-v2.o
> >> +obj-y += $(KVM)/arm/vgic/vgic-mmio-v3.o
> >>  obj-y += $(KVM)/arm/vgic/vgic-kvm-device.o
> >>  obj-y += $(KVM)/irqchip.o
> >>  obj-y += $(KVM)/arm/arch_timer.o
> >> diff --git a/arch/arm/kvm/hyp/Makefile b/arch/arm/kvm/hyp/Makefile
> >> index 8dfa5f7..3023bb5 100644
> >> --- a/arch/arm/kvm/hyp/Makefile
> >> +++ b/arch/arm/kvm/hyp/Makefile
> >> @@ -5,6 +5,7 @@
> >>  KVM=../../../../virt/kvm
> >>  
> >>  obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
> >> +obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
> >>  obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
> >>  
> >>  obj-$(CONFIG_KVM_ARM_HOST) += tlb.o
> >> diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
> >> index b13caa9..9666bae 100644
> >> --- a/arch/arm/kvm/hyp/switch.c
> >> +++ b/arch/arm/kvm/hyp/switch.c
> >> @@ -15,6 +15,8 @@
> >>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> >>   */
> >>  
> >> +#include <linux/irqchip/arm-gic-v3.h>
> >> +
> >>  #include <asm/kvm_asm.h>
> >>  #include <asm/kvm_hyp.h>
> >>  
> >> @@ -74,14 +76,28 @@ static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
> >>  	write_sysreg(read_sysreg(MIDR), VPIDR);
> >>  }
> >>  
> >> +static bool __hyp_text __has_useable_gicv3_cpuif(void)
> >> +{
> >> +	if (IS_ENABLED(CONFIG_ARM_GIC_V3) && (read_sysreg(ID_PFR1) >> 28))
> > 
> > Do we have a define for bit 28 we could use?
> > 
> > Does this actually work on all v7 boards?  The v7 ARM ARM seems to state
> > that this bitfield is Reserved, UNK.  Does that somehow mean 'is going
> > to be zero'?
> > 

Any thoughts on this one?

> >> +		return !!(read_sysreg(ICC_HSRE) & ICC_SRE_EL2_SRE);
> >> +	else
> >> +		return false;
> >> +}
> >> +
> >>  static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
> >>  {
> >> -	__vgic_v2_save_state(vcpu);
> >> +	if (__has_useable_gicv3_cpuif())
> >> +		__vgic_v3_save_state(vcpu);
> >> +	else
> >> +		__vgic_v2_save_state(vcpu);
> >>  }
> >>  
> >>  static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
> >>  {
> >> -	__vgic_v2_restore_state(vcpu);
> >> +	if (__has_useable_gicv3_cpuif())
> >> +		__vgic_v3_restore_state(vcpu);
> >> +	else
> >> +		__vgic_v2_restore_state(vcpu);
> >>  }
> >>  
> >>  static bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu)
> >> diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
> >> index 9c9edc9..6eaf12c 100644
> >> --- a/arch/arm64/kvm/Kconfig
> >> +++ b/arch/arm64/kvm/Kconfig
> >> @@ -16,7 +16,7 @@ menuconfig VIRTUALIZATION
> >>  
> >>  if VIRTUALIZATION
> >>  
> >> -config KVM_ARM_VGIC_V3
> >> +config KVM_ARM_VGIC_V3_ITS
> > 
> > I feel like this could have been simplified with adding the new ITS
> > guard in a separate patch.
> > 
> > In fact, you could then have a separate patch that enables compilation
> > of the gicv3 code and removes the static inlines etc., and then finally
> > a patch that adds in the logic for the world switch.
> 
> Agree. Will rework this patch.
> 

Thanks, sounds good!

-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 7/7] ARM: KVM: Unlock vgic-v3 support
  2016-09-06 13:23       ` Vladimir Murzin
@ 2016-09-06 16:55         ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-06 16:55 UTC (permalink / raw)
  To: Vladimir Murzin; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

On Tue, Sep 06, 2016 at 02:23:16PM +0100, Vladimir Murzin wrote:
> 
> Sorry, missed this one
> 
> On 05/09/16 12:29, Christoffer Dall wrote:
> >>  
> >> > +static bool __hyp_text __has_useable_gicv3_cpuif(void)
> >> > +{
> >> > +	if (IS_ENABLED(CONFIG_ARM_GIC_V3) && (read_sysreg(ID_PFR1) >> 28))
> > Do we have a define for bit 28 we could use?
> 
> I'll check it.
> 
> > 
> > Does this actually work on all v7 boards?  The v7 ARM ARM seems to state
> > that this bitfield is Reserved, UNK.  Does that somehow mean 'is going
> > to be zero'?
> 
> It is how v7ARM ARM I have defines UNK
> 
> An abbreviation indicating that software must treat a field as
> containing an UNKNOWN value. Hardware must implement the bit as read as
> 0, or all 0s for a bit field. Software must not rely on the field
> reading as zero.
> 
> It seems goes under 'is going to be zero' case, no?
> 
The last sentence is disturbing to me, and feels slightly contradicting
itself.  Reading the UNKNOWN description doesn't help much either.

Perhaps you can ask around internally and figure out what the precise
answer to this is?

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 7/7] ARM: KVM: Unlock vgic-v3 support
@ 2016-09-06 16:55         ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-06 16:55 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Sep 06, 2016 at 02:23:16PM +0100, Vladimir Murzin wrote:
> 
> Sorry, missed this one
> 
> On 05/09/16 12:29, Christoffer Dall wrote:
> >>  
> >> > +static bool __hyp_text __has_useable_gicv3_cpuif(void)
> >> > +{
> >> > +	if (IS_ENABLED(CONFIG_ARM_GIC_V3) && (read_sysreg(ID_PFR1) >> 28))
> > Do we have a define for bit 28 we could use?
> 
> I'll check it.
> 
> > 
> > Does this actually work on all v7 boards?  The v7 ARM ARM seems to state
> > that this bitfield is Reserved, UNK.  Does that somehow mean 'is going
> > to be zero'?
> 
> It is how v7ARM ARM I have defines UNK
> 
> An abbreviation indicating that software must treat a field as
> containing an UNKNOWN value. Hardware must implement the bit as read as
> 0, or all 0s for a bit field. Software must not rely on the field
> reading as zero.
> 
> It seems goes under 'is going to be zero' case, no?
> 
The last sentence is disturbing to me, and feels slightly contradicting
itself.  Reading the UNKNOWN description doesn't help much either.

Perhaps you can ask around internally and figure out what the precise
answer to this is?

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 3/7] KVM: arm: vgic-new: improve compatibility with 32-bit
  2016-09-06 16:31             ` Christoffer Dall
@ 2016-09-07  9:06               ` Vladimir Murzin
  -1 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-09-07  9:06 UTC (permalink / raw)
  To: Christoffer Dall; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

On 06/09/16 17:31, Christoffer Dall wrote:
> On Tue, Sep 06, 2016 at 02:54:01PM +0100, Vladimir Murzin wrote:
>> On 06/09/16 14:22, Christoffer Dall wrote:
>>> On Tue, Sep 06, 2016 at 01:41:37PM +0100, Vladimir Murzin wrote:
>>>> Hi Christoffer,
>>>>
>>>> On 05/09/16 12:29, Christoffer Dall wrote:
>>>>> Hi Vladimir,
>>>>>
>>>>> I think commit title is too vague, can you be more specific?
>>>>>
>>>>
>>>> KVM: arm: vgic-new: make extract_bytes to always work on 64-bit data
>>>>
>>>> is it better?
>>>
>>> I would suggest:
>>>
>>> KVM: arm: vgic: Support 64-bit data manipulation on 32-bit host systems
>>>
>>>>
>>>>> On Tue, Aug 16, 2016 at 11:46:54AM +0100, Vladimir Murzin wrote:
>>>>>> We have couple of 64-bit register defined in GICv3 architecture, so
>>>>>
>>>>> 'a couple',  'registers' (plural)
>>>>>
>>>>>> "unsigned long" kind of accessors wouldn't work for 32-bit. However,
>>>>>
>>>>> 'wouldn't work for 32-bit' is kind of generic as well.  Perhaps you mean
>>>>> that unsigned long accesses to these registers will only access a single
>>>>> 32-bit work of that register.
>>>>>
>>>>>> these registers can't be access as 64-bit in a one go if we run 32-bit
>>>>>
>>>>> 'accessed', 's/in one go/with a single instruction/' ?
>>>>>
>>>>> 'a 32-bit host'
>>>>>
>>>>>> host simply because KVM doesn't support multiple load/store on MMIO
>>>>>
>>>>> by 'multiple load/store' you mean the 'load/store multiple' instructions
>>>>> specifically, right?  Not a sequence of multiple loads and stores.  I
>>>>> think you should be more specific here as well, for example, I think
>>>>> ldrd and strd are problematic as well.
>>>>>
>>>>>> space.
>>>>>>
>>>>>> It means that 32-bit guest access these registers in 32-bit chunks, so
>>>>>
>>>>> 'a 32-bit guest', 'accesses'
>>>>>
>>>>
>>>> all suggestions you've made above are true. I'll rework commit message
>>>> to be more precise.
>>>>
>>>
>>> Thanks!
>>>
>>>>>> the only thing we need to do is to ensure that extract_bytes() always
>>>>>> takes 64-bit data.
>>>>>>
>>>>>> Since we are here fix couple of other width related issues by using
>>>>>> ULL variants over UL.
>>>>>>
>>>>>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>>>>>> ---
>>>>>>  virt/kvm/arm/vgic/vgic-mmio-v3.c |    6 +++---
>>>>>>  virt/kvm/arm/vgic/vgic-mmio.h    |    2 +-
>>>>>>  2 files changed, 4 insertions(+), 4 deletions(-)
>>>>>>
>>>>>> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>>>>>> index ff668e0..cc20b60 100644
>>>>>> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
>>>>>> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>>>>>> @@ -23,7 +23,7 @@
>>>>>>  #include "vgic-mmio.h"
>>>>>>  
>>>>>>  /* extract @num bytes at @offset bytes offset in data */
>>>>>> -unsigned long extract_bytes(unsigned long data, unsigned int offset,
>>>>>> +unsigned long extract_bytes(u64 data, unsigned int offset,
>>>>>>  			    unsigned int num)
>>>>>>  {
>>>>>>  	return (data >> (offset * 8)) & GENMASK_ULL(num * 8 - 1, 0);
>>>>>> @@ -179,7 +179,7 @@ static unsigned long vgic_mmio_read_v3r_typer(struct kvm_vcpu *vcpu,
>>>>>>  	int target_vcpu_id = vcpu->vcpu_id;
>>>>>>  	u64 value;
>>>>>>  
>>>>>> -	value = (mpidr & GENMASK(23, 0)) << 32;
>>>>>> +	value = (mpidr & GENMASK_ULL(23, 0)) << 32;
>>>>>
>>>>> why does this make a difference when mpidr is an unsigned long?
>>>>
>>>> because we access a little bit further than unsigned long can accommodate
>>>>
>>>>   CC      arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.o
>>>> arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c: In function
>>>> 'vgic_mmio_read_v3r_typer':
>>>> arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c:184:35: warning:
>>>> left shift count >= width of type [-Wshift-count-overflow]
>>>>   value = (mpidr & GENMASK(23, 0)) << 32;
>>>>                                    ^
>>>>
>>>> I can include this warning in commit message or maybe you want a
>>>> separate patch?
>>>>
>>> My point was that the code doesn't really make sense when compiled on a
>>> 32-bit platform without also modifing the type for the mpidr variable.
>>> Am I missing something?
>>
>> I've not seen any difference in generated code, but for consistency I'll
>> update mpidr variable to u64.
>>
> 
> That could be because you need to update kvm_vcpu_get_mpidr_aff() to
> return a u64 as well.

I think we don't need to update the type of mpidr. mpidr fits in
"unsigned long" nicely and kvm_vcpu_get_mpidr_aff() applies
MPIDR_HWID_BITMASK mask anyway.

In my patch I just abused GENMASK_ULL() and the proper fix for warning
produced by gcc should be

+	value = (u64)(mpidr & GENMASK(23, 0)) << 32;


Cheers
Vladimir

> 
> -Christoffer
> 
> 

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 3/7] KVM: arm: vgic-new: improve compatibility with 32-bit
@ 2016-09-07  9:06               ` Vladimir Murzin
  0 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-09-07  9:06 UTC (permalink / raw)
  To: linux-arm-kernel

On 06/09/16 17:31, Christoffer Dall wrote:
> On Tue, Sep 06, 2016 at 02:54:01PM +0100, Vladimir Murzin wrote:
>> On 06/09/16 14:22, Christoffer Dall wrote:
>>> On Tue, Sep 06, 2016 at 01:41:37PM +0100, Vladimir Murzin wrote:
>>>> Hi Christoffer,
>>>>
>>>> On 05/09/16 12:29, Christoffer Dall wrote:
>>>>> Hi Vladimir,
>>>>>
>>>>> I think commit title is too vague, can you be more specific?
>>>>>
>>>>
>>>> KVM: arm: vgic-new: make extract_bytes to always work on 64-bit data
>>>>
>>>> is it better?
>>>
>>> I would suggest:
>>>
>>> KVM: arm: vgic: Support 64-bit data manipulation on 32-bit host systems
>>>
>>>>
>>>>> On Tue, Aug 16, 2016 at 11:46:54AM +0100, Vladimir Murzin wrote:
>>>>>> We have couple of 64-bit register defined in GICv3 architecture, so
>>>>>
>>>>> 'a couple',  'registers' (plural)
>>>>>
>>>>>> "unsigned long" kind of accessors wouldn't work for 32-bit. However,
>>>>>
>>>>> 'wouldn't work for 32-bit' is kind of generic as well.  Perhaps you mean
>>>>> that unsigned long accesses to these registers will only access a single
>>>>> 32-bit work of that register.
>>>>>
>>>>>> these registers can't be access as 64-bit in a one go if we run 32-bit
>>>>>
>>>>> 'accessed', 's/in one go/with a single instruction/' ?
>>>>>
>>>>> 'a 32-bit host'
>>>>>
>>>>>> host simply because KVM doesn't support multiple load/store on MMIO
>>>>>
>>>>> by 'multiple load/store' you mean the 'load/store multiple' instructions
>>>>> specifically, right?  Not a sequence of multiple loads and stores.  I
>>>>> think you should be more specific here as well, for example, I think
>>>>> ldrd and strd are problematic as well.
>>>>>
>>>>>> space.
>>>>>>
>>>>>> It means that 32-bit guest access these registers in 32-bit chunks, so
>>>>>
>>>>> 'a 32-bit guest', 'accesses'
>>>>>
>>>>
>>>> all suggestions you've made above are true. I'll rework commit message
>>>> to be more precise.
>>>>
>>>
>>> Thanks!
>>>
>>>>>> the only thing we need to do is to ensure that extract_bytes() always
>>>>>> takes 64-bit data.
>>>>>>
>>>>>> Since we are here fix couple of other width related issues by using
>>>>>> ULL variants over UL.
>>>>>>
>>>>>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
>>>>>> ---
>>>>>>  virt/kvm/arm/vgic/vgic-mmio-v3.c |    6 +++---
>>>>>>  virt/kvm/arm/vgic/vgic-mmio.h    |    2 +-
>>>>>>  2 files changed, 4 insertions(+), 4 deletions(-)
>>>>>>
>>>>>> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>>>>>> index ff668e0..cc20b60 100644
>>>>>> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
>>>>>> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>>>>>> @@ -23,7 +23,7 @@
>>>>>>  #include "vgic-mmio.h"
>>>>>>  
>>>>>>  /* extract @num bytes at @offset bytes offset in data */
>>>>>> -unsigned long extract_bytes(unsigned long data, unsigned int offset,
>>>>>> +unsigned long extract_bytes(u64 data, unsigned int offset,
>>>>>>  			    unsigned int num)
>>>>>>  {
>>>>>>  	return (data >> (offset * 8)) & GENMASK_ULL(num * 8 - 1, 0);
>>>>>> @@ -179,7 +179,7 @@ static unsigned long vgic_mmio_read_v3r_typer(struct kvm_vcpu *vcpu,
>>>>>>  	int target_vcpu_id = vcpu->vcpu_id;
>>>>>>  	u64 value;
>>>>>>  
>>>>>> -	value = (mpidr & GENMASK(23, 0)) << 32;
>>>>>> +	value = (mpidr & GENMASK_ULL(23, 0)) << 32;
>>>>>
>>>>> why does this make a difference when mpidr is an unsigned long?
>>>>
>>>> because we access a little bit further than unsigned long can accommodate
>>>>
>>>>   CC      arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.o
>>>> arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c: In function
>>>> 'vgic_mmio_read_v3r_typer':
>>>> arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c:184:35: warning:
>>>> left shift count >= width of type [-Wshift-count-overflow]
>>>>   value = (mpidr & GENMASK(23, 0)) << 32;
>>>>                                    ^
>>>>
>>>> I can include this warning in commit message or maybe you want a
>>>> separate patch?
>>>>
>>> My point was that the code doesn't really make sense when compiled on a
>>> 32-bit platform without also modifing the type for the mpidr variable.
>>> Am I missing something?
>>
>> I've not seen any difference in generated code, but for consistency I'll
>> update mpidr variable to u64.
>>
> 
> That could be because you need to update kvm_vcpu_get_mpidr_aff() to
> return a u64 as well.

I think we don't need to update the type of mpidr. mpidr fits in
"unsigned long" nicely and kvm_vcpu_get_mpidr_aff() applies
MPIDR_HWID_BITMASK mask anyway.

In my patch I just abused GENMASK_ULL() and the proper fix for warning
produced by gcc should be

+	value = (u64)(mpidr & GENMASK(23, 0)) << 32;


Cheers
Vladimir

> 
> -Christoffer
> 
> 

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 3/7] KVM: arm: vgic-new: improve compatibility with 32-bit
  2016-09-07  9:06               ` Vladimir Murzin
@ 2016-09-07  9:43                 ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-07  9:43 UTC (permalink / raw)
  To: Vladimir Murzin; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

On Wed, Sep 07, 2016 at 10:06:33AM +0100, Vladimir Murzin wrote:
> On 06/09/16 17:31, Christoffer Dall wrote:
> > On Tue, Sep 06, 2016 at 02:54:01PM +0100, Vladimir Murzin wrote:
> >> On 06/09/16 14:22, Christoffer Dall wrote:
> >>> On Tue, Sep 06, 2016 at 01:41:37PM +0100, Vladimir Murzin wrote:
> >>>> Hi Christoffer,
> >>>>
> >>>> On 05/09/16 12:29, Christoffer Dall wrote:
> >>>>> Hi Vladimir,
> >>>>>
> >>>>> I think commit title is too vague, can you be more specific?
> >>>>>
> >>>>
> >>>> KVM: arm: vgic-new: make extract_bytes to always work on 64-bit data
> >>>>
> >>>> is it better?
> >>>
> >>> I would suggest:
> >>>
> >>> KVM: arm: vgic: Support 64-bit data manipulation on 32-bit host systems
> >>>
> >>>>
> >>>>> On Tue, Aug 16, 2016 at 11:46:54AM +0100, Vladimir Murzin wrote:
> >>>>>> We have couple of 64-bit register defined in GICv3 architecture, so
> >>>>>
> >>>>> 'a couple',  'registers' (plural)
> >>>>>
> >>>>>> "unsigned long" kind of accessors wouldn't work for 32-bit. However,
> >>>>>
> >>>>> 'wouldn't work for 32-bit' is kind of generic as well.  Perhaps you mean
> >>>>> that unsigned long accesses to these registers will only access a single
> >>>>> 32-bit work of that register.
> >>>>>
> >>>>>> these registers can't be access as 64-bit in a one go if we run 32-bit
> >>>>>
> >>>>> 'accessed', 's/in one go/with a single instruction/' ?
> >>>>>
> >>>>> 'a 32-bit host'
> >>>>>
> >>>>>> host simply because KVM doesn't support multiple load/store on MMIO
> >>>>>
> >>>>> by 'multiple load/store' you mean the 'load/store multiple' instructions
> >>>>> specifically, right?  Not a sequence of multiple loads and stores.  I
> >>>>> think you should be more specific here as well, for example, I think
> >>>>> ldrd and strd are problematic as well.
> >>>>>
> >>>>>> space.
> >>>>>>
> >>>>>> It means that 32-bit guest access these registers in 32-bit chunks, so
> >>>>>
> >>>>> 'a 32-bit guest', 'accesses'
> >>>>>
> >>>>
> >>>> all suggestions you've made above are true. I'll rework commit message
> >>>> to be more precise.
> >>>>
> >>>
> >>> Thanks!
> >>>
> >>>>>> the only thing we need to do is to ensure that extract_bytes() always
> >>>>>> takes 64-bit data.
> >>>>>>
> >>>>>> Since we are here fix couple of other width related issues by using
> >>>>>> ULL variants over UL.
> >>>>>>
> >>>>>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> >>>>>> ---
> >>>>>>  virt/kvm/arm/vgic/vgic-mmio-v3.c |    6 +++---
> >>>>>>  virt/kvm/arm/vgic/vgic-mmio.h    |    2 +-
> >>>>>>  2 files changed, 4 insertions(+), 4 deletions(-)
> >>>>>>
> >>>>>> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >>>>>> index ff668e0..cc20b60 100644
> >>>>>> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >>>>>> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >>>>>> @@ -23,7 +23,7 @@
> >>>>>>  #include "vgic-mmio.h"
> >>>>>>  
> >>>>>>  /* extract @num bytes at @offset bytes offset in data */
> >>>>>> -unsigned long extract_bytes(unsigned long data, unsigned int offset,
> >>>>>> +unsigned long extract_bytes(u64 data, unsigned int offset,
> >>>>>>  			    unsigned int num)
> >>>>>>  {
> >>>>>>  	return (data >> (offset * 8)) & GENMASK_ULL(num * 8 - 1, 0);
> >>>>>> @@ -179,7 +179,7 @@ static unsigned long vgic_mmio_read_v3r_typer(struct kvm_vcpu *vcpu,
> >>>>>>  	int target_vcpu_id = vcpu->vcpu_id;
> >>>>>>  	u64 value;
> >>>>>>  
> >>>>>> -	value = (mpidr & GENMASK(23, 0)) << 32;
> >>>>>> +	value = (mpidr & GENMASK_ULL(23, 0)) << 32;
> >>>>>
> >>>>> why does this make a difference when mpidr is an unsigned long?
> >>>>
> >>>> because we access a little bit further than unsigned long can accommodate
> >>>>
> >>>>   CC      arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.o
> >>>> arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c: In function
> >>>> 'vgic_mmio_read_v3r_typer':
> >>>> arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c:184:35: warning:
> >>>> left shift count >= width of type [-Wshift-count-overflow]
> >>>>   value = (mpidr & GENMASK(23, 0)) << 32;
> >>>>                                    ^
> >>>>
> >>>> I can include this warning in commit message or maybe you want a
> >>>> separate patch?
> >>>>
> >>> My point was that the code doesn't really make sense when compiled on a
> >>> 32-bit platform without also modifing the type for the mpidr variable.
> >>> Am I missing something?
> >>
> >> I've not seen any difference in generated code, but for consistency I'll
> >> update mpidr variable to u64.
> >>
> > 
> > That could be because you need to update kvm_vcpu_get_mpidr_aff() to
> > return a u64 as well.
> 
> I think we don't need to update the type of mpidr. mpidr fits in
> "unsigned long" nicely and kvm_vcpu_get_mpidr_aff() applies
> MPIDR_HWID_BITMASK mask anyway.
> 
> In my patch I just abused GENMASK_ULL() and the proper fix for warning
> produced by gcc should be
> 
> +	value = (u64)(mpidr & GENMASK(23, 0)) << 32;
> 
> 
Ah, right, this is all the 32-bit include files I should be looking at,
so this makes sense.

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 3/7] KVM: arm: vgic-new: improve compatibility with 32-bit
@ 2016-09-07  9:43                 ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-07  9:43 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Sep 07, 2016 at 10:06:33AM +0100, Vladimir Murzin wrote:
> On 06/09/16 17:31, Christoffer Dall wrote:
> > On Tue, Sep 06, 2016 at 02:54:01PM +0100, Vladimir Murzin wrote:
> >> On 06/09/16 14:22, Christoffer Dall wrote:
> >>> On Tue, Sep 06, 2016 at 01:41:37PM +0100, Vladimir Murzin wrote:
> >>>> Hi Christoffer,
> >>>>
> >>>> On 05/09/16 12:29, Christoffer Dall wrote:
> >>>>> Hi Vladimir,
> >>>>>
> >>>>> I think commit title is too vague, can you be more specific?
> >>>>>
> >>>>
> >>>> KVM: arm: vgic-new: make extract_bytes to always work on 64-bit data
> >>>>
> >>>> is it better?
> >>>
> >>> I would suggest:
> >>>
> >>> KVM: arm: vgic: Support 64-bit data manipulation on 32-bit host systems
> >>>
> >>>>
> >>>>> On Tue, Aug 16, 2016 at 11:46:54AM +0100, Vladimir Murzin wrote:
> >>>>>> We have couple of 64-bit register defined in GICv3 architecture, so
> >>>>>
> >>>>> 'a couple',  'registers' (plural)
> >>>>>
> >>>>>> "unsigned long" kind of accessors wouldn't work for 32-bit. However,
> >>>>>
> >>>>> 'wouldn't work for 32-bit' is kind of generic as well.  Perhaps you mean
> >>>>> that unsigned long accesses to these registers will only access a single
> >>>>> 32-bit work of that register.
> >>>>>
> >>>>>> these registers can't be access as 64-bit in a one go if we run 32-bit
> >>>>>
> >>>>> 'accessed', 's/in one go/with a single instruction/' ?
> >>>>>
> >>>>> 'a 32-bit host'
> >>>>>
> >>>>>> host simply because KVM doesn't support multiple load/store on MMIO
> >>>>>
> >>>>> by 'multiple load/store' you mean the 'load/store multiple' instructions
> >>>>> specifically, right?  Not a sequence of multiple loads and stores.  I
> >>>>> think you should be more specific here as well, for example, I think
> >>>>> ldrd and strd are problematic as well.
> >>>>>
> >>>>>> space.
> >>>>>>
> >>>>>> It means that 32-bit guest access these registers in 32-bit chunks, so
> >>>>>
> >>>>> 'a 32-bit guest', 'accesses'
> >>>>>
> >>>>
> >>>> all suggestions you've made above are true. I'll rework commit message
> >>>> to be more precise.
> >>>>
> >>>
> >>> Thanks!
> >>>
> >>>>>> the only thing we need to do is to ensure that extract_bytes() always
> >>>>>> takes 64-bit data.
> >>>>>>
> >>>>>> Since we are here fix couple of other width related issues by using
> >>>>>> ULL variants over UL.
> >>>>>>
> >>>>>> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> >>>>>> ---
> >>>>>>  virt/kvm/arm/vgic/vgic-mmio-v3.c |    6 +++---
> >>>>>>  virt/kvm/arm/vgic/vgic-mmio.h    |    2 +-
> >>>>>>  2 files changed, 4 insertions(+), 4 deletions(-)
> >>>>>>
> >>>>>> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >>>>>> index ff668e0..cc20b60 100644
> >>>>>> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >>>>>> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >>>>>> @@ -23,7 +23,7 @@
> >>>>>>  #include "vgic-mmio.h"
> >>>>>>  
> >>>>>>  /* extract @num bytes at @offset bytes offset in data */
> >>>>>> -unsigned long extract_bytes(unsigned long data, unsigned int offset,
> >>>>>> +unsigned long extract_bytes(u64 data, unsigned int offset,
> >>>>>>  			    unsigned int num)
> >>>>>>  {
> >>>>>>  	return (data >> (offset * 8)) & GENMASK_ULL(num * 8 - 1, 0);
> >>>>>> @@ -179,7 +179,7 @@ static unsigned long vgic_mmio_read_v3r_typer(struct kvm_vcpu *vcpu,
> >>>>>>  	int target_vcpu_id = vcpu->vcpu_id;
> >>>>>>  	u64 value;
> >>>>>>  
> >>>>>> -	value = (mpidr & GENMASK(23, 0)) << 32;
> >>>>>> +	value = (mpidr & GENMASK_ULL(23, 0)) << 32;
> >>>>>
> >>>>> why does this make a difference when mpidr is an unsigned long?
> >>>>
> >>>> because we access a little bit further than unsigned long can accommodate
> >>>>
> >>>>   CC      arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.o
> >>>> arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c: In function
> >>>> 'vgic_mmio_read_v3r_typer':
> >>>> arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c:184:35: warning:
> >>>> left shift count >= width of type [-Wshift-count-overflow]
> >>>>   value = (mpidr & GENMASK(23, 0)) << 32;
> >>>>                                    ^
> >>>>
> >>>> I can include this warning in commit message or maybe you want a
> >>>> separate patch?
> >>>>
> >>> My point was that the code doesn't really make sense when compiled on a
> >>> 32-bit platform without also modifing the type for the mpidr variable.
> >>> Am I missing something?
> >>
> >> I've not seen any difference in generated code, but for consistency I'll
> >> update mpidr variable to u64.
> >>
> > 
> > That could be because you need to update kvm_vcpu_get_mpidr_aff() to
> > return a u64 as well.
> 
> I think we don't need to update the type of mpidr. mpidr fits in
> "unsigned long" nicely and kvm_vcpu_get_mpidr_aff() applies
> MPIDR_HWID_BITMASK mask anyway.
> 
> In my patch I just abused GENMASK_ULL() and the proper fix for warning
> produced by gcc should be
> 
> +	value = (u64)(mpidr & GENMASK(23, 0)) << 32;
> 
> 
Ah, right, this is all the 32-bit include files I should be looking at,
so this makes sense.

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 7/7] ARM: KVM: Unlock vgic-v3 support
  2016-09-06 16:55         ` Christoffer Dall
@ 2016-09-07 10:48           ` Vladimir Murzin
  -1 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-09-07 10:48 UTC (permalink / raw)
  To: Christoffer Dall; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

On 06/09/16 17:55, Christoffer Dall wrote:
> On Tue, Sep 06, 2016 at 02:23:16PM +0100, Vladimir Murzin wrote:
>>
>> Sorry, missed this one
>>
>> On 05/09/16 12:29, Christoffer Dall wrote:
>>>>  
>>>>> +static bool __hyp_text __has_useable_gicv3_cpuif(void)
>>>>> +{
>>>>> +	if (IS_ENABLED(CONFIG_ARM_GIC_V3) && (read_sysreg(ID_PFR1) >> 28))
>>> Do we have a define for bit 28 we could use?
>>
>> I'll check it.
>>
>>>
>>> Does this actually work on all v7 boards?  The v7 ARM ARM seems to state
>>> that this bitfield is Reserved, UNK.  Does that somehow mean 'is going
>>> to be zero'?
>>
>> It is how v7ARM ARM I have defines UNK
>>
>> An abbreviation indicating that software must treat a field as
>> containing an UNKNOWN value. Hardware must implement the bit as read as
>> 0, or all 0s for a bit field. Software must not rely on the field
>> reading as zero.
>>
>> It seems goes under 'is going to be zero' case, no?
>>
> The last sentence is disturbing to me, and feels slightly contradicting
> itself.  Reading the UNKNOWN description doesn't help much either.
> 
> Perhaps you can ask around internally and figure out what the precise
> answer to this is?

Since it is kind of implementation dependant thing the precise answer
from here hardly help, IMO. We still have non-zero chance to see
something scary.

OTOH, why do we care of all v7 boards if none of them have
CONFIG_ARM_GIC_V3 defined?

Cheers
Vladimir

> 
> Thanks,
> -Christoffer
> 
> 

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 7/7] ARM: KVM: Unlock vgic-v3 support
@ 2016-09-07 10:48           ` Vladimir Murzin
  0 siblings, 0 replies; 74+ messages in thread
From: Vladimir Murzin @ 2016-09-07 10:48 UTC (permalink / raw)
  To: linux-arm-kernel

On 06/09/16 17:55, Christoffer Dall wrote:
> On Tue, Sep 06, 2016 at 02:23:16PM +0100, Vladimir Murzin wrote:
>>
>> Sorry, missed this one
>>
>> On 05/09/16 12:29, Christoffer Dall wrote:
>>>>  
>>>>> +static bool __hyp_text __has_useable_gicv3_cpuif(void)
>>>>> +{
>>>>> +	if (IS_ENABLED(CONFIG_ARM_GIC_V3) && (read_sysreg(ID_PFR1) >> 28))
>>> Do we have a define for bit 28 we could use?
>>
>> I'll check it.
>>
>>>
>>> Does this actually work on all v7 boards?  The v7 ARM ARM seems to state
>>> that this bitfield is Reserved, UNK.  Does that somehow mean 'is going
>>> to be zero'?
>>
>> It is how v7ARM ARM I have defines UNK
>>
>> An abbreviation indicating that software must treat a field as
>> containing an UNKNOWN value. Hardware must implement the bit as read as
>> 0, or all 0s for a bit field. Software must not rely on the field
>> reading as zero.
>>
>> It seems goes under 'is going to be zero' case, no?
>>
> The last sentence is disturbing to me, and feels slightly contradicting
> itself.  Reading the UNKNOWN description doesn't help much either.
> 
> Perhaps you can ask around internally and figure out what the precise
> answer to this is?

Since it is kind of implementation dependant thing the precise answer
from here hardly help, IMO. We still have non-zero chance to see
something scary.

OTOH, why do we care of all v7 boards if none of them have
CONFIG_ARM_GIC_V3 defined?

Cheers
Vladimir

> 
> Thanks,
> -Christoffer
> 
> 

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 7/7] ARM: KVM: Unlock vgic-v3 support
  2016-09-07 10:48           ` Vladimir Murzin
@ 2016-09-07 12:58             ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-07 12:58 UTC (permalink / raw)
  To: Vladimir Murzin; +Cc: marc.zyngier, andre.przywara, kvmarm, linux-arm-kernel

On Wed, Sep 07, 2016 at 11:48:52AM +0100, Vladimir Murzin wrote:
> On 06/09/16 17:55, Christoffer Dall wrote:
> > On Tue, Sep 06, 2016 at 02:23:16PM +0100, Vladimir Murzin wrote:
> >>
> >> Sorry, missed this one
> >>
> >> On 05/09/16 12:29, Christoffer Dall wrote:
> >>>>  
> >>>>> +static bool __hyp_text __has_useable_gicv3_cpuif(void)
> >>>>> +{
> >>>>> +	if (IS_ENABLED(CONFIG_ARM_GIC_V3) && (read_sysreg(ID_PFR1) >> 28))
> >>> Do we have a define for bit 28 we could use?
> >>
> >> I'll check it.
> >>
> >>>
> >>> Does this actually work on all v7 boards?  The v7 ARM ARM seems to state
> >>> that this bitfield is Reserved, UNK.  Does that somehow mean 'is going
> >>> to be zero'?
> >>
> >> It is how v7ARM ARM I have defines UNK
> >>
> >> An abbreviation indicating that software must treat a field as
> >> containing an UNKNOWN value. Hardware must implement the bit as read as
> >> 0, or all 0s for a bit field. Software must not rely on the field
> >> reading as zero.
> >>
> >> It seems goes under 'is going to be zero' case, no?
> >>
> > The last sentence is disturbing to me, and feels slightly contradicting
> > itself.  Reading the UNKNOWN description doesn't help much either.
> > 
> > Perhaps you can ask around internally and figure out what the precise
> > answer to this is?
> 
> Since it is kind of implementation dependant thing the precise answer
> from here hardly help, IMO. We still have non-zero chance to see
> something scary.

Well, if the precise answer is: This will actually always return 0
because of X and Y, then your code is fine.

If the answer is:  It's implementation defined and may return something
bogus, then it's dodgy.  At the very least we should know that for all
boards with virt support out there, the read actually does return 0.

> 
> OTOH, why do we care of all v7 boards if none of them have
> CONFIG_ARM_GIC_V3 defined?
> 

I think it's pretty bad to have a multiarch config where running KVM
crashes your system, if this can be avoided.

If our way out is CONFIG_ARM_GIC_V3, then that config option should say
"May crash your system if you don't actually have a GICv3 and try to run
KVM on there".

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 7/7] ARM: KVM: Unlock vgic-v3 support
@ 2016-09-07 12:58             ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-07 12:58 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Sep 07, 2016 at 11:48:52AM +0100, Vladimir Murzin wrote:
> On 06/09/16 17:55, Christoffer Dall wrote:
> > On Tue, Sep 06, 2016 at 02:23:16PM +0100, Vladimir Murzin wrote:
> >>
> >> Sorry, missed this one
> >>
> >> On 05/09/16 12:29, Christoffer Dall wrote:
> >>>>  
> >>>>> +static bool __hyp_text __has_useable_gicv3_cpuif(void)
> >>>>> +{
> >>>>> +	if (IS_ENABLED(CONFIG_ARM_GIC_V3) && (read_sysreg(ID_PFR1) >> 28))
> >>> Do we have a define for bit 28 we could use?
> >>
> >> I'll check it.
> >>
> >>>
> >>> Does this actually work on all v7 boards?  The v7 ARM ARM seems to state
> >>> that this bitfield is Reserved, UNK.  Does that somehow mean 'is going
> >>> to be zero'?
> >>
> >> It is how v7ARM ARM I have defines UNK
> >>
> >> An abbreviation indicating that software must treat a field as
> >> containing an UNKNOWN value. Hardware must implement the bit as read as
> >> 0, or all 0s for a bit field. Software must not rely on the field
> >> reading as zero.
> >>
> >> It seems goes under 'is going to be zero' case, no?
> >>
> > The last sentence is disturbing to me, and feels slightly contradicting
> > itself.  Reading the UNKNOWN description doesn't help much either.
> > 
> > Perhaps you can ask around internally and figure out what the precise
> > answer to this is?
> 
> Since it is kind of implementation dependant thing the precise answer
> from here hardly help, IMO. We still have non-zero chance to see
> something scary.

Well, if the precise answer is: This will actually always return 0
because of X and Y, then your code is fine.

If the answer is:  It's implementation defined and may return something
bogus, then it's dodgy.  At the very least we should know that for all
boards with virt support out there, the read actually does return 0.

> 
> OTOH, why do we care of all v7 boards if none of them have
> CONFIG_ARM_GIC_V3 defined?
> 

I think it's pretty bad to have a multiarch config where running KVM
crashes your system, if this can be avoided.

If our way out is CONFIG_ARM_GIC_V3, then that config option should say
"May crash your system if you don't actually have a GICv3 and try to run
KVM on there".

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 7/7] ARM: KVM: Unlock vgic-v3 support
  2016-09-07 12:58             ` Christoffer Dall
@ 2016-09-07 14:20               ` Peter Maydell
  -1 siblings, 0 replies; 74+ messages in thread
From: Peter Maydell @ 2016-09-07 14:20 UTC (permalink / raw)
  To: Christoffer Dall; +Cc: Marc Zyngier, Andre Przywara, kvmarm, arm-mail-list

On 7 September 2016 at 13:58, Christoffer Dall
<christoffer.dall@linaro.org> wrote:
> On Wed, Sep 07, 2016 at 11:48:52AM +0100, Vladimir Murzin wrote:
>> On 06/09/16 17:55, Christoffer Dall wrote:
>> > On Tue, Sep 06, 2016 at 02:23:16PM +0100, Vladimir Murzin wrote:
>> >>
>> >> Sorry, missed this one
>> >>
>> >> On 05/09/16 12:29, Christoffer Dall wrote:
>> >>>>
>> >>>>> +static bool __hyp_text __has_useable_gicv3_cpuif(void)
>> >>>>> +{
>> >>>>> +       if (IS_ENABLED(CONFIG_ARM_GIC_V3) && (read_sysreg(ID_PFR1) >> 28))
>> >>> Do we have a define for bit 28 we could use?
>> >>
>> >> I'll check it.
>> >>
>> >>>
>> >>> Does this actually work on all v7 boards?  The v7 ARM ARM seems to state
>> >>> that this bitfield is Reserved, UNK.  Does that somehow mean 'is going
>> >>> to be zero'?
>> >>
>> >> It is how v7ARM ARM I have defines UNK
>> >>
>> >> An abbreviation indicating that software must treat a field as
>> >> containing an UNKNOWN value. Hardware must implement the bit as read as
>> >> 0, or all 0s for a bit field. Software must not rely on the field
>> >> reading as zero.
>> >>
>> >> It seems goes under 'is going to be zero' case, no?
>> >>
>> > The last sentence is disturbing to me, and feels slightly contradicting
>> > itself.  Reading the UNKNOWN description doesn't help much either.
>> >
>> > Perhaps you can ask around internally and figure out what the precise
>> > answer to this is?
>>
>> Since it is kind of implementation dependant thing the precise answer
>> from here hardly help, IMO. We still have non-zero chance to see
>> something scary.
>
> Well, if the precise answer is: This will actually always return 0
> because of X and Y, then your code is fine.

I think the "must not rely on the field reading as zero" wording
in the case of the ID registers is intended to mean "in a
future rev of the architecture we may assign these bits,
and your code mustn't do something that will break if the
bits then read as something other than zero". (And indeed
in v8 bits 28..31 have an assigned meaning.) It doesn't
mean there'll be v7 hardware out there with non-zero
values, because that would be breaking the hardware's part
of the UNKNOWN contract ("must implement the bit as read as 0").

thanks
-- PMM

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 7/7] ARM: KVM: Unlock vgic-v3 support
@ 2016-09-07 14:20               ` Peter Maydell
  0 siblings, 0 replies; 74+ messages in thread
From: Peter Maydell @ 2016-09-07 14:20 UTC (permalink / raw)
  To: linux-arm-kernel

On 7 September 2016 at 13:58, Christoffer Dall
<christoffer.dall@linaro.org> wrote:
> On Wed, Sep 07, 2016 at 11:48:52AM +0100, Vladimir Murzin wrote:
>> On 06/09/16 17:55, Christoffer Dall wrote:
>> > On Tue, Sep 06, 2016 at 02:23:16PM +0100, Vladimir Murzin wrote:
>> >>
>> >> Sorry, missed this one
>> >>
>> >> On 05/09/16 12:29, Christoffer Dall wrote:
>> >>>>
>> >>>>> +static bool __hyp_text __has_useable_gicv3_cpuif(void)
>> >>>>> +{
>> >>>>> +       if (IS_ENABLED(CONFIG_ARM_GIC_V3) && (read_sysreg(ID_PFR1) >> 28))
>> >>> Do we have a define for bit 28 we could use?
>> >>
>> >> I'll check it.
>> >>
>> >>>
>> >>> Does this actually work on all v7 boards?  The v7 ARM ARM seems to state
>> >>> that this bitfield is Reserved, UNK.  Does that somehow mean 'is going
>> >>> to be zero'?
>> >>
>> >> It is how v7ARM ARM I have defines UNK
>> >>
>> >> An abbreviation indicating that software must treat a field as
>> >> containing an UNKNOWN value. Hardware must implement the bit as read as
>> >> 0, or all 0s for a bit field. Software must not rely on the field
>> >> reading as zero.
>> >>
>> >> It seems goes under 'is going to be zero' case, no?
>> >>
>> > The last sentence is disturbing to me, and feels slightly contradicting
>> > itself.  Reading the UNKNOWN description doesn't help much either.
>> >
>> > Perhaps you can ask around internally and figure out what the precise
>> > answer to this is?
>>
>> Since it is kind of implementation dependant thing the precise answer
>> from here hardly help, IMO. We still have non-zero chance to see
>> something scary.
>
> Well, if the precise answer is: This will actually always return 0
> because of X and Y, then your code is fine.

I think the "must not rely on the field reading as zero" wording
in the case of the ID registers is intended to mean "in a
future rev of the architecture we may assign these bits,
and your code mustn't do something that will break if the
bits then read as something other than zero". (And indeed
in v8 bits 28..31 have an assigned meaning.) It doesn't
mean there'll be v7 hardware out there with non-zero
values, because that would be breaking the hardware's part
of the UNKNOWN contract ("must implement the bit as read as 0").

thanks
-- PMM

^ permalink raw reply	[flat|nested] 74+ messages in thread
* Re: [PATCH v2 7/7] ARM: KVM: Unlock vgic-v3 support
  2016-09-07 14:20               ` Peter Maydell
@ 2016-09-07 14:47                 ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-07 14:47 UTC (permalink / raw)
  To: Peter Maydell; +Cc: Marc Zyngier, Andre Przywara, kvmarm, arm-mail-list

On Wed, Sep 07, 2016 at 03:20:14PM +0100, Peter Maydell wrote:
> On 7 September 2016 at 13:58, Christoffer Dall
> <christoffer.dall@linaro.org> wrote:
> > On Wed, Sep 07, 2016 at 11:48:52AM +0100, Vladimir Murzin wrote:
> >> On 06/09/16 17:55, Christoffer Dall wrote:
> >> > On Tue, Sep 06, 2016 at 02:23:16PM +0100, Vladimir Murzin wrote:
> >> >>
> >> >> Sorry, missed this one
> >> >>
> >> >> On 05/09/16 12:29, Christoffer Dall wrote:
> >> >>>>
> >> >>>>> +static bool __hyp_text __has_useable_gicv3_cpuif(void)
> >> >>>>> +{
> >> >>>>> +       if (IS_ENABLED(CONFIG_ARM_GIC_V3) && (read_sysreg(ID_PFR1) >> 28))
> >> >>> Do we have a define for bit 28 we could use?
> >> >>
> >> >> I'll check it.
> >> >>
> >> >>>
> >> >>> Does this actually work on all v7 boards?  The v7 ARM ARM seems to state
> >> >>> that this bitfield is Reserved, UNK.  Does that somehow mean 'is going
> >> >>> to be zero'?
> >> >>
> >> >> It is how v7ARM ARM I have defines UNK
> >> >>
> >> >> An abbreviation indicating that software must treat a field as
> >> >> containing an UNKNOWN value. Hardware must implement the bit as read as
> >> >> 0, or all 0s for a bit field. Software must not rely on the field
> >> >> reading as zero.
> >> >>
> >> >> It seems goes under 'is going to be zero' case, no?
> >> >>
> >> > The last sentence is disturbing to me, and feels slightly contradicting
> >> > itself.  Reading the UNKNOWN description doesn't help much either.
> >> >
> >> > Perhaps you can ask around internally and figure out what the precise
> >> > answer to this is?
> >>
> >> Since it is kind of implementation dependant thing the precise answer
> >> from here hardly help, IMO. We still have non-zero chance to see
> >> something scary.
> >
> > Well, if the precise answer is: This will actually always return 0
> > because of X and Y, then your code is fine.
> 
> I think the "must not rely on the field reading as zero" wording
> in the case of the ID registers is intended to mean "in a
> future rev of the architecture we may assign these bits,
> and your code mustn't do something that will break if the
> bits then read as something other than zero". (And indeed
> in v8 bits 28..31 have an assigned meaning.) It doesn't
> mean there'll be v7 hardware out there with non-zero
> values, because that would be breaking the hardware's part
> of the UNKNOWN contract ("must implement the bit as read as 0").
> 
ok, that was the kind of answer I was hoping for.  In that case, this is
a non-issue.

Thanks for the clarification.

-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread
* [PATCH v2 7/7] ARM: KVM: Unlock vgic-v3 support
@ 2016-09-07 14:47                 ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2016-09-07 14:47 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Sep 07, 2016 at 03:20:14PM +0100, Peter Maydell wrote:
> On 7 September 2016 at 13:58, Christoffer Dall
> <christoffer.dall@linaro.org> wrote:
> > On Wed, Sep 07, 2016 at 11:48:52AM +0100, Vladimir Murzin wrote:
> >> On 06/09/16 17:55, Christoffer Dall wrote:
> >> > On Tue, Sep 06, 2016 at 02:23:16PM +0100, Vladimir Murzin wrote:
> >> >>
> >> >> Sorry, missed this one
> >> >>
> >> >> On 05/09/16 12:29, Christoffer Dall wrote:
> >> >>>>
> >> >>>>> +static bool __hyp_text __has_useable_gicv3_cpuif(void)
> >> >>>>> +{
> >> >>>>> +       if (IS_ENABLED(CONFIG_ARM_GIC_V3) && (read_sysreg(ID_PFR1) >> 28))
> >> >>> Do we have a define for bit 28 we could use?
> >> >>
> >> >> I'll check it.
> >> >>
> >> >>>
> >> >>> Does this actually work on all v7 boards?  The v7 ARM ARM seems to state
> >> >>> that this bitfield is Reserved, UNK.  Does that somehow mean 'is going
> >> >>> to be zero'?
> >> >>
> >> >> It is how v7ARM ARM I have defines UNK
> >> >>
> >> >> An abbreviation indicating that software must treat a field as
> >> >> containing an UNKNOWN value. Hardware must implement the bit as read as
> >> >> 0, or all 0s for a bit field. Software must not rely on the field
> >> >> reading as zero.
> >> >>
> >> >> It seems goes under 'is going to be zero' case, no?
> >> >>
> >> > The last sentence is disturbing to me, and feels slightly contradicting
> >> > itself.  Reading the UNKNOWN description doesn't help much either.
> >> >
> >> > Perhaps you can ask around internally and figure out what the precise
> >> > answer to this is?
> >>
> >> Since it is kind of implementation dependant thing the precise answer
> >> from here hardly help, IMO. We still have non-zero chance to see
> >> something scary.
> >
> > Well, if the precise answer is: This will actually always return 0
> > because of X and Y, then your code is fine.
> 
> I think the "must not rely on the field reading as zero" wording
> in the case of the ID registers is intended to mean "in a
> future rev of the architecture we may assign these bits,
> and your code mustn't do something that will break if the
> bits then read as something other than zero". (And indeed
> in v8 bits 28..31 have an assigned meaning.) It doesn't
> mean there'll be v7 hardware out there with non-zero
> values, because that would be breaking the hardware's part
> of the UNKNOWN contract ("must implement the bit as read as 0").
> 
ok, that was the kind of answer I was hoping for.  In that case, this is
a non-issue.

Thanks for the clarification.

-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread
end of thread, other threads:[~2016-09-07 14:47 UTC | newest]

Thread overview: 74+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-08-16 10:46 [PATCH v2 0/7] ARM: KVM: Support for vgic-v3 Vladimir Murzin
2016-08-16 10:46 ` Vladimir Murzin
2016-08-16 10:46 ` [PATCH v2 1/7] arm64: KVM: Move GIC accessors to arch_gicv3.h Vladimir Murzin
2016-08-16 10:46   ` Vladimir Murzin
2016-09-05 11:28   ` Christoffer Dall
2016-09-05 11:28     ` Christoffer Dall
2016-09-06 12:33     ` Vladimir Murzin
2016-09-06 12:33       ` Vladimir Murzin
2016-08-16 10:46 ` [PATCH v2 2/7] arm64: KVM: Move vgic-v3 save/restore to virt/kvm/arm/hyp Vladimir Murzin
2016-08-16 10:46   ` Vladimir Murzin
2016-08-16 10:46 ` [PATCH v2 3/7] KVM: arm: vgic-new: improve compatibility with 32-bit Vladimir Murzin
2016-08-16 10:46   ` Vladimir Murzin
2016-09-05 11:29   ` Christoffer Dall
2016-09-05 11:29     ` Christoffer Dall
2016-09-06 12:41     ` Vladimir Murzin
2016-09-06 12:41       ` Vladimir Murzin
2016-09-06 13:22       ` Christoffer Dall
2016-09-06 13:22         ` Christoffer Dall
2016-09-06 13:54         ` Vladimir Murzin
2016-09-06 13:54           ` Vladimir Murzin
2016-09-06 16:31           ` Christoffer Dall
2016-09-06 16:31             ` Christoffer Dall
2016-09-07  9:06             ` Vladimir Murzin
2016-09-07  9:06               ` Vladimir Murzin
2016-09-07  9:43               ` Christoffer Dall
2016-09-07  9:43                 ` Christoffer Dall
2016-08-16 10:46 ` [PATCH v2 4/7] ARM: update MPIDR accessors macro Vladimir Murzin
2016-08-16 10:46   ` Vladimir Murzin
2016-09-05 11:29   ` Christoffer Dall
2016-09-05 11:29     ` Christoffer Dall
2016-09-06 12:42     ` Vladimir Murzin
2016-09-06 12:42       ` Vladimir Murzin
2016-08-16 10:46 ` [PATCH v2 5/7] ARM: move system register accessors to asm/cp15.h Vladimir Murzin
2016-08-16 10:46   ` Vladimir Murzin
2016-09-05 11:29   ` Christoffer Dall
2016-09-05 11:29     ` Christoffer Dall
2016-09-06 13:05     ` Vladimir Murzin
2016-09-06 13:05       ` Vladimir Murzin
2016-09-06 16:34       ` Christoffer Dall
2016-09-06 16:34         ` Christoffer Dall
2016-08-16 10:46 ` [PATCH v2 6/7] ARM: KVM: Get ready to use vgic-v3 Vladimir Murzin
2016-08-16 10:46   ` Vladimir Murzin
2016-09-05 11:29   ` Christoffer Dall
2016-09-05 11:29     ` Christoffer Dall
2016-09-06 13:12     ` Vladimir Murzin
2016-09-06 13:12       ` Vladimir Murzin
2016-09-06 16:49       ` Christoffer Dall
2016-09-06 16:49         ` Christoffer Dall
2016-08-16 10:46 ` [PATCH v2 7/7] ARM: KVM: Unlock vgic-v3 support Vladimir Murzin
2016-08-16 10:46   ` Vladimir Murzin
2016-09-05 11:29   ` Christoffer Dall
2016-09-05 11:29     ` Christoffer Dall
2016-09-06 13:08     ` Marc Zyngier
2016-09-06 13:08       ` Marc Zyngier
2016-09-06 13:18     ` Vladimir Murzin
2016-09-06 13:18       ` Vladimir Murzin
2016-09-06 16:52       ` Christoffer Dall
2016-09-06 16:52         ` Christoffer Dall
2016-09-06 13:23     ` Vladimir Murzin
2016-09-06 13:23       ` Vladimir Murzin
2016-09-06 16:55       ` Christoffer Dall
2016-09-06 16:55         ` Christoffer Dall
2016-09-07 10:48         ` Vladimir Murzin
2016-09-07 10:48           ` Vladimir Murzin
2016-09-07 12:58           ` Christoffer Dall
2016-09-07 12:58             ` Christoffer Dall
2016-09-07 14:20             ` Peter Maydell
2016-09-07 14:20               ` Peter Maydell
2016-09-07 14:47               ` Christoffer Dall
2016-09-07 14:47                 ` Christoffer Dall
2016-09-05 11:28 ` [PATCH v2 0/7] ARM: KVM: Support for vgic-v3 Christoffer Dall
2016-09-05 11:28   ` Christoffer Dall
2016-09-06 12:32   ` Vladimir Murzin
2016-09-06 12:32     ` Vladimir Murzin

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.