All of lore.kernel.org
 help / color / mirror / Atom feed
* [Qemu-devel] [PATCH v2 00/11] KVM/ARM: virt-3.0: Multiple redistributor regions and 256MB ECAM region
@ 2018-06-15 14:28 Eric Auger
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 01/11] linux-headers: Update to 4.18-rc0 Eric Auger
                   ` (11 more replies)
  0 siblings, 12 replies; 23+ messages in thread
From: Eric Auger @ 2018-06-15 14:28 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell
  Cc: drjones, wei, zhaoshenglong, lersek, ard.biesheuvel,
	christoffer.dall, marc.zyngier

This series increases the number of vcpus usable in accelerated mode
along with GICv3 and allows up to 256 PCIe busses.

It is a combination of 2 series:
[1] [RFC v3 0/8] KVM/ARM: Relax the max 123 vcpus limitation along
    with KVM GICv3
[2] [PATCH 0/2] ARM virt: Support up to 256 PCIe buses

Both add features to the new 3.0 virt machine model.

VCPU changes:
-------------

Currently the max number of VCPUs usable along with the KVM GICv3
device is limited to 123. The rationale is a single redistributor
region was supported and this latter was set to [0x80A0000, 0x9000000]
within the guest physical address space, surrounded with DIST and UART
MMIO regions.

The 4.18 host kernel now allows to register several redistributor regions.
So this series overcomes the max 123 vcpu limitation by registering
a new redistributor region located just after the VIRT_MEM RAM region.
This second redistributor region has a capacity of 512 redistributors.

The max supported VCPUs in non accelerated mode is not modified.

PCIe BUS changes:
-----------------

Current Machvirt PCI host controller's ECAM region is 16MB large.
This limits the number of PCIe buses to 16.

PC/Q35 machines have a 256MB region allowing up to 256 buses.
This series tries to bridge the gap.

It declares a new ECAM region located beyond 256GB, of size 256MB
The new ECAM region is used if:
- highmem option is set (default) and,
- either FW is not loaded or we are run an aarch64 guest
- machine type >= 3.0.

aarch32 FW does not support this highmem ECAM region. For guests
without LPAE support the highmem option must be turned off.

Best Regards

Eric

This QEMU series can be found at:
https://github.com/eauger/qemu/tree/v2.12.0-virt3.0-v2
Previous version:
https://github.com/eauger/qemu/tree/v2.12.0-virt3.0-v1

History:
v1 -> v2:
- split virt machine model patch into 3 patches.
  Dared to keep Laszlo's and Drew's R-b.
- Took into account Drew's comments on v1 and collected
  his R-b.


Eric Auger (11):
  linux-headers: Update to 4.18-rc0
  target/arm: Allow KVM device address overwriting
  hw/intc/arm_gicv3: Introduce redist-region-count array property
  hw/intc/arm_gicv3_kvm: Get prepared to handle multiple redist regions
  hw/arm/virt: GICv3 DT node with one or two redistributor regions
  hw/arm/virt-acpi-build: Advertise one or two GICR structures
  hw/arm/virt: Register two redistributor regions when necessary
  hw/arm/virt: Add a new 256MB ECAM region
  hw/arm/virt: Add virt-3.0 machine type
  hw/arm/virt: Use 256MB ECAM region by default
  hw/arm/virt: Increase max_cpus to 512

 hw/arm/virt-acpi-build.c                           |  30 +++++--
 hw/arm/virt.c                                      | 100 +++++++++++++++++----
 hw/intc/arm_gic_kvm.c                              |   4 +-
 hw/intc/arm_gicv3.c                                |  12 ++-
 hw/intc/arm_gicv3_common.c                         |  38 ++++++--
 hw/intc/arm_gicv3_its_kvm.c                        |   2 +-
 hw/intc/arm_gicv3_kvm.c                            |  44 ++++++++-
 include/hw/arm/virt.h                              |  19 ++++
 include/hw/intc/arm_gicv3_common.h                 |   8 +-
 include/standard-headers/linux/pci_regs.h          |   8 ++
 include/standard-headers/linux/virtio_gpu.h        |   1 +
 include/standard-headers/linux/virtio_net.h        |   3 +
 .../LICENSES/exceptions/Linux-syscall-note         |   2 +-
 linux-headers/LICENSES/preferred/GPL-2.0           |   6 ++
 linux-headers/asm-arm/kvm.h                        |   1 +
 linux-headers/asm-arm/unistd-common.h              |   1 +
 linux-headers/asm-arm64/kvm.h                      |   1 +
 linux-headers/asm-generic/unistd.h                 |   4 +-
 linux-headers/asm-powerpc/unistd.h                 |   1 +
 linux-headers/asm-x86/unistd_32.h                  |   2 +
 linux-headers/asm-x86/unistd_64.h                  |   2 +
 linux-headers/asm-x86/unistd_x32.h                 |   2 +
 linux-headers/linux/kvm.h                          |   5 +-
 linux-headers/linux/psp-sev.h                      |  12 +++
 target/arm/kvm.c                                   |  10 ++-
 target/arm/kvm_arm.h                               |   3 +-
 26 files changed, 277 insertions(+), 44 deletions(-)

-- 
2.5.5

^ permalink raw reply	[flat|nested] 23+ messages in thread

* [Qemu-devel] [PATCH v2 01/11] linux-headers: Update to 4.18-rc0
  2018-06-15 14:28 [Qemu-devel] [PATCH v2 00/11] KVM/ARM: virt-3.0: Multiple redistributor regions and 256MB ECAM region Eric Auger
@ 2018-06-15 14:28 ` Eric Auger
  2018-06-20 13:57   ` Peter Maydell
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 02/11] target/arm: Allow KVM device address overwriting Eric Auger
                   ` (10 subsequent siblings)
  11 siblings, 1 reply; 23+ messages in thread
From: Eric Auger @ 2018-06-15 14:28 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell
  Cc: drjones, wei, zhaoshenglong, lersek, ard.biesheuvel,
	christoffer.dall, marc.zyngier

commit b357bf6023a948cf6a9472f07a1b0caac0e4f8e8
Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm

Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
 include/standard-headers/linux/pci_regs.h            |  8 ++++++++
 include/standard-headers/linux/virtio_gpu.h          |  1 +
 include/standard-headers/linux/virtio_net.h          |  3 +++
 linux-headers/LICENSES/exceptions/Linux-syscall-note |  2 +-
 linux-headers/LICENSES/preferred/GPL-2.0             |  6 ++++++
 linux-headers/asm-arm/kvm.h                          |  1 +
 linux-headers/asm-arm/unistd-common.h                |  1 +
 linux-headers/asm-arm64/kvm.h                        |  1 +
 linux-headers/asm-generic/unistd.h                   |  4 +++-
 linux-headers/asm-powerpc/unistd.h                   |  1 +
 linux-headers/asm-x86/unistd_32.h                    |  2 ++
 linux-headers/asm-x86/unistd_64.h                    |  2 ++
 linux-headers/asm-x86/unistd_x32.h                   |  2 ++
 linux-headers/linux/kvm.h                            |  5 +++--
 linux-headers/linux/psp-sev.h                        | 12 ++++++++++++
 15 files changed, 47 insertions(+), 4 deletions(-)

diff --git a/include/standard-headers/linux/pci_regs.h b/include/standard-headers/linux/pci_regs.h
index 103ba79..4da87e2 100644
--- a/include/standard-headers/linux/pci_regs.h
+++ b/include/standard-headers/linux/pci_regs.h
@@ -506,6 +506,8 @@
 #define  PCI_EXP_DEVCTL_READRQ_256B  0x1000 /* 256 Bytes */
 #define  PCI_EXP_DEVCTL_READRQ_512B  0x2000 /* 512 Bytes */
 #define  PCI_EXP_DEVCTL_READRQ_1024B 0x3000 /* 1024 Bytes */
+#define  PCI_EXP_DEVCTL_READRQ_2048B 0x4000 /* 2048 Bytes */
+#define  PCI_EXP_DEVCTL_READRQ_4096B 0x5000 /* 4096 Bytes */
 #define  PCI_EXP_DEVCTL_BCR_FLR 0x8000  /* Bridge Configuration Retry / FLR */
 #define PCI_EXP_DEVSTA		10	/* Device Status */
 #define  PCI_EXP_DEVSTA_CED	0x0001	/* Correctable Error Detected */
@@ -655,6 +657,11 @@
 #define  PCI_EXP_LNKCAP2_SLS_16_0GB	0x00000010 /* Supported Speed 16GT/s */
 #define  PCI_EXP_LNKCAP2_CROSSLINK	0x00000100 /* Crosslink supported */
 #define PCI_EXP_LNKCTL2		48	/* Link Control 2 */
+#define PCI_EXP_LNKCTL2_TLS		0x000f
+#define PCI_EXP_LNKCTL2_TLS_2_5GT	0x0001 /* Supported Speed 2.5GT/s */
+#define PCI_EXP_LNKCTL2_TLS_5_0GT	0x0002 /* Supported Speed 5GT/s */
+#define PCI_EXP_LNKCTL2_TLS_8_0GT	0x0003 /* Supported Speed 8GT/s */
+#define PCI_EXP_LNKCTL2_TLS_16_0GT	0x0004 /* Supported Speed 16GT/s */
 #define PCI_EXP_LNKSTA2		50	/* Link Status 2 */
 #define PCI_CAP_EXP_ENDPOINT_SIZEOF_V2	52	/* v2 endpoints with link end here */
 #define PCI_EXP_SLTCAP2		52	/* Slot Capabilities 2 */
@@ -981,6 +988,7 @@
 #define  PCI_EXP_DPC_CAP_DL_ACTIVE	0x1000	/* ERR_COR signal on DL_Active supported */
 
 #define PCI_EXP_DPC_CTL			6	/* DPC control */
+#define  PCI_EXP_DPC_CTL_EN_FATAL 	0x0001	/* Enable trigger on ERR_FATAL message */
 #define  PCI_EXP_DPC_CTL_EN_NONFATAL 	0x0002	/* Enable trigger on ERR_NONFATAL message */
 #define  PCI_EXP_DPC_CTL_INT_EN 	0x0008	/* DPC Interrupt Enable */
 
diff --git a/include/standard-headers/linux/virtio_gpu.h b/include/standard-headers/linux/virtio_gpu.h
index c1c8f07..52a830d 100644
--- a/include/standard-headers/linux/virtio_gpu.h
+++ b/include/standard-headers/linux/virtio_gpu.h
@@ -260,6 +260,7 @@ struct virtio_gpu_cmd_submit {
 };
 
 #define VIRTIO_GPU_CAPSET_VIRGL 1
+#define VIRTIO_GPU_CAPSET_VIRGL2 2
 
 /* VIRTIO_GPU_CMD_GET_CAPSET_INFO */
 struct virtio_gpu_get_capset_info {
diff --git a/include/standard-headers/linux/virtio_net.h b/include/standard-headers/linux/virtio_net.h
index e9f255e..260c368 100644
--- a/include/standard-headers/linux/virtio_net.h
+++ b/include/standard-headers/linux/virtio_net.h
@@ -57,6 +57,9 @@
 					 * Steering */
 #define VIRTIO_NET_F_CTRL_MAC_ADDR 23	/* Set MAC address */
 
+#define VIRTIO_NET_F_STANDBY	  62	/* Act as standby for another device
+					 * with the same MAC.
+					 */
 #define VIRTIO_NET_F_SPEED_DUPLEX 63	/* Device set linkspeed and duplex */
 
 #ifndef VIRTIO_NET_NO_LEGACY
diff --git a/linux-headers/LICENSES/exceptions/Linux-syscall-note b/linux-headers/LICENSES/exceptions/Linux-syscall-note
index 6b60b61..9abdad7 100644
--- a/linux-headers/LICENSES/exceptions/Linux-syscall-note
+++ b/linux-headers/LICENSES/exceptions/Linux-syscall-note
@@ -1,6 +1,6 @@
 SPDX-Exception-Identifier: Linux-syscall-note
 SPDX-URL: https://spdx.org/licenses/Linux-syscall-note.html
-SPDX-Licenses: GPL-2.0, GPL-2.0+, GPL-1.0+, LGPL-2.0, LGPL-2.0+, LGPL-2.1, LGPL-2.1+
+SPDX-Licenses: GPL-2.0, GPL-2.0+, GPL-1.0+, LGPL-2.0, LGPL-2.0+, LGPL-2.1, LGPL-2.1+, GPL-2.0-only, GPL-2.0-or-later
 Usage-Guide:
   This exception is used together with one of the above SPDX-Licenses
   to mark user space API (uapi) header files so they can be included
diff --git a/linux-headers/LICENSES/preferred/GPL-2.0 b/linux-headers/LICENSES/preferred/GPL-2.0
index b8db91d..ff0812f 100644
--- a/linux-headers/LICENSES/preferred/GPL-2.0
+++ b/linux-headers/LICENSES/preferred/GPL-2.0
@@ -1,5 +1,7 @@
 Valid-License-Identifier: GPL-2.0
+Valid-License-Identifier: GPL-2.0-only
 Valid-License-Identifier: GPL-2.0+
+Valid-License-Identifier: GPL-2.0-or-later
 SPDX-URL: https://spdx.org/licenses/GPL-2.0.html
 Usage-Guide:
   To use this license in source code, put one of the following SPDX
@@ -7,8 +9,12 @@ Usage-Guide:
   guidelines in the licensing rules documentation.
   For 'GNU General Public License (GPL) version 2 only' use:
     SPDX-License-Identifier: GPL-2.0
+  or
+    SPDX-License-Identifier: GPL-2.0-only
   For 'GNU General Public License (GPL) version 2 or any later version' use:
     SPDX-License-Identifier: GPL-2.0+
+  or
+    SPDX-License-Identifier: GPL-2.0-or-later
 License-Text:
 
 		    GNU GENERAL PUBLIC LICENSE
diff --git a/linux-headers/asm-arm/kvm.h b/linux-headers/asm-arm/kvm.h
index 670b43c..72aa226 100644
--- a/linux-headers/asm-arm/kvm.h
+++ b/linux-headers/asm-arm/kvm.h
@@ -91,6 +91,7 @@ struct kvm_regs {
 #define KVM_VGIC_V3_ADDR_TYPE_DIST	2
 #define KVM_VGIC_V3_ADDR_TYPE_REDIST	3
 #define KVM_VGIC_ITS_ADDR_TYPE		4
+#define KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION	5
 
 #define KVM_VGIC_V3_DIST_SIZE		SZ_64K
 #define KVM_VGIC_V3_REDIST_SIZE		(2 * SZ_64K)
diff --git a/linux-headers/asm-arm/unistd-common.h b/linux-headers/asm-arm/unistd-common.h
index 8d5ceae..60c2d93 100644
--- a/linux-headers/asm-arm/unistd-common.h
+++ b/linux-headers/asm-arm/unistd-common.h
@@ -354,5 +354,6 @@
 #define __NR_pkey_alloc (__NR_SYSCALL_BASE + 395)
 #define __NR_pkey_free (__NR_SYSCALL_BASE + 396)
 #define __NR_statx (__NR_SYSCALL_BASE + 397)
+#define __NR_rseq (__NR_SYSCALL_BASE + 398)
 
 #endif /* _ASM_ARM_UNISTD_COMMON_H */
diff --git a/linux-headers/asm-arm64/kvm.h b/linux-headers/asm-arm64/kvm.h
index 17315ab..99cb9ad 100644
--- a/linux-headers/asm-arm64/kvm.h
+++ b/linux-headers/asm-arm64/kvm.h
@@ -91,6 +91,7 @@ struct kvm_regs {
 #define KVM_VGIC_V3_ADDR_TYPE_DIST	2
 #define KVM_VGIC_V3_ADDR_TYPE_REDIST	3
 #define KVM_VGIC_ITS_ADDR_TYPE		4
+#define KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION	5
 
 #define KVM_VGIC_V3_DIST_SIZE		SZ_64K
 #define KVM_VGIC_V3_REDIST_SIZE		(2 * SZ_64K)
diff --git a/linux-headers/asm-generic/unistd.h b/linux-headers/asm-generic/unistd.h
index 8bcb186..4299067 100644
--- a/linux-headers/asm-generic/unistd.h
+++ b/linux-headers/asm-generic/unistd.h
@@ -732,9 +732,11 @@ __SYSCALL(__NR_pkey_alloc,    sys_pkey_alloc)
 __SYSCALL(__NR_pkey_free,     sys_pkey_free)
 #define __NR_statx 291
 __SYSCALL(__NR_statx,     sys_statx)
+#define __NR_io_pgetevents 292
+__SC_COMP(__NR_io_pgetevents, sys_io_pgetevents, compat_sys_io_pgetevents)
 
 #undef __NR_syscalls
-#define __NR_syscalls 292
+#define __NR_syscalls 293
 
 /*
  * 32 bit systems traditionally used different
diff --git a/linux-headers/asm-powerpc/unistd.h b/linux-headers/asm-powerpc/unistd.h
index 0c08edc..3629858 100644
--- a/linux-headers/asm-powerpc/unistd.h
+++ b/linux-headers/asm-powerpc/unistd.h
@@ -398,5 +398,6 @@
 #define __NR_pkey_alloc		384
 #define __NR_pkey_free		385
 #define __NR_pkey_mprotect	386
+#define __NR_rseq		387
 
 #endif /* _ASM_POWERPC_UNISTD_H_ */
diff --git a/linux-headers/asm-x86/unistd_32.h b/linux-headers/asm-x86/unistd_32.h
index 8a206df..c1b30a0 100644
--- a/linux-headers/asm-x86/unistd_32.h
+++ b/linux-headers/asm-x86/unistd_32.h
@@ -382,5 +382,7 @@
 #define __NR_pkey_free 382
 #define __NR_statx 383
 #define __NR_arch_prctl 384
+#define __NR_io_pgetevents 385
+#define __NR_rseq 386
 
 #endif /* _ASM_X86_UNISTD_32_H */
diff --git a/linux-headers/asm-x86/unistd_64.h b/linux-headers/asm-x86/unistd_64.h
index 336c2e4..c2e464c1 100644
--- a/linux-headers/asm-x86/unistd_64.h
+++ b/linux-headers/asm-x86/unistd_64.h
@@ -334,5 +334,7 @@
 #define __NR_pkey_alloc 330
 #define __NR_pkey_free 331
 #define __NR_statx 332
+#define __NR_io_pgetevents 333
+#define __NR_rseq 334
 
 #endif /* _ASM_X86_UNISTD_64_H */
diff --git a/linux-headers/asm-x86/unistd_x32.h b/linux-headers/asm-x86/unistd_x32.h
index cb98a52..3722902 100644
--- a/linux-headers/asm-x86/unistd_x32.h
+++ b/linux-headers/asm-x86/unistd_x32.h
@@ -287,6 +287,8 @@
 #define __NR_pkey_alloc (__X32_SYSCALL_BIT + 330)
 #define __NR_pkey_free (__X32_SYSCALL_BIT + 331)
 #define __NR_statx (__X32_SYSCALL_BIT + 332)
+#define __NR_io_pgetevents (__X32_SYSCALL_BIT + 333)
+#define __NR_rseq (__X32_SYSCALL_BIT + 334)
 #define __NR_rt_sigaction (__X32_SYSCALL_BIT + 512)
 #define __NR_rt_sigreturn (__X32_SYSCALL_BIT + 513)
 #define __NR_ioctl (__X32_SYSCALL_BIT + 514)
diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
index cdb148e..98f389a 100644
--- a/linux-headers/linux/kvm.h
+++ b/linux-headers/linux/kvm.h
@@ -677,10 +677,10 @@ struct kvm_ioeventfd {
 };
 
 #define KVM_X86_DISABLE_EXITS_MWAIT          (1 << 0)
-#define KVM_X86_DISABLE_EXITS_HTL            (1 << 1)
+#define KVM_X86_DISABLE_EXITS_HLT            (1 << 1)
 #define KVM_X86_DISABLE_EXITS_PAUSE          (1 << 2)
 #define KVM_X86_DISABLE_VALID_EXITS          (KVM_X86_DISABLE_EXITS_MWAIT | \
-                                              KVM_X86_DISABLE_EXITS_HTL | \
+                                              KVM_X86_DISABLE_EXITS_HLT | \
                                               KVM_X86_DISABLE_EXITS_PAUSE)
 
 /* for KVM_ENABLE_CAP */
@@ -948,6 +948,7 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_S390_BPB 152
 #define KVM_CAP_GET_MSR_FEATURES 153
 #define KVM_CAP_HYPERV_EVENTFD 154
+#define KVM_CAP_HYPERV_TLBFLUSH 155
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
diff --git a/linux-headers/linux/psp-sev.h b/linux-headers/linux/psp-sev.h
index 33e2474..b7b933f 100644
--- a/linux-headers/linux/psp-sev.h
+++ b/linux-headers/linux/psp-sev.h
@@ -30,6 +30,7 @@ enum {
 	SEV_PDH_GEN,
 	SEV_PDH_CERT_EXPORT,
 	SEV_PEK_CERT_IMPORT,
+	SEV_GET_ID,
 
 	SEV_MAX,
 };
@@ -124,6 +125,17 @@ struct sev_user_data_pdh_cert_export {
 } __attribute__((packed));
 
 /**
+ * struct sev_user_data_get_id - GET_ID command parameters
+ *
+ * @socket1: Buffer to pass unique ID of first socket
+ * @socket2: Buffer to pass unique ID of second socket
+ */
+struct sev_user_data_get_id {
+	__u8 socket1[64];			/* Out */
+	__u8 socket2[64];			/* Out */
+} __attribute__((packed));
+
+/**
  * struct sev_issue_cmd - SEV ioctl parameters
  *
  * @cmd: SEV commands to execute
-- 
2.5.5

^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [Qemu-devel] [PATCH v2 02/11] target/arm: Allow KVM device address overwriting
  2018-06-15 14:28 [Qemu-devel] [PATCH v2 00/11] KVM/ARM: virt-3.0: Multiple redistributor regions and 256MB ECAM region Eric Auger
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 01/11] linux-headers: Update to 4.18-rc0 Eric Auger
@ 2018-06-15 14:28 ` Eric Auger
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 03/11] hw/intc/arm_gicv3: Introduce redist-region-count array property Eric Auger
                   ` (9 subsequent siblings)
  11 siblings, 0 replies; 23+ messages in thread
From: Eric Auger @ 2018-06-15 14:28 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell
  Cc: drjones, wei, zhaoshenglong, lersek, ard.biesheuvel,
	christoffer.dall, marc.zyngier

for KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION attribute, the attribute
data pointed to by kvm_device_attr.addr is a OR of the
redistributor region address and other fields such as the index
of the redistributor region and the number of redistributors the
region can contain.

The existing machine init done notifier framework sets the address
field to the actual address of the device and does not allow to OR
this value with other fields.

This patch extends the KVMDevice struct with a new kda_addr_ormask
member. Its value is passed at registration time and OR'ed with the
resolved address on kvm_arm_set_device_addr().

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>

---

v2 -> v3:
- s/addr_fixup/add_ormask
- Added Peter's R-b
---
 hw/intc/arm_gic_kvm.c       |  4 ++--
 hw/intc/arm_gicv3_its_kvm.c |  2 +-
 hw/intc/arm_gicv3_kvm.c     |  4 ++--
 target/arm/kvm.c            | 10 +++++++++-
 target/arm/kvm_arm.h        |  3 ++-
 5 files changed, 16 insertions(+), 7 deletions(-)

diff --git a/hw/intc/arm_gic_kvm.c b/hw/intc/arm_gic_kvm.c
index 204369d..8666508 100644
--- a/hw/intc/arm_gic_kvm.c
+++ b/hw/intc/arm_gic_kvm.c
@@ -558,7 +558,7 @@ static void kvm_arm_gic_realize(DeviceState *dev, Error **errp)
                             | KVM_VGIC_V2_ADDR_TYPE_DIST,
                             KVM_DEV_ARM_VGIC_GRP_ADDR,
                             KVM_VGIC_V2_ADDR_TYPE_DIST,
-                            s->dev_fd);
+                            s->dev_fd, 0);
     /* CPU interface for current core. Unlike arm_gic, we don't
      * provide the "interface for core #N" memory regions, because
      * cores with a VGIC don't have those.
@@ -568,7 +568,7 @@ static void kvm_arm_gic_realize(DeviceState *dev, Error **errp)
                             | KVM_VGIC_V2_ADDR_TYPE_CPU,
                             KVM_DEV_ARM_VGIC_GRP_ADDR,
                             KVM_VGIC_V2_ADDR_TYPE_CPU,
-                            s->dev_fd);
+                            s->dev_fd, 0);
 
     if (kvm_has_gsi_routing()) {
         /* set up irq routing */
diff --git a/hw/intc/arm_gicv3_its_kvm.c b/hw/intc/arm_gicv3_its_kvm.c
index eea6a73..271ebe4 100644
--- a/hw/intc/arm_gicv3_its_kvm.c
+++ b/hw/intc/arm_gicv3_its_kvm.c
@@ -103,7 +103,7 @@ static void kvm_arm_its_realize(DeviceState *dev, Error **errp)
 
     /* register the base address */
     kvm_arm_register_device(&s->iomem_its_cntrl, -1, KVM_DEV_ARM_VGIC_GRP_ADDR,
-                            KVM_VGIC_ITS_ADDR_TYPE, s->dev_fd);
+                            KVM_VGIC_ITS_ADDR_TYPE, s->dev_fd, 0);
 
     gicv3_its_init_mmio(s, NULL);
 
diff --git a/hw/intc/arm_gicv3_kvm.c b/hw/intc/arm_gicv3_kvm.c
index 5649cac..46d9afb 100644
--- a/hw/intc/arm_gicv3_kvm.c
+++ b/hw/intc/arm_gicv3_kvm.c
@@ -793,9 +793,9 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp)
                       KVM_DEV_ARM_VGIC_CTRL_INIT, NULL, true, &error_abort);
 
     kvm_arm_register_device(&s->iomem_dist, -1, KVM_DEV_ARM_VGIC_GRP_ADDR,
-                            KVM_VGIC_V3_ADDR_TYPE_DIST, s->dev_fd);
+                            KVM_VGIC_V3_ADDR_TYPE_DIST, s->dev_fd, 0);
     kvm_arm_register_device(&s->iomem_redist, -1, KVM_DEV_ARM_VGIC_GRP_ADDR,
-                            KVM_VGIC_V3_ADDR_TYPE_REDIST, s->dev_fd);
+                            KVM_VGIC_V3_ADDR_TYPE_REDIST, s->dev_fd, 0);
 
     if (kvm_has_gsi_routing()) {
         /* set up irq routing */
diff --git a/target/arm/kvm.c b/target/arm/kvm.c
index 98f5006..867fef8 100644
--- a/target/arm/kvm.c
+++ b/target/arm/kvm.c
@@ -184,10 +184,15 @@ unsigned long kvm_arch_vcpu_id(CPUState *cpu)
  * We use a MemoryListener to track mapping and unmapping of
  * the regions during board creation, so the board models don't
  * need to do anything special for the KVM case.
+ *
+ * Sometimes the address must be OR'ed with some other fields
+ * (for example for KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION).
+ * @kda_addr_ormask aims at storing the value of those fields.
  */
 typedef struct KVMDevice {
     struct kvm_arm_device_addr kda;
     struct kvm_device_attr kdattr;
+    uint64_t kda_addr_ormask;
     MemoryRegion *mr;
     QSLIST_ENTRY(KVMDevice) entries;
     int dev_fd;
@@ -234,6 +239,8 @@ static void kvm_arm_set_device_addr(KVMDevice *kd)
      */
     if (kd->dev_fd >= 0) {
         uint64_t addr = kd->kda.addr;
+
+        addr |= kd->kda_addr_ormask;
         attr->addr = (uintptr_t)&addr;
         ret = kvm_device_ioctl(kd->dev_fd, KVM_SET_DEVICE_ATTR, attr);
     } else {
@@ -266,7 +273,7 @@ static Notifier notify = {
 };
 
 void kvm_arm_register_device(MemoryRegion *mr, uint64_t devid, uint64_t group,
-                             uint64_t attr, int dev_fd)
+                             uint64_t attr, int dev_fd, uint64_t addr_ormask)
 {
     KVMDevice *kd;
 
@@ -286,6 +293,7 @@ void kvm_arm_register_device(MemoryRegion *mr, uint64_t devid, uint64_t group,
     kd->kdattr.group = group;
     kd->kdattr.attr = attr;
     kd->dev_fd = dev_fd;
+    kd->kda_addr_ormask = addr_ormask;
     QSLIST_INSERT_HEAD(&kvm_devices_head, kd, entries);
     memory_region_ref(kd->mr);
 }
diff --git a/target/arm/kvm_arm.h b/target/arm/kvm_arm.h
index 1e23640..863f205 100644
--- a/target/arm/kvm_arm.h
+++ b/target/arm/kvm_arm.h
@@ -34,6 +34,7 @@ int kvm_arm_vcpu_init(CPUState *cs);
  * @group: device control API group for setting addresses
  * @attr: device control API address type
  * @dev_fd: device control device file descriptor (or -1 if not supported)
+ * @addr_ormask: value to be OR'ed with resolved address
  *
  * Remember the memory region @mr, and when it is mapped by the
  * machine model, tell the kernel that base address using the
@@ -45,7 +46,7 @@ int kvm_arm_vcpu_init(CPUState *cs);
  * address at the point where machine init is complete.
  */
 void kvm_arm_register_device(MemoryRegion *mr, uint64_t devid, uint64_t group,
-                             uint64_t attr, int dev_fd);
+                             uint64_t attr, int dev_fd, uint64_t addr_ormask);
 
 /**
  * kvm_arm_init_cpreg_list:
-- 
2.5.5

^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [Qemu-devel] [PATCH v2 03/11] hw/intc/arm_gicv3: Introduce redist-region-count array property
  2018-06-15 14:28 [Qemu-devel] [PATCH v2 00/11] KVM/ARM: virt-3.0: Multiple redistributor regions and 256MB ECAM region Eric Auger
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 01/11] linux-headers: Update to 4.18-rc0 Eric Auger
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 02/11] target/arm: Allow KVM device address overwriting Eric Auger
@ 2018-06-15 14:28 ` Eric Auger
  2018-06-15 15:27   ` Andrew Jones
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 04/11] hw/intc/arm_gicv3_kvm: Get prepared to handle multiple redist regions Eric Auger
                   ` (8 subsequent siblings)
  11 siblings, 1 reply; 23+ messages in thread
From: Eric Auger @ 2018-06-15 14:28 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell
  Cc: drjones, wei, zhaoshenglong, lersek, ard.biesheuvel,
	christoffer.dall, marc.zyngier

To prepare for multiple redistributor regions, we introduce
an array of uint32_t properties that stores the redistributor
count of each redistributor region.

Non accelerated VGICv3 only supports a single redistributor region.
The capacity of all redist regions is checked against the number of
vcpus.

Machvirt is updated to set those properties, ie. a single
redistributor region with count set to the number of vcpus
capped by 123.

Signed-off-by: Eric Auger <eric.auger@redhat.com>

---
v2 -> v3:
- add missing return in arm_gic_realize
- in gicv3_init_irqs_and_mmio, compute/check rdist_capacity first
- rdist region 0 size set to MIN(smp_cpus, redist0_capacity)
- add GICV3_REDIST_SIZE
---
 hw/arm/virt.c                      | 11 ++++++++++-
 hw/intc/arm_gicv3.c                | 12 +++++++++++-
 hw/intc/arm_gicv3_common.c         | 38 +++++++++++++++++++++++++++++++++-----
 hw/intc/arm_gicv3_kvm.c            |  9 +++++++--
 include/hw/intc/arm_gicv3_common.h |  8 ++++++--
 5 files changed, 67 insertions(+), 11 deletions(-)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index f0a4fa0..2885d18 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -522,6 +522,15 @@ static void create_gic(VirtMachineState *vms, qemu_irq *pic)
     if (!kvm_irqchip_in_kernel()) {
         qdev_prop_set_bit(gicdev, "has-security-extensions", vms->secure);
     }
+
+    if (type == 3) {
+        uint32_t redist0_capacity =
+                    vms->memmap[VIRT_GIC_REDIST].size / GICV3_REDIST_SIZE;
+        uint32_t redist0_count = MIN(smp_cpus, redist0_capacity);
+
+        qdev_prop_set_uint32(gicdev, "len-redist-region-count", 1);
+        qdev_prop_set_uint32(gicdev, "redist-region-count[0]", redist0_count);
+    }
     qdev_init_nofail(gicdev);
     gicbusdev = SYS_BUS_DEVICE(gicdev);
     sysbus_mmio_map(gicbusdev, 0, vms->memmap[VIRT_GIC_DIST].base);
@@ -1321,7 +1330,7 @@ static void machvirt_init(MachineState *machine)
      * many redistributors we can fit into the memory map.
      */
     if (vms->gic_version == 3) {
-        virt_max_cpus = vms->memmap[VIRT_GIC_REDIST].size / 0x20000;
+        virt_max_cpus = vms->memmap[VIRT_GIC_REDIST].size / GICV3_REDIST_SIZE;
     } else {
         virt_max_cpus = GIC_NCPU;
     }
diff --git a/hw/intc/arm_gicv3.c b/hw/intc/arm_gicv3.c
index 479c667..7044133 100644
--- a/hw/intc/arm_gicv3.c
+++ b/hw/intc/arm_gicv3.c
@@ -373,7 +373,17 @@ static void arm_gic_realize(DeviceState *dev, Error **errp)
         return;
     }
 
-    gicv3_init_irqs_and_mmio(s, gicv3_set_irq, gic_ops);
+    if (s->nb_redist_regions != 1) {
+        error_setg(errp, "VGICv3 redist region number(%d) not equal to 1",
+                   s->nb_redist_regions);
+        return;
+    }
+
+    gicv3_init_irqs_and_mmio(s, gicv3_set_irq, gic_ops, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        return;
+    }
 
     gicv3_init_cpuif(s);
 }
diff --git a/hw/intc/arm_gicv3_common.c b/hw/intc/arm_gicv3_common.c
index 864b7c6..ff326b3 100644
--- a/hw/intc/arm_gicv3_common.c
+++ b/hw/intc/arm_gicv3_common.c
@@ -247,11 +247,22 @@ static const VMStateDescription vmstate_gicv3 = {
 };
 
 void gicv3_init_irqs_and_mmio(GICv3State *s, qemu_irq_handler handler,
-                              const MemoryRegionOps *ops)
+                              const MemoryRegionOps *ops, Error **errp)
 {
     SysBusDevice *sbd = SYS_BUS_DEVICE(s);
+    int rdist_capacity = 0;
     int i;
 
+    for (i = 0; i < s->nb_redist_regions; i++) {
+        rdist_capacity += s->redist_region_count[i];
+    }
+    if (rdist_capacity < s->num_cpu) {
+        error_setg(errp, "Capacity of the redist regions(%d) "
+                   "is less than number of vcpus(%d)",
+                   rdist_capacity, s->num_cpu);
+        return;
+    }
+
     /* For the GIC, also expose incoming GPIO lines for PPIs for each CPU.
      * GPIO array layout is thus:
      *  [0..N-1] spi
@@ -277,11 +288,18 @@ void gicv3_init_irqs_and_mmio(GICv3State *s, qemu_irq_handler handler,
 
     memory_region_init_io(&s->iomem_dist, OBJECT(s), ops, s,
                           "gicv3_dist", 0x10000);
-    memory_region_init_io(&s->iomem_redist, OBJECT(s), ops ? &ops[1] : NULL, s,
-                          "gicv3_redist", 0x20000 * s->num_cpu);
-
     sysbus_init_mmio(sbd, &s->iomem_dist);
-    sysbus_init_mmio(sbd, &s->iomem_redist);
+
+    s->iomem_redist = g_new0(MemoryRegion, s->nb_redist_regions);
+    for (i = 0; i < s->nb_redist_regions; i++) {
+        char *name = g_strdup_printf("gicv3_redist_region[%d]", i);
+
+        memory_region_init_io(&s->iomem_redist[i], OBJECT(s),
+                              ops ? &ops[1] : NULL, s, name,
+                              s->redist_region_count[i] * GICV3_REDIST_SIZE);
+        sysbus_init_mmio(sbd, &s->iomem_redist[i]);
+        g_free(name);
+    }
 }
 
 static void arm_gicv3_common_realize(DeviceState *dev, Error **errp)
@@ -363,6 +381,13 @@ static void arm_gicv3_common_realize(DeviceState *dev, Error **errp)
     }
 }
 
+static void arm_gicv3_finalize(Object *obj)
+{
+    GICv3State *s = ARM_GICV3_COMMON(obj);
+
+    g_free(s->redist_region_count);
+}
+
 static void arm_gicv3_common_reset(DeviceState *dev)
 {
     GICv3State *s = ARM_GICV3_COMMON(dev);
@@ -467,6 +492,8 @@ static Property arm_gicv3_common_properties[] = {
     DEFINE_PROP_UINT32("num-irq", GICv3State, num_irq, 32),
     DEFINE_PROP_UINT32("revision", GICv3State, revision, 3),
     DEFINE_PROP_BOOL("has-security-extensions", GICv3State, security_extn, 0),
+    DEFINE_PROP_ARRAY("redist-region-count", GICv3State, nb_redist_regions,
+                      redist_region_count, qdev_prop_uint32, uint32_t),
     DEFINE_PROP_END_OF_LIST(),
 };
 
@@ -488,6 +515,7 @@ static const TypeInfo arm_gicv3_common_type = {
     .instance_size = sizeof(GICv3State),
     .class_size = sizeof(ARMGICv3CommonClass),
     .class_init = arm_gicv3_common_class_init,
+    .instance_finalize = arm_gicv3_finalize,
     .abstract = true,
     .interfaces = (InterfaceInfo []) {
         { TYPE_ARM_LINUX_BOOT_IF },
diff --git a/hw/intc/arm_gicv3_kvm.c b/hw/intc/arm_gicv3_kvm.c
index 46d9afb..ed7b719 100644
--- a/hw/intc/arm_gicv3_kvm.c
+++ b/hw/intc/arm_gicv3_kvm.c
@@ -770,7 +770,11 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp)
         return;
     }
 
-    gicv3_init_irqs_and_mmio(s, kvm_arm_gicv3_set_irq, NULL);
+    gicv3_init_irqs_and_mmio(s, kvm_arm_gicv3_set_irq, NULL, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        return;
+    }
 
     for (i = 0; i < s->num_cpu; i++) {
         ARMCPU *cpu = ARM_CPU(qemu_get_cpu(i));
@@ -794,7 +798,8 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp)
 
     kvm_arm_register_device(&s->iomem_dist, -1, KVM_DEV_ARM_VGIC_GRP_ADDR,
                             KVM_VGIC_V3_ADDR_TYPE_DIST, s->dev_fd, 0);
-    kvm_arm_register_device(&s->iomem_redist, -1, KVM_DEV_ARM_VGIC_GRP_ADDR,
+    kvm_arm_register_device(&s->iomem_redist[0], -1,
+                            KVM_DEV_ARM_VGIC_GRP_ADDR,
                             KVM_VGIC_V3_ADDR_TYPE_REDIST, s->dev_fd, 0);
 
     if (kvm_has_gsi_routing()) {
diff --git a/include/hw/intc/arm_gicv3_common.h b/include/hw/intc/arm_gicv3_common.h
index d75b49d..b798486 100644
--- a/include/hw/intc/arm_gicv3_common.h
+++ b/include/hw/intc/arm_gicv3_common.h
@@ -35,6 +35,8 @@
 #define GICV3_MAXIRQ 1020
 #define GICV3_MAXSPI (GICV3_MAXIRQ - GIC_INTERNAL)
 
+#define GICV3_REDIST_SIZE 0x20000
+
 /* Number of SGI target-list bits */
 #define GICV3_TARGETLIST_BITS 16
 
@@ -210,7 +212,9 @@ struct GICv3State {
     /*< public >*/
 
     MemoryRegion iomem_dist; /* Distributor */
-    MemoryRegion iomem_redist; /* Redistributors */
+    MemoryRegion *iomem_redist; /* Redistributor Regions */
+    uint32_t *redist_region_count; /* redistributor count within each region */
+    uint32_t nb_redist_regions; /* number of redist regions */
 
     uint32_t num_cpu;
     uint32_t num_irq;
@@ -292,6 +296,6 @@ typedef struct ARMGICv3CommonClass {
 } ARMGICv3CommonClass;
 
 void gicv3_init_irqs_and_mmio(GICv3State *s, qemu_irq_handler handler,
-                              const MemoryRegionOps *ops);
+                              const MemoryRegionOps *ops, Error **errp);
 
 #endif
-- 
2.5.5

^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [Qemu-devel] [PATCH v2 04/11] hw/intc/arm_gicv3_kvm: Get prepared to handle multiple redist regions
  2018-06-15 14:28 [Qemu-devel] [PATCH v2 00/11] KVM/ARM: virt-3.0: Multiple redistributor regions and 256MB ECAM region Eric Auger
                   ` (2 preceding siblings ...)
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 03/11] hw/intc/arm_gicv3: Introduce redist-region-count array property Eric Auger
@ 2018-06-15 14:28 ` Eric Auger
  2018-06-15 15:19   ` Andrew Jones
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 05/11] hw/arm/virt: GICv3 DT node with one or two redistributor regions Eric Auger
                   ` (7 subsequent siblings)
  11 siblings, 1 reply; 23+ messages in thread
From: Eric Auger @ 2018-06-15 14:28 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell
  Cc: drjones, wei, zhaoshenglong, lersek, ard.biesheuvel,
	christoffer.dall, marc.zyngier

Let's check if KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION is supported.
If not, we check the number of redist region is equal to 1 and use the
legacy KVM_VGIC_V3_ADDR_TYPE_REDIST attribute. Otherwise we use
the new attribute and allow to register multiple regions to the
KVM device.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>

---

v1 (virt3.0)-> v2
- added a comment to explain why the regions are registered in
  reverse order

v2 -> v3:
- In kvm_arm_gicv3_realize rename val into add_ormask local variable and
  add a comment
- start the redist region registration  from s->nb_redist_regions - 1
  downwards
---
 hw/intc/arm_gicv3_kvm.c | 37 ++++++++++++++++++++++++++++++++++---
 1 file changed, 34 insertions(+), 3 deletions(-)

diff --git a/hw/intc/arm_gicv3_kvm.c b/hw/intc/arm_gicv3_kvm.c
index ed7b719..5cb94fc 100644
--- a/hw/intc/arm_gicv3_kvm.c
+++ b/hw/intc/arm_gicv3_kvm.c
@@ -753,6 +753,7 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp)
 {
     GICv3State *s = KVM_ARM_GICV3(dev);
     KVMARMGICv3Class *kgc = KVM_ARM_GICV3_GET_CLASS(s);
+    bool multiple_redist_region_allowed;
     Error *local_err = NULL;
     int i;
 
@@ -789,6 +790,18 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp)
         return;
     }
 
+    multiple_redist_region_allowed =
+        kvm_device_check_attr(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+                              KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION);
+
+    if (!multiple_redist_region_allowed && s->nb_redist_regions > 1) {
+        error_setg(errp, "Multiple VGICv3 redistributor regions are not "
+                   "supported by this host kernel");
+        error_append_hint(errp, "A maximum of %d VCPUs can be used",
+                          s->redist_region_count[0]);
+        return;
+    }
+
     kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS,
                       0, &s->num_irq, true, &error_abort);
 
@@ -798,9 +811,27 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp)
 
     kvm_arm_register_device(&s->iomem_dist, -1, KVM_DEV_ARM_VGIC_GRP_ADDR,
                             KVM_VGIC_V3_ADDR_TYPE_DIST, s->dev_fd, 0);
-    kvm_arm_register_device(&s->iomem_redist[0], -1,
-                            KVM_DEV_ARM_VGIC_GRP_ADDR,
-                            KVM_VGIC_V3_ADDR_TYPE_REDIST, s->dev_fd, 0);
+
+    if (!multiple_redist_region_allowed) {
+        kvm_arm_register_device(&s->iomem_redist[0], -1,
+                                KVM_DEV_ARM_VGIC_GRP_ADDR,
+                                KVM_VGIC_V3_ADDR_TYPE_REDIST, s->dev_fd, 0);
+    } else {
+        /* we register regions in reverse order as "devices" are inserted at
+         * the head of a QSLIST and the list is then popped from the head
+         * onwards by kvm_arm_machine_init_done()
+         */
+        for (i = s->nb_redist_regions - 1; i >= 0; i--) {
+            /* Address mask made of the rdist region index and count */
+            uint64_t addr_ormask =
+                        i | ((uint64_t)s->redist_region_count[i] << 52);
+
+            kvm_arm_register_device(&s->iomem_redist[i], -1,
+                                    KVM_DEV_ARM_VGIC_GRP_ADDR,
+                                    KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION,
+                                    s->dev_fd, addr_ormask);
+        }
+    }
 
     if (kvm_has_gsi_routing()) {
         /* set up irq routing */
-- 
2.5.5

^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [Qemu-devel] [PATCH v2 05/11] hw/arm/virt: GICv3 DT node with one or two redistributor regions
  2018-06-15 14:28 [Qemu-devel] [PATCH v2 00/11] KVM/ARM: virt-3.0: Multiple redistributor regions and 256MB ECAM region Eric Auger
                   ` (3 preceding siblings ...)
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 04/11] hw/intc/arm_gicv3_kvm: Get prepared to handle multiple redist regions Eric Auger
@ 2018-06-15 14:28 ` Eric Auger
  2018-06-19 18:53   ` Laszlo Ersek
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 06/11] hw/arm/virt-acpi-build: Advertise one or two GICR structures Eric Auger
                   ` (6 subsequent siblings)
  11 siblings, 1 reply; 23+ messages in thread
From: Eric Auger @ 2018-06-15 14:28 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell
  Cc: drjones, wei, zhaoshenglong, lersek, ard.biesheuvel,
	christoffer.dall, marc.zyngier

This patch allows the creation of a GICv3 node with 1 or 2
redistributor regions depending on the number of smu_cpus.
The second redistributor region is located just after the
existing RAM region, at 256GB and contains up to up to 512 vcpus.

Please refer to kernel documentation for further node details:
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>

---
v1 (virt3.0) -> v2
- Added Drew's R-b

v2 -> v3:
- VIRT_GIC_REDIST2 is now 64MB large, ie. 512 redistributor capacity
- virt_gicv3_redist_region_count does not test kvm_irqchip_in_kernel
  anymore
---
 hw/arm/virt.c         | 29 ++++++++++++++++++++++++-----
 include/hw/arm/virt.h | 14 ++++++++++++++
 2 files changed, 38 insertions(+), 5 deletions(-)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 2885d18..d9f72eb 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -148,6 +148,8 @@ static const MemMapEntry a15memmap[] = {
     [VIRT_PCIE_PIO] =           { 0x3eff0000, 0x00010000 },
     [VIRT_PCIE_ECAM] =          { 0x3f000000, 0x01000000 },
     [VIRT_MEM] =                { 0x40000000, RAMLIMIT_BYTES },
+    /* Additional 64 MB redist region (can contain up to 512 redistributors) */
+    [VIRT_GIC_REDIST2] =        { 0x4000000000ULL, 0x4000000 },
     /* Second PCIe window, 512GB wide at the 512GB boundary */
     [VIRT_PCIE_MMIO_HIGH] =   { 0x8000000000ULL, 0x8000000000ULL },
 };
@@ -401,13 +403,30 @@ static void fdt_add_gic_node(VirtMachineState *vms)
     qemu_fdt_setprop_cell(vms->fdt, "/intc", "#size-cells", 0x2);
     qemu_fdt_setprop(vms->fdt, "/intc", "ranges", NULL, 0);
     if (vms->gic_version == 3) {
+        int nb_redist_regions = virt_gicv3_redist_region_count(vms);
+
         qemu_fdt_setprop_string(vms->fdt, "/intc", "compatible",
                                 "arm,gic-v3");
-        qemu_fdt_setprop_sized_cells(vms->fdt, "/intc", "reg",
-                                     2, vms->memmap[VIRT_GIC_DIST].base,
-                                     2, vms->memmap[VIRT_GIC_DIST].size,
-                                     2, vms->memmap[VIRT_GIC_REDIST].base,
-                                     2, vms->memmap[VIRT_GIC_REDIST].size);
+
+        qemu_fdt_setprop_cell(vms->fdt, "/intc",
+                              "#redistributor-regions", nb_redist_regions);
+
+        if (nb_redist_regions == 1) {
+            qemu_fdt_setprop_sized_cells(vms->fdt, "/intc", "reg",
+                                         2, vms->memmap[VIRT_GIC_DIST].base,
+                                         2, vms->memmap[VIRT_GIC_DIST].size,
+                                         2, vms->memmap[VIRT_GIC_REDIST].base,
+                                         2, vms->memmap[VIRT_GIC_REDIST].size);
+        } else {
+            qemu_fdt_setprop_sized_cells(vms->fdt, "/intc", "reg",
+                                         2, vms->memmap[VIRT_GIC_DIST].base,
+                                         2, vms->memmap[VIRT_GIC_DIST].size,
+                                         2, vms->memmap[VIRT_GIC_REDIST].base,
+                                         2, vms->memmap[VIRT_GIC_REDIST].size,
+                                         2, vms->memmap[VIRT_GIC_REDIST2].base,
+                                         2, vms->memmap[VIRT_GIC_REDIST2].size);
+        }
+
         if (vms->virt) {
             qemu_fdt_setprop_cells(vms->fdt, "/intc", "interrupts",
                                    GIC_FDT_IRQ_TYPE_PPI, ARCH_GICV3_MAINT_IRQ,
diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h
index 4ac7ef6..308156f 100644
--- a/include/hw/arm/virt.h
+++ b/include/hw/arm/virt.h
@@ -35,6 +35,8 @@
 #include "qemu/notify.h"
 #include "hw/boards.h"
 #include "hw/arm/arm.h"
+#include "sysemu/kvm.h"
+#include "hw/intc/arm_gicv3_common.h"
 
 #define NUM_GICV2M_SPIS       64
 #define NUM_VIRTIO_TRANSPORTS 32
@@ -60,6 +62,7 @@ enum {
     VIRT_GIC_V2M,
     VIRT_GIC_ITS,
     VIRT_GIC_REDIST,
+    VIRT_GIC_REDIST2,
     VIRT_SMMU,
     VIRT_UART,
     VIRT_MMIO,
@@ -130,4 +133,15 @@ typedef struct {
 
 void virt_acpi_setup(VirtMachineState *vms);
 
+/* Return the number of used redistributor regions  */
+static inline int virt_gicv3_redist_region_count(VirtMachineState *vms)
+{
+    uint32_t redist0_capacity =
+                vms->memmap[VIRT_GIC_REDIST].size / GICV3_REDIST_SIZE;
+
+    assert(vms->gic_version == 3);
+
+    return vms->smp_cpus > redist0_capacity ? 2 : 1;
+}
+
 #endif /* QEMU_ARM_VIRT_H */
-- 
2.5.5

^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [Qemu-devel] [PATCH v2 06/11] hw/arm/virt-acpi-build: Advertise one or two GICR structures
  2018-06-15 14:28 [Qemu-devel] [PATCH v2 00/11] KVM/ARM: virt-3.0: Multiple redistributor regions and 256MB ECAM region Eric Auger
                   ` (4 preceding siblings ...)
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 05/11] hw/arm/virt: GICv3 DT node with one or two redistributor regions Eric Auger
@ 2018-06-15 14:28 ` Eric Auger
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 07/11] hw/arm/virt: Register two redistributor regions when necessary Eric Auger
                   ` (5 subsequent siblings)
  11 siblings, 0 replies; 23+ messages in thread
From: Eric Auger @ 2018-06-15 14:28 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell
  Cc: drjones, wei, zhaoshenglong, lersek, ard.biesheuvel,
	christoffer.dall, marc.zyngier

Depending on the number of smp_cpus we now register one or two
GICR structures.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>

v1(virt 3.0) -> v2
- Added Drew's R-b
---
 hw/arm/virt-acpi-build.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
index 74f5744..eefd1d4 100644
--- a/hw/arm/virt-acpi-build.c
+++ b/hw/arm/virt-acpi-build.c
@@ -670,6 +670,7 @@ build_madt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
 
     if (vms->gic_version == 3) {
         AcpiMadtGenericTranslator *gic_its;
+        int nb_redist_regions = virt_gicv3_redist_region_count(vms);
         AcpiMadtGenericRedistributor *gicr = acpi_data_push(table_data,
                                                          sizeof *gicr);
 
@@ -678,6 +679,14 @@ build_madt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
         gicr->base_address = cpu_to_le64(memmap[VIRT_GIC_REDIST].base);
         gicr->range_length = cpu_to_le32(memmap[VIRT_GIC_REDIST].size);
 
+        if (nb_redist_regions == 2) {
+            gicr = acpi_data_push(table_data, sizeof(*gicr));
+            gicr->type = ACPI_APIC_GENERIC_REDISTRIBUTOR;
+            gicr->length = sizeof(*gicr);
+            gicr->base_address = cpu_to_le64(memmap[VIRT_GIC_REDIST2].base);
+            gicr->range_length = cpu_to_le32(memmap[VIRT_GIC_REDIST2].size);
+        }
+
         if (its_class_name() && !vmc->no_its) {
             gic_its = acpi_data_push(table_data, sizeof *gic_its);
             gic_its->type = ACPI_APIC_GENERIC_TRANSLATOR;
-- 
2.5.5

^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [Qemu-devel] [PATCH v2 07/11] hw/arm/virt: Register two redistributor regions when necessary
  2018-06-15 14:28 [Qemu-devel] [PATCH v2 00/11] KVM/ARM: virt-3.0: Multiple redistributor regions and 256MB ECAM region Eric Auger
                   ` (5 preceding siblings ...)
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 06/11] hw/arm/virt-acpi-build: Advertise one or two GICR structures Eric Auger
@ 2018-06-15 14:28 ` Eric Auger
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 08/11] hw/arm/virt: Add a new 256MB ECAM region Eric Auger
                   ` (4 subsequent siblings)
  11 siblings, 0 replies; 23+ messages in thread
From: Eric Auger @ 2018-06-15 14:28 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell
  Cc: drjones, wei, zhaoshenglong, lersek, ard.biesheuvel,
	christoffer.dall, marc.zyngier

With a VGICv3 KVM device, if the number of vcpus exceeds the
capacity of the legacy redistributor region (123 redistributors),
we now attempt to register a second redistributor region. Up to
512 redistributors can fit in this latter on top of the 123 allowed
by the legacy redistributor region.

Registering this second redistributor region is possible if the
host kernel supports the following VGICv3 KVM device group/attribute:
KVM_DEV_ARM_VGIC_GRP_ADDR/KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION.

In case the host kernel does not support the registration of several
redistributor regions and the requested number of vcpus exceeds the
capacity of the legacy redistributor region, the GICv3 device
initialization fails with a proper error message and qemu exits.

At the moment the max number of vcpus still is capped by the
virt machine class max_cpus.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>

---

v2 -> v3:
- remove spare space
---
 hw/arm/virt.c | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index d9f72eb..936cb3a 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -528,6 +528,7 @@ static void create_gic(VirtMachineState *vms, qemu_irq *pic)
     SysBusDevice *gicbusdev;
     const char *gictype;
     int type = vms->gic_version, i;
+    uint32_t nb_redist_regions = 0;
 
     gictype = (type == 3) ? gicv3_class_name() : gic_class_name();
 
@@ -547,14 +548,28 @@ static void create_gic(VirtMachineState *vms, qemu_irq *pic)
                     vms->memmap[VIRT_GIC_REDIST].size / GICV3_REDIST_SIZE;
         uint32_t redist0_count = MIN(smp_cpus, redist0_capacity);
 
-        qdev_prop_set_uint32(gicdev, "len-redist-region-count", 1);
+        nb_redist_regions = virt_gicv3_redist_region_count(vms);
+
+        qdev_prop_set_uint32(gicdev, "len-redist-region-count",
+                             nb_redist_regions);
         qdev_prop_set_uint32(gicdev, "redist-region-count[0]", redist0_count);
+
+        if (nb_redist_regions == 2) {
+            uint32_t redist1_capacity =
+                        vms->memmap[VIRT_GIC_REDIST2].size / GICV3_REDIST_SIZE;
+
+            qdev_prop_set_uint32(gicdev, "redist-region-count[1]",
+                MIN(smp_cpus - redist0_count, redist1_capacity));
+        }
     }
     qdev_init_nofail(gicdev);
     gicbusdev = SYS_BUS_DEVICE(gicdev);
     sysbus_mmio_map(gicbusdev, 0, vms->memmap[VIRT_GIC_DIST].base);
     if (type == 3) {
         sysbus_mmio_map(gicbusdev, 1, vms->memmap[VIRT_GIC_REDIST].base);
+        if (nb_redist_regions == 2) {
+            sysbus_mmio_map(gicbusdev, 2, vms->memmap[VIRT_GIC_REDIST2].base);
+        }
     } else {
         sysbus_mmio_map(gicbusdev, 1, vms->memmap[VIRT_GIC_CPU].base);
     }
@@ -1350,6 +1365,7 @@ static void machvirt_init(MachineState *machine)
      */
     if (vms->gic_version == 3) {
         virt_max_cpus = vms->memmap[VIRT_GIC_REDIST].size / GICV3_REDIST_SIZE;
+        virt_max_cpus += vms->memmap[VIRT_GIC_REDIST2].size / GICV3_REDIST_SIZE;
     } else {
         virt_max_cpus = GIC_NCPU;
     }
-- 
2.5.5

^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [Qemu-devel] [PATCH v2 08/11] hw/arm/virt: Add a new 256MB ECAM region
  2018-06-15 14:28 [Qemu-devel] [PATCH v2 00/11] KVM/ARM: virt-3.0: Multiple redistributor regions and 256MB ECAM region Eric Auger
                   ` (6 preceding siblings ...)
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 07/11] hw/arm/virt: Register two redistributor regions when necessary Eric Auger
@ 2018-06-15 14:28 ` Eric Auger
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 09/11] hw/arm/virt: Add virt-3.0 machine type Eric Auger
                   ` (3 subsequent siblings)
  11 siblings, 0 replies; 23+ messages in thread
From: Eric Auger @ 2018-06-15 14:28 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell
  Cc: drjones, wei, zhaoshenglong, lersek, ard.biesheuvel,
	christoffer.dall, marc.zyngier

This patch defines a new ECAM region located after the 256GB limit.

The virt machine state is augmented with a new highmem_ecam field
which guards the usage of this new ECAM region instead of the legacy
16MB one. With the highmem ECAM region, up to 256 PCIe buses can be
used.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>

---

1.0 (virt3.0) -> 2.0
- introduce VIRT_ECAM_ID and renamed ecam_memmap_id variable into
  ecam_id
- Added Drew's R-b

RFC -> PATCH:
- remove the newline at the end of acpi_dsdt_add_pci
- use vms->highmem_ecam to select the memmap id
---
 hw/arm/virt-acpi-build.c | 21 +++++++++++++--------
 hw/arm/virt.c            | 12 ++++++++----
 include/hw/arm/virt.h    |  4 ++++
 3 files changed, 25 insertions(+), 12 deletions(-)

diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
index eefd1d4..6ea47e2 100644
--- a/hw/arm/virt-acpi-build.c
+++ b/hw/arm/virt-acpi-build.c
@@ -150,16 +150,17 @@ static void acpi_dsdt_add_virtio(Aml *scope,
 }
 
 static void acpi_dsdt_add_pci(Aml *scope, const MemMapEntry *memmap,
-                              uint32_t irq, bool use_highmem)
+                              uint32_t irq, bool use_highmem, bool highmem_ecam)
 {
+    int ecam_id = VIRT_ECAM_ID(highmem_ecam);
     Aml *method, *crs, *ifctx, *UUID, *ifctx1, *elsectx, *buf;
     int i, bus_no;
     hwaddr base_mmio = memmap[VIRT_PCIE_MMIO].base;
     hwaddr size_mmio = memmap[VIRT_PCIE_MMIO].size;
     hwaddr base_pio = memmap[VIRT_PCIE_PIO].base;
     hwaddr size_pio = memmap[VIRT_PCIE_PIO].size;
-    hwaddr base_ecam = memmap[VIRT_PCIE_ECAM].base;
-    hwaddr size_ecam = memmap[VIRT_PCIE_ECAM].size;
+    hwaddr base_ecam = memmap[ecam_id].base;
+    hwaddr size_ecam = memmap[ecam_id].size;
     int nr_pcie_buses = size_ecam / PCIE_MMCFG_SIZE_MIN;
 
     Aml *dev = aml_device("%s", "PCI0");
@@ -173,7 +174,7 @@ static void acpi_dsdt_add_pci(Aml *scope, const MemMapEntry *memmap,
     aml_append(dev, aml_name_decl("_CCA", aml_int(1)));
 
     /* Declare the PCI Routing Table. */
-    Aml *rt_pkg = aml_package(nr_pcie_buses * PCI_NUM_PINS);
+    Aml *rt_pkg = aml_varpackage(nr_pcie_buses * PCI_NUM_PINS);
     for (bus_no = 0; bus_no < nr_pcie_buses; bus_no++) {
         for (i = 0; i < PCI_NUM_PINS; i++) {
             int gsi = (i + bus_no) % PCI_NUM_PINS;
@@ -316,7 +317,10 @@ static void acpi_dsdt_add_pci(Aml *scope, const MemMapEntry *memmap,
     Aml *dev_res0 = aml_device("%s", "RES0");
     aml_append(dev_res0, aml_name_decl("_HID", aml_string("PNP0C02")));
     crs = aml_resource_template();
-    aml_append(crs, aml_memory32_fixed(base_ecam, size_ecam, AML_READ_WRITE));
+    aml_append(crs,
+        aml_qword_memory(AML_POS_DECODE, AML_MIN_FIXED, AML_MAX_FIXED,
+                         AML_NON_CACHEABLE, AML_READ_WRITE, 0x0000, base_ecam,
+                         base_ecam + size_ecam - 1, 0x0000, size_ecam));
     aml_append(dev_res0, aml_name_decl("_CRS", crs));
     aml_append(dev, dev_res0);
     aml_append(scope, dev);
@@ -573,16 +577,17 @@ build_mcfg(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
 {
     AcpiTableMcfg *mcfg;
     const MemMapEntry *memmap = vms->memmap;
+    int ecam_id = VIRT_ECAM_ID(vms->highmem_ecam);
     int len = sizeof(*mcfg) + sizeof(mcfg->allocation[0]);
     int mcfg_start = table_data->len;
 
     mcfg = acpi_data_push(table_data, len);
-    mcfg->allocation[0].address = cpu_to_le64(memmap[VIRT_PCIE_ECAM].base);
+    mcfg->allocation[0].address = cpu_to_le64(memmap[ecam_id].base);
 
     /* Only a single allocation so no need to play with segments */
     mcfg->allocation[0].pci_segment = cpu_to_le16(0);
     mcfg->allocation[0].start_bus_number = 0;
-    mcfg->allocation[0].end_bus_number = (memmap[VIRT_PCIE_ECAM].size
+    mcfg->allocation[0].end_bus_number = (memmap[ecam_id].size
                                           / PCIE_MMCFG_SIZE_MIN) - 1;
 
     build_header(linker, table_data, (void *)(table_data->data + mcfg_start),
@@ -766,7 +771,7 @@ build_dsdt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
     acpi_dsdt_add_virtio(scope, &memmap[VIRT_MMIO],
                     (irqmap[VIRT_MMIO] + ARM_SPI_BASE), NUM_VIRTIO_TRANSPORTS);
     acpi_dsdt_add_pci(scope, memmap, (irqmap[VIRT_PCIE] + ARM_SPI_BASE),
-                      vms->highmem);
+                      vms->highmem, vms->highmem_ecam);
     acpi_dsdt_add_gpio(scope, &memmap[VIRT_GPIO],
                        (irqmap[VIRT_GPIO] + ARM_SPI_BASE));
     acpi_dsdt_add_power_button(scope);
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 936cb3a..45d5ed0 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -150,6 +150,7 @@ static const MemMapEntry a15memmap[] = {
     [VIRT_MEM] =                { 0x40000000, RAMLIMIT_BYTES },
     /* Additional 64 MB redist region (can contain up to 512 redistributors) */
     [VIRT_GIC_REDIST2] =        { 0x4000000000ULL, 0x4000000 },
+    [VIRT_PCIE_ECAM_HIGH] =     { 0x4010000000ULL, 0x10000000 },
     /* Second PCIe window, 512GB wide at the 512GB boundary */
     [VIRT_PCIE_MMIO_HIGH] =   { 0x8000000000ULL, 0x8000000000ULL },
 };
@@ -1043,10 +1044,9 @@ static void create_pcie(VirtMachineState *vms, qemu_irq *pic)
     hwaddr size_mmio_high = vms->memmap[VIRT_PCIE_MMIO_HIGH].size;
     hwaddr base_pio = vms->memmap[VIRT_PCIE_PIO].base;
     hwaddr size_pio = vms->memmap[VIRT_PCIE_PIO].size;
-    hwaddr base_ecam = vms->memmap[VIRT_PCIE_ECAM].base;
-    hwaddr size_ecam = vms->memmap[VIRT_PCIE_ECAM].size;
+    hwaddr base_ecam, size_ecam;
     hwaddr base = base_mmio;
-    int nr_pcie_buses = size_ecam / PCIE_MMCFG_SIZE_MIN;
+    int nr_pcie_buses;
     int irq = vms->irqmap[VIRT_PCIE];
     MemoryRegion *mmio_alias;
     MemoryRegion *mmio_reg;
@@ -1054,12 +1054,16 @@ static void create_pcie(VirtMachineState *vms, qemu_irq *pic)
     MemoryRegion *ecam_reg;
     DeviceState *dev;
     char *nodename;
-    int i;
+    int i, ecam_id;
     PCIHostState *pci;
 
     dev = qdev_create(NULL, TYPE_GPEX_HOST);
     qdev_init_nofail(dev);
 
+    ecam_id = VIRT_ECAM_ID(vms->highmem_ecam);
+    base_ecam = vms->memmap[ecam_id].base;
+    size_ecam = vms->memmap[ecam_id].size;
+    nr_pcie_buses = size_ecam / PCIE_MMCFG_SIZE_MIN;
     /* Map only the first size_ecam bytes of ECAM space */
     ecam_alias = g_new0(MemoryRegion, 1);
     ecam_reg = sysbus_mmio_get_region(SYS_BUS_DEVICE(dev), 0);
diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h
index 308156f..085fdcc 100644
--- a/include/hw/arm/virt.h
+++ b/include/hw/arm/virt.h
@@ -72,6 +72,7 @@ enum {
     VIRT_PCIE_MMIO,
     VIRT_PCIE_PIO,
     VIRT_PCIE_ECAM,
+    VIRT_PCIE_ECAM_HIGH,
     VIRT_PLATFORM_BUS,
     VIRT_PCIE_MMIO_HIGH,
     VIRT_GPIO,
@@ -106,6 +107,7 @@ typedef struct {
     FWCfgState *fw_cfg;
     bool secure;
     bool highmem;
+    bool highmem_ecam;
     bool its;
     bool virt;
     int32_t gic_version;
@@ -123,6 +125,8 @@ typedef struct {
     int psci_conduit;
 } VirtMachineState;
 
+#define VIRT_ECAM_ID(high) (high ? VIRT_PCIE_ECAM_HIGH : VIRT_PCIE_ECAM)
+
 #define TYPE_VIRT_MACHINE   MACHINE_TYPE_NAME("virt")
 #define VIRT_MACHINE(obj) \
     OBJECT_CHECK(VirtMachineState, (obj), TYPE_VIRT_MACHINE)
-- 
2.5.5

^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [Qemu-devel] [PATCH v2 09/11] hw/arm/virt: Add virt-3.0 machine type
  2018-06-15 14:28 [Qemu-devel] [PATCH v2 00/11] KVM/ARM: virt-3.0: Multiple redistributor regions and 256MB ECAM region Eric Auger
                   ` (7 preceding siblings ...)
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 08/11] hw/arm/virt: Add a new 256MB ECAM region Eric Auger
@ 2018-06-15 14:28 ` Eric Auger
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 10/11] hw/arm/virt: Use 256MB ECAM region by default Eric Auger
                   ` (2 subsequent siblings)
  11 siblings, 0 replies; 23+ messages in thread
From: Eric Auger @ 2018-06-15 14:28 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell
  Cc: drjones, wei, zhaoshenglong, lersek, ard.biesheuvel,
	christoffer.dall, marc.zyngier

Add virt-3.0 machine type.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
---
 hw/arm/virt.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 45d5ed0..75a1a14 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1743,7 +1743,7 @@ type_init(machvirt_machine_init);
 #define VIRT_COMPAT_2_12 \
     HW_COMPAT_2_12
 
-static void virt_2_12_instance_init(Object *obj)
+static void virt_3_0_instance_init(Object *obj)
 {
     VirtMachineState *vms = VIRT_MACHINE(obj);
     VirtMachineClass *vmc = VIRT_MACHINE_GET_CLASS(vms);
@@ -1811,11 +1811,22 @@ static void virt_2_12_instance_init(Object *obj)
     vms->irqmap = a15irqmap;
 }
 
+static void virt_machine_3_0_options(MachineClass *mc)
+{
+}
+DEFINE_VIRT_MACHINE_AS_LATEST(3, 0)
+
+static void virt_2_12_instance_init(Object *obj)
+{
+    virt_3_0_instance_init(obj);
+}
+
 static void virt_machine_2_12_options(MachineClass *mc)
 {
+    virt_machine_3_0_options(mc);
     SET_MACHINE_COMPAT(mc, VIRT_COMPAT_2_12);
 }
-DEFINE_VIRT_MACHINE_AS_LATEST(2, 12)
+DEFINE_VIRT_MACHINE(2, 12)
 
 #define VIRT_COMPAT_2_11 \
     HW_COMPAT_2_11
-- 
2.5.5

^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [Qemu-devel] [PATCH v2 10/11] hw/arm/virt: Use 256MB ECAM region by default
  2018-06-15 14:28 [Qemu-devel] [PATCH v2 00/11] KVM/ARM: virt-3.0: Multiple redistributor regions and 256MB ECAM region Eric Auger
                   ` (8 preceding siblings ...)
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 09/11] hw/arm/virt: Add virt-3.0 machine type Eric Auger
@ 2018-06-15 14:28 ` Eric Auger
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 11/11] hw/arm/virt: Increase max_cpus to 512 Eric Auger
  2018-06-20 14:36 ` [Qemu-devel] [PATCH v2 00/11] KVM/ARM: virt-3.0: Multiple redistributor regions and 256MB ECAM region Peter Maydell
  11 siblings, 0 replies; 23+ messages in thread
From: Eric Auger @ 2018-06-15 14:28 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell
  Cc: drjones, wei, zhaoshenglong, lersek, ard.biesheuvel,
	christoffer.dall, marc.zyngier

With this patch, virt-3.0 machine uses a new 256MB ECAM region
by default instead of the legacy 16MB one, if highmem is set
(LPAE supported by the guest) and (!firmware_loaded || aarch64).

Indeed aarch32 mode FW may not support this high ECAM region.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
---
 hw/arm/virt.c         | 10 ++++++++++
 include/hw/arm/virt.h |  1 +
 2 files changed, 11 insertions(+)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 75a1a14..5a16e6c 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1317,6 +1317,7 @@ static void machvirt_init(MachineState *machine)
     int n, virt_max_cpus;
     MemoryRegion *ram = g_new(MemoryRegion, 1);
     bool firmware_loaded = bios_name || drive_get(IF_PFLASH, 0, 0);
+    bool aarch64 = true;
 
     /* We can probe only here because during property set
      * KVM is not available yet
@@ -1432,6 +1433,8 @@ static void machvirt_init(MachineState *machine)
         numa_cpu_pre_plug(&possible_cpus->cpus[cs->cpu_index], DEVICE(cpuobj),
                           &error_fatal);
 
+        aarch64 &= object_property_get_bool(cpuobj, "aarch64", NULL);
+
         if (!vms->secure) {
             object_property_set_bool(cpuobj, false, "has_el3", NULL);
         }
@@ -1490,6 +1493,8 @@ static void machvirt_init(MachineState *machine)
         create_uart(vms, pic, VIRT_SECURE_UART, secure_sysmem, serial_hd(1));
     }
 
+    vms->highmem_ecam &= vms->highmem && (!firmware_loaded || aarch64);
+
     create_rtc(vms, pic);
 
     create_pcie(vms, pic);
@@ -1786,6 +1791,8 @@ static void virt_3_0_instance_init(Object *obj)
                                     "Set GIC version. "
                                     "Valid values are 2, 3 and host", NULL);
 
+    vms->highmem_ecam = !vmc->no_highmem_ecam;
+
     if (vmc->no_its) {
         vms->its = false;
     } else {
@@ -1823,8 +1830,11 @@ static void virt_2_12_instance_init(Object *obj)
 
 static void virt_machine_2_12_options(MachineClass *mc)
 {
+    VirtMachineClass *vmc = VIRT_MACHINE_CLASS(OBJECT_CLASS(mc));
+
     virt_machine_3_0_options(mc);
     SET_MACHINE_COMPAT(mc, VIRT_COMPAT_2_12);
+    vmc->no_highmem_ecam = true;
 }
 DEFINE_VIRT_MACHINE(2, 12)
 
diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h
index 085fdcc..9a870cc 100644
--- a/include/hw/arm/virt.h
+++ b/include/hw/arm/virt.h
@@ -98,6 +98,7 @@ typedef struct {
     bool no_pmu;
     bool claim_edge_triggered_timers;
     bool smbios_old_sys_ver;
+    bool no_highmem_ecam;
 } VirtMachineClass;
 
 typedef struct {
-- 
2.5.5

^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [Qemu-devel] [PATCH v2 11/11] hw/arm/virt: Increase max_cpus to 512
  2018-06-15 14:28 [Qemu-devel] [PATCH v2 00/11] KVM/ARM: virt-3.0: Multiple redistributor regions and 256MB ECAM region Eric Auger
                   ` (9 preceding siblings ...)
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 10/11] hw/arm/virt: Use 256MB ECAM region by default Eric Auger
@ 2018-06-15 14:28 ` Eric Auger
  2018-06-20 14:36 ` [Qemu-devel] [PATCH v2 00/11] KVM/ARM: virt-3.0: Multiple redistributor regions and 256MB ECAM region Peter Maydell
  11 siblings, 0 replies; 23+ messages in thread
From: Eric Auger @ 2018-06-15 14:28 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell
  Cc: drjones, wei, zhaoshenglong, lersek, ard.biesheuvel,
	christoffer.dall, marc.zyngier

virt 3.0 now allows up to 512 vcpus whereas for earlier machine
types, max_cpus was set to 255 and any attempt to start the
machine with vcpus > 255 was rejected at a very early stage,
in vl.c/main level.

512 is the max supported by KVM. Anyway the actual vcpu count
that can be achieved depends on other parameters such as the
acceleration mode, the vgic version, the host kernel version.
Those are discovered later on.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
---
 hw/arm/virt.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 5a16e6c..e899475 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1705,11 +1705,11 @@ static void virt_machine_class_init(ObjectClass *oc, void *data)
     HotplugHandlerClass *hc = HOTPLUG_HANDLER_CLASS(oc);
 
     mc->init = machvirt_init;
-    /* Start max_cpus at the maximum QEMU supports. We'll further restrict
-     * it later in machvirt_init, where we have more information about the
+    /* Start with max_cpus set to 512, which is the maximum supported by KVM.
+     * The value may be reduced later when we have more information about the
      * configuration of the particular instance.
      */
-    mc->max_cpus = 255;
+    mc->max_cpus = 512;
     machine_class_allow_dynamic_sysbus_dev(mc, TYPE_VFIO_CALXEDA_XGMAC);
     machine_class_allow_dynamic_sysbus_dev(mc, TYPE_VFIO_AMD_XGBE);
     mc->block_default_type = IF_VIRTIO;
@@ -1835,6 +1835,7 @@ static void virt_machine_2_12_options(MachineClass *mc)
     virt_machine_3_0_options(mc);
     SET_MACHINE_COMPAT(mc, VIRT_COMPAT_2_12);
     vmc->no_highmem_ecam = true;
+    mc->max_cpus = 255;
 }
 DEFINE_VIRT_MACHINE(2, 12)
 
-- 
2.5.5

^ permalink raw reply related	[flat|nested] 23+ messages in thread

* Re: [Qemu-devel] [PATCH v2 04/11] hw/intc/arm_gicv3_kvm: Get prepared to handle multiple redist regions
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 04/11] hw/intc/arm_gicv3_kvm: Get prepared to handle multiple redist regions Eric Auger
@ 2018-06-15 15:19   ` Andrew Jones
  0 siblings, 0 replies; 23+ messages in thread
From: Andrew Jones @ 2018-06-15 15:19 UTC (permalink / raw)
  To: Eric Auger
  Cc: eric.auger.pro, qemu-devel, qemu-arm, peter.maydell, wei,
	zhaoshenglong, lersek, ard.biesheuvel, christoffer.dall,
	marc.zyngier

On Fri, Jun 15, 2018 at 04:28:23PM +0200, Eric Auger wrote:
> Let's check if KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION is supported.
> If not, we check the number of redist region is equal to 1 and use the
> legacy KVM_VGIC_V3_ADDR_TYPE_REDIST attribute. Otherwise we use
> the new attribute and allow to register multiple regions to the
> KVM device.
> 
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
> 
> ---
> 
> v1 (virt3.0)-> v2
> - added a comment to explain why the regions are registered in
>   reverse order
> 
> v2 -> v3:
> - In kvm_arm_gicv3_realize rename val into add_ormask local variable and
>   add a comment
> - start the redist region registration  from s->nb_redist_regions - 1
>   downwards
> ---
>  hw/intc/arm_gicv3_kvm.c | 37 ++++++++++++++++++++++++++++++++++---
>  1 file changed, 34 insertions(+), 3 deletions(-)
> 
> diff --git a/hw/intc/arm_gicv3_kvm.c b/hw/intc/arm_gicv3_kvm.c
> index ed7b719..5cb94fc 100644
> --- a/hw/intc/arm_gicv3_kvm.c
> +++ b/hw/intc/arm_gicv3_kvm.c
> @@ -753,6 +753,7 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp)
>  {
>      GICv3State *s = KVM_ARM_GICV3(dev);
>      KVMARMGICv3Class *kgc = KVM_ARM_GICV3_GET_CLASS(s);
> +    bool multiple_redist_region_allowed;
>      Error *local_err = NULL;
>      int i;
>  
> @@ -789,6 +790,18 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp)
>          return;
>      }
>  
> +    multiple_redist_region_allowed =
> +        kvm_device_check_attr(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
> +                              KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION);
> +
> +    if (!multiple_redist_region_allowed && s->nb_redist_regions > 1) {
> +        error_setg(errp, "Multiple VGICv3 redistributor regions are not "
> +                   "supported by this host kernel");
> +        error_append_hint(errp, "A maximum of %d VCPUs can be used",
> +                          s->redist_region_count[0]);
> +        return;
> +    }
> +
>      kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS,
>                        0, &s->num_irq, true, &error_abort);
>  
> @@ -798,9 +811,27 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp)
>  
>      kvm_arm_register_device(&s->iomem_dist, -1, KVM_DEV_ARM_VGIC_GRP_ADDR,
>                              KVM_VGIC_V3_ADDR_TYPE_DIST, s->dev_fd, 0);
> -    kvm_arm_register_device(&s->iomem_redist[0], -1,
> -                            KVM_DEV_ARM_VGIC_GRP_ADDR,
> -                            KVM_VGIC_V3_ADDR_TYPE_REDIST, s->dev_fd, 0);
> +
> +    if (!multiple_redist_region_allowed) {
> +        kvm_arm_register_device(&s->iomem_redist[0], -1,
> +                                KVM_DEV_ARM_VGIC_GRP_ADDR,
> +                                KVM_VGIC_V3_ADDR_TYPE_REDIST, s->dev_fd, 0);
> +    } else {
> +        /* we register regions in reverse order as "devices" are inserted at

Capital w on we would be nicer, but probably not worth respinning for

> +         * the head of a QSLIST and the list is then popped from the head
> +         * onwards by kvm_arm_machine_init_done()
> +         */
> +        for (i = s->nb_redist_regions - 1; i >= 0; i--) {
> +            /* Address mask made of the rdist region index and count */
> +            uint64_t addr_ormask =
> +                        i | ((uint64_t)s->redist_region_count[i] << 52);
> +
> +            kvm_arm_register_device(&s->iomem_redist[i], -1,
> +                                    KVM_DEV_ARM_VGIC_GRP_ADDR,
> +                                    KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION,
> +                                    s->dev_fd, addr_ormask);
> +        }
> +    }
>  
>      if (kvm_has_gsi_routing()) {
>          /* set up irq routing */
> -- 
> 2.5.5
>

Reviewed-by: Andrew Jones <drjones@redhat.com>

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [Qemu-devel] [PATCH v2 03/11] hw/intc/arm_gicv3: Introduce redist-region-count array property
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 03/11] hw/intc/arm_gicv3: Introduce redist-region-count array property Eric Auger
@ 2018-06-15 15:27   ` Andrew Jones
  0 siblings, 0 replies; 23+ messages in thread
From: Andrew Jones @ 2018-06-15 15:27 UTC (permalink / raw)
  To: Eric Auger
  Cc: eric.auger.pro, qemu-devel, qemu-arm, peter.maydell, wei,
	zhaoshenglong, lersek, ard.biesheuvel, christoffer.dall,
	marc.zyngier

On Fri, Jun 15, 2018 at 04:28:22PM +0200, Eric Auger wrote:
> To prepare for multiple redistributor regions, we introduce
> an array of uint32_t properties that stores the redistributor
> count of each redistributor region.
> 
> Non accelerated VGICv3 only supports a single redistributor region.
> The capacity of all redist regions is checked against the number of
> vcpus.
> 
> Machvirt is updated to set those properties, ie. a single
> redistributor region with count set to the number of vcpus
> capped by 123.
> 
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> 
> ---
> v2 -> v3:
> - add missing return in arm_gic_realize
> - in gicv3_init_irqs_and_mmio, compute/check rdist_capacity first
> - rdist region 0 size set to MIN(smp_cpus, redist0_capacity)
> - add GICV3_REDIST_SIZE
> ---
>  hw/arm/virt.c                      | 11 ++++++++++-
>  hw/intc/arm_gicv3.c                | 12 +++++++++++-
>  hw/intc/arm_gicv3_common.c         | 38 +++++++++++++++++++++++++++++++++-----
>  hw/intc/arm_gicv3_kvm.c            |  9 +++++++--
>  include/hw/intc/arm_gicv3_common.h |  8 ++++++--
>  5 files changed, 67 insertions(+), 11 deletions(-)
>

Reviewed-by: Andrew Jones <drjones@redhat.com>

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [Qemu-devel] [PATCH v2 05/11] hw/arm/virt: GICv3 DT node with one or two redistributor regions
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 05/11] hw/arm/virt: GICv3 DT node with one or two redistributor regions Eric Auger
@ 2018-06-19 18:53   ` Laszlo Ersek
  2018-06-19 19:02     ` Ard Biesheuvel
  0 siblings, 1 reply; 23+ messages in thread
From: Laszlo Ersek @ 2018-06-19 18:53 UTC (permalink / raw)
  To: Eric Auger, ard.biesheuvel
  Cc: eric.auger.pro, qemu-devel, qemu-arm, peter.maydell, wei,
	drjones, marc.zyngier, christoffer.dall, zhaoshenglong

Hi Eric,

sorry about the late followup. I have one question (mainly for Ard):

On 06/15/18 16:28, Eric Auger wrote:
> This patch allows the creation of a GICv3 node with 1 or 2
> redistributor regions depending on the number of smu_cpus.
> The second redistributor region is located just after the
> existing RAM region, at 256GB and contains up to up to 512 vcpus.
>
> Please refer to kernel documentation for further node details:
> Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> Reviewed-by: Andrew Jones <drjones@redhat.com>
>
> ---
> v1 (virt3.0) -> v2
> - Added Drew's R-b
>
> v2 -> v3:
> - VIRT_GIC_REDIST2 is now 64MB large, ie. 512 redistributor capacity
> - virt_gicv3_redist_region_count does not test kvm_irqchip_in_kernel
>   anymore
> ---
>  hw/arm/virt.c         | 29 ++++++++++++++++++++++++-----
>  include/hw/arm/virt.h | 14 ++++++++++++++
>  2 files changed, 38 insertions(+), 5 deletions(-)
>
> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> index 2885d18..d9f72eb 100644
> --- a/hw/arm/virt.c
> +++ b/hw/arm/virt.c
> @@ -148,6 +148,8 @@ static const MemMapEntry a15memmap[] = {
>      [VIRT_PCIE_PIO] =           { 0x3eff0000, 0x00010000 },
>      [VIRT_PCIE_ECAM] =          { 0x3f000000, 0x01000000 },
>      [VIRT_MEM] =                { 0x40000000, RAMLIMIT_BYTES },
> +    /* Additional 64 MB redist region (can contain up to 512 redistributors) */
> +    [VIRT_GIC_REDIST2] =        { 0x4000000000ULL, 0x4000000 },
>      /* Second PCIe window, 512GB wide at the 512GB boundary */
>      [VIRT_PCIE_MMIO_HIGH] =   { 0x8000000000ULL, 0x8000000000ULL },
>  };
> @@ -401,13 +403,30 @@ static void fdt_add_gic_node(VirtMachineState *vms)
>      qemu_fdt_setprop_cell(vms->fdt, "/intc", "#size-cells", 0x2);
>      qemu_fdt_setprop(vms->fdt, "/intc", "ranges", NULL, 0);
>      if (vms->gic_version == 3) {
> +        int nb_redist_regions = virt_gicv3_redist_region_count(vms);
> +
>          qemu_fdt_setprop_string(vms->fdt, "/intc", "compatible",
>                                  "arm,gic-v3");
> -        qemu_fdt_setprop_sized_cells(vms->fdt, "/intc", "reg",
> -                                     2, vms->memmap[VIRT_GIC_DIST].base,
> -                                     2, vms->memmap[VIRT_GIC_DIST].size,
> -                                     2, vms->memmap[VIRT_GIC_REDIST].base,
> -                                     2, vms->memmap[VIRT_GIC_REDIST].size);
> +
> +        qemu_fdt_setprop_cell(vms->fdt, "/intc",
> +                              "#redistributor-regions", nb_redist_regions);
> +
> +        if (nb_redist_regions == 1) {
> +            qemu_fdt_setprop_sized_cells(vms->fdt, "/intc", "reg",
> +                                         2, vms->memmap[VIRT_GIC_DIST].base,
> +                                         2, vms->memmap[VIRT_GIC_DIST].size,
> +                                         2, vms->memmap[VIRT_GIC_REDIST].base,
> +                                         2, vms->memmap[VIRT_GIC_REDIST].size);
> +        } else {
> +            qemu_fdt_setprop_sized_cells(vms->fdt, "/intc", "reg",
> +                                         2, vms->memmap[VIRT_GIC_DIST].base,
> +                                         2, vms->memmap[VIRT_GIC_DIST].size,
> +                                         2, vms->memmap[VIRT_GIC_REDIST].base,
> +                                         2, vms->memmap[VIRT_GIC_REDIST].size,
> +                                         2, vms->memmap[VIRT_GIC_REDIST2].base,
> +                                         2, vms->memmap[VIRT_GIC_REDIST2].size);
> +        }
> +
>          if (vms->virt) {
>              qemu_fdt_setprop_cells(vms->fdt, "/intc", "interrupts",
>                                     GIC_FDT_IRQ_TYPE_PPI, ARCH_GICV3_MAINT_IRQ,

In edk2, we have the following code in
"ArmVirtPkg/Library/ArmVirtGicArchLib/ArmVirtGicArchLib.c":

  switch (GicRevision) {

  case 3:
    //
    // The GIC v3 DT binding describes a series of at least 3 physical (base
    // addresses, size) pairs: the distributor interface (GICD), at least one
    // redistributor region (GICR) containing dedicated redistributor
    // interfaces for all individual CPUs, and the CPU interface (GICC).
    // Under virtualization, we assume that the first redistributor region
    // listed covers the boot CPU. Also, our GICv3 driver only supports the
    // system register CPU interface, so we can safely ignore the MMIO version
    // which is listed after the sequence of redistributor interfaces.
    // This means we are only interested in the first two memory regions
    // supplied, and ignore everything else.
    //
    ASSERT (RegSize >= 32);

    // RegProp[0..1] == { GICD base, GICD size }
    DistBase = SwapBytes64 (Reg[0]);
    ASSERT (DistBase < MAX_UINTN);

    // RegProp[2..3] == { GICR base, GICR size }
    RedistBase = SwapBytes64 (Reg[2]);
    ASSERT (RedistBase < MAX_UINTN);

    PcdStatus = PcdSet64S (PcdGicDistributorBase, DistBase);
    ASSERT_RETURN_ERROR (PcdStatus);
    PcdStatus = PcdSet64S (PcdGicRedistributorsBase, RedistBase);
    ASSERT_RETURN_ERROR (PcdStatus);

    DEBUG ((EFI_D_INFO, "Found GIC v3 (re)distributor @ 0x%Lx (0x%Lx)\n",
      DistBase, RedistBase));

(I vaguely recall that I may have quoted this already, but I'm not
sure.)

My understanding is that this will continue working -- as long as we
don't want to boot up multiple CPUs in UEFI, anyway.

I think we have no plans for adding (well, for implementing)
EFI_PEI_MP_SERVICES_PPI or EFI_MP_SERVICES_PROTOCOL for ARM -- do you
agree, Ard?

(That PPI and PROTOCOL are totally unnecessary unless it is
architecturally required to configure various CPU features on each
(V)CPU *individually*, before leaving the firmware. Unfortunately, x86
has some CPU warts like that, and said PPI and PROTOCOL implementations
for x86 have given us a good dose of grief over time. I very much hope
the PPI and the PROTOCOL will never be needed on ARM!)


Eric, I reckon you tested this patch with the ArmVirtQemu firmware too,
covering both branches of the code, and with multiple VCPUs in both
cases. Can you please confirm?

Thanks
Laszlo

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [Qemu-devel] [PATCH v2 05/11] hw/arm/virt: GICv3 DT node with one or two redistributor regions
  2018-06-19 18:53   ` Laszlo Ersek
@ 2018-06-19 19:02     ` Ard Biesheuvel
  2018-06-20  7:10       ` Auger Eric
  0 siblings, 1 reply; 23+ messages in thread
From: Ard Biesheuvel @ 2018-06-19 19:02 UTC (permalink / raw)
  To: Laszlo Ersek
  Cc: Eric Auger, Eric Auger, QEMU Developers, qemu-arm, Peter Maydell,
	Wei Huang, Andrew Jones, Marc Zyngier, Christoffer Dall,
	Shannon Zhao

On 19 June 2018 at 20:53, Laszlo Ersek <lersek@redhat.com> wrote:
> Hi Eric,
>
> sorry about the late followup. I have one question (mainly for Ard):
>
> On 06/15/18 16:28, Eric Auger wrote:
>> This patch allows the creation of a GICv3 node with 1 or 2
>> redistributor regions depending on the number of smu_cpus.
>> The second redistributor region is located just after the
>> existing RAM region, at 256GB and contains up to up to 512 vcpus.
>>
>> Please refer to kernel documentation for further node details:
>> Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>> Reviewed-by: Andrew Jones <drjones@redhat.com>
>>
>> ---
>> v1 (virt3.0) -> v2
>> - Added Drew's R-b
>>
>> v2 -> v3:
>> - VIRT_GIC_REDIST2 is now 64MB large, ie. 512 redistributor capacity
>> - virt_gicv3_redist_region_count does not test kvm_irqchip_in_kernel
>>   anymore
>> ---
>>  hw/arm/virt.c         | 29 ++++++++++++++++++++++++-----
>>  include/hw/arm/virt.h | 14 ++++++++++++++
>>  2 files changed, 38 insertions(+), 5 deletions(-)
>>
>> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
>> index 2885d18..d9f72eb 100644
>> --- a/hw/arm/virt.c
>> +++ b/hw/arm/virt.c
>> @@ -148,6 +148,8 @@ static const MemMapEntry a15memmap[] = {
>>      [VIRT_PCIE_PIO] =           { 0x3eff0000, 0x00010000 },
>>      [VIRT_PCIE_ECAM] =          { 0x3f000000, 0x01000000 },
>>      [VIRT_MEM] =                { 0x40000000, RAMLIMIT_BYTES },
>> +    /* Additional 64 MB redist region (can contain up to 512 redistributors) */
>> +    [VIRT_GIC_REDIST2] =        { 0x4000000000ULL, 0x4000000 },
>>      /* Second PCIe window, 512GB wide at the 512GB boundary */
>>      [VIRT_PCIE_MMIO_HIGH] =   { 0x8000000000ULL, 0x8000000000ULL },
>>  };
>> @@ -401,13 +403,30 @@ static void fdt_add_gic_node(VirtMachineState *vms)
>>      qemu_fdt_setprop_cell(vms->fdt, "/intc", "#size-cells", 0x2);
>>      qemu_fdt_setprop(vms->fdt, "/intc", "ranges", NULL, 0);
>>      if (vms->gic_version == 3) {
>> +        int nb_redist_regions = virt_gicv3_redist_region_count(vms);
>> +
>>          qemu_fdt_setprop_string(vms->fdt, "/intc", "compatible",
>>                                  "arm,gic-v3");
>> -        qemu_fdt_setprop_sized_cells(vms->fdt, "/intc", "reg",
>> -                                     2, vms->memmap[VIRT_GIC_DIST].base,
>> -                                     2, vms->memmap[VIRT_GIC_DIST].size,
>> -                                     2, vms->memmap[VIRT_GIC_REDIST].base,
>> -                                     2, vms->memmap[VIRT_GIC_REDIST].size);
>> +
>> +        qemu_fdt_setprop_cell(vms->fdt, "/intc",
>> +                              "#redistributor-regions", nb_redist_regions);
>> +
>> +        if (nb_redist_regions == 1) {
>> +            qemu_fdt_setprop_sized_cells(vms->fdt, "/intc", "reg",
>> +                                         2, vms->memmap[VIRT_GIC_DIST].base,
>> +                                         2, vms->memmap[VIRT_GIC_DIST].size,
>> +                                         2, vms->memmap[VIRT_GIC_REDIST].base,
>> +                                         2, vms->memmap[VIRT_GIC_REDIST].size);
>> +        } else {
>> +            qemu_fdt_setprop_sized_cells(vms->fdt, "/intc", "reg",
>> +                                         2, vms->memmap[VIRT_GIC_DIST].base,
>> +                                         2, vms->memmap[VIRT_GIC_DIST].size,
>> +                                         2, vms->memmap[VIRT_GIC_REDIST].base,
>> +                                         2, vms->memmap[VIRT_GIC_REDIST].size,
>> +                                         2, vms->memmap[VIRT_GIC_REDIST2].base,
>> +                                         2, vms->memmap[VIRT_GIC_REDIST2].size);
>> +        }
>> +
>>          if (vms->virt) {
>>              qemu_fdt_setprop_cells(vms->fdt, "/intc", "interrupts",
>>                                     GIC_FDT_IRQ_TYPE_PPI, ARCH_GICV3_MAINT_IRQ,
>
> In edk2, we have the following code in
> "ArmVirtPkg/Library/ArmVirtGicArchLib/ArmVirtGicArchLib.c":
>
>   switch (GicRevision) {
>
>   case 3:
>     //
>     // The GIC v3 DT binding describes a series of at least 3 physical (base
>     // addresses, size) pairs: the distributor interface (GICD), at least one
>     // redistributor region (GICR) containing dedicated redistributor
>     // interfaces for all individual CPUs, and the CPU interface (GICC).
>     // Under virtualization, we assume that the first redistributor region
>     // listed covers the boot CPU. Also, our GICv3 driver only supports the
>     // system register CPU interface, so we can safely ignore the MMIO version
>     // which is listed after the sequence of redistributor interfaces.
>     // This means we are only interested in the first two memory regions
>     // supplied, and ignore everything else.
>     //
>     ASSERT (RegSize >= 32);
>
>     // RegProp[0..1] == { GICD base, GICD size }
>     DistBase = SwapBytes64 (Reg[0]);
>     ASSERT (DistBase < MAX_UINTN);
>
>     // RegProp[2..3] == { GICR base, GICR size }
>     RedistBase = SwapBytes64 (Reg[2]);
>     ASSERT (RedistBase < MAX_UINTN);
>
>     PcdStatus = PcdSet64S (PcdGicDistributorBase, DistBase);
>     ASSERT_RETURN_ERROR (PcdStatus);
>     PcdStatus = PcdSet64S (PcdGicRedistributorsBase, RedistBase);
>     ASSERT_RETURN_ERROR (PcdStatus);
>
>     DEBUG ((EFI_D_INFO, "Found GIC v3 (re)distributor @ 0x%Lx (0x%Lx)\n",
>       DistBase, RedistBase));
>
> (I vaguely recall that I may have quoted this already, but I'm not
> sure.)
>
> My understanding is that this will continue working -- as long as we
> don't want to boot up multiple CPUs in UEFI, anyway.
>
> I think we have no plans for adding (well, for implementing)
> EFI_PEI_MP_SERVICES_PPI or EFI_MP_SERVICES_PROTOCOL for ARM -- do you
> agree, Ard?
>

Yes I agree. Only the boot cpu should enter UEFI, and although there
are some remnants of multi core ARM stuff in the EDK2 tree, I would
prefer to get rid of that as soon as we can.

> (That PPI and PROTOCOL are totally unnecessary unless it is
> architecturally required to configure various CPU features on each
> (V)CPU *individually*, before leaving the firmware. Unfortunately, x86
> has some CPU warts like that, and said PPI and PROTOCOL implementations
> for x86 have given us a good dose of grief over time. I very much hope
> the PPI and the PROTOCOL will never be needed on ARM!)
>

That kind of configuration typically occurs in the secure firmware on
an ARM system, and not at all on a virtual machine.

>
> Eric, I reckon you tested this patch with the ArmVirtQemu firmware too,
> covering both branches of the code, and with multiple VCPUs in both
> cases. Can you please confirm?
>
> Thanks
> Laszlo

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [Qemu-devel] [PATCH v2 05/11] hw/arm/virt: GICv3 DT node with one or two redistributor regions
  2018-06-19 19:02     ` Ard Biesheuvel
@ 2018-06-20  7:10       ` Auger Eric
  2018-06-20 10:38         ` Laszlo Ersek
  0 siblings, 1 reply; 23+ messages in thread
From: Auger Eric @ 2018-06-20  7:10 UTC (permalink / raw)
  To: Ard Biesheuvel, Laszlo Ersek
  Cc: Eric Auger, QEMU Developers, qemu-arm, Peter Maydell, Wei Huang,
	Andrew Jones, Marc Zyngier, Christoffer Dall, Shannon Zhao

Hi Laszlo,

On 06/19/2018 09:02 PM, Ard Biesheuvel wrote:
> On 19 June 2018 at 20:53, Laszlo Ersek <lersek@redhat.com> wrote:
>> Hi Eric,
>>
>> sorry about the late followup. I have one question (mainly for Ard):
>>
>> On 06/15/18 16:28, Eric Auger wrote:
>>> This patch allows the creation of a GICv3 node with 1 or 2
>>> redistributor regions depending on the number of smu_cpus.
>>> The second redistributor region is located just after the
>>> existing RAM region, at 256GB and contains up to up to 512 vcpus.
>>>
>>> Please refer to kernel documentation for further node details:
>>> Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
>>>
>>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>> Reviewed-by: Andrew Jones <drjones@redhat.com>
>>>
>>> ---
>>> v1 (virt3.0) -> v2
>>> - Added Drew's R-b
>>>
>>> v2 -> v3:
>>> - VIRT_GIC_REDIST2 is now 64MB large, ie. 512 redistributor capacity
>>> - virt_gicv3_redist_region_count does not test kvm_irqchip_in_kernel
>>>   anymore
>>> ---
>>>  hw/arm/virt.c         | 29 ++++++++++++++++++++++++-----
>>>  include/hw/arm/virt.h | 14 ++++++++++++++
>>>  2 files changed, 38 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
>>> index 2885d18..d9f72eb 100644
>>> --- a/hw/arm/virt.c
>>> +++ b/hw/arm/virt.c
>>> @@ -148,6 +148,8 @@ static const MemMapEntry a15memmap[] = {
>>>      [VIRT_PCIE_PIO] =           { 0x3eff0000, 0x00010000 },
>>>      [VIRT_PCIE_ECAM] =          { 0x3f000000, 0x01000000 },
>>>      [VIRT_MEM] =                { 0x40000000, RAMLIMIT_BYTES },
>>> +    /* Additional 64 MB redist region (can contain up to 512 redistributors) */
>>> +    [VIRT_GIC_REDIST2] =        { 0x4000000000ULL, 0x4000000 },
>>>      /* Second PCIe window, 512GB wide at the 512GB boundary */
>>>      [VIRT_PCIE_MMIO_HIGH] =   { 0x8000000000ULL, 0x8000000000ULL },
>>>  };
>>> @@ -401,13 +403,30 @@ static void fdt_add_gic_node(VirtMachineState *vms)
>>>      qemu_fdt_setprop_cell(vms->fdt, "/intc", "#size-cells", 0x2);
>>>      qemu_fdt_setprop(vms->fdt, "/intc", "ranges", NULL, 0);
>>>      if (vms->gic_version == 3) {
>>> +        int nb_redist_regions = virt_gicv3_redist_region_count(vms);
>>> +
>>>          qemu_fdt_setprop_string(vms->fdt, "/intc", "compatible",
>>>                                  "arm,gic-v3");
>>> -        qemu_fdt_setprop_sized_cells(vms->fdt, "/intc", "reg",
>>> -                                     2, vms->memmap[VIRT_GIC_DIST].base,
>>> -                                     2, vms->memmap[VIRT_GIC_DIST].size,
>>> -                                     2, vms->memmap[VIRT_GIC_REDIST].base,
>>> -                                     2, vms->memmap[VIRT_GIC_REDIST].size);
>>> +
>>> +        qemu_fdt_setprop_cell(vms->fdt, "/intc",
>>> +                              "#redistributor-regions", nb_redist_regions);
>>> +
>>> +        if (nb_redist_regions == 1) {
>>> +            qemu_fdt_setprop_sized_cells(vms->fdt, "/intc", "reg",
>>> +                                         2, vms->memmap[VIRT_GIC_DIST].base,
>>> +                                         2, vms->memmap[VIRT_GIC_DIST].size,
>>> +                                         2, vms->memmap[VIRT_GIC_REDIST].base,
>>> +                                         2, vms->memmap[VIRT_GIC_REDIST].size);
>>> +        } else {
>>> +            qemu_fdt_setprop_sized_cells(vms->fdt, "/intc", "reg",
>>> +                                         2, vms->memmap[VIRT_GIC_DIST].base,
>>> +                                         2, vms->memmap[VIRT_GIC_DIST].size,
>>> +                                         2, vms->memmap[VIRT_GIC_REDIST].base,
>>> +                                         2, vms->memmap[VIRT_GIC_REDIST].size,
>>> +                                         2, vms->memmap[VIRT_GIC_REDIST2].base,
>>> +                                         2, vms->memmap[VIRT_GIC_REDIST2].size);
>>> +        }
>>> +
>>>          if (vms->virt) {
>>>              qemu_fdt_setprop_cells(vms->fdt, "/intc", "interrupts",
>>>                                     GIC_FDT_IRQ_TYPE_PPI, ARCH_GICV3_MAINT_IRQ,
>>
>> In edk2, we have the following code in
>> "ArmVirtPkg/Library/ArmVirtGicArchLib/ArmVirtGicArchLib.c":
>>
>>   switch (GicRevision) {
>>
>>   case 3:
>>     //
>>     // The GIC v3 DT binding describes a series of at least 3 physical (base
>>     // addresses, size) pairs: the distributor interface (GICD), at least one
>>     // redistributor region (GICR) containing dedicated redistributor
>>     // interfaces for all individual CPUs, and the CPU interface (GICC).
>>     // Under virtualization, we assume that the first redistributor region
>>     // listed covers the boot CPU. Also, our GICv3 driver only supports the
>>     // system register CPU interface, so we can safely ignore the MMIO version
>>     // which is listed after the sequence of redistributor interfaces.
>>     // This means we are only interested in the first two memory regions
>>     // supplied, and ignore everything else.
>>     //
>>     ASSERT (RegSize >= 32);
>>
>>     // RegProp[0..1] == { GICD base, GICD size }
>>     DistBase = SwapBytes64 (Reg[0]);
>>     ASSERT (DistBase < MAX_UINTN);
>>
>>     // RegProp[2..3] == { GICR base, GICR size }
>>     RedistBase = SwapBytes64 (Reg[2]);
>>     ASSERT (RedistBase < MAX_UINTN);
>>
>>     PcdStatus = PcdSet64S (PcdGicDistributorBase, DistBase);
>>     ASSERT_RETURN_ERROR (PcdStatus);
>>     PcdStatus = PcdSet64S (PcdGicRedistributorsBase, RedistBase);
>>     ASSERT_RETURN_ERROR (PcdStatus);
>>
>>     DEBUG ((EFI_D_INFO, "Found GIC v3 (re)distributor @ 0x%Lx (0x%Lx)\n",
>>       DistBase, RedistBase));
>>
>> (I vaguely recall that I may have quoted this already, but I'm not
>> sure.)
I don't remember you did ;-)
>>
>> My understanding is that this will continue working -- as long as we
>> don't want to boot up multiple CPUs in UEFI, anyway.
>>
>> I think we have no plans for adding (well, for implementing)
>> EFI_PEI_MP_SERVICES_PPI or EFI_MP_SERVICES_PROTOCOL for ARM -- do you
>> agree, Ard?
>>
> 
> Yes I agree. Only the boot cpu should enter UEFI, and although there
> are some remnants of multi core ARM stuff in the EDK2 tree, I would
> prefer to get rid of that as soon as we can.
> 
>> (That PPI and PROTOCOL are totally unnecessary unless it is
>> architecturally required to configure various CPU features on each
>> (V)CPU *individually*, before leaving the firmware. Unfortunately, x86
>> has some CPU warts like that, and said PPI and PROTOCOL implementations
>> for x86 have given us a good dose of grief over time. I very much hope
>> the PPI and the PROTOCOL will never be needed on ARM!)
>>
> 
> That kind of configuration typically occurs in the secure firmware on
> an ARM system, and not at all on a virtual machine.
> 
>>
>> Eric, I reckon you tested this patch with the ArmVirtQemu firmware too,
>> covering both branches of the code, and with multiple VCPUs in both
>> cases. Can you please confirm?
I tested the high ECAM part with both 32b and 64b FW. However the
multiple redistributor region part only was tested with 64b FW. The
reason is Moonshot which I used to test accelerated aarch32 guest has a
GICv2 so I can't test the GICv3 redist part on it.

However if my understanding is correct as the above check only is done
on the 1st redist, which is located in the low redist region, this
series shouldn't introduce any regression.

Thanks

Eric
>>
>> Thanks
>> Laszlo

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [Qemu-devel] [PATCH v2 05/11] hw/arm/virt: GICv3 DT node with one or two redistributor regions
  2018-06-20  7:10       ` Auger Eric
@ 2018-06-20 10:38         ` Laszlo Ersek
  0 siblings, 0 replies; 23+ messages in thread
From: Laszlo Ersek @ 2018-06-20 10:38 UTC (permalink / raw)
  To: Auger Eric, Ard Biesheuvel
  Cc: Eric Auger, QEMU Developers, qemu-arm, Peter Maydell, Wei Huang,
	Andrew Jones, Marc Zyngier, Christoffer Dall, Shannon Zhao

On 06/20/18 09:10, Auger Eric wrote:
> Hi Laszlo,
> 
> On 06/19/2018 09:02 PM, Ard Biesheuvel wrote:
>> On 19 June 2018 at 20:53, Laszlo Ersek <lersek@redhat.com> wrote:
>>> Hi Eric,
>>>
>>> sorry about the late followup. I have one question (mainly for Ard):
>>>
>>> On 06/15/18 16:28, Eric Auger wrote:
>>>> This patch allows the creation of a GICv3 node with 1 or 2
>>>> redistributor regions depending on the number of smu_cpus.
>>>> The second redistributor region is located just after the
>>>> existing RAM region, at 256GB and contains up to up to 512 vcpus.
>>>>
>>>> Please refer to kernel documentation for further node details:
>>>> Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
>>>>
>>>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>>> Reviewed-by: Andrew Jones <drjones@redhat.com>
>>>>
>>>> ---
>>>> v1 (virt3.0) -> v2
>>>> - Added Drew's R-b
>>>>
>>>> v2 -> v3:
>>>> - VIRT_GIC_REDIST2 is now 64MB large, ie. 512 redistributor capacity
>>>> - virt_gicv3_redist_region_count does not test kvm_irqchip_in_kernel
>>>>   anymore
>>>> ---
>>>>  hw/arm/virt.c         | 29 ++++++++++++++++++++++++-----
>>>>  include/hw/arm/virt.h | 14 ++++++++++++++
>>>>  2 files changed, 38 insertions(+), 5 deletions(-)
>>>>
>>>> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
>>>> index 2885d18..d9f72eb 100644
>>>> --- a/hw/arm/virt.c
>>>> +++ b/hw/arm/virt.c
>>>> @@ -148,6 +148,8 @@ static const MemMapEntry a15memmap[] = {
>>>>      [VIRT_PCIE_PIO] =           { 0x3eff0000, 0x00010000 },
>>>>      [VIRT_PCIE_ECAM] =          { 0x3f000000, 0x01000000 },
>>>>      [VIRT_MEM] =                { 0x40000000, RAMLIMIT_BYTES },
>>>> +    /* Additional 64 MB redist region (can contain up to 512 redistributors) */
>>>> +    [VIRT_GIC_REDIST2] =        { 0x4000000000ULL, 0x4000000 },
>>>>      /* Second PCIe window, 512GB wide at the 512GB boundary */
>>>>      [VIRT_PCIE_MMIO_HIGH] =   { 0x8000000000ULL, 0x8000000000ULL },
>>>>  };
>>>> @@ -401,13 +403,30 @@ static void fdt_add_gic_node(VirtMachineState *vms)
>>>>      qemu_fdt_setprop_cell(vms->fdt, "/intc", "#size-cells", 0x2);
>>>>      qemu_fdt_setprop(vms->fdt, "/intc", "ranges", NULL, 0);
>>>>      if (vms->gic_version == 3) {
>>>> +        int nb_redist_regions = virt_gicv3_redist_region_count(vms);
>>>> +
>>>>          qemu_fdt_setprop_string(vms->fdt, "/intc", "compatible",
>>>>                                  "arm,gic-v3");
>>>> -        qemu_fdt_setprop_sized_cells(vms->fdt, "/intc", "reg",
>>>> -                                     2, vms->memmap[VIRT_GIC_DIST].base,
>>>> -                                     2, vms->memmap[VIRT_GIC_DIST].size,
>>>> -                                     2, vms->memmap[VIRT_GIC_REDIST].base,
>>>> -                                     2, vms->memmap[VIRT_GIC_REDIST].size);
>>>> +
>>>> +        qemu_fdt_setprop_cell(vms->fdt, "/intc",
>>>> +                              "#redistributor-regions", nb_redist_regions);
>>>> +
>>>> +        if (nb_redist_regions == 1) {
>>>> +            qemu_fdt_setprop_sized_cells(vms->fdt, "/intc", "reg",
>>>> +                                         2, vms->memmap[VIRT_GIC_DIST].base,
>>>> +                                         2, vms->memmap[VIRT_GIC_DIST].size,
>>>> +                                         2, vms->memmap[VIRT_GIC_REDIST].base,
>>>> +                                         2, vms->memmap[VIRT_GIC_REDIST].size);
>>>> +        } else {
>>>> +            qemu_fdt_setprop_sized_cells(vms->fdt, "/intc", "reg",
>>>> +                                         2, vms->memmap[VIRT_GIC_DIST].base,
>>>> +                                         2, vms->memmap[VIRT_GIC_DIST].size,
>>>> +                                         2, vms->memmap[VIRT_GIC_REDIST].base,
>>>> +                                         2, vms->memmap[VIRT_GIC_REDIST].size,
>>>> +                                         2, vms->memmap[VIRT_GIC_REDIST2].base,
>>>> +                                         2, vms->memmap[VIRT_GIC_REDIST2].size);
>>>> +        }
>>>> +
>>>>          if (vms->virt) {
>>>>              qemu_fdt_setprop_cells(vms->fdt, "/intc", "interrupts",
>>>>                                     GIC_FDT_IRQ_TYPE_PPI, ARCH_GICV3_MAINT_IRQ,
>>>
>>> In edk2, we have the following code in
>>> "ArmVirtPkg/Library/ArmVirtGicArchLib/ArmVirtGicArchLib.c":
>>>
>>>   switch (GicRevision) {
>>>
>>>   case 3:
>>>     //
>>>     // The GIC v3 DT binding describes a series of at least 3 physical (base
>>>     // addresses, size) pairs: the distributor interface (GICD), at least one
>>>     // redistributor region (GICR) containing dedicated redistributor
>>>     // interfaces for all individual CPUs, and the CPU interface (GICC).
>>>     // Under virtualization, we assume that the first redistributor region
>>>     // listed covers the boot CPU. Also, our GICv3 driver only supports the
>>>     // system register CPU interface, so we can safely ignore the MMIO version
>>>     // which is listed after the sequence of redistributor interfaces.
>>>     // This means we are only interested in the first two memory regions
>>>     // supplied, and ignore everything else.
>>>     //
>>>     ASSERT (RegSize >= 32);
>>>
>>>     // RegProp[0..1] == { GICD base, GICD size }
>>>     DistBase = SwapBytes64 (Reg[0]);
>>>     ASSERT (DistBase < MAX_UINTN);
>>>
>>>     // RegProp[2..3] == { GICR base, GICR size }
>>>     RedistBase = SwapBytes64 (Reg[2]);
>>>     ASSERT (RedistBase < MAX_UINTN);
>>>
>>>     PcdStatus = PcdSet64S (PcdGicDistributorBase, DistBase);
>>>     ASSERT_RETURN_ERROR (PcdStatus);
>>>     PcdStatus = PcdSet64S (PcdGicRedistributorsBase, RedistBase);
>>>     ASSERT_RETURN_ERROR (PcdStatus);
>>>
>>>     DEBUG ((EFI_D_INFO, "Found GIC v3 (re)distributor @ 0x%Lx (0x%Lx)\n",
>>>       DistBase, RedistBase));
>>>
>>> (I vaguely recall that I may have quoted this already, but I'm not
>>> sure.)
> I don't remember you did ;-)
>>>
>>> My understanding is that this will continue working -- as long as we
>>> don't want to boot up multiple CPUs in UEFI, anyway.
>>>
>>> I think we have no plans for adding (well, for implementing)
>>> EFI_PEI_MP_SERVICES_PPI or EFI_MP_SERVICES_PROTOCOL for ARM -- do you
>>> agree, Ard?
>>>
>>
>> Yes I agree. Only the boot cpu should enter UEFI, and although there
>> are some remnants of multi core ARM stuff in the EDK2 tree, I would
>> prefer to get rid of that as soon as we can.
>>
>>> (That PPI and PROTOCOL are totally unnecessary unless it is
>>> architecturally required to configure various CPU features on each
>>> (V)CPU *individually*, before leaving the firmware. Unfortunately, x86
>>> has some CPU warts like that, and said PPI and PROTOCOL implementations
>>> for x86 have given us a good dose of grief over time. I very much hope
>>> the PPI and the PROTOCOL will never be needed on ARM!)
>>>
>>
>> That kind of configuration typically occurs in the secure firmware on
>> an ARM system, and not at all on a virtual machine.
>>
>>>
>>> Eric, I reckon you tested this patch with the ArmVirtQemu firmware too,
>>> covering both branches of the code, and with multiple VCPUs in both
>>> cases. Can you please confirm?
> I tested the high ECAM part with both 32b and 64b FW. However the
> multiple redistributor region part only was tested with 64b FW. The
> reason is Moonshot which I used to test accelerated aarch32 guest has a
> GICv2 so I can't test the GICv3 redist part on it.
> 
> However if my understanding is correct as the above check only is done
> on the 1st redist, which is located in the low redist region, this
> series shouldn't introduce any regression.

I agree. Thank you!
Laszlo

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [Qemu-devel] [PATCH v2 01/11] linux-headers: Update to 4.18-rc0
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 01/11] linux-headers: Update to 4.18-rc0 Eric Auger
@ 2018-06-20 13:57   ` Peter Maydell
  2018-06-20 14:03     ` Auger Eric
  0 siblings, 1 reply; 23+ messages in thread
From: Peter Maydell @ 2018-06-20 13:57 UTC (permalink / raw)
  To: Eric Auger
  Cc: Eric Auger, QEMU Developers, qemu-arm, Andrew Jones, Wei Huang,
	Shannon Zhao, Laszlo Ersek, Ard Biesheuvel, Christoffer Dall,
	Marc Zyngier

On 15 June 2018 at 15:28, Eric Auger <eric.auger@redhat.com> wrote:
> commit b357bf6023a948cf6a9472f07a1b0caac0e4f8e8
> Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>

What's 4.18-rc0? The upstream kernel doesn't have a tag by that
name and looking at the Makefile the last thing before 4.18-rc1
was 4.17.

Still, since it is a commit on mainline I guess we're fine?

thanks
-- PMM

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [Qemu-devel] [PATCH v2 01/11] linux-headers: Update to 4.18-rc0
  2018-06-20 13:57   ` Peter Maydell
@ 2018-06-20 14:03     ` Auger Eric
  2018-06-20 14:06       ` Peter Maydell
  0 siblings, 1 reply; 23+ messages in thread
From: Auger Eric @ 2018-06-20 14:03 UTC (permalink / raw)
  To: Peter Maydell
  Cc: Wei Huang, Andrew Jones, Ard Biesheuvel, Marc Zyngier,
	QEMU Developers, Christoffer Dall, qemu-arm, Shannon Zhao,
	Laszlo Ersek, Eric Auger

Hi Peter,
On 06/20/2018 03:57 PM, Peter Maydell wrote:
> On 15 June 2018 at 15:28, Eric Auger <eric.auger@redhat.com> wrote:
>> commit b357bf6023a948cf6a9472f07a1b0caac0e4f8e8
>> Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> 
> What's 4.18-rc0? The upstream kernel doesn't have a tag by that
> name and looking at the Makefile the last thing before 4.18-rc1
> was 4.17.
> 
> Still, since it is a commit on mainline I guess we're fine?
Yes it is post 4.17 and before 4.18-rc1. I didn't know how to call that.

Thanks

Eric
> 
> thanks
> -- PMM
> 

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [Qemu-devel] [PATCH v2 01/11] linux-headers: Update to 4.18-rc0
  2018-06-20 14:03     ` Auger Eric
@ 2018-06-20 14:06       ` Peter Maydell
  2018-06-20 14:08         ` Auger Eric
  0 siblings, 1 reply; 23+ messages in thread
From: Peter Maydell @ 2018-06-20 14:06 UTC (permalink / raw)
  To: Auger Eric
  Cc: Wei Huang, Andrew Jones, Ard Biesheuvel, Marc Zyngier,
	QEMU Developers, Christoffer Dall, qemu-arm, Shannon Zhao,
	Laszlo Ersek, Eric Auger

On 20 June 2018 at 15:03, Auger Eric <eric.auger@redhat.com> wrote:
> Hi Peter,
> On 06/20/2018 03:57 PM, Peter Maydell wrote:
>> On 15 June 2018 at 15:28, Eric Auger <eric.auger@redhat.com> wrote:
>>> commit b357bf6023a948cf6a9472f07a1b0caac0e4f8e8
>>> Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm
>>>
>>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> What's 4.18-rc0? The upstream kernel doesn't have a tag by that
>> name and looking at the Makefile the last thing before 4.18-rc1
>> was 4.17.
>>
>> Still, since it is a commit on mainline I guess we're fine?
> Yes it is post 4.17 and before 4.18-rc1. I didn't know how to call that.

I've tweaked the commit message to read:
> linux-headers: Update to kernel mainline commit b357bf602
>
> Update our kernel headers to mainline commit
> b357bf6023a948cf6a9472f07a1b0caac0e4f8e8
> ("Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm")

thanks
-- PMM

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [Qemu-devel] [PATCH v2 01/11] linux-headers: Update to 4.18-rc0
  2018-06-20 14:06       ` Peter Maydell
@ 2018-06-20 14:08         ` Auger Eric
  0 siblings, 0 replies; 23+ messages in thread
From: Auger Eric @ 2018-06-20 14:08 UTC (permalink / raw)
  To: Peter Maydell
  Cc: Wei Huang, Andrew Jones, Ard Biesheuvel, Marc Zyngier,
	QEMU Developers, Christoffer Dall, qemu-arm, Shannon Zhao,
	Laszlo Ersek, Eric Auger

Hi Peter,

On 06/20/2018 04:06 PM, Peter Maydell wrote:
> On 20 June 2018 at 15:03, Auger Eric <eric.auger@redhat.com> wrote:
>> Hi Peter,
>> On 06/20/2018 03:57 PM, Peter Maydell wrote:
>>> On 15 June 2018 at 15:28, Eric Auger <eric.auger@redhat.com> wrote:
>>>> commit b357bf6023a948cf6a9472f07a1b0caac0e4f8e8
>>>> Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm
>>>>
>>>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>>
>>> What's 4.18-rc0? The upstream kernel doesn't have a tag by that
>>> name and looking at the Makefile the last thing before 4.18-rc1
>>> was 4.17.
>>>
>>> Still, since it is a commit on mainline I guess we're fine?
>> Yes it is post 4.17 and before 4.18-rc1. I didn't know how to call that.
> 
> I've tweaked the commit message to read:
>> linux-headers: Update to kernel mainline commit b357bf602
thank you!

Eric
>>
>> Update our kernel headers to mainline commit
>> b357bf6023a948cf6a9472f07a1b0caac0e4f8e8
>> ("Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm")
> 
> thanks
> -- PMM
> 

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [Qemu-devel] [PATCH v2 00/11] KVM/ARM: virt-3.0: Multiple redistributor regions and 256MB ECAM region
  2018-06-15 14:28 [Qemu-devel] [PATCH v2 00/11] KVM/ARM: virt-3.0: Multiple redistributor regions and 256MB ECAM region Eric Auger
                   ` (10 preceding siblings ...)
  2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 11/11] hw/arm/virt: Increase max_cpus to 512 Eric Auger
@ 2018-06-20 14:36 ` Peter Maydell
  11 siblings, 0 replies; 23+ messages in thread
From: Peter Maydell @ 2018-06-20 14:36 UTC (permalink / raw)
  To: Eric Auger
  Cc: Eric Auger, QEMU Developers, qemu-arm, Andrew Jones, Wei Huang,
	Shannon Zhao, Laszlo Ersek, Ard Biesheuvel, Christoffer Dall,
	Marc Zyngier

On 15 June 2018 at 15:28, Eric Auger <eric.auger@redhat.com> wrote:
> This series increases the number of vcpus usable in accelerated mode
> along with GICv3 and allows up to 256 PCIe busses.
>
> It is a combination of 2 series:
> [1] [RFC v3 0/8] KVM/ARM: Relax the max 123 vcpus limitation along
>     with KVM GICv3
> [2] [PATCH 0/2] ARM virt: Support up to 256 PCIe buses
>
> Both add features to the new 3.0 virt machine model.

Applied to target-arm.next, thanks.

-- PMM

^ permalink raw reply	[flat|nested] 23+ messages in thread

end of thread, other threads:[~2018-06-20 14:36 UTC | newest]

Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-06-15 14:28 [Qemu-devel] [PATCH v2 00/11] KVM/ARM: virt-3.0: Multiple redistributor regions and 256MB ECAM region Eric Auger
2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 01/11] linux-headers: Update to 4.18-rc0 Eric Auger
2018-06-20 13:57   ` Peter Maydell
2018-06-20 14:03     ` Auger Eric
2018-06-20 14:06       ` Peter Maydell
2018-06-20 14:08         ` Auger Eric
2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 02/11] target/arm: Allow KVM device address overwriting Eric Auger
2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 03/11] hw/intc/arm_gicv3: Introduce redist-region-count array property Eric Auger
2018-06-15 15:27   ` Andrew Jones
2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 04/11] hw/intc/arm_gicv3_kvm: Get prepared to handle multiple redist regions Eric Auger
2018-06-15 15:19   ` Andrew Jones
2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 05/11] hw/arm/virt: GICv3 DT node with one or two redistributor regions Eric Auger
2018-06-19 18:53   ` Laszlo Ersek
2018-06-19 19:02     ` Ard Biesheuvel
2018-06-20  7:10       ` Auger Eric
2018-06-20 10:38         ` Laszlo Ersek
2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 06/11] hw/arm/virt-acpi-build: Advertise one or two GICR structures Eric Auger
2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 07/11] hw/arm/virt: Register two redistributor regions when necessary Eric Auger
2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 08/11] hw/arm/virt: Add a new 256MB ECAM region Eric Auger
2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 09/11] hw/arm/virt: Add virt-3.0 machine type Eric Auger
2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 10/11] hw/arm/virt: Use 256MB ECAM region by default Eric Auger
2018-06-15 14:28 ` [Qemu-devel] [PATCH v2 11/11] hw/arm/virt: Increase max_cpus to 512 Eric Auger
2018-06-20 14:36 ` [Qemu-devel] [PATCH v2 00/11] KVM/ARM: virt-3.0: Multiple redistributor regions and 256MB ECAM region Peter Maydell

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.