[kvm-unit-tests PATCH v2 00/23] EFI and ACPI support for arm64

[kvm-unit-tests PATCH v2 00/23] EFI and ACPI support for arm64

[Nikos Nikoleris]

Hello,

This patch series adds initial support for building arm64 tests as EFI
tests and running them under QEMU. Much like x86_64 we import external
dependencies from gnu-efi and adopt them to work with types and other
assumptions from kvm-unit-tests. In addition, this series adds support
for discovering parts of the machine using ACPI.

The first set of patches moves the existing ACPI code to the common
lib path. Then, it extends definitions and functions to allow for more
robust discovery of ACPI tables. In arm64, we add support for setting
up the PSCI conduit, discovering the UART, timers and cpus via
ACPI. The code retains existing behavior and gives priority to
discovery through DT when one has been provided.

In the second set of patches, we add support for getting the command
line from the EFI shell. This is a requirement for many of the
existing arm64 tests.

In the third set of patches, we import code from gnu-efi, make minor
changes and add an alternative setup sequence from arm64 systems that
boot through EFI. Finally, we add support in the build system and a
run script which is used to run an EFI app.

After this set of patches one can build arm64 EFI tests:

$> ./configure --enable-efi
$> make

And use the run script to run an EFI tests:

$> ./arm/efi/run ./arm/selftest.efi -smp 2 -m 256 -append "setup smp=2 mem=256"

Or all tests:

$> ./run_tests.sh

There are a few items that this series does not address but they would
be useful to have:

* Support for booting the system from EL2. Currently, we assume that a
tests starts running at EL1. This the case when we run with EFI, it's
not always the case in hardware.

* Support for reading environment variables and populating __envp.

* Support for discovering the chr-testdev through ACPI.

PS: Apologies for the mess with v1. Due to a mistake in my git
send-email configuration some patches didn't make it to the list and
the right recipients.

Thanks,

Nikos

Andrew Jones (3):
  arm/arm64: mmu_disable: Clean and invalidate before disabling
  arm/arm64: Rename etext to _etext
  arm64: Add a new type of memory type flag MR_F_RESERVED

Nikos Nikoleris (20):
  lib: Move acpi header and implementation to lib
  lib: Ensure all struct definition for ACPI tables are packed
  lib: Add support for the XSDT ACPI table
  lib: Extend the definition of the ACPI table FADT
  arm/arm64: Add support for setting up the PSCI conduit through ACPI
  arm/arm64: Add support for discovering the UART through ACPI
  arm/arm64: Add support for timer initialization through ACPI
  arm/arm64: Add support for cpu initialization through ACPI
  lib/printf: Support for precision modifier in printing strings
  lib/printf: Add support for printing wide strings
  lib/efi: Add support for getting the cmdline
  lib: Avoid ms_abi for calls related to EFI on arm64
  arm/arm64: Add a setup sequence for systems that boot through EFI
  arm64: Copy code from GNU-EFI
  arm64: Change GNU-EFI imported file to use defined types
  arm64: Use code from the gnu-efi when booting with EFI
  lib: Avoid external dependency in libelf
  x86: Move x86_64-specific EFI CFLAGS to x86_64 Makefile
  arm64: Add support for efi in Makefile
  arm64: Add an efi/run script

 scripts/runtime.bash        |  14 +-
 arm/efi/run                 |  61 +++++++++
 arm/run                     |   8 +-
 configure                   |  15 ++-
 Makefile                    |   4 -
 arm/Makefile.arm            |   6 +
 arm/Makefile.arm64          |  18 ++-
 arm/Makefile.common         |  48 +++++--
 x86/Makefile.common         |   2 +-
 x86/Makefile.x86_64         |   4 +
 lib/linux/efi.h             |  44 ++++++
 lib/arm/asm/setup.h         |   3 +
 lib/arm/asm/timer.h         |   2 +
 lib/x86/asm/setup.h         |   2 +-
 lib/acpi.h                  | 260 ++++++++++++++++++++++++++++++++++++
 lib/stdlib.h                |   1 +
 lib/x86/acpi.h              | 112 ----------------
 lib/acpi.c                  | 124 +++++++++++++++++
 lib/arm/io.c                |  21 ++-
 lib/arm/mmu.c               |   4 +
 lib/arm/psci.c              |  25 +++-
 lib/arm/setup.c             | 247 +++++++++++++++++++++++++++-------
 lib/arm/timer.c             |  73 ++++++++++
 lib/devicetree.c            |   2 +-
 lib/efi.c                   | 123 +++++++++++++++++
 lib/printf.c                | 183 +++++++++++++++++++++++--
 lib/string.c                |   2 +-
 lib/x86/acpi.c              |  82 ------------
 arm/efi/elf_aarch64_efi.lds |  63 +++++++++
 arm/flat.lds                |   2 +-
 arm/cstart.S                |  29 +++-
 arm/cstart64.S              |  28 +++-
 arm/efi/crt0-efi-aarch64.S  | 143 ++++++++++++++++++++
 arm/dummy.c                 |   4 +
 arm/efi/reloc_aarch64.c     |  93 +++++++++++++
 x86/s3.c                    |  20 +--
 x86/vmexit.c                |   4 +-
 37 files changed, 1556 insertions(+), 320 deletions(-)
 create mode 100755 arm/efi/run
 create mode 100644 lib/acpi.h
 delete mode 100644 lib/x86/acpi.h
 create mode 100644 lib/acpi.c
 create mode 100644 lib/arm/timer.c
 delete mode 100644 lib/x86/acpi.c
 create mode 100644 arm/efi/elf_aarch64_efi.lds
 create mode 100644 arm/efi/crt0-efi-aarch64.S
 create mode 100644 arm/dummy.c
 create mode 100644 arm/efi/reloc_aarch64.c

-- 
2.25.1

[kvm-unit-tests PATCH v2 01/23] lib: Move acpi header and implementation to lib

[Nikos Nikoleris]

This change is in preparation of using ACPI in arm64 systems booting
with EFI.

Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
---
 x86/Makefile.common  | 2 +-
 lib/x86/asm/setup.h  | 2 +-
 lib/{x86 => }/acpi.h | 4 ++--
 lib/{x86 => }/acpi.c | 0
 x86/s3.c             | 2 +-
 x86/vmexit.c         | 2 +-
 6 files changed, 6 insertions(+), 6 deletions(-)
 rename lib/{x86 => }/acpi.h (99%)
 rename lib/{x86 => }/acpi.c (100%)

diff --git a/x86/Makefile.common b/x86/Makefile.common
index b903988..4cdba79 100644
--- a/x86/Makefile.common
+++ b/x86/Makefile.common
@@ -2,6 +2,7 @@
 
 all: directories test_cases
 
+cflatobjs += lib/acpi.o
 cflatobjs += lib/pci.o
 cflatobjs += lib/pci-edu.o
 cflatobjs += lib/alloc.o
@@ -18,7 +19,6 @@ cflatobjs += lib/x86/apic.o
 cflatobjs += lib/x86/atomic.o
 cflatobjs += lib/x86/desc.o
 cflatobjs += lib/x86/isr.o
-cflatobjs += lib/x86/acpi.o
 cflatobjs += lib/x86/stack.o
 cflatobjs += lib/x86/fault_test.o
 cflatobjs += lib/x86/delay.o
diff --git a/lib/x86/asm/setup.h b/lib/x86/asm/setup.h
index 24d4fa9..f46462c 100644
--- a/lib/x86/asm/setup.h
+++ b/lib/x86/asm/setup.h
@@ -4,7 +4,7 @@
 unsigned long setup_tss(u8 *stacktop);
 
 #ifdef CONFIG_EFI
-#include "x86/acpi.h"
+#include "acpi.h"
 #include "x86/apic.h"
 #include "x86/processor.h"
 #include "x86/smp.h"
diff --git a/lib/x86/acpi.h b/lib/acpi.h
similarity index 99%
rename from lib/x86/acpi.h
rename to lib/acpi.h
index 67ba389..1e89840 100644
--- a/lib/x86/acpi.h
+++ b/lib/acpi.h
@@ -1,5 +1,5 @@
-#ifndef _X86_ACPI_H_
-#define _X86_ACPI_H_
+#ifndef _ACPI_H_
+#define _ACPI_H_
 
 #include "libcflat.h"
 
diff --git a/lib/x86/acpi.c b/lib/acpi.c
similarity index 100%
rename from lib/x86/acpi.c
rename to lib/acpi.c
diff --git a/x86/s3.c b/x86/s3.c
index 6e41d0c..378d37a 100644
--- a/x86/s3.c
+++ b/x86/s3.c
@@ -1,5 +1,5 @@
 #include "libcflat.h"
-#include "x86/acpi.h"
+#include "acpi.h"
 #include "asm/io.h"
 
 static u32* find_resume_vector_addr(void)
diff --git a/x86/vmexit.c b/x86/vmexit.c
index 4adec78..2bac049 100644
--- a/x86/vmexit.c
+++ b/x86/vmexit.c
@@ -1,9 +1,9 @@
+#include "acpi.h"
 #include "libcflat.h"
 #include "smp.h"
 #include "pci.h"
 #include "x86/vm.h"
 #include "x86/desc.h"
-#include "x86/acpi.h"
 #include "x86/apic.h"
 #include "x86/isr.h"
 
-- 
2.25.1

[kvm-unit-tests PATCH v2 02/23] lib: Ensure all struct definition for ACPI tables are packed

[Nikos Nikoleris]

All ACPI table definitions are provided with precise definitions of
field sizes and offsets, make sure that no compiler optimization can
interfere with the memory layout of the corresponding structs.

Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
---
 lib/acpi.h | 11 ++++++++---
 x86/s3.c   | 16 ++++------------
 2 files changed, 12 insertions(+), 15 deletions(-)

diff --git a/lib/acpi.h b/lib/acpi.h
index 1e89840..42a2c16 100644
--- a/lib/acpi.h
+++ b/lib/acpi.h
@@ -3,6 +3,11 @@
 
 #include "libcflat.h"
 
+/*
+ * All tables and structures must be byte-packed to match the ACPI
+ * specification, since the tables are provided by the system BIOS
+ */
+
 #define ACPI_SIGNATURE(c1, c2, c3, c4) \
 	((c1) | ((c2) << 8) | ((c3) << 16) | ((c4) << 24))
 
@@ -44,12 +49,12 @@ struct rsdp_descriptor {        /* Root System Descriptor Pointer */
 struct acpi_table {
     ACPI_TABLE_HEADER_DEF
     char data[0];
-};
+} __attribute__ ((packed));
 
 struct rsdt_descriptor_rev1 {
     ACPI_TABLE_HEADER_DEF
     u32 table_offset_entry[0];
-};
+} __attribute__ ((packed));
 
 struct fadt_descriptor_rev1
 {
@@ -104,7 +109,7 @@ struct facs_descriptor_rev1
     u32 S4bios_f        : 1;    /* Indicates if S4BIOS support is present */
     u32 reserved1       : 31;   /* Must be 0 */
     u8  reserved3 [40];         /* Reserved - must be zero */
-};
+} __attribute__ ((packed));
 
 void set_efi_rsdp(struct rsdp_descriptor *rsdp);
 void* find_acpi_table_addr(u32 sig);
diff --git a/x86/s3.c b/x86/s3.c
index 378d37a..89d69fc 100644
--- a/x86/s3.c
+++ b/x86/s3.c
@@ -2,15 +2,6 @@
 #include "acpi.h"
 #include "asm/io.h"
 
-static u32* find_resume_vector_addr(void)
-{
-    struct facs_descriptor_rev1 *facs = find_acpi_table_addr(FACS_SIGNATURE);
-    if (!facs)
-        return 0;
-    printf("FACS is at %p\n", facs);
-    return &facs->firmware_waking_vector;
-}
-
 #define RTC_SECONDS_ALARM       1
 #define RTC_MINUTES_ALARM       3
 #define RTC_HOURS_ALARM         5
@@ -40,12 +31,13 @@ extern char resume_start, resume_end;
 int main(int argc, char **argv)
 {
 	struct fadt_descriptor_rev1 *fadt = find_acpi_table_addr(FACP_SIGNATURE);
-	volatile u32 *resume_vector_ptr = find_resume_vector_addr();
+	struct facs_descriptor_rev1 *facs = find_acpi_table_addr(FACS_SIGNATURE);
 	char *addr, *resume_vec = (void*)0x1000;
 
-	*resume_vector_ptr = (u32)(ulong)resume_vec;
+	facs->firmware_waking_vector = (u32)(ulong)resume_vec;
 
-	printf("resume vector addr is %p\n", resume_vector_ptr);
+	printf("FACS is at %p\n", facs);
+	printf("resume vector addr is %p\n", &facs->firmware_waking_vector);
 	for (addr = &resume_start; addr < &resume_end; addr++)
 		*resume_vec++ = *addr;
 	printf("copy resume code from %p\n", &resume_start);
-- 
2.25.1

[kvm-unit-tests PATCH v2 03/23] lib: Add support for the XSDT ACPI table

[Nikos Nikoleris]

XSDT provides pointers to other ACPI tables much like RSDT. However,
contrary to RSDT that provides 32-bit addresses, XSDT provides 64-bit
pointers. ACPI requires that if XSDT is valid then it takes precedence
over RSDT.

Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
---
 lib/acpi.h |   6 ++++
 lib/acpi.c | 103 ++++++++++++++++++++++++++++++++---------------------
 2 files changed, 68 insertions(+), 41 deletions(-)

diff --git a/lib/acpi.h b/lib/acpi.h
index 42a2c16..d80b983 100644
--- a/lib/acpi.h
+++ b/lib/acpi.h
@@ -13,6 +13,7 @@
 
 #define RSDP_SIGNATURE ACPI_SIGNATURE('R','S','D','P')
 #define RSDT_SIGNATURE ACPI_SIGNATURE('R','S','D','T')
+#define XSDT_SIGNATURE ACPI_SIGNATURE('X','S','D','T')
 #define FACP_SIGNATURE ACPI_SIGNATURE('F','A','C','P')
 #define FACS_SIGNATURE ACPI_SIGNATURE('F','A','C','S')
 
@@ -56,6 +57,11 @@ struct rsdt_descriptor_rev1 {
     u32 table_offset_entry[0];
 } __attribute__ ((packed));
 
+struct acpi_table_xsdt {
+    ACPI_TABLE_HEADER_DEF
+    u64 table_offset_entry[1];
+} __attribute__ ((packed));
+
 struct fadt_descriptor_rev1
 {
     ACPI_TABLE_HEADER_DEF     /* ACPI common table header */
diff --git a/lib/acpi.c b/lib/acpi.c
index de275ca..9b8700c 100644
--- a/lib/acpi.c
+++ b/lib/acpi.c
@@ -38,45 +38,66 @@ static struct rsdp_descriptor *get_rsdp(void)
 
 void* find_acpi_table_addr(u32 sig)
 {
-    struct rsdp_descriptor *rsdp;
-    struct rsdt_descriptor_rev1 *rsdt;
-    void *end;
-    int i;
-
-    /* FACS is special... */
-    if (sig == FACS_SIGNATURE) {
-        struct fadt_descriptor_rev1 *fadt;
-        fadt = find_acpi_table_addr(FACP_SIGNATURE);
-        if (!fadt) {
-            return NULL;
-        }
-        return (void*)(ulong)fadt->firmware_ctrl;
-    }
-
-    rsdp = get_rsdp();
-    if (rsdp == NULL) {
-        printf("Can't find RSDP\n");
-        return 0;
-    }
-
-    if (sig == RSDP_SIGNATURE) {
-        return rsdp;
-    }
-
-    rsdt = (void*)(ulong)rsdp->rsdt_physical_address;
-    if (!rsdt || rsdt->signature != RSDT_SIGNATURE)
-        return 0;
-
-    if (sig == RSDT_SIGNATURE) {
-        return rsdt;
-    }
-
-    end = (void*)rsdt + rsdt->length;
-    for (i=0; (void*)&rsdt->table_offset_entry[i] < end; i++) {
-        struct acpi_table *t = (void*)(ulong)rsdt->table_offset_entry[i];
-        if (t && t->signature == sig) {
-            return t;
-        }
-    }
-   return NULL;
+	struct rsdp_descriptor *rsdp;
+	struct rsdt_descriptor_rev1 *rsdt;
+	struct acpi_table_xsdt *xsdt = NULL;
+	void *end;
+	int i;
+
+	/* FACS is special... */
+	if (sig == FACS_SIGNATURE) {
+		struct fadt_descriptor_rev1 *fadt;
+
+		fadt = find_acpi_table_addr(FACP_SIGNATURE);
+		if (!fadt)
+			return NULL;
+
+		return (void*)(ulong)fadt->firmware_ctrl;
+	}
+
+	rsdp = get_rsdp();
+	if (rsdp == NULL) {
+		printf("Can't find RSDP\n");
+		return 0;
+	}
+
+	if (sig == RSDP_SIGNATURE)
+		return rsdp;
+
+	rsdt = (void *)(ulong)rsdp->rsdt_physical_address;
+	if (!rsdt || rsdt->signature != RSDT_SIGNATURE)
+		rsdt = NULL;
+
+	if (sig == RSDT_SIGNATURE)
+		return rsdt;
+
+	if (rsdp->revision > 1)
+		xsdt = (void *)(ulong)rsdp->xsdt_physical_address;
+	if (!xsdt || xsdt->signature != XSDT_SIGNATURE)
+		xsdt = NULL;
+
+	if (sig == XSDT_SIGNATURE)
+		return xsdt;
+
+	// APCI requires that we first try to use XSDT if it's valid,
+	//  we use to find other tables, otherwise we use RSDT.
+	if (xsdt) {
+		end = (void *)(ulong)xsdt + xsdt->length;
+		for (i = 0; (void *)&xsdt->table_offset_entry[i] < end; i++) {
+			struct acpi_table *t =
+				(void *)xsdt->table_offset_entry[i];
+			if (t && t->signature == sig)
+				return t;
+		}
+	} else if (rsdt) {
+		end = (void *)rsdt + rsdt->length;
+		for (i = 0; (void *)&rsdt->table_offset_entry[i] < end; i++) {
+			struct acpi_table *t =
+				(void *)(ulong)rsdt->table_offset_entry[i];
+			if (t && t->signature == sig)
+				return t;
+		}
+	}
+
+	return NULL;
 }
-- 
2.25.1

[kvm-unit-tests PATCH v2 04/23] lib: Extend the definition of the ACPI table FADT

[Nikos Nikoleris]

This change add more fields in the APCI table FADT to allow for the
discovery of the PSCI conduit in arm64 systems. The definition for
FADT is similar to the one in include/acpi/actbl.h in Linux.

Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
---
 lib/acpi.h   | 35 ++++++++++++++++++++++++++++++-----
 lib/acpi.c   |  2 +-
 x86/s3.c     |  2 +-
 x86/vmexit.c |  2 +-
 4 files changed, 33 insertions(+), 8 deletions(-)

diff --git a/lib/acpi.h b/lib/acpi.h
index d80b983..9f27eb1 100644
--- a/lib/acpi.h
+++ b/lib/acpi.h
@@ -62,7 +62,15 @@ struct acpi_table_xsdt {
     u64 table_offset_entry[1];
 } __attribute__ ((packed));
 
-struct fadt_descriptor_rev1
+struct acpi_generic_address {
+    u8 space_id;            /* Address space where struct or register exists */
+    u8 bit_width;           /* Size in bits of given register */
+    u8 bit_offset;          /* Bit offset within the register */
+    u8 access_width;        /* Minimum Access size (ACPI 3.0) */
+    u64 address;            /* 64-bit address of struct or register */
+} __attribute__ ((packed));
+
+struct acpi_table_fadt
 {
     ACPI_TABLE_HEADER_DEF     /* ACPI common table header */
     u32 firmware_ctrl;          /* Physical address of FACS */
@@ -100,10 +108,27 @@ struct fadt_descriptor_rev1
     u8  day_alrm;               /* Index to day-of-month alarm in RTC CMOS RAM */
     u8  mon_alrm;               /* Index to month-of-year alarm in RTC CMOS RAM */
     u8  century;                /* Index to century in RTC CMOS RAM */
-    u8  reserved4;              /* Reserved */
-    u8  reserved4a;             /* Reserved */
-    u8  reserved4b;             /* Reserved */
-};
+    u16 boot_flags;             /* IA-PC Boot Architecture Flags (see below for individual flags) */
+    u8 reserved;                /* Reserved, must be zero */
+    u32 flags;                  /* Miscellaneous flag bits (see below for individual flags) */
+    struct acpi_generic_address reset_register;     /* 64-bit address of the Reset register */
+    u8 reset_value;             /* Value to write to the reset_register port to reset the system */
+    u16 arm_boot_flags;         /* ARM-Specific Boot Flags (see below for individual flags) (ACPI 5.1) */
+    u8 minor_revision;          /* FADT Minor Revision (ACPI 5.1) */
+    u64 Xfacs;                  /* 64-bit physical address of FACS */
+    u64 Xdsdt;                  /* 64-bit physical address of DSDT */
+    struct acpi_generic_address xpm1a_event_block;  /* 64-bit Extended Power Mgt 1a Event Reg Blk address */
+    struct acpi_generic_address xpm1b_event_block;  /* 64-bit Extended Power Mgt 1b Event Reg Blk address */
+    struct acpi_generic_address xpm1a_control_block;        /* 64-bit Extended Power Mgt 1a Control Reg Blk address */
+    struct acpi_generic_address xpm1b_control_block;        /* 64-bit Extended Power Mgt 1b Control Reg Blk address */
+    struct acpi_generic_address xpm2_control_block; /* 64-bit Extended Power Mgt 2 Control Reg Blk address */
+    struct acpi_generic_address xpm_timer_block;    /* 64-bit Extended Power Mgt Timer Ctrl Reg Blk address */
+    struct acpi_generic_address xgpe0_block;        /* 64-bit Extended General Purpose Event 0 Reg Blk address */
+    struct acpi_generic_address xgpe1_block;        /* 64-bit Extended General Purpose Event 1 Reg Blk address */
+    struct acpi_generic_address sleep_control;      /* 64-bit Sleep Control register (ACPI 5.0) */
+    struct acpi_generic_address sleep_status;       /* 64-bit Sleep Status register (ACPI 5.0) */
+    u64 hypervisor_id;      /* Hypervisor Vendor ID (ACPI 6.0) */
+}  __attribute__ ((packed));
 
 struct facs_descriptor_rev1
 {
diff --git a/lib/acpi.c b/lib/acpi.c
index 9b8700c..e8440ae 100644
--- a/lib/acpi.c
+++ b/lib/acpi.c
@@ -46,7 +46,7 @@ void* find_acpi_table_addr(u32 sig)
 
 	/* FACS is special... */
 	if (sig == FACS_SIGNATURE) {
-		struct fadt_descriptor_rev1 *fadt;
+		struct acpi_table_fadt *fadt;
 
 		fadt = find_acpi_table_addr(FACP_SIGNATURE);
 		if (!fadt)
diff --git a/x86/s3.c b/x86/s3.c
index 89d69fc..16e79f8 100644
--- a/x86/s3.c
+++ b/x86/s3.c
@@ -30,7 +30,7 @@ extern char resume_start, resume_end;
 
 int main(int argc, char **argv)
 {
-	struct fadt_descriptor_rev1 *fadt = find_acpi_table_addr(FACP_SIGNATURE);
+	struct acpi_table_fadt *fadt = find_acpi_table_addr(FACP_SIGNATURE);
 	struct facs_descriptor_rev1 *facs = find_acpi_table_addr(FACS_SIGNATURE);
 	char *addr, *resume_vec = (void*)0x1000;
 
diff --git a/x86/vmexit.c b/x86/vmexit.c
index 2bac049..fcc0760 100644
--- a/x86/vmexit.c
+++ b/x86/vmexit.c
@@ -206,7 +206,7 @@ int pm_tmr_blk;
 static void inl_pmtimer(void)
 {
     if (!pm_tmr_blk) {
-	struct fadt_descriptor_rev1 *fadt;
+	struct acpi_table_fadt *fadt;
 
 	fadt = find_acpi_table_addr(FACP_SIGNATURE);
 	pm_tmr_blk = fadt->pm_tmr_blk;
-- 
2.25.1

[kvm-unit-tests PATCH v2 05/23] arm/arm64: Add support for setting up the PSCI conduit through ACPI

[Nikos Nikoleris]

In systems with ACPI support and when a DT is not provided, we can use
the FADT to discover whether PSCI calls need to use smc or hvc
calls. This change implements this but retains the default behavior;
we check if a valid DT is provided, if not, we try to setup the PSCI
conduit using ACPI.

Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
---
 arm/Makefile.common |  1 +
 lib/acpi.h          |  5 +++++
 lib/arm/psci.c      | 23 ++++++++++++++++++++++-
 lib/devicetree.c    |  2 +-
 4 files changed, 29 insertions(+), 2 deletions(-)

diff --git a/arm/Makefile.common b/arm/Makefile.common
index 38385e0..8e9b3bb 100644
--- a/arm/Makefile.common
+++ b/arm/Makefile.common
@@ -38,6 +38,7 @@ cflatobjs += lib/alloc_page.o
 cflatobjs += lib/vmalloc.o
 cflatobjs += lib/alloc.o
 cflatobjs += lib/devicetree.o
+cflatobjs += lib/acpi.o
 cflatobjs += lib/pci.o
 cflatobjs += lib/pci-host-generic.o
 cflatobjs += lib/pci-testdev.o
diff --git a/lib/acpi.h b/lib/acpi.h
index 9f27eb1..139ccba 100644
--- a/lib/acpi.h
+++ b/lib/acpi.h
@@ -130,6 +130,11 @@ struct acpi_table_fadt
     u64 hypervisor_id;      /* Hypervisor Vendor ID (ACPI 6.0) */
 }  __attribute__ ((packed));
 
+/* Masks for FADT ARM Boot Architecture Flags (arm_boot_flags) ACPI 5.1 */
+
+#define ACPI_FADT_PSCI_COMPLIANT    (1)         /* 00: [V5+] PSCI 0.2+ is implemented */
+#define ACPI_FADT_PSCI_USE_HVC      (1<<1)      /* 01: [V5+] HVC must be used instead of SMC as the PSCI conduit */
+
 struct facs_descriptor_rev1
 {
     u32 signature;           /* ACPI Signature */
diff --git a/lib/arm/psci.c b/lib/arm/psci.c
index 9c031a1..0e96d19 100644
--- a/lib/arm/psci.c
+++ b/lib/arm/psci.c
@@ -6,6 +6,7 @@
  *
  * This work is licensed under the terms of the GNU LGPL, version 2.
  */
+#include <acpi.h>
 #include <devicetree.h>
 #include <asm/psci.h>
 #include <asm/setup.h>
@@ -56,7 +57,7 @@ void psci_system_off(void)
 	printf("CPU%d unable to do system off (error = %d)\n", smp_processor_id(), err);
 }
 
-void psci_set_conduit(void)
+static void psci_set_conduit_fdt(void)
 {
 	const void *fdt = dt_fdt();
 	const struct fdt_property *method;
@@ -75,3 +76,23 @@ void psci_set_conduit(void)
 	else
 		assert_msg(false, "Unknown PSCI conduit: %s", method->data);
 }
+
+static void psci_set_conduit_acpi(void)
+{
+	struct acpi_table_fadt *fadt = find_acpi_table_addr(FACP_SIGNATURE);
+	assert_msg(fadt, "Unable to find ACPI FADT");
+	assert_msg(fadt->arm_boot_flags & ACPI_FADT_PSCI_COMPLIANT,
+		   "PSCI is not supported in this platfrom");
+	if (fadt->arm_boot_flags & ACPI_FADT_PSCI_USE_HVC)
+		psci_invoke = psci_invoke_hvc;
+	else
+		psci_invoke = psci_invoke_smc;
+}
+
+void psci_set_conduit(void)
+{
+	if (dt_available())
+		psci_set_conduit_fdt();
+	else
+		psci_set_conduit_acpi();
+}
diff --git a/lib/devicetree.c b/lib/devicetree.c
index 78c1f6f..3ff9d16 100644
--- a/lib/devicetree.c
+++ b/lib/devicetree.c
@@ -16,7 +16,7 @@ const void *dt_fdt(void)
 
 bool dt_available(void)
 {
-	return fdt_check_header(fdt) == 0;
+	return fdt && fdt_check_header(fdt) == 0;
 }
 
 int dt_get_nr_cells(int fdtnode, u32 *nr_address_cells, u32 *nr_size_cells)
-- 
2.25.1

[kvm-unit-tests PATCH v2 06/23] arm/arm64: Add support for discovering the UART through ACPI

[Nikos Nikoleris]

In systems with ACPI support and when a DT is not provided, we can use
the SPCR to discover the serial port address range. This change
implements this but retains the default behavior; we check if a valid
DT is provided, if not, we try to discover the UART using ACPI.

Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
---
 lib/acpi.h     | 25 +++++++++++++++++++++++++
 lib/arm/io.c   | 21 +++++++++++++++++++--
 lib/arm/psci.c |  4 +++-
 3 files changed, 47 insertions(+), 3 deletions(-)

diff --git a/lib/acpi.h b/lib/acpi.h
index 139ccba..5213299 100644
--- a/lib/acpi.h
+++ b/lib/acpi.h
@@ -16,6 +16,7 @@
 #define XSDT_SIGNATURE ACPI_SIGNATURE('X','S','D','T')
 #define FACP_SIGNATURE ACPI_SIGNATURE('F','A','C','P')
 #define FACS_SIGNATURE ACPI_SIGNATURE('F','A','C','S')
+#define SPCR_SIGNATURE ACPI_SIGNATURE('S','P','C','R')
 
 
 #define ACPI_SIGNATURE_8BYTE(c1, c2, c3, c4, c5, c6, c7, c8) \
@@ -147,6 +148,30 @@ struct facs_descriptor_rev1
     u8  reserved3 [40];         /* Reserved - must be zero */
 } __attribute__ ((packed));
 
+struct spcr_descriptor {
+    ACPI_TABLE_HEADER_DEF   /* ACPI common table header */
+    u8 interface_type;      /* 0=full 16550, 1=subset of 16550 */
+    u8 reserved[3];
+    struct acpi_generic_address serial_port;
+    u8 interrupt_type;
+    u8 pc_interrupt;
+    u32 interrupt;
+    u8 baud_rate;
+    u8 parity;
+    u8 stop_bits;
+    u8 flow_control;
+    u8 terminal_type;
+    u8 reserved1;
+    u16 pci_device_id;
+    u16 pci_vendor_id;
+    u8 pci_bus;
+    u8 pci_device;
+    u8 pci_function;
+    u32 pci_flags;
+    u8 pci_segment;
+    u32 reserved2;
+} __attribute__ ((packed));
+
 void set_efi_rsdp(struct rsdp_descriptor *rsdp);
 void* find_acpi_table_addr(u32 sig);
 
diff --git a/lib/arm/io.c b/lib/arm/io.c
index 343e108..893bdfc 100644
--- a/lib/arm/io.c
+++ b/lib/arm/io.c
@@ -8,6 +8,7 @@
  *
  * This work is licensed under the terms of the GNU LGPL, version 2.
  */
+#include <acpi.h>
 #include <libcflat.h>
 #include <devicetree.h>
 #include <chr-testdev.h>
@@ -29,7 +30,7 @@ static struct spinlock uart_lock;
 #define UART_EARLY_BASE (u8 *)(unsigned long)CONFIG_UART_EARLY_BASE
 static volatile u8 *uart0_base = UART_EARLY_BASE;
 
-static void uart0_init(void)
+static void uart0_init_fdt(void)
 {
 	/*
 	 * kvm-unit-tests uses the uart only for output. Both uart models have
@@ -73,9 +74,25 @@ static void uart0_init(void)
 	}
 }
 
+static void uart0_init_acpi(void)
+{
+	struct spcr_descriptor *spcr = find_acpi_table_addr(SPCR_SIGNATURE);
+	assert_msg(spcr, "Unable to find ACPI SPCR");
+	uart0_base = ioremap(spcr->serial_port.address, spcr->serial_port.bit_width);
+
+	if (uart0_base != UART_EARLY_BASE) {
+		printf("WARNING: early print support may not work. "
+		       "Found uart at %p, but early base is %p.\n",
+			uart0_base, UART_EARLY_BASE);
+	}
+}
+
 void io_init(void)
 {
-	uart0_init();
+	if (dt_available())
+		uart0_init_fdt();
+	else
+		uart0_init_acpi();
 	chr_testdev_init();
 }
 
diff --git a/lib/arm/psci.c b/lib/arm/psci.c
index 0e96d19..afbc33d 100644
--- a/lib/arm/psci.c
+++ b/lib/arm/psci.c
@@ -80,9 +80,11 @@ static void psci_set_conduit_fdt(void)
 static void psci_set_conduit_acpi(void)
 {
 	struct acpi_table_fadt *fadt = find_acpi_table_addr(FACP_SIGNATURE);
+
 	assert_msg(fadt, "Unable to find ACPI FADT");
 	assert_msg(fadt->arm_boot_flags & ACPI_FADT_PSCI_COMPLIANT,
-		   "PSCI is not supported in this platfrom");
+		   "PSCI is not supported in this platform");
+
 	if (fadt->arm_boot_flags & ACPI_FADT_PSCI_USE_HVC)
 		psci_invoke = psci_invoke_hvc;
 	else
-- 
2.25.1

[kvm-unit-tests PATCH v2 07/23] arm/arm64: Add support for timer initialization through ACPI

[Nikos Nikoleris]

For systems with ACPI support, we can discover timers through the ACPI
GTDT table. This change implements the code to discover timers through
the GTDT and adds ACPI support in timer_save_state. This change
retains the default behavior; we check if a valid DT is provided, if
not, we try to discover timers using ACPI.

Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
---
 arm/Makefile.common |  1 +
 lib/arm/asm/timer.h |  2 ++
 lib/acpi.h          | 18 +++++++++++
 lib/arm/setup.c     | 39 ------------------------
 lib/arm/timer.c     | 73 +++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 94 insertions(+), 39 deletions(-)
 create mode 100644 lib/arm/timer.c

diff --git a/arm/Makefile.common b/arm/Makefile.common
index 8e9b3bb..5be42c0 100644
--- a/arm/Makefile.common
+++ b/arm/Makefile.common
@@ -53,6 +53,7 @@ cflatobjs += lib/arm/psci.o
 cflatobjs += lib/arm/smp.o
 cflatobjs += lib/arm/delay.o
 cflatobjs += lib/arm/gic.o lib/arm/gic-v2.o lib/arm/gic-v3.o
+cflatobjs += lib/arm/timer.o
 
 OBJDIRS += lib/arm
 
diff --git a/lib/arm/asm/timer.h b/lib/arm/asm/timer.h
index f75cc67..aaf839f 100644
--- a/lib/arm/asm/timer.h
+++ b/lib/arm/asm/timer.h
@@ -27,5 +27,7 @@ extern struct timer_state __timer_state;
 #define TIMER_PTIMER_IRQ (__timer_state.ptimer.irq)
 #define TIMER_VTIMER_IRQ (__timer_state.vtimer.irq)
 
+void timer_save_state(void);
+
 #endif /* !__ASSEMBLY__ */
 #endif /* _ASMARM_TIMER_H_ */
diff --git a/lib/acpi.h b/lib/acpi.h
index 5213299..297ad87 100644
--- a/lib/acpi.h
+++ b/lib/acpi.h
@@ -17,6 +17,7 @@
 #define FACP_SIGNATURE ACPI_SIGNATURE('F','A','C','P')
 #define FACS_SIGNATURE ACPI_SIGNATURE('F','A','C','S')
 #define SPCR_SIGNATURE ACPI_SIGNATURE('S','P','C','R')
+#define GTDT_SIGNATURE ACPI_SIGNATURE('G','T','D','T')
 
 
 #define ACPI_SIGNATURE_8BYTE(c1, c2, c3, c4, c5, c6, c7, c8) \
@@ -172,6 +173,23 @@ struct spcr_descriptor {
     u32 reserved2;
 } __attribute__ ((packed));
 
+struct acpi_table_gtdt {
+    ACPI_TABLE_HEADER_DEF   /* ACPI common table header */
+    u64 counter_block_addresss;
+    u32 reserved;
+    u32 secure_el1_interrupt;
+    u32 secure_el1_flags;
+    u32 non_secure_el1_interrupt;
+    u32 non_secure_el1_flags;
+    u32 virtual_timer_interrupt;
+    u32 virtual_timer_flags;
+    u32 non_secure_el2_interrupt;
+    u32 non_secure_el2_flags;
+    u64 counter_read_block_address;
+    u32 platform_timer_count;
+    u32 platform_timer_offset;
+} __attribute__ ((packed));
+
 void set_efi_rsdp(struct rsdp_descriptor *rsdp);
 void* find_acpi_table_addr(u32 sig);
 
diff --git a/lib/arm/setup.c b/lib/arm/setup.c
index bcdf0d7..1572c64 100644
--- a/lib/arm/setup.c
+++ b/lib/arm/setup.c
@@ -35,8 +35,6 @@
 
 extern unsigned long etext;
 
-struct timer_state __timer_state;
-
 char *initrd;
 u32 initrd_size;
 
@@ -199,43 +197,6 @@ static void mem_init(phys_addr_t freemem_start)
 	page_alloc_ops_enable();
 }
 
-static void timer_save_state(void)
-{
-	const struct fdt_property *prop;
-	const void *fdt = dt_fdt();
-	int node, len;
-	u32 *data;
-
-	node = fdt_node_offset_by_compatible(fdt, -1, "arm,armv8-timer");
-	assert(node >= 0 || node == -FDT_ERR_NOTFOUND);
-
-	if (node == -FDT_ERR_NOTFOUND) {
-		__timer_state.ptimer.irq = -1;
-		__timer_state.vtimer.irq = -1;
-		return;
-	}
-
-	/*
-	 * From Linux devicetree timer binding documentation
-	 *
-	 * interrupts <type irq flags>:
-	 *	secure timer irq
-	 *	non-secure timer irq		(ptimer)
-	 *	virtual timer irq		(vtimer)
-	 *	hypervisor timer irq
-	 */
-	prop = fdt_get_property(fdt, node, "interrupts", &len);
-	assert(prop && len == (4 * 3 * sizeof(u32)));
-
-	data = (u32 *)prop->data;
-	assert(fdt32_to_cpu(data[3]) == 1 /* PPI */);
-	__timer_state.ptimer.irq = fdt32_to_cpu(data[4]);
-	__timer_state.ptimer.irq_flags = fdt32_to_cpu(data[5]);
-	assert(fdt32_to_cpu(data[6]) == 1 /* PPI */);
-	__timer_state.vtimer.irq = fdt32_to_cpu(data[7]);
-	__timer_state.vtimer.irq_flags = fdt32_to_cpu(data[8]);
-}
-
 void setup(const void *fdt, phys_addr_t freemem_start)
 {
 	void *freemem;
diff --git a/lib/arm/timer.c b/lib/arm/timer.c
new file mode 100644
index 0000000..eceabdf
--- /dev/null
+++ b/lib/arm/timer.c
@@ -0,0 +1,73 @@
+/*
+ * Initialize timers.
+ *
+ * Copyright (C) 2022, Arm Ltd., Nikos Nikoleris <nikos.nikoleris@arm.com>
+ * Copyright (C) 2014, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#include <libcflat.h>
+#include <asm/timer.h>
+
+#include <acpi.h>
+#include <devicetree.h>
+#include <libfdt/libfdt.h>
+
+struct timer_state __timer_state;
+
+static void timer_save_state_fdt(void)
+{
+	const struct fdt_property *prop;
+	const void *fdt = dt_fdt();
+	int node, len;
+	u32 *data;
+
+	node = fdt_node_offset_by_compatible(fdt, -1, "arm,armv8-timer");
+	assert(node >= 0 || node == -FDT_ERR_NOTFOUND);
+
+	if (node == -FDT_ERR_NOTFOUND) {
+		__timer_state.ptimer.irq = -1;
+		__timer_state.vtimer.irq = -1;
+		return;
+	}
+
+	/*
+	 * From Linux devicetree timer binding documentation
+	 *
+	 * interrupts <type irq flags>:
+	 *	secure timer irq
+	 *	non-secure timer irq		(ptimer)
+	 *	virtual timer irq		(vtimer)
+	 *	hypervisor timer irq
+	 */
+	prop = fdt_get_property(fdt, node, "interrupts", &len);
+	assert(prop && len == (4 * 3 * sizeof(u32)));
+
+	data = (u32 *)prop->data;
+	assert(fdt32_to_cpu(data[3]) == 1 /* PPI */);
+	__timer_state.ptimer.irq = fdt32_to_cpu(data[4]);
+	__timer_state.ptimer.irq_flags = fdt32_to_cpu(data[5]);
+	assert(fdt32_to_cpu(data[6]) == 1 /* PPI */);
+	__timer_state.vtimer.irq = fdt32_to_cpu(data[7]);
+	__timer_state.vtimer.irq_flags = fdt32_to_cpu(data[8]);
+}
+
+static void timer_save_state_acpi(void)
+{
+	struct acpi_table_gtdt *gtdt = find_acpi_table_addr(GTDT_SIGNATURE);
+
+	assert_msg(gtdt, "Unable to find ACPI GTDT");
+	__timer_state.ptimer.irq = gtdt->non_secure_el1_interrupt;
+	__timer_state.ptimer.irq_flags = gtdt->non_secure_el1_flags;
+
+	__timer_state.vtimer.irq = gtdt->virtual_timer_interrupt;
+	__timer_state.vtimer.irq_flags = gtdt->virtual_timer_flags;
+}
+
+void timer_save_state(void)
+{
+	if (dt_available())
+		timer_save_state_fdt();
+	else
+		timer_save_state_acpi();
+}
-- 
2.25.1

[kvm-unit-tests PATCH v2 08/23] arm/arm64: Add support for cpu initialization through ACPI

[Nikos Nikoleris]

In systems with ACPI support and when a DT is not provided, we can use
the MADTs to discover the number of CPUs and their corresponding MIDR.
This change implements this but retains the default behavior; we check
if a valid DT is provided, if not, we try to discover the cores in the
system using ACPI.

Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
---
 lib/acpi.h      | 64 +++++++++++++++++++++++++++++++++++++++++++++++++
 lib/acpi.c      | 21 ++++++++++++++++
 lib/arm/setup.c | 25 ++++++++++++++++---
 3 files changed, 107 insertions(+), 3 deletions(-)

diff --git a/lib/acpi.h b/lib/acpi.h
index 297ad87..55de54a 100644
--- a/lib/acpi.h
+++ b/lib/acpi.h
@@ -16,6 +16,7 @@
 #define XSDT_SIGNATURE ACPI_SIGNATURE('X','S','D','T')
 #define FACP_SIGNATURE ACPI_SIGNATURE('F','A','C','P')
 #define FACS_SIGNATURE ACPI_SIGNATURE('F','A','C','S')
+#define MADT_SIGNATURE ACPI_SIGNATURE('A','P','I','C')
 #define SPCR_SIGNATURE ACPI_SIGNATURE('S','P','C','R')
 #define GTDT_SIGNATURE ACPI_SIGNATURE('G','T','D','T')
 
@@ -149,6 +150,67 @@ struct facs_descriptor_rev1
     u8  reserved3 [40];         /* Reserved - must be zero */
 } __attribute__ ((packed));
 
+struct acpi_table_madt {
+    ACPI_TABLE_HEADER_DEF     /* ACPI common table header */
+    u32 address;               /* Physical address of local APIC */
+    u32 flags;
+} __attribute__ ((packed));
+
+struct acpi_subtable_header {
+    u8 type;
+    u8 length;
+}  __attribute__ ((packed));
+
+typedef int (*acpi_table_handler)(struct acpi_subtable_header *header);
+
+/* 11: Generic interrupt - GICC (ACPI 5.0 + ACPI 6.0 + ACPI 6.3 changes) */
+
+struct acpi_madt_generic_interrupt {
+    u8 type;
+    u8 length;
+    u16 reserved;           /* reserved - must be zero */
+    u32 cpu_interface_number;
+    u32 uid;
+    u32 flags;
+    u32 parking_version;
+    u32 performance_interrupt;
+    u64 parked_address;
+    u64 base_address;
+    u64 gicv_base_address;
+    u64 gich_base_address;
+    u32 vgic_interrupt;
+    u64 gicr_base_address;
+    u64 arm_mpidr;
+    u8 efficiency_class;
+    u8 reserved2[1];
+    u16 spe_interrupt;      /* ACPI 6.3 */
+} __attribute__ ((packed));
+
+/* Values for MADT subtable type in struct acpi_subtable_header */
+
+enum acpi_madt_type {
+    ACPI_MADT_TYPE_LOCAL_APIC = 0,
+    ACPI_MADT_TYPE_IO_APIC = 1,
+    ACPI_MADT_TYPE_INTERRUPT_OVERRIDE = 2,
+    ACPI_MADT_TYPE_NMI_SOURCE = 3,
+    ACPI_MADT_TYPE_LOCAL_APIC_NMI = 4,
+    ACPI_MADT_TYPE_LOCAL_APIC_OVERRIDE = 5,
+    ACPI_MADT_TYPE_IO_SAPIC = 6,
+    ACPI_MADT_TYPE_LOCAL_SAPIC = 7,
+    ACPI_MADT_TYPE_INTERRUPT_SOURCE = 8,
+    ACPI_MADT_TYPE_LOCAL_X2APIC = 9,
+    ACPI_MADT_TYPE_LOCAL_X2APIC_NMI = 10,
+    ACPI_MADT_TYPE_GENERIC_INTERRUPT = 11,
+    ACPI_MADT_TYPE_GENERIC_DISTRIBUTOR = 12,
+    ACPI_MADT_TYPE_GENERIC_MSI_FRAME = 13,
+    ACPI_MADT_TYPE_GENERIC_REDISTRIBUTOR = 14,
+    ACPI_MADT_TYPE_GENERIC_TRANSLATOR = 15,
+    ACPI_MADT_TYPE_RESERVED = 16    /* 16 and greater are reserved */
+};
+
+/* MADT Local APIC flags */
+#define ACPI_MADT_ENABLED           (1) /* 00: Processor is usable if set */
+
 struct spcr_descriptor {
     ACPI_TABLE_HEADER_DEF   /* ACPI common table header */
     u8 interface_type;      /* 0=full 16550, 1=subset of 16550 */
@@ -192,5 +254,7 @@ struct acpi_table_gtdt {
 
 void set_efi_rsdp(struct rsdp_descriptor *rsdp);
 void* find_acpi_table_addr(u32 sig);
+void acpi_table_parse_madt(enum acpi_madt_type mtype,
+			   acpi_table_handler handler);
 
 #endif
diff --git a/lib/acpi.c b/lib/acpi.c
index e8440ae..ad3ae8d 100644
--- a/lib/acpi.c
+++ b/lib/acpi.c
@@ -101,3 +101,24 @@ void* find_acpi_table_addr(u32 sig)
 
 	return NULL;
 }
+
+void acpi_table_parse_madt(enum acpi_madt_type mtype,
+			   acpi_table_handler handler)
+{
+	struct acpi_table_madt *madt;
+	void *end;
+
+	madt = find_acpi_table_addr(MADT_SIGNATURE);
+	assert(madt);
+
+	struct acpi_subtable_header *header =
+		(void *)(ulong)madt + sizeof(struct acpi_table_madt);
+	end = (void *)((ulong)madt + madt->length);
+
+	while ((void *)header < end) {
+		if (header->type == mtype)
+			handler(header);
+
+		header = (void *)(ulong)header + header->length;
+	}
+}
diff --git a/lib/arm/setup.c b/lib/arm/setup.c
index 1572c64..3c24c75 100644
--- a/lib/arm/setup.c
+++ b/lib/arm/setup.c
@@ -13,6 +13,7 @@
 #include <libcflat.h>
 #include <libfdt/libfdt.h>
 #include <devicetree.h>
+#include <acpi.h>
 #include <alloc.h>
 #include <alloc_phys.h>
 #include <alloc_page.h>
@@ -55,7 +56,7 @@ int mpidr_to_cpu(uint64_t mpidr)
 	return -1;
 }
 
-static void cpu_set(int fdtnode __unused, u64 regval, void *info __unused)
+static void cpu_set_fdt(int fdtnode __unused, u64 regval, void *info __unused)
 {
 	int cpu = nr_cpus++;
 
@@ -65,13 +66,31 @@ static void cpu_set(int fdtnode __unused, u64 regval, void *info __unused)
 	set_cpu_present(cpu, true);
 }
 
+static int cpu_set_acpi(struct acpi_subtable_header *header)
+{
+	int cpu = nr_cpus++;
+	struct acpi_madt_generic_interrupt *gicc = (void *)header;
+
+	assert_msg(cpu < NR_CPUS, "Number cpus exceeds maximum supported (%d).", NR_CPUS);
+
+	cpus[cpu] = gicc->arm_mpidr;
+	set_cpu_present(cpu, true);
+
+	return 0;
+}
+
 static void cpu_init(void)
 {
 	int ret;
 
 	nr_cpus = 0;
-	ret = dt_for_each_cpu_node(cpu_set, NULL);
-	assert(ret == 0);
+	if (dt_available()) {
+		ret = dt_for_each_cpu_node(cpu_set_fdt, NULL);
+		assert(ret == 0);
+	} else
+		acpi_table_parse_madt(ACPI_MADT_TYPE_GENERIC_INTERRUPT,
+				      cpu_set_acpi);
+
 	set_cpu_online(0, true);
 }
 
-- 
2.25.1

[kvm-unit-tests PATCH v2 09/23] lib/printf: Support for precision modifier in printing strings

[Nikos Nikoleris]

This follows the typical format of:

printf("%.Ns", *str);

Where N might be a decimal digit string or '*'. This feature is used
by a future change.

See also: man 3 printf

Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
---
 lib/printf.c | 84 ++++++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 71 insertions(+), 13 deletions(-)

diff --git a/lib/printf.c b/lib/printf.c
index 1269723..724befa 100644
--- a/lib/printf.c
+++ b/lib/printf.c
@@ -19,6 +19,7 @@ typedef struct strprops {
     char pad;
     int npad;
     bool alternate;
+    int precision;
 } strprops_t;
 
 static void addchar(pstream_t *p, char c)
@@ -43,7 +44,7 @@ static void print_str(pstream_t *p, const char *s, strprops_t props)
 	}
     }
 
-    while (*s)
+    while (*s && props.precision--)
 	addchar(p, *s++);
 
     if (npad < 0) {
@@ -147,9 +148,61 @@ static int fmtnum(const char **fmt)
     return num;
 }
 
+static inline int isdigit(int c)
+{
+    return '0' <= c && c <= '9';
+}
+
+/*
+ * Adapted from drivers/firmware/efi/libstub/vsprintf.c
+ */
+static int skip_atoi(const char **s)
+{
+    int i = 0;
+
+    do {
+	i = i*10 + *((*s)++) - '0';
+    } while (isdigit(**s));
+
+    return i;
+}
+
+/*
+ * Adapted from drivers/firmware/efi/libstub/vsprintf.c
+ */
+static int get_int(const char **fmt, va_list *ap)
+{
+    if (isdigit(**fmt)) {
+	return skip_atoi(fmt);
+    }
+    if (**fmt == '*') {
+	++(*fmt);
+	/* it's the next argument */
+	return va_arg(*ap, int);
+    }
+    return 0;
+}
+
 int vsnprintf(char *buf, int size, const char *fmt, va_list va)
 {
     pstream_t s;
+    va_list args;
+
+    /*
+     * We want to pass our input va_list to helper functions by reference,
+     * but there's an annoying edge case. If va_list was originally passed
+     * to us by value, we could just pass &ap down to the helpers. This is
+     * the case on, for example, X86_32.
+     * However, on X86_64 (and possibly others), va_list is actually a
+     * size-1 array containing a structure. Our function parameter ap has
+     * decayed from T[1] to T*, and &ap has type T** rather than T(*)[1],
+     * which is what will be expected by a function taking a va_list *
+     * parameter.
+     * One standard way to solve this mess is by creating a copy in a local
+     * variable of type va_list and then passing a pointer to that local
+     * copy instead, which is what we do here.
+     */
+    va_copy(args, va);
 
     s.buffer = buf;
     s.remain = size - 1;
@@ -160,6 +213,7 @@ int vsnprintf(char *buf, int size, const char *fmt, va_list va)
 	strprops_t props;
 	memset(&props, 0, sizeof(props));
 	props.pad = ' ';
+	props.precision = -1;
 
 	if (f != '%') {
 	    addchar(&s, f);
@@ -172,11 +226,14 @@ int vsnprintf(char *buf, int size, const char *fmt, va_list va)
 	    addchar(&s, '%');
 	    break;
 	case 'c':
-            addchar(&s, va_arg(va, int));
+	    addchar(&s, va_arg(args, int));
 	    break;
 	case '\0':
 	    --fmt;
 	    break;
+	case '.':
+	    props.precision = get_int(&fmt, &args);
+	    goto morefmt;
 	case '#':
 	    props.alternate = true;
 	    goto morefmt;
@@ -204,54 +261,55 @@ int vsnprintf(char *buf, int size, const char *fmt, va_list va)
 	case 'd':
 	    switch (nlong) {
 	    case 0:
-		print_int(&s, va_arg(va, int), 10, props);
+		print_int(&s, va_arg(args, int), 10, props);
 		break;
 	    case 1:
-		print_int(&s, va_arg(va, long), 10, props);
+		print_int(&s, va_arg(args, long), 10, props);
 		break;
 	    default:
-		print_int(&s, va_arg(va, long long), 10, props);
+		print_int(&s, va_arg(args, long long), 10, props);
 		break;
 	    }
 	    break;
 	case 'u':
 	    switch (nlong) {
 	    case 0:
-		print_unsigned(&s, va_arg(va, unsigned), 10, props);
+		print_unsigned(&s, va_arg(args, unsigned), 10, props);
 		break;
 	    case 1:
-		print_unsigned(&s, va_arg(va, unsigned long), 10, props);
+		print_unsigned(&s, va_arg(args, unsigned long), 10, props);
 		break;
 	    default:
-		print_unsigned(&s, va_arg(va, unsigned long long), 10, props);
+		print_unsigned(&s, va_arg(args, unsigned long long), 10, props);
 		break;
 	    }
 	    break;
 	case 'x':
 	    switch (nlong) {
 	    case 0:
-		print_unsigned(&s, va_arg(va, unsigned), 16, props);
+		print_unsigned(&s, va_arg(args, unsigned), 16, props);
 		break;
 	    case 1:
-		print_unsigned(&s, va_arg(va, unsigned long), 16, props);
+		print_unsigned(&s, va_arg(args, unsigned long), 16, props);
 		break;
 	    default:
-		print_unsigned(&s, va_arg(va, unsigned long long), 16, props);
+		print_unsigned(&s, va_arg(args, unsigned long long), 16, props);
 		break;
 	    }
 	    break;
 	case 'p':
 	    props.alternate = true;
-	    print_unsigned(&s, (unsigned long)va_arg(va, void *), 16, props);
+	    print_unsigned(&s, (unsigned long)va_arg(args, void *), 16, props);
 	    break;
 	case 's':
-	    print_str(&s, va_arg(va, const char *), props);
+	    print_str(&s, va_arg(args, const char *), props);
 	    break;
 	default:
 	    addchar(&s, f);
 	    break;
 	}
     }
+    va_end(args);
     *s.buffer = 0;
     return s.added;
 }
-- 
2.25.1

[kvm-unit-tests PATCH v2 10/23] lib/printf: Add support for printing wide strings

[Nikos Nikoleris]

This change adds support for wide strings (u16*) to printf()
variants. This feature is used by a future change.

Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
---
 lib/printf.c | 101 ++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 100 insertions(+), 1 deletion(-)

diff --git a/lib/printf.c b/lib/printf.c
index 724befa..14a3fed 100644
--- a/lib/printf.c
+++ b/lib/printf.c
@@ -57,6 +57,102 @@ static void print_str(pstream_t *p, const char *s, strprops_t props)
     }
 }
 
+/*
+ * Adapted from drivers/firmware/efi/libstub/vsprintf.c
+ */
+static u32 utf16_to_utf32(const u16 **s16)
+{
+    u16 c0, c1;
+
+    c0 = *(*s16)++;
+    /* not a surrogate */
+    if ((c0 & 0xf800) != 0xd800)
+	return c0;
+    /* invalid: low surrogate instead of high */
+    if (c0 & 0x0400)
+	return 0xfffd;
+    c1 = **s16;
+    /* invalid: missing low surrogate */
+    if ((c1 & 0xfc00) != 0xdc00)
+	return 0xfffd;
+    /* valid surrogate pair */
+    ++(*s16);
+    return (0x10000 - (0xd800 << 10) - 0xdc00) + (c0 << 10) + c1;
+}
+
+/*
+ * Adapted from drivers/firmware/efi/libstub/vsprintf.c
+ */
+static size_t utf16s_utf8nlen(const u16 *s16, size_t maxlen)
+{
+    size_t len, clen;
+
+    for (len = 0; len < maxlen && *s16; len += clen) {
+	u16 c0 = *s16++;
+
+	/* First, get the length for a BMP character */
+	clen = 1 + (c0 >= 0x80) + (c0 >= 0x800);
+	if (len + clen > maxlen)
+	    break;
+	/*
+	 * If this is a high surrogate, and we're already at maxlen, we
+	 * can't include the character if it's a valid surrogate pair.
+	 * Avoid accessing one extra word just to check if it's valid
+	 * or not.
+	 */
+	if ((c0 & 0xfc00) == 0xd800) {
+	    if (len + clen == maxlen)
+		break;
+	    if ((*s16 & 0xfc00) == 0xdc00) {
+		++s16;
+		++clen;
+	    }
+	}
+    }
+
+    return len;
+}
+
+/*
+ * Adapted from drivers/firmware/efi/libstub/vsprintf.c
+ */
+static void print_wstring(pstream_t *p, const u16 *s, strprops_t props)
+{
+    const u16 *ws = (const u16 *)s;
+    size_t pos = 0, size = p->remain + 1, len = utf16s_utf8nlen(ws, props.precision);
+
+    while (len-- > 0) {
+	u32 c32 = utf16_to_utf32(&ws);
+	u8 *s8;
+	size_t clen;
+
+	if (c32 < 0x80) {
+	    addchar(p, c32);
+	    continue;
+	}
+
+	/* Number of trailing octets */
+	clen = 1 + (c32 >= 0x800) + (c32 >= 0x10000);
+
+	len -= clen;
+	s8 = (u8 *)(p->buffer - p->added + pos);
+
+	/* Avoid writing partial character */
+	addchar(p, '\0');
+	pos += clen;
+	if (pos >= size)
+	    continue;
+
+	/* Set high bits of leading octet */
+	*s8 = (0xf00 >> 1) >> clen;
+	/* Write trailing octets in reverse order */
+	for (s8 += clen; clen; --clen, c32 >>= 6)
+	    *s8-- = 0x80 | (c32 & 0x3f);
+	/* Set low bits of leading octet */
+	*s8 |= c32;
+    }
+}
+
 static char digits[16] = "0123456789abcdef";
 
 static void print_int(pstream_t *ps, long long n, int base, strprops_t props)
@@ -302,7 +398,10 @@ int vsnprintf(char *buf, int size, const char *fmt, va_list va)
 	    print_unsigned(&s, (unsigned long)va_arg(args, void *), 16, props);
 	    break;
 	case 's':
-	    print_str(&s, va_arg(args, const char *), props);
+	    if (nlong)
+		print_wstring(&s, va_arg(args, const u16 *), props);
+	    else
+		print_str(&s, va_arg(args, const char *), props);
 	    break;
 	default:
 	    addchar(&s, f);
-- 
2.25.1

[kvm-unit-tests PATCH v2 11/23] lib/efi: Add support for getting the cmdline

[Nikos Nikoleris]

This change adds support for discovering the command line arguments,
as a string. Then, we parse this string to populate __argc and __argv
for EFI tests.

Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
---
 lib/linux/efi.h |  39 +++++++++++++++
 lib/stdlib.h    |   1 +
 lib/efi.c       | 123 ++++++++++++++++++++++++++++++++++++++++++++++++
 lib/string.c    |   2 +-
 4 files changed, 164 insertions(+), 1 deletion(-)

diff --git a/lib/linux/efi.h b/lib/linux/efi.h
index 455625a..e3aba1d 100644
--- a/lib/linux/efi.h
+++ b/lib/linux/efi.h
@@ -60,6 +60,10 @@ typedef guid_t efi_guid_t;
 
 #define ACPI_TABLE_GUID EFI_GUID(0xeb9d2d30, 0x2d88, 0x11d3, 0x9a, 0x16, 0x00, 0x90, 0x27, 0x3f, 0xc1, 0x4d)
 
+#define LOADED_IMAGE_PROTOCOL_GUID EFI_GUID(0x5b1b31a1, 0x9562, 0x11d2,  0x8e, 0x3f, 0x00, 0xa0, 0xc9, 0x69, 0x72, 0x3b)
+
+#define efi_table_attr(inst, attr) (inst->attr)
+
 typedef struct {
 	efi_guid_t guid;
 	void *table;
@@ -416,6 +420,41 @@ struct efi_boot_memmap {
 	unsigned long           *buff_size;
 };
 
+#define __aligned_u64 u64 __attribute__((aligned(8)))
+
+typedef union {
+	struct {
+		u32			revision;
+		efi_handle_t		parent_handle;
+		efi_system_table_t	*system_table;
+		efi_handle_t		device_handle;
+		void			*file_path;
+		void			*reserved;
+		u32			load_options_size;
+		void			*load_options;
+		void			*image_base;
+		__aligned_u64		image_size;
+		unsigned int		image_code_type;
+		unsigned int		image_data_type;
+		efi_status_t		(__efiapi *unload)(efi_handle_t image_handle);
+	};
+	struct {
+		u32		revision;
+		u32		parent_handle;
+		u32		system_table;
+		u32		device_handle;
+		u32		file_path;
+		u32		reserved;
+		u32		load_options_size;
+		u32		load_options;
+		u32		image_base;
+		__aligned_u64	image_size;
+		u32		image_code_type;
+		u32		image_data_type;
+		u32		unload;
+	} mixed_mode;
+} efi_loaded_image_t;
+
 #define efi_bs_call(func, ...) efi_system_table->boottime->func(__VA_ARGS__)
 #define efi_rs_call(func, ...) efi_system_table->runtime->func(__VA_ARGS__)
 
diff --git a/lib/stdlib.h b/lib/stdlib.h
index 28496d7..2c524d7 100644
--- a/lib/stdlib.h
+++ b/lib/stdlib.h
@@ -7,6 +7,7 @@
 #ifndef _STDLIB_H_
 #define _STDLIB_H_
 
+int isspace(int c);
 long int strtol(const char *nptr, char **endptr, int base);
 unsigned long int strtoul(const char *nptr, char **endptr, int base);
 long long int strtoll(const char *nptr, char **endptr, int base);
diff --git a/lib/efi.c b/lib/efi.c
index 64cc978..5341942 100644
--- a/lib/efi.c
+++ b/lib/efi.c
@@ -8,6 +8,7 @@
  */
 
 #include "efi.h"
+#include <stdlib.h>
 #include <libcflat.h>
 #include <asm/setup.h>
 
@@ -96,6 +97,97 @@ static void efi_exit(efi_status_t code)
 	efi_rs_call(reset_system, EFI_RESET_SHUTDOWN, code, 0, NULL);
 }
 
+static void efi_cmdline_to_argv(char *cmdline_ptr)
+{
+	char *c = cmdline_ptr;
+	bool narg = true;
+	while (*c) {
+		if (isspace(*c)) {
+			*c = '\0';
+			narg = true;
+		} else if (narg) {
+			__argv[__argc++] = c;
+			narg = false;
+		}
+		c++;
+	}
+}
+
+static char *efi_convert_cmdline(efi_loaded_image_t *image, int *cmd_line_len)
+{
+	const u16 *s2;
+	unsigned long cmdline_addr = 0;
+	int options_chars = efi_table_attr(image, load_options_size) / 2;
+	const u16 *options = efi_table_attr(image, load_options);
+	int options_bytes = 0, safe_options_bytes = 0;  /* UTF-8 bytes */
+	bool in_quote = false;
+	efi_status_t status;
+	const int COMMAND_LINE_SIZE = 2048;
+
+	if (options) {
+		s2 = options;
+		while (options_bytes < COMMAND_LINE_SIZE && options_chars--) {
+			u16 c = *s2++;
+
+			if (c < 0x80) {
+				if (c == L'\0' || c == L'\n')
+					break;
+				if (c == L'"')
+					in_quote = !in_quote;
+				else if (!in_quote && isspace((char)c))
+					safe_options_bytes = options_bytes;
+
+				options_bytes++;
+				continue;
+			}
+
+			/*
+			 * Get the number of UTF-8 bytes corresponding to a
+			 * UTF-16 character.
+			 * The first part handles everything in the BMP.
+			 */
+			options_bytes += 2 + (c >= 0x800);
+			/*
+			 * Add one more byte for valid surrogate pairs. Invalid
+			 * surrogates will be replaced with 0xfffd and take up
+			 * only 3 bytes.
+			 */
+			if ((c & 0xfc00) == 0xd800) {
+				/*
+				 * If the very last word is a high surrogate,
+				 * we must ignore it since we can't access the
+				 * low surrogate.
+				 */
+				if (!options_chars) {
+					options_bytes -= 3;
+				} else if ((*s2 & 0xfc00) == 0xdc00) {
+					options_bytes++;
+					options_chars--;
+					s2++;
+				}
+			}
+		}
+		if (options_bytes >= COMMAND_LINE_SIZE) {
+			options_bytes = safe_options_bytes;
+			printf("Command line is too long: truncated to %d bytes\n",
+			       options_bytes);
+		}
+        }
+
+	options_bytes++;        /* NUL termination */
+
+	status = efi_bs_call(allocate_pool, EFI_LOADER_DATA, options_bytes,
+			     (void **)&cmdline_addr);
+	if (status != EFI_SUCCESS)
+		return NULL;
+
+	snprintf((char *)cmdline_addr, options_bytes, "%.*ls",
+		 options_bytes - 1, options);
+
+	*cmd_line_len = options_bytes;
+	return (char *)cmdline_addr;
+}
+
 efi_status_t efi_main(efi_handle_t handle, efi_system_table_t *sys_tab)
 {
 	int ret;
@@ -109,6 +201,37 @@ efi_status_t efi_main(efi_handle_t handle, efi_system_table_t *sys_tab)
 	unsigned long map_size = 0, desc_size = 0, key = 0, buff_size = 0;
 	u32 desc_ver;
 
+	/* Helper variables needed to get the cmdline */
+	efi_loaded_image_t *image;
+	efi_guid_t loaded_image_proto = LOADED_IMAGE_PROTOCOL_GUID;
+	char *cmdline_ptr = NULL;
+	int cmdline_size = 0;
+
+	/*
+	 * Get a handle to the loaded image protocol.  This is used to get
+	 * information about the running image, such as size and the command
+	 * line.
+	 */
+	status = efi_bs_call(handle_protocol, handle, &loaded_image_proto,
+			     (void *)&image);
+	if (status != EFI_SUCCESS) {
+		printf("Failed to get loaded image protocol\n");
+		goto efi_main_error;
+	}
+
+	/*
+	 * Get the command line from EFI, using the LOADED_IMAGE
+	 * protocol. We are going to copy the command line into the
+	 * device tree, so this can be allocated anywhere.
+	 */
+	cmdline_ptr = efi_convert_cmdline(image, &cmdline_size);
+	if (!cmdline_ptr) {
+		printf("getting command line via LOADED_IMAGE_PROTOCOL\n");
+		status = EFI_OUT_OF_RESOURCES;
+		goto efi_main_error;
+	}
+	efi_cmdline_to_argv(cmdline_ptr);
+
 	/* Set up efi_bootinfo */
 	efi_bootinfo.mem_map.map = &map;
 	efi_bootinfo.mem_map.map_size = &map_size;
diff --git a/lib/string.c b/lib/string.c
index 27106da..b191ab1 100644
--- a/lib/string.c
+++ b/lib/string.c
@@ -163,7 +163,7 @@ void *memchr(const void *s, int c, size_t n)
     return NULL;
 }
 
-static int isspace(int c)
+int isspace(int c)
 {
     return c == ' ' || c == '\t' || c == '\r' || c == '\n' || c == '\v' || c == '\f';
 }
-- 
2.25.1

[kvm-unit-tests PATCH v2 12/23] arm/arm64: mmu_disable: Clean and invalidate before disabling

[Nikos Nikoleris]

From: Andrew Jones <drjones@redhat.com>

The commit message of commit 410b3bf09e76 ("arm/arm64: Perform dcache
clean + invalidate after turning MMU off") justifies cleaning and
invalidating the dcache after disabling the MMU by saying it's nice
not to rely on the current page tables and that it should still work
(per the spec), as long as there's an identity map in the current
tables. Doing the invalidation after also somewhat helped with
reenabling the MMU without seeing stale data, but the real problem
with reenabling was because the cache needs to be disabled with
the MMU, but it wasn't.

Since we have to trust/validate that the current page tables have an
identity map anyway, then there's no harm in doing the clean
and invalidate first (it feels a little better to do so, anyway,
considering the cache maintenance instructions take virtual
addresses). Then, also disable the cache with the MMU to avoid
problems when reenabling. We invalidate the Icache and disable
that too for good measure. And, a final TLB invalidation ensures
we're crystal clean when we return from asm_mmu_disable().

Cc: Alexandru Elisei <alexandru.elisei@arm.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>
Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
---
 arm/cstart.S   | 28 +++++++++++++++++++++-------
 arm/cstart64.S | 21 ++++++++++++++++-----
 2 files changed, 37 insertions(+), 12 deletions(-)

diff --git a/arm/cstart.S b/arm/cstart.S
index 7036e67..dc324c5 100644
--- a/arm/cstart.S
+++ b/arm/cstart.S
@@ -179,6 +179,7 @@ halt:
 .globl asm_mmu_enable
 asm_mmu_enable:
 	/* TLBIALL */
+	mov	r2, #0
 	mcr	p15, 0, r2, c8, c7, 0
 	dsb	nsh
 
@@ -211,12 +212,7 @@ asm_mmu_enable:
 
 .globl asm_mmu_disable
 asm_mmu_disable:
-	/* SCTLR */
-	mrc	p15, 0, r0, c1, c0, 0
-	bic	r0, #CR_M
-	mcr	p15, 0, r0, c1, c0, 0
-	isb
-
+	/* Clean + invalidate the entire memory */
 	ldr	r0, =__phys_offset
 	ldr	r0, [r0]
 	ldr	r1, =__phys_end
@@ -224,7 +220,25 @@ asm_mmu_disable:
 	sub	r1, r1, r0
 	dcache_by_line_op dccimvac, sy, r0, r1, r2, r3
 
-	mov     pc, lr
+	/* Invalidate Icache */
+	mov	r0, #0
+	mcr	p15, 0, r0, c7, c5, 0
+	isb
+
+	/*  Disable cache, Icache and MMU */
+	mrc	p15, 0, r0, c1, c0, 0
+	bic	r0, #CR_C
+	bic	r0, #CR_I
+	bic	r0, #CR_M
+	mcr	p15, 0, r0, c1, c0, 0
+	isb
+
+	/* Invalidate TLB */
+	mov	r0, #0
+	mcr	p15, 0, r0, c8, c7, 0
+	dsb	nsh
+
+	mov	pc, lr
 
 /*
  * Vectors
diff --git a/arm/cstart64.S b/arm/cstart64.S
index e4ab7d0..390feb9 100644
--- a/arm/cstart64.S
+++ b/arm/cstart64.S
@@ -246,11 +246,6 @@ asm_mmu_enable:
 
 .globl asm_mmu_disable
 asm_mmu_disable:
-	mrs	x0, sctlr_el1
-	bic	x0, x0, SCTLR_EL1_M
-	msr	sctlr_el1, x0
-	isb
-
 	/* Clean + invalidate the entire memory */
 	adrp	x0, __phys_offset
 	ldr	x0, [x0, :lo12:__phys_offset]
@@ -259,6 +254,22 @@ asm_mmu_disable:
 	sub	x1, x1, x0
 	dcache_by_line_op civac, sy, x0, x1, x2, x3
 
+	/* Invalidate Icache */
+	ic	iallu
+	isb
+
+	/* Disable cache, Icache and MMU */
+	mrs	x0, sctlr_el1
+	bic	x0, x0, SCTLR_EL1_C
+	bic	x0, x0, SCTLR_EL1_I
+	bic	x0, x0, SCTLR_EL1_M
+	msr	sctlr_el1, x0
+	isb
+
+	/* Invalidate TLB */
+	tlbi	vmalle1
+	dsb	nsh
+
 	ret
 
 /*
-- 
2.25.1

[kvm-unit-tests PATCH v2 13/23] arm/arm64: Rename etext to _etext

[Nikos Nikoleris]

From: Andrew Jones <drjones@redhat.com>

Rename etext to the more popular _etext allowing different linker
scripts to more easily be used.

Signed-off-by: Andrew Jones <drjones@redhat.com>
Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
---
 lib/arm/setup.c | 4 ++--
 arm/flat.lds    | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/lib/arm/setup.c b/lib/arm/setup.c
index 3c24c75..2d67292 100644
--- a/lib/arm/setup.c
+++ b/lib/arm/setup.c
@@ -34,7 +34,7 @@
 #define NR_EXTRA_MEM_REGIONS	16
 #define NR_INITIAL_MEM_REGIONS	(MAX_DT_MEM_REGIONS + NR_EXTRA_MEM_REGIONS)
 
-extern unsigned long etext;
+extern unsigned long _etext;
 
 char *initrd;
 u32 initrd_size;
@@ -140,7 +140,7 @@ unsigned int mem_region_get_flags(phys_addr_t paddr)
 
 static void mem_regions_add_assumed(void)
 {
-	phys_addr_t code_end = (phys_addr_t)(unsigned long)&etext;
+	phys_addr_t code_end = (phys_addr_t)(unsigned long)&_etext;
 	struct mem_region *r;
 
 	r = mem_region_find(code_end - 1);
diff --git a/arm/flat.lds b/arm/flat.lds
index 47fcb64..9016ac9 100644
--- a/arm/flat.lds
+++ b/arm/flat.lds
@@ -27,7 +27,7 @@ SECTIONS
     PROVIDE(_text = .);
     .text : { *(.init) *(.text) *(.text.*) }
     . = ALIGN(64K);
-    PROVIDE(etext = .);
+    PROVIDE(_etext = .);
 
     PROVIDE(reloc_start = .);
     .rela.dyn : { *(.rela.dyn) }
-- 
2.25.1

[kvm-unit-tests PATCH v2 14/23] lib: Avoid ms_abi for calls related to EFI on arm64

[Nikos Nikoleris]

x86_64 requires that EFI calls use the ms_abi calling convention. For
arm64 this is unnecessary.

Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
---
 lib/linux/efi.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/lib/linux/efi.h b/lib/linux/efi.h
index e3aba1d..594eaca 100644
--- a/lib/linux/efi.h
+++ b/lib/linux/efi.h
@@ -33,7 +33,11 @@ typedef u16 efi_char16_t;		/* UNICODE character */
 typedef u64 efi_physical_addr_t;
 typedef void *efi_handle_t;
 
+#ifdef __x86_64__
 #define __efiapi __attribute__((ms_abi))
+#else
+#define __efiapi
+#endif
 
 /*
  * The UEFI spec and EDK2 reference implementation both define EFI_GUID as
-- 
2.25.1

[kvm-unit-tests PATCH v2 15/23] arm64: Add a new type of memory type flag MR_F_RESERVED

[Nikos Nikoleris]

From: Andrew Jones <drjones@redhat.com>

This will be used by future change to add PTE entries for special EFI
memory regions.

Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
---
 lib/arm/asm/setup.h | 1 +
 lib/arm/mmu.c       | 4 ++++
 2 files changed, 5 insertions(+)

diff --git a/lib/arm/asm/setup.h b/lib/arm/asm/setup.h
index f0e70b1..64cd379 100644
--- a/lib/arm/asm/setup.h
+++ b/lib/arm/asm/setup.h
@@ -15,6 +15,7 @@ extern int nr_cpus;
 
 #define MR_F_IO			(1U << 0)
 #define MR_F_CODE		(1U << 1)
+#define MR_F_RESERVED		(1U << 2)
 #define MR_F_UNKNOWN		(1U << 31)
 
 struct mem_region {
diff --git a/lib/arm/mmu.c b/lib/arm/mmu.c
index e1a72fe..931be98 100644
--- a/lib/arm/mmu.c
+++ b/lib/arm/mmu.c
@@ -174,6 +174,10 @@ void *setup_mmu(phys_addr_t phys_end, void *unused)
 	for (r = mem_regions; r->end; ++r) {
 		if (r->flags & MR_F_IO) {
 			continue;
+		} else if (r->flags & MR_F_RESERVED) {
+			/* Reserved pages need to be writable for whatever reserved them */
+			mmu_set_range_ptes(mmu_idmap, r->start, r->start, r->end,
+					   __pgprot(PTE_WBWA));
 		} else if (r->flags & MR_F_CODE) {
 			/* armv8 requires code shared between EL1 and EL0 to be read-only */
 			mmu_set_range_ptes(mmu_idmap, r->start, r->start, r->end,
-- 
2.25.1

[kvm-unit-tests PATCH v2 16/23] arm/arm64: Add a setup sequence for systems that boot through EFI

[Nikos Nikoleris]

This change implements an alternative setup sequence for the system
when we are booting through EFI. The memory map is discovered through
EFI boot services and devices through ACPI.

This change is based on a change initially proposed by
Andrew Jones <drjones@redhat.com>

Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
---
 lib/linux/efi.h     |   1 +
 lib/arm/asm/setup.h |   2 +
 lib/arm/setup.c     | 181 +++++++++++++++++++++++++++++++++++++++++++-
 arm/cstart.S        |   1 +
 arm/cstart64.S      |   1 +
 5 files changed, 184 insertions(+), 2 deletions(-)

diff --git a/lib/linux/efi.h b/lib/linux/efi.h
index 594eaca..9b77c39 100644
--- a/lib/linux/efi.h
+++ b/lib/linux/efi.h
@@ -63,6 +63,7 @@ typedef guid_t efi_guid_t;
 	(c) & 0xff, ((c) >> 8) & 0xff, d } }
 
 #define ACPI_TABLE_GUID EFI_GUID(0xeb9d2d30, 0x2d88, 0x11d3, 0x9a, 0x16, 0x00, 0x90, 0x27, 0x3f, 0xc1, 0x4d)
+#define ACPI_20_TABLE_GUID EFI_GUID(0x8868e871, 0xe4f1, 0x11d3,  0xbc, 0x22, 0x00, 0x80, 0xc7, 0x3c, 0x88, 0x81)
 
 #define LOADED_IMAGE_PROTOCOL_GUID EFI_GUID(0x5b1b31a1, 0x9562, 0x11d2,  0x8e, 0x3f, 0x00, 0xa0, 0xc9, 0x69, 0x72, 0x3b)
 
diff --git a/lib/arm/asm/setup.h b/lib/arm/asm/setup.h
index 64cd379..1a7e734 100644
--- a/lib/arm/asm/setup.h
+++ b/lib/arm/asm/setup.h
@@ -5,6 +5,7 @@
  *
  * This work is licensed under the terms of the GNU LGPL, version 2.
  */
+#include <efi.h>
 #include <libcflat.h>
 #include <asm/page.h>
 #include <asm/pgtable-hwdef.h>
@@ -37,5 +38,6 @@ extern unsigned int mem_region_get_flags(phys_addr_t paddr);
 #define SMP_CACHE_BYTES		L1_CACHE_BYTES
 
 void setup(const void *fdt, phys_addr_t freemem_start);
+efi_status_t setup_efi(efi_bootinfo_t *efi_bootinfo);
 
 #endif /* _ASMARM_SETUP_H_ */
diff --git a/lib/arm/setup.c b/lib/arm/setup.c
index 2d67292..542e2ff 100644
--- a/lib/arm/setup.c
+++ b/lib/arm/setup.c
@@ -34,7 +34,7 @@
 #define NR_EXTRA_MEM_REGIONS	16
 #define NR_INITIAL_MEM_REGIONS	(MAX_DT_MEM_REGIONS + NR_EXTRA_MEM_REGIONS)
 
-extern unsigned long _etext;
+extern unsigned long _text, _etext, _data, _edata;
 
 char *initrd;
 u32 initrd_size;
@@ -44,7 +44,10 @@ int nr_cpus;
 
 static struct mem_region __initial_mem_regions[NR_INITIAL_MEM_REGIONS + 1];
 struct mem_region *mem_regions = __initial_mem_regions;
-phys_addr_t __phys_offset, __phys_end;
+phys_addr_t __phys_offset = (phys_addr_t)-1, __phys_end = 0;
+
+extern void exceptions_init(void);
+extern void asm_mmu_disable(void);
 
 int mpidr_to_cpu(uint64_t mpidr)
 {
@@ -272,3 +275,177 @@ void setup(const void *fdt, phys_addr_t freemem_start)
 	if (!(auxinfo.flags & AUXINFO_MMU_OFF))
 		setup_vm();
 }
+
+#ifdef CONFIG_EFI
+
+#include <efi.h>
+
+static efi_status_t setup_rsdp(efi_bootinfo_t *efi_bootinfo)
+{
+	efi_status_t status;
+	struct rsdp_descriptor *rsdp;
+
+	/*
+	 * RSDP resides in an EFI_ACPI_RECLAIM_MEMORY region, which is not used
+	 * by kvm-unit-tests arm64 memory allocator. So it is not necessary to
+	 * copy the data structure to another memory region to prevent
+	 * unintentional overwrite.
+	 */
+	status = efi_get_system_config_table(ACPI_20_TABLE_GUID, (void **)&rsdp);
+	if (status != EFI_SUCCESS)
+		return status;
+
+	set_efi_rsdp(rsdp);
+
+	return EFI_SUCCESS;
+}
+
+static efi_status_t efi_mem_init(efi_bootinfo_t *efi_bootinfo)
+{
+	int i;
+	unsigned long free_mem_pages = 0;
+	unsigned long free_mem_start = 0;
+	struct efi_boot_memmap *map = &(efi_bootinfo->mem_map);
+	efi_memory_desc_t *buffer = *map->map;
+	efi_memory_desc_t *d = NULL;
+	phys_addr_t base, top;
+	struct mem_region *r;
+	uintptr_t text = (uintptr_t)&_text, etext = __ALIGN((uintptr_t)&_etext, 4096);
+	uintptr_t data = (uintptr_t)&_data, edata = __ALIGN((uintptr_t)&_edata, 4096);
+
+	/*
+	 * Record the largest free EFI_CONVENTIONAL_MEMORY region
+	 * which will be used to set up the memory allocator, so that
+	 * the memory allocator can work in the largest free
+	 * continuous memory region.
+	 */
+	for (i = 0, r = &mem_regions[0]; i < *(map->map_size); i += *(map->desc_size), ++r) {
+		d = (efi_memory_desc_t *)(&((u8 *)buffer)[i]);
+
+		r->start = d->phys_addr;
+		r->end = d->phys_addr + d->num_pages * EFI_PAGE_SIZE;
+
+		switch (d->type) {
+		case EFI_RESERVED_TYPE:
+		case EFI_LOADER_DATA:
+		case EFI_BOOT_SERVICES_CODE:
+		case EFI_BOOT_SERVICES_DATA:
+		case EFI_RUNTIME_SERVICES_CODE:
+		case EFI_RUNTIME_SERVICES_DATA:
+		case EFI_UNUSABLE_MEMORY:
+		case EFI_ACPI_RECLAIM_MEMORY:
+		case EFI_ACPI_MEMORY_NVS:
+		case EFI_PAL_CODE:
+			r->flags = MR_F_RESERVED;
+			break;
+		case EFI_MEMORY_MAPPED_IO:
+		case EFI_MEMORY_MAPPED_IO_PORT_SPACE:
+			r->flags = MR_F_IO;
+			break;
+		case EFI_LOADER_CODE:
+			if (r->start <= text && r->end > text) {
+				/* This is the unit test region. Flag the code separately. */
+				phys_addr_t tmp = r->end;
+
+				assert(etext <= data);
+				assert(edata <= r->end);
+				r->flags = MR_F_CODE;
+				r->end = data;
+				++r;
+				r->start = data;
+				r->end = tmp;
+			} else {
+				r->flags = MR_F_RESERVED;
+			}
+			break;
+		case EFI_CONVENTIONAL_MEMORY:
+			if (free_mem_pages < d->num_pages) {
+				free_mem_pages = d->num_pages;
+				free_mem_start = d->phys_addr;
+			}
+			break;
+		}
+
+		if (!(r->flags & MR_F_IO)) {
+			if (r->start < __phys_offset)
+				__phys_offset = r->start;
+			if (r->end > __phys_end)
+				__phys_end = r->end;
+		}
+	}
+	__phys_end &= PHYS_MASK;
+	asm_mmu_disable();
+
+	if (free_mem_pages == 0)
+		return EFI_OUT_OF_RESOURCES;
+
+	assert(sizeof(long) == 8 || free_mem_start < (3ul << 30));
+
+	phys_alloc_init(free_mem_start, free_mem_pages << EFI_PAGE_SHIFT);
+	phys_alloc_set_minimum_alignment(SMP_CACHE_BYTES);
+
+	phys_alloc_get_unused(&base, &top);
+	base = PAGE_ALIGN(base);
+	top = top & PAGE_MASK;
+	assert(sizeof(long) == 8 || !(base >> 32));
+	if (sizeof(long) != 8 && (top >> 32) != 0)
+		top = ((uint64_t)1 << 32);
+	page_alloc_init_area(0, base >> PAGE_SHIFT, top >> PAGE_SHIFT);
+	page_alloc_ops_enable();
+
+	return EFI_SUCCESS;
+}
+
+efi_status_t setup_efi(efi_bootinfo_t *efi_bootinfo)
+{
+	efi_status_t status;
+
+	struct thread_info *ti = current_thread_info();
+
+	memset(ti, 0, sizeof(*ti));
+
+	exceptions_init();
+
+	status = efi_mem_init(efi_bootinfo);
+	if (status != EFI_SUCCESS) {
+		printf("Failed to initialize memory: ");
+		switch (status) {
+		case EFI_OUT_OF_RESOURCES:
+			printf("No free memory region\n");
+			break;
+		default:
+			printf("Unknown error\n");
+			break;
+		}
+		return status;
+	}
+
+	status = setup_rsdp(efi_bootinfo);
+	if (status != EFI_SUCCESS) {
+		printf("Cannot find RSDP in EFI system table\n");
+		return status;
+	}
+
+	psci_set_conduit();
+	cpu_init();
+	/* cpu_init must be called before thread_info_init */
+	thread_info_init(current_thread_info(), 0);
+	/* mem_init must be called before io_init */
+	io_init();
+
+	timer_save_state();
+	if (initrd) {
+		/* environ is currently the only file in the initrd */
+		char *env = malloc(initrd_size);
+
+		memcpy(env, initrd, initrd_size);
+		setup_env(env, initrd_size);
+	}
+
+	if (!(auxinfo.flags & AUXINFO_MMU_OFF))
+		setup_vm();
+
+	return EFI_SUCCESS;
+}
+
+#endif
diff --git a/arm/cstart.S b/arm/cstart.S
index dc324c5..66a55b9 100644
--- a/arm/cstart.S
+++ b/arm/cstart.S
@@ -256,6 +256,7 @@ asm_mmu_disable:
  *
  * Input r0 is the stack top, which is the exception stacks base
  */
+.globl exceptions_init
 exceptions_init:
 	mrc	p15, 0, r2, c1, c0, 0	@ read SCTLR
 	bic	r2, #CR_V		@ SCTLR.V := 0
diff --git a/arm/cstart64.S b/arm/cstart64.S
index 390feb9..55b41ea 100644
--- a/arm/cstart64.S
+++ b/arm/cstart64.S
@@ -276,6 +276,7 @@ asm_mmu_disable:
  * Vectors
  */
 
+.globl exceptions_init
 exceptions_init:
 	adrp	x4, vector_table
 	add	x4, x4, :lo12:vector_table
-- 
2.25.1

[kvm-unit-tests PATCH v2 17/23] arm64: Copy code from GNU-EFI

[Nikos Nikoleris]

This change adds unmodified dependencies that we need from GNU-EFI in
order to build arm64 EFI apps.

GNU-EFI sources from  https://git.code.sf.net/p/gnu-efi/code v3.0.14

Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
---
 arm/efi/elf_aarch64_efi.lds |  63 +++++++++++++++++
 arm/efi/crt0-efi-aarch64.S  | 130 ++++++++++++++++++++++++++++++++++++
 arm/efi/reloc_aarch64.c     |  97 +++++++++++++++++++++++++++
 3 files changed, 290 insertions(+)
 create mode 100644 arm/efi/elf_aarch64_efi.lds
 create mode 100644 arm/efi/crt0-efi-aarch64.S
 create mode 100644 arm/efi/reloc_aarch64.c

diff --git a/arm/efi/elf_aarch64_efi.lds b/arm/efi/elf_aarch64_efi.lds
new file mode 100644
index 0000000..836d982
--- /dev/null
+++ b/arm/efi/elf_aarch64_efi.lds
@@ -0,0 +1,63 @@
+OUTPUT_FORMAT("elf64-littleaarch64", "elf64-littleaarch64", "elf64-littleaarch64")
+OUTPUT_ARCH(aarch64)
+ENTRY(_start)
+SECTIONS
+{
+  .text 0x0 : {
+    _text = .;
+    *(.text.head)
+    *(.text)
+    *(.text.*)
+    *(.gnu.linkonce.t.*)
+    *(.srodata)
+    *(.rodata*)
+    . = ALIGN(16);
+  }
+  _etext = .;
+  _text_size = . - _text;
+  .dynamic  : { *(.dynamic) }
+  .data : ALIGN(4096)
+  {
+   _data = .;
+   *(.sdata)
+   *(.data)
+   *(.data1)
+   *(.data.*)
+   *(.got.plt)
+   *(.got)
+
+   /* the EFI loader doesn't seem to like a .bss section, so we stick
+      it all into .data: */
+   . = ALIGN(16);
+   _bss = .;
+   *(.sbss)
+   *(.scommon)
+   *(.dynbss)
+   *(.bss)
+   *(COMMON)
+   . = ALIGN(16);
+   _bss_end = .;
+  }
+
+  .rela.dyn : { *(.rela.dyn) }
+  .rela.plt : { *(.rela.plt) }
+  .rela.got : { *(.rela.got) }
+  .rela.data : { *(.rela.data) *(.rela.data*) }
+  . = ALIGN(512);
+  _edata = .;
+  _data_size = . - _data;
+
+  . = ALIGN(4096);
+  .dynsym   : { *(.dynsym) }
+  . = ALIGN(4096);
+  .dynstr   : { *(.dynstr) }
+  . = ALIGN(4096);
+  .note.gnu.build-id : { *(.note.gnu.build-id) }
+  /DISCARD/ :
+  {
+    *(.rel.reloc)
+    *(.eh_frame)
+    *(.note.GNU-stack)
+  }
+  .comment 0 : { *(.comment) }
+}
diff --git a/arm/efi/crt0-efi-aarch64.S b/arm/efi/crt0-efi-aarch64.S
new file mode 100644
index 0000000..d50e78d
--- /dev/null
+++ b/arm/efi/crt0-efi-aarch64.S
@@ -0,0 +1,130 @@
+/*
+ * crt0-efi-aarch64.S - PE/COFF header for AArch64 EFI applications
+ *
+ * Copright (C) 2014 Linaro Ltd. <ard.biesheuvel@linaro.org>
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice and this list of conditions, without modification.
+ * 2. The name of the author may not be used to endorse or promote products
+ *    derived from this software without specific prior written permission.
+ *
+ * Alternatively, this software may be distributed under the terms of the
+ * GNU General Public License as published by the Free Software Foundation;
+ * either version 2 of the License, or (at your option) any later version.
+ */
+
+	.section	.text.head
+
+	/*
+	 * Magic "MZ" signature for PE/COFF
+	 */
+	.globl	ImageBase
+ImageBase:
+	.ascii	"MZ"
+	.skip	58				// 'MZ' + pad + offset == 64
+	.long	pe_header - ImageBase		// Offset to the PE header.
+pe_header:
+	.ascii	"PE"
+	.short 	0
+coff_header:
+	.short	0xaa64				// AArch64
+	.short	2				// nr_sections
+	.long	0 				// TimeDateStamp
+	.long	0				// PointerToSymbolTable
+	.long	0				// NumberOfSymbols
+	.short	section_table - optional_header	// SizeOfOptionalHeader
+	.short	0x206				// Characteristics.
+						// IMAGE_FILE_DEBUG_STRIPPED |
+						// IMAGE_FILE_EXECUTABLE_IMAGE |
+						// IMAGE_FILE_LINE_NUMS_STRIPPED
+optional_header:
+	.short	0x20b				// PE32+ format
+	.byte	0x02				// MajorLinkerVersion
+	.byte	0x14				// MinorLinkerVersion
+	.long	_data - _start			// SizeOfCode
+	.long	_data_size			// SizeOfInitializedData
+	.long	0				// SizeOfUninitializedData
+	.long	_start - ImageBase		// AddressOfEntryPoint
+	.long	_start - ImageBase		// BaseOfCode
+
+extra_header_fields:
+	.quad	0				// ImageBase
+	.long	0x1000				// SectionAlignment
+	.long	0x200				// FileAlignment
+	.short	0				// MajorOperatingSystemVersion
+	.short	0				// MinorOperatingSystemVersion
+	.short	0				// MajorImageVersion
+	.short	0				// MinorImageVersion
+	.short	0				// MajorSubsystemVersion
+	.short	0				// MinorSubsystemVersion
+	.long	0				// Win32VersionValue
+
+	.long	_edata - ImageBase		// SizeOfImage
+
+	// Everything before the kernel image is considered part of the header
+	.long	_start - ImageBase		// SizeOfHeaders
+	.long	0				// CheckSum
+	.short	EFI_SUBSYSTEM			// Subsystem
+	.short	0				// DllCharacteristics
+	.quad	0				// SizeOfStackReserve
+	.quad	0				// SizeOfStackCommit
+	.quad	0				// SizeOfHeapReserve
+	.quad	0				// SizeOfHeapCommit
+	.long	0				// LoaderFlags
+	.long	0x6				// NumberOfRvaAndSizes
+
+	.quad	0				// ExportTable
+	.quad	0				// ImportTable
+	.quad	0				// ResourceTable
+	.quad	0				// ExceptionTable
+	.quad	0				// CertificationTable
+	.quad	0				// BaseRelocationTable
+
+	// Section table
+section_table:
+	.ascii	".text\0\0\0"
+	.long	_data - _start		// VirtualSize
+	.long	_start - ImageBase	// VirtualAddress
+	.long	_data - _start		// SizeOfRawData
+	.long	_start - ImageBase	// PointerToRawData
+
+	.long	0		// PointerToRelocations (0 for executables)
+	.long	0		// PointerToLineNumbers (0 for executables)
+	.short	0		// NumberOfRelocations  (0 for executables)
+	.short	0		// NumberOfLineNumbers  (0 for executables)
+	.long	0x60000020	// Characteristics (section flags)
+
+	.ascii	".data\0\0\0"
+	.long	_data_size		// VirtualSize
+	.long	_data - ImageBase	// VirtualAddress
+	.long	_data_size		// SizeOfRawData
+	.long	_data - ImageBase	// PointerToRawData
+
+	.long	0		// PointerToRelocations (0 for executables)
+	.long	0		// PointerToLineNumbers (0 for executables)
+	.short	0		// NumberOfRelocations  (0 for executables)
+	.short	0		// NumberOfLineNumbers  (0 for executables)
+	.long	0xc0000040	// Characteristics (section flags)
+
+	.align		12
+_start:
+	stp		x29, x30, [sp, #-32]!
+	mov		x29, sp
+
+	stp		x0, x1, [sp, #16]
+	mov		x2, x0
+	mov		x3, x1
+	adr		x0, ImageBase
+	adrp		x1, _DYNAMIC
+	add		x1, x1, #:lo12:_DYNAMIC
+	bl		_relocate
+	cbnz		x0, 0f
+
+	ldp		x0, x1, [sp, #16]
+	bl		efi_main
+
+0:	ldp		x29, x30, [sp], #32
+	ret
diff --git a/arm/efi/reloc_aarch64.c b/arm/efi/reloc_aarch64.c
new file mode 100644
index 0000000..0867279
--- /dev/null
+++ b/arm/efi/reloc_aarch64.c
@@ -0,0 +1,97 @@
+/* reloc_aarch64.c - position independent x86 ELF shared object relocator
+   Copyright (C) 2014 Linaro Ltd. <ard.biesheuvel@linaro.org>
+   Copyright (C) 1999 Hewlett-Packard Co.
+	Contributed by David Mosberger <davidm@hpl.hp.com>.
+
+    All rights reserved.
+
+    Redistribution and use in source and binary forms, with or without
+    modification, are permitted provided that the following conditions
+    are met:
+
+    * Redistributions of source code must retain the above copyright
+      notice, this list of conditions and the following disclaimer.
+    * Redistributions in binary form must reproduce the above
+      copyright notice, this list of conditions and the following
+      disclaimer in the documentation and/or other materials
+      provided with the distribution.
+    * Neither the name of Hewlett-Packard Co. nor the names of its
+      contributors may be used to endorse or promote products derived
+      from this software without specific prior written permission.
+
+    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+    CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+    INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+    MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+    DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS
+    BE LIABLE FOR ANYDIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY,
+    OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+    PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+    PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR
+    TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+    THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+    SUCH DAMAGE.
+*/
+
+#include <efi.h>
+#include <efilib.h>
+
+#include <elf.h>
+
+EFI_STATUS _relocate (long ldbase, Elf64_Dyn *dyn,
+		      EFI_HANDLE image EFI_UNUSED,
+		      EFI_SYSTEM_TABLE *systab EFI_UNUSED)
+{
+	long relsz = 0, relent = 0;
+	Elf64_Rela *rel = 0;
+	unsigned long *addr;
+	int i;
+
+	for (i = 0; dyn[i].d_tag != DT_NULL; ++i) {
+		switch (dyn[i].d_tag) {
+			case DT_RELA:
+				rel = (Elf64_Rela*)
+					((unsigned long)dyn[i].d_un.d_ptr
+					 + ldbase);
+				break;
+
+			case DT_RELASZ:
+				relsz = dyn[i].d_un.d_val;
+				break;
+
+			case DT_RELAENT:
+				relent = dyn[i].d_un.d_val;
+				break;
+
+			default:
+				break;
+		}
+	}
+
+	if (!rel && relent == 0)
+		return EFI_SUCCESS;
+
+	if (!rel || relent == 0)
+		return EFI_LOAD_ERROR;
+
+	while (relsz > 0) {
+		/* apply the relocs */
+		switch (ELF64_R_TYPE (rel->r_info)) {
+			case R_AARCH64_NONE:
+				break;
+
+			case R_AARCH64_RELATIVE:
+				addr = (unsigned long *)
+					(ldbase + rel->r_offset);
+				*addr = ldbase + rel->r_addend;
+				break;
+
+			default:
+				break;
+		}
+		rel = (Elf64_Rela*) ((char *) rel + relent);
+		relsz -= relent;
+	}
+	return EFI_SUCCESS;
+}
-- 
2.25.1

[kvm-unit-tests PATCH v2 18/23] arm64: Change gnu-efi imported file to use defined types

[Nikos Nikoleris]

Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
---
 arm/efi/reloc_aarch64.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/arm/efi/reloc_aarch64.c b/arm/efi/reloc_aarch64.c
index 0867279..fa0cd6b 100644
--- a/arm/efi/reloc_aarch64.c
+++ b/arm/efi/reloc_aarch64.c
@@ -34,14 +34,11 @@
     SUCH DAMAGE.
 */
 
-#include <efi.h>
-#include <efilib.h>
-
+#include "efi.h"
 #include <elf.h>
 
-EFI_STATUS _relocate (long ldbase, Elf64_Dyn *dyn,
-		      EFI_HANDLE image EFI_UNUSED,
-		      EFI_SYSTEM_TABLE *systab EFI_UNUSED)
+efi_status_t _relocate(long ldbase, Elf64_Dyn *dyn, efi_handle_t image,
+		       efi_system_table_t *sys_tab)
 {
 	long relsz = 0, relent = 0;
 	Elf64_Rela *rel = 0;
-- 
2.25.1

[kvm-unit-tests PATCH v2 19/23] arm64: Use code from the gnu-efi when booting with EFI

[Nikos Nikoleris]

arm/efi/crt0-efi-aarch64.S defines the header and the handover
sequence from EFI to a efi_main. This change includes the whole file
in arm/cstart64.S when we compile with EFI support.

In addition, we change the handover code in arm/efi/crt0-efi-aarch64.S
to align the stack pointer. This alignment is necessary because we
make assumptions about cpu0's stack alignment and most importantly we
place its thread_info at the bottom of this stack.

Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
---
 arm/cstart64.S             |  6 ++++++
 arm/efi/crt0-efi-aarch64.S | 17 +++++++++++++++--
 2 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/arm/cstart64.S b/arm/cstart64.S
index 55b41ea..08cf02f 100644
--- a/arm/cstart64.S
+++ b/arm/cstart64.S
@@ -15,6 +15,10 @@
 #include <asm/thread_info.h>
 #include <asm/sysreg.h>
 
+#ifdef CONFIG_EFI
+#include "efi/crt0-efi-aarch64.S"
+#else
+
 .macro zero_range, tmp1, tmp2
 9998:	cmp	\tmp1, \tmp2
 	b.eq	9997f
@@ -107,6 +111,8 @@ start:
 	bl	exit
 	b	halt
 
+#endif
+
 .text
 
 /*
diff --git a/arm/efi/crt0-efi-aarch64.S b/arm/efi/crt0-efi-aarch64.S
index d50e78d..11a062d 100644
--- a/arm/efi/crt0-efi-aarch64.S
+++ b/arm/efi/crt0-efi-aarch64.S
@@ -111,10 +111,19 @@ section_table:
 
 	.align		12
 _start:
-	stp		x29, x30, [sp, #-32]!
+	stp		x29, x30, [sp, #-16]!
+
+	// Align sp; this is necessary due to way we store cpu0's thread_info
 	mov		x29, sp
+	and		x29, x29, #THREAD_MASK
+	mov		x30, sp
+	mov		sp, x29
+	str		x30, [sp, #-32]!
+
+	mov             x29, sp
 
 	stp		x0, x1, [sp, #16]
+
 	mov		x2, x0
 	mov		x3, x1
 	adr		x0, ImageBase
@@ -126,5 +135,9 @@ _start:
 	ldp		x0, x1, [sp, #16]
 	bl		efi_main
 
-0:	ldp		x29, x30, [sp], #32
+	// Restore sp
+	ldr		x30, [sp]
+	mov             sp, x30
+
+0:	ldp		x29, x30, [sp], #16
 	ret
-- 
2.25.1

[kvm-unit-tests PATCH v2 20/23] lib: Avoid external dependency in libelf

[Nikos Nikoleris]

There is just a small number of definitions we need from
uapi/linux/elf.h and asm/elf.h and the relocation code is
self-contained.

Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
---
 arm/efi/reloc_aarch64.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/arm/efi/reloc_aarch64.c b/arm/efi/reloc_aarch64.c
index fa0cd6b..3f6d9a6 100644
--- a/arm/efi/reloc_aarch64.c
+++ b/arm/efi/reloc_aarch64.c
@@ -34,8 +34,7 @@
     SUCH DAMAGE.
 */
 
-#include "efi.h"
-#include <elf.h>
+#include <efi.h>
 
 efi_status_t _relocate(long ldbase, Elf64_Dyn *dyn, efi_handle_t image,
 		       efi_system_table_t *sys_tab)
-- 
2.25.1

[kvm-unit-tests PATCH v2 21/23] x86: Move x86_64-specific EFI CFLAGS to x86_64 Makefile

[Nikos Nikoleris]

Compiler flag -macculate-outgoing-args is only needed by the x86_64
ABI. Move it to the relevant Makefile.

Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
---
 Makefile            | 4 ----
 x86/Makefile.x86_64 | 4 ++++
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/Makefile b/Makefile
index 6ed5dea..307bc29 100644
--- a/Makefile
+++ b/Makefile
@@ -40,14 +40,10 @@ OBJDIRS += $(LIBFDT_objdir)
 
 # EFI App
 ifeq ($(CONFIG_EFI),y)
-EFI_ARCH = x86_64
 EFI_CFLAGS := -DCONFIG_EFI
 # The following CFLAGS and LDFLAGS come from:
 #   - GNU-EFI/Makefile.defaults
 #   - GNU-EFI/apps/Makefile
-# Function calls must include the number of arguments passed to the functions
-# More details: https://wiki.osdev.org/GNU-EFI
-EFI_CFLAGS += -maccumulate-outgoing-args
 # GCC defines wchar to be 32 bits, but EFI expects 16 bits
 EFI_CFLAGS += -fshort-wchar
 # EFI applications use PIC as they are loaded to dynamic addresses, not a fixed
diff --git a/x86/Makefile.x86_64 b/x86/Makefile.x86_64
index f18c1e2..ac588ed 100644
--- a/x86/Makefile.x86_64
+++ b/x86/Makefile.x86_64
@@ -2,6 +2,10 @@ cstart.o = $(TEST_DIR)/cstart64.o
 bits = 64
 ldarch = elf64-x86-64
 ifeq ($(CONFIG_EFI),y)
+# Function calls must include the number of arguments passed to the functions
+# More details: https://wiki.osdev.org/GNU-EFI
+CFLAGS += -maccumulate-outgoing-args
+
 exe = efi
 bin = so
 FORMAT = efi-app-x86_64
-- 
2.25.1

[kvm-unit-tests PATCH v2 22/23] arm64: Add support for efi in Makefile

[Nikos Nikoleris]

Users can now build kvm-unit-tests as efi apps by supplying an extra
argument when invoking configure:

$> ./configure --enable-efi

This patch is based on an earlier version by
Andrew Jones <drjones@redhat.com>

Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
---
 configure           | 15 ++++++++++++---
 arm/Makefile.arm    |  6 ++++++
 arm/Makefile.arm64  | 18 ++++++++++++++----
 arm/Makefile.common | 45 ++++++++++++++++++++++++++++++++++-----------
 4 files changed, 66 insertions(+), 18 deletions(-)

diff --git a/configure b/configure
index 86c3095..beef655 100755
--- a/configure
+++ b/configure
@@ -195,14 +195,19 @@ else
     fi
 fi
 
-if [ "$efi" ] && [ "$arch" != "x86_64" ]; then
+if [ "$efi" ] && [ "$arch" != "x86_64" ] && [ "$arch" != "arm64" ]; then
     echo "--[enable|disable]-efi is not supported for $arch"
     usage
 fi
 
 if [ -z "$page_size" ]; then
-    [ "$arch" = "arm64" ] && page_size="65536"
-    [ "$arch" = "arm" ] && page_size="4096"
+    if [ "$efi" = 'y' ] && [ "$arch" = "arm64" ]; then
+        page_size="4096"
+    elif [ "$arch" = "arm64" ]; then
+        page_size="65536"
+    elif [ "$arch" = "arm" ]; then
+        page_size="4096"
+    fi
 else
     if [ "$arch" != "arm64" ]; then
         echo "--page-size is not supported for $arch"
@@ -217,6 +222,10 @@ else
         echo "arm64 doesn't support page size of $page_size"
         usage
     fi
+    if [ "$efi" = 'y' ] && [ "$page_size" != "4096" ]; then
+        echo "efi must use 4K pages"
+        exit 1
+    fi
 fi
 
 [ -z "$processor" ] && processor="$arch"
diff --git a/arm/Makefile.arm b/arm/Makefile.arm
index 01fd4c7..2ce00f5 100644
--- a/arm/Makefile.arm
+++ b/arm/Makefile.arm
@@ -7,6 +7,10 @@ bits = 32
 ldarch = elf32-littlearm
 machine = -marm -mfpu=vfp
 
+ifeq ($(CONFIG_EFI),y)
+$(error Cannot build arm32 tests as EFI apps)
+endif
+
 # stack.o relies on frame pointers.
 KEEP_FRAME_POINTER := y
 
@@ -32,6 +36,8 @@ cflatobjs += lib/arm/stack.o
 cflatobjs += lib/ldiv32.o
 cflatobjs += lib/arm/ldivmod.o
 
+exe = flat
+
 # arm specific tests
 tests =
 
diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
index 6feac76..550e1b2 100644
--- a/arm/Makefile.arm64
+++ b/arm/Makefile.arm64
@@ -27,11 +27,21 @@ cflatobjs += lib/arm64/gic-v3-its.o lib/arm64/gic-v3-its-cmd.o
 
 OBJDIRS += lib/arm64
 
+ifeq ($(CONFIG_EFI),y)
+# avoid jump tables before all relocations have been processed
+arm/efi/reloc_aarch64.o: CFLAGS += -fno-jump-tables
+cflatobjs += arm/efi/reloc_aarch64.o
+
+exe = efi
+else
+exe = flat
+endif
+
 # arm64 specific tests
-tests = $(TEST_DIR)/timer.flat
-tests += $(TEST_DIR)/micro-bench.flat
-tests += $(TEST_DIR)/cache.flat
-tests += $(TEST_DIR)/debug.flat
+tests = $(TEST_DIR)/timer.$(exe)
+tests += $(TEST_DIR)/micro-bench.$(exe)
+tests += $(TEST_DIR)/cache.$(exe)
+tests += $(TEST_DIR)/debug.$(exe)
 
 include $(SRCDIR)/$(TEST_DIR)/Makefile.common
 
diff --git a/arm/Makefile.common b/arm/Makefile.common
index 5be42c0..a8007f4 100644
--- a/arm/Makefile.common
+++ b/arm/Makefile.common
@@ -4,14 +4,14 @@
 # Authors: Andrew Jones <drjones@redhat.com>
 #
 
-tests-common  = $(TEST_DIR)/selftest.flat
-tests-common += $(TEST_DIR)/spinlock-test.flat
-tests-common += $(TEST_DIR)/pci-test.flat
-tests-common += $(TEST_DIR)/pmu.flat
-tests-common += $(TEST_DIR)/gic.flat
-tests-common += $(TEST_DIR)/psci.flat
-tests-common += $(TEST_DIR)/sieve.flat
-tests-common += $(TEST_DIR)/pl031.flat
+tests-common  = $(TEST_DIR)/selftest.$(exe)
+tests-common += $(TEST_DIR)/spinlock-test.$(exe)
+tests-common += $(TEST_DIR)/pci-test.$(exe)
+tests-common += $(TEST_DIR)/pmu.$(exe)
+tests-common += $(TEST_DIR)/gic.$(exe)
+tests-common += $(TEST_DIR)/psci.$(exe)
+tests-common += $(TEST_DIR)/sieve.$(exe)
+tests-common += $(TEST_DIR)/pl031.$(exe)
 
 tests-all = $(tests-common) $(tests)
 all: directories $(tests-all)
@@ -54,6 +54,9 @@ cflatobjs += lib/arm/smp.o
 cflatobjs += lib/arm/delay.o
 cflatobjs += lib/arm/gic.o lib/arm/gic-v2.o lib/arm/gic-v3.o
 cflatobjs += lib/arm/timer.o
+ifeq ($(CONFIG_EFI),y)
+cflatobjs += lib/efi.o
+endif
 
 OBJDIRS += lib/arm
 
@@ -61,6 +64,25 @@ libeabi = lib/arm/libeabi.a
 eabiobjs = lib/arm/eabi_compat.o
 
 FLATLIBS = $(libcflat) $(LIBFDT_archive) $(libeabi)
+
+ifeq ($(CONFIG_EFI),y)
+%.so: EFI_LDFLAGS += -defsym=EFI_SUBSYSTEM=0xa --no-undefined
+%.so: %.o $(FLATLIBS) $(SRCDIR)/arm/efi/elf_aarch64_efi.lds $(cstart.o)
+	$(CC) $(CFLAGS) -c -o $(@:.so=.aux.o) $(SRCDIR)/lib/auxinfo.c \
+		-DPROGNAME=\"$(@:.so=.efi)\" -DAUXFLAGS=$(AUXFLAGS)
+	$(LD) $(EFI_LDFLAGS) -o $@ -T $(SRCDIR)/arm/efi/elf_aarch64_efi.lds \
+		$(filter %.o, $^) $(FLATLIBS) $(@:.so=.aux.o) \
+		$(EFI_LIBS)
+	$(RM) $(@:.so=.aux.o)
+
+%.efi: %.so
+	$(call arch_elf_check, $^)
+	$(OBJCOPY) \
+		-j .text -j .sdata -j .data -j .dynamic -j .dynsym \
+		-j .rel -j .rela -j .rel.* -j .rela.* -j .rel* -j .rela* \
+		-j .reloc \
+		-O binary $^ $@
+else
 %.elf: LDFLAGS = -nostdlib $(arch_LDFLAGS)
 %.elf: %.o $(FLATLIBS) $(SRCDIR)/arm/flat.lds $(cstart.o)
 	$(CC) $(CFLAGS) -c -o $(@:.elf=.aux.o) $(SRCDIR)/lib/auxinfo.c \
@@ -74,13 +96,14 @@ FLATLIBS = $(libcflat) $(LIBFDT_archive) $(libeabi)
 	$(call arch_elf_check, $^)
 	$(OBJCOPY) -O binary $^ $@
 	@chmod a-x $@
+endif
 
 $(libeabi): $(eabiobjs)
 	$(AR) rcs $@ $^
 
 arm_clean: asm_offsets_clean
-	$(RM) $(TEST_DIR)/*.{o,flat,elf} $(libeabi) $(eabiobjs) \
-	      $(TEST_DIR)/.*.d lib/arm/.*.d
+	$(RM) $(TEST_DIR)/*.{o,flat,elf,so,efi} $(libeabi) $(eabiobjs) \
+	      $(TEST_DIR)/.*.d $(TEST_DIR)/efi/.*.d lib/arm/.*.d
 
 generated-files = $(asm-offsets)
-$(tests-all:.flat=.o) $(cstart.o) $(cflatobjs): $(generated-files)
+$(tests-all:.$(exe)=.o) $(cstart.o) $(cflatobjs): $(generated-files)
-- 
2.25.1

[kvm-unit-tests PATCH v2 23/23] arm64: Add an efi/run script

[Nikos Nikoleris]

This change adds a efi/run script inspired by the one in x86. This
script will setup a folder with the test compiled as an EFI app and a
startup.nsh script. The script launches QEMU providing an image with
EDKII and the path to the folder with the test which is executed
automatically.

For example:

$> ./arm/efi/run ./arm/selftest.efi setup smp=2 mem=256

Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
---
 scripts/runtime.bash | 14 +++++-----
 arm/efi/run          | 61 ++++++++++++++++++++++++++++++++++++++++++++
 arm/run              |  8 ++++--
 arm/Makefile.common  |  1 +
 arm/dummy.c          |  4 +++
 5 files changed, 78 insertions(+), 10 deletions(-)
 create mode 100755 arm/efi/run
 create mode 100644 arm/dummy.c

diff --git a/scripts/runtime.bash b/scripts/runtime.bash
index 7d0180b..dc28f24 100644
--- a/scripts/runtime.bash
+++ b/scripts/runtime.bash
@@ -131,14 +131,12 @@ function run()
     fi
 
     last_line=$(premature_failure > >(tail -1)) && {
-        skip=true
-        if [ "${CONFIG_EFI}" == "y" ] && [[ "${last_line}" =~ "enabling apic" ]]; then
-            skip=false
-        fi
-        if [ ${skip} == true ]; then
-            print_result "SKIP" $testname "" "$last_line"
-            return 77
-        fi
+        if [ "${CONFIG_EFI}" == "y" ] && [ "${ARCH}" = x86_64 ]; then
+		if ! [[ "${last_line}" =~ "enabling apic" ]]; then
+			print_result "SKIP" $testname "" "$last_line"
+			return 77
+		fi
+	fi
     }
 
     cmdline=$(get_cmdline $kernel)
diff --git a/arm/efi/run b/arm/efi/run
new file mode 100755
index 0000000..dfff717
--- /dev/null
+++ b/arm/efi/run
@@ -0,0 +1,61 @@
+#!/bin/bash
+
+set -e
+
+if [ $# -eq 0 ]; then
+	echo "Usage $0 TEST_CASE [QEMU_ARGS]"
+	exit 2
+fi
+
+if [ ! -f config.mak ]; then
+	echo "run './configure --enable-efi && make' first. See ./configure -h"
+	exit 2
+fi
+source config.mak
+source scripts/arch-run.bash
+source scripts/common.bash
+
+: "${EFI_SRC:=$(realpath "$(dirname "$0")/../")}"
+: "${EFI_UEFI:=/usr/share/qemu-efi-aarch64/QEMU_EFI.fd}"
+: "${EFI_TEST:=efi-tests}"
+: "${EFI_CASE:=$(basename $1 .efi)}"
+
+if [ ! -f "$EFI_UEFI" ]; then
+	echo "UEFI firmware not found: $EFI_UEFI"
+	echo "Please install the UEFI firmware to this path"
+	echo "Or specify the correct path with the env variable EFI_UEFI"
+	exit 2
+fi
+
+# Remove the TEST_CASE from $@
+shift 1
+
+# Fish out the arguments for the test, they should be the next string
+# after the "-append" option
+qemu_args=()
+cmd_args=()
+while (( "$#" )); do
+	if [ "$1" = "-append" ]; then
+		cmd_args=$2
+		shift 2
+	else
+		qemu_args+=("$1")
+		shift 1
+	fi
+done
+
+if [ "$EFI_CASE" = "_NO_FILE_4Uhere_" ]; then
+	EFI_CASE=dummy
+fi
+
+: "${EFI_CASE_DIR:="$EFI_TEST/$EFI_CASE"}"
+mkdir -p "$EFI_CASE_DIR"
+
+cp "$EFI_SRC/$EFI_CASE.efi" "$EFI_TEST/$EFI_CASE/"
+echo "@echo -off" > "$EFI_TEST/$EFI_CASE/startup.nsh"
+echo "$EFI_CASE.efi" "${cmd_args[@]}" >> "$EFI_TEST/$EFI_CASE/startup.nsh"
+
+EFI_RUN=y $TEST_DIR/run \
+       -bios "$EFI_UEFI" \
+       -drive file.dir="$EFI_TEST/$EFI_CASE/",file.driver=vvfat,file.rw=on,format=raw,if=virtio \
+       "${qemu_args[@]}"
diff --git a/arm/run b/arm/run
index 28a0b4a..e96875e 100755
--- a/arm/run
+++ b/arm/run
@@ -67,7 +67,11 @@ fi
 
 A="-accel $ACCEL"
 command="$qemu -nodefaults $M $A -cpu $processor $chr_testdev $pci_testdev"
-command+=" -display none -serial stdio -kernel"
+command+=" -display none -serial stdio"
 command="$(migration_cmd) $(timeout_cmd) $command"
 
-run_qemu $command "$@"
+if [ "$EFI_RUN" = "y" ]; then
+	ENVIRON_DEFAULT=n run_qemu $command "$@"
+else
+	run_qemu $command -kernel "$@"
+fi
diff --git a/arm/Makefile.common b/arm/Makefile.common
index a8007f4..aabd335 100644
--- a/arm/Makefile.common
+++ b/arm/Makefile.common
@@ -12,6 +12,7 @@ tests-common += $(TEST_DIR)/gic.$(exe)
 tests-common += $(TEST_DIR)/psci.$(exe)
 tests-common += $(TEST_DIR)/sieve.$(exe)
 tests-common += $(TEST_DIR)/pl031.$(exe)
+tests-common += $(TEST_DIR)/dummy.$(exe)
 
 tests-all = $(tests-common) $(tests)
 all: directories $(tests-all)
diff --git a/arm/dummy.c b/arm/dummy.c
new file mode 100644
index 0000000..5019e79
--- /dev/null
+++ b/arm/dummy.c
@@ -0,0 +1,4 @@
+int main(int argc, char **argv)
+{
+	return 0;
+}
-- 
2.25.1

Re: [kvm-unit-tests PATCH v2 12/23] arm/arm64: mmu_disable: Clean and invalidate before disabling

[Alexandru Elisei]

Hi,

On Fri, May 06, 2022 at 09:55:54PM +0100, Nikos Nikoleris wrote:
> From: Andrew Jones <drjones@redhat.com>
> 
> The commit message of commit 410b3bf09e76 ("arm/arm64: Perform dcache
> clean + invalidate after turning MMU off") justifies cleaning and
> invalidating the dcache after disabling the MMU by saying it's nice
> not to rely on the current page tables and that it should still work
> (per the spec), as long as there's an identity map in the current
> tables. Doing the invalidation after also somewhat helped with

That's not what the commit says (well, that's now what I wanted to say in
the commit, it might be that I haven't been clear enough):

"Data caches are PIPT and the VAs are translated using the current
translation tables, or an identity mapping (what Arm calls a "flat
mapping") when the MMU is off".

That "flat mapping" does not rely on the TTBRx_EL1 tables, it means that
the output address (the physical address) is the same as the input address
(the virtual address). No actual translation is taking place.

> reenabling the MMU without seeing stale data, but the real problem
> with reenabling was because the cache needs to be disabled with
> the MMU, but it wasn't.

That's not what ARM DDI 0487H.a says on page D5-4826 when HCR_EL2.DC == 0
(which is how KVM configures HCR_EL2):

"For all other accesses, when stage 1 address translation is disabled, the
assigned attributes depend on whether the access is a data access or an
instruction access, as follows:
Data access
The stage 1 translation assigns the Device-nGnRnE memory type."

When the MMU is off, data accesses are non-cacheable.

> 
> Since we have to trust/validate that the current page tables have an
> identity map anyway, then there's no harm in doing the clean
> and invalidate first (it feels a little better to do so, anyway,
> considering the cache maintenance instructions take virtual
> addresses). Then, also disable the cache with the MMU to avoid

That's questionable, CPU can speculate reads which allocate a new dcache
entry after clean + invalidate and before the MMU is turned off, thus
making the clean + invalidate rather useless.

> problems when reenabling. We invalidate the Icache and disable

ARM DDI 0487H.a is pretty clear when icache maintainance is required on
page D5-4933:

"Any permitted instruction cache implementation can be described as
implementing the IVIPT Extension to the Arm architecture.

The formal definition of the Arm IVIPT Extension is that it reduces the
instruction cache maintenance requirement to the following condition:

- Instruction cache maintenance is required only after writing new data to
  a PA that holds an instruction."

If you are seeing issues that are solved by doing an icache invalidation, I
would look first at what the EFI spec guarantees regarding icache
maintenance, because kvm-unit-tests doesn't modify its instructions.

> that too for good measure. And, a final TLB invalidation ensures
> we're crystal clean when we return from asm_mmu_disable().

If I were to guess, any issues that you are seeing are caused by the fact
that EFI apps start with the MMU enabled, and kvm-unit-tests so far has
assumed that the tests start with the MMU disabled.

Thanks,
Alex

> 
> Cc: Alexandru Elisei <alexandru.elisei@arm.com>
> Signed-off-by: Andrew Jones <drjones@redhat.com>
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  arm/cstart.S   | 28 +++++++++++++++++++++-------
>  arm/cstart64.S | 21 ++++++++++++++++-----
>  2 files changed, 37 insertions(+), 12 deletions(-)
> 
> diff --git a/arm/cstart.S b/arm/cstart.S
> index 7036e67..dc324c5 100644
> --- a/arm/cstart.S
> +++ b/arm/cstart.S
> @@ -179,6 +179,7 @@ halt:
>  .globl asm_mmu_enable
>  asm_mmu_enable:
>  	/* TLBIALL */
> +	mov	r2, #0
>  	mcr	p15, 0, r2, c8, c7, 0
>  	dsb	nsh
>  
> @@ -211,12 +212,7 @@ asm_mmu_enable:
>  
>  .globl asm_mmu_disable
>  asm_mmu_disable:
> -	/* SCTLR */
> -	mrc	p15, 0, r0, c1, c0, 0
> -	bic	r0, #CR_M
> -	mcr	p15, 0, r0, c1, c0, 0
> -	isb
> -
> +	/* Clean + invalidate the entire memory */
>  	ldr	r0, =__phys_offset
>  	ldr	r0, [r0]
>  	ldr	r1, =__phys_end
> @@ -224,7 +220,25 @@ asm_mmu_disable:
>  	sub	r1, r1, r0
>  	dcache_by_line_op dccimvac, sy, r0, r1, r2, r3
>  
> -	mov     pc, lr
> +	/* Invalidate Icache */
> +	mov	r0, #0
> +	mcr	p15, 0, r0, c7, c5, 0
> +	isb
> +
> +	/*  Disable cache, Icache and MMU */
> +	mrc	p15, 0, r0, c1, c0, 0
> +	bic	r0, #CR_C
> +	bic	r0, #CR_I
> +	bic	r0, #CR_M
> +	mcr	p15, 0, r0, c1, c0, 0
> +	isb
> +
> +	/* Invalidate TLB */
> +	mov	r0, #0
> +	mcr	p15, 0, r0, c8, c7, 0
> +	dsb	nsh
> +
> +	mov	pc, lr
>  
>  /*
>   * Vectors
> diff --git a/arm/cstart64.S b/arm/cstart64.S
> index e4ab7d0..390feb9 100644
> --- a/arm/cstart64.S
> +++ b/arm/cstart64.S
> @@ -246,11 +246,6 @@ asm_mmu_enable:
>  
>  .globl asm_mmu_disable
>  asm_mmu_disable:
> -	mrs	x0, sctlr_el1
> -	bic	x0, x0, SCTLR_EL1_M
> -	msr	sctlr_el1, x0
> -	isb
> -
>  	/* Clean + invalidate the entire memory */
>  	adrp	x0, __phys_offset
>  	ldr	x0, [x0, :lo12:__phys_offset]
> @@ -259,6 +254,22 @@ asm_mmu_disable:
>  	sub	x1, x1, x0
>  	dcache_by_line_op civac, sy, x0, x1, x2, x3
>  
> +	/* Invalidate Icache */
> +	ic	iallu
> +	isb
> +
> +	/* Disable cache, Icache and MMU */
> +	mrs	x0, sctlr_el1
> +	bic	x0, x0, SCTLR_EL1_C
> +	bic	x0, x0, SCTLR_EL1_I
> +	bic	x0, x0, SCTLR_EL1_M
> +	msr	sctlr_el1, x0
> +	isb
> +
> +	/* Invalidate TLB */
> +	tlbi	vmalle1
> +	dsb	nsh
> +
>  	ret
>  
>  /*
> -- 
> 2.25.1
> 

Re: [kvm-unit-tests PATCH v2 16/23] arm/arm64: Add a setup sequence for systems that boot through EFI

[Alexandru Elisei]

Hi,

On Fri, May 06, 2022 at 09:55:58PM +0100, Nikos Nikoleris wrote:
> This change implements an alternative setup sequence for the system
> when we are booting through EFI. The memory map is discovered through
> EFI boot services and devices through ACPI.
> 
> This change is based on a change initially proposed by
> Andrew Jones <drjones@redhat.com>
> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  lib/linux/efi.h     |   1 +
>  lib/arm/asm/setup.h |   2 +
>  lib/arm/setup.c     | 181 +++++++++++++++++++++++++++++++++++++++++++-
>  arm/cstart.S        |   1 +
>  arm/cstart64.S      |   1 +
>  5 files changed, 184 insertions(+), 2 deletions(-)
> 
> diff --git a/lib/linux/efi.h b/lib/linux/efi.h
> index 594eaca..9b77c39 100644
> --- a/lib/linux/efi.h
> +++ b/lib/linux/efi.h
> @@ -63,6 +63,7 @@ typedef guid_t efi_guid_t;
>  	(c) & 0xff, ((c) >> 8) & 0xff, d } }
>  
>  #define ACPI_TABLE_GUID EFI_GUID(0xeb9d2d30, 0x2d88, 0x11d3, 0x9a, 0x16, 0x00, 0x90, 0x27, 0x3f, 0xc1, 0x4d)
> +#define ACPI_20_TABLE_GUID EFI_GUID(0x8868e871, 0xe4f1, 0x11d3,  0xbc, 0x22, 0x00, 0x80, 0xc7, 0x3c, 0x88, 0x81)
>  
>  #define LOADED_IMAGE_PROTOCOL_GUID EFI_GUID(0x5b1b31a1, 0x9562, 0x11d2,  0x8e, 0x3f, 0x00, 0xa0, 0xc9, 0x69, 0x72, 0x3b)
>  
> diff --git a/lib/arm/asm/setup.h b/lib/arm/asm/setup.h
> index 64cd379..1a7e734 100644
> --- a/lib/arm/asm/setup.h
> +++ b/lib/arm/asm/setup.h
> @@ -5,6 +5,7 @@
>   *
>   * This work is licensed under the terms of the GNU LGPL, version 2.
>   */
> +#include <efi.h>
>  #include <libcflat.h>
>  #include <asm/page.h>
>  #include <asm/pgtable-hwdef.h>
> @@ -37,5 +38,6 @@ extern unsigned int mem_region_get_flags(phys_addr_t paddr);
>  #define SMP_CACHE_BYTES		L1_CACHE_BYTES
>  
>  void setup(const void *fdt, phys_addr_t freemem_start);
> +efi_status_t setup_efi(efi_bootinfo_t *efi_bootinfo);
>  
>  #endif /* _ASMARM_SETUP_H_ */
> diff --git a/lib/arm/setup.c b/lib/arm/setup.c
> index 2d67292..542e2ff 100644
> --- a/lib/arm/setup.c
> +++ b/lib/arm/setup.c
> @@ -34,7 +34,7 @@
>  #define NR_EXTRA_MEM_REGIONS	16
>  #define NR_INITIAL_MEM_REGIONS	(MAX_DT_MEM_REGIONS + NR_EXTRA_MEM_REGIONS)
>  
> -extern unsigned long _etext;
> +extern unsigned long _text, _etext, _data, _edata;
>  
>  char *initrd;
>  u32 initrd_size;
> @@ -44,7 +44,10 @@ int nr_cpus;
>  
>  static struct mem_region __initial_mem_regions[NR_INITIAL_MEM_REGIONS + 1];
>  struct mem_region *mem_regions = __initial_mem_regions;
> -phys_addr_t __phys_offset, __phys_end;
> +phys_addr_t __phys_offset = (phys_addr_t)-1, __phys_end = 0;
> +
> +extern void exceptions_init(void);
> +extern void asm_mmu_disable(void);
>  
>  int mpidr_to_cpu(uint64_t mpidr)
>  {
> @@ -272,3 +275,177 @@ void setup(const void *fdt, phys_addr_t freemem_start)
>  	if (!(auxinfo.flags & AUXINFO_MMU_OFF))
>  		setup_vm();
>  }
> +
> +#ifdef CONFIG_EFI
> +
> +#include <efi.h>
> +
> +static efi_status_t setup_rsdp(efi_bootinfo_t *efi_bootinfo)
> +{
> +	efi_status_t status;
> +	struct rsdp_descriptor *rsdp;
> +
> +	/*
> +	 * RSDP resides in an EFI_ACPI_RECLAIM_MEMORY region, which is not used
> +	 * by kvm-unit-tests arm64 memory allocator. So it is not necessary to
> +	 * copy the data structure to another memory region to prevent
> +	 * unintentional overwrite.
> +	 */
> +	status = efi_get_system_config_table(ACPI_20_TABLE_GUID, (void **)&rsdp);
> +	if (status != EFI_SUCCESS)
> +		return status;
> +
> +	set_efi_rsdp(rsdp);
> +
> +	return EFI_SUCCESS;
> +}
> +
> +static efi_status_t efi_mem_init(efi_bootinfo_t *efi_bootinfo)
> +{
> +	int i;
> +	unsigned long free_mem_pages = 0;
> +	unsigned long free_mem_start = 0;
> +	struct efi_boot_memmap *map = &(efi_bootinfo->mem_map);
> +	efi_memory_desc_t *buffer = *map->map;
> +	efi_memory_desc_t *d = NULL;
> +	phys_addr_t base, top;
> +	struct mem_region *r;
> +	uintptr_t text = (uintptr_t)&_text, etext = __ALIGN((uintptr_t)&_etext, 4096);
> +	uintptr_t data = (uintptr_t)&_data, edata = __ALIGN((uintptr_t)&_edata, 4096);
> +
> +	/*
> +	 * Record the largest free EFI_CONVENTIONAL_MEMORY region
> +	 * which will be used to set up the memory allocator, so that
> +	 * the memory allocator can work in the largest free
> +	 * continuous memory region.
> +	 */
> +	for (i = 0, r = &mem_regions[0]; i < *(map->map_size); i += *(map->desc_size), ++r) {
> +		d = (efi_memory_desc_t *)(&((u8 *)buffer)[i]);
> +
> +		r->start = d->phys_addr;
> +		r->end = d->phys_addr + d->num_pages * EFI_PAGE_SIZE;
> +
> +		switch (d->type) {
> +		case EFI_RESERVED_TYPE:
> +		case EFI_LOADER_DATA:
> +		case EFI_BOOT_SERVICES_CODE:
> +		case EFI_BOOT_SERVICES_DATA:
> +		case EFI_RUNTIME_SERVICES_CODE:
> +		case EFI_RUNTIME_SERVICES_DATA:
> +		case EFI_UNUSABLE_MEMORY:
> +		case EFI_ACPI_RECLAIM_MEMORY:
> +		case EFI_ACPI_MEMORY_NVS:
> +		case EFI_PAL_CODE:
> +			r->flags = MR_F_RESERVED;
> +			break;
> +		case EFI_MEMORY_MAPPED_IO:
> +		case EFI_MEMORY_MAPPED_IO_PORT_SPACE:
> +			r->flags = MR_F_IO;
> +			break;
> +		case EFI_LOADER_CODE:
> +			if (r->start <= text && r->end > text) {
> +				/* This is the unit test region. Flag the code separately. */
> +				phys_addr_t tmp = r->end;
> +
> +				assert(etext <= data);
> +				assert(edata <= r->end);
> +				r->flags = MR_F_CODE;
> +				r->end = data;
> +				++r;
> +				r->start = data;
> +				r->end = tmp;
> +			} else {
> +				r->flags = MR_F_RESERVED;
> +			}
> +			break;
> +		case EFI_CONVENTIONAL_MEMORY:
> +			if (free_mem_pages < d->num_pages) {
> +				free_mem_pages = d->num_pages;
> +				free_mem_start = d->phys_addr;
> +			}
> +			break;
> +		}
> +
> +		if (!(r->flags & MR_F_IO)) {
> +			if (r->start < __phys_offset)
> +				__phys_offset = r->start;
> +			if (r->end > __phys_end)
> +				__phys_end = r->end;
> +		}
> +	}
> +	__phys_end &= PHYS_MASK;
> +	asm_mmu_disable();

And here's why moving the dcache clean + invalidate *before* turning the MMU off
is fixing something: __phys_offset and __phys_end are written with MMU on and
are still in the cache; when the CPU reads them with the MMU off it gets back
stale values and the CMO doesn't execute correctly.

Doing a dcache clean for just those two variables should be enough and you can
drop patch #12 ("arm/arm64: mmu_disable: Clean and invalidate before
disabling").

Thanks,
Alex

Re: [kvm-unit-tests PATCH v2 00/23] EFI and ACPI support for arm64

[Alexandru Elisei]

Hi,

On Fri, May 06, 2022 at 09:55:42PM +0100, Nikos Nikoleris wrote:
> Hello,
> 
> This patch series adds initial support for building arm64 tests as EFI
> tests and running them under QEMU. Much like x86_64 we import external

I would like to see kvm-unit-tests run as an EFI app on real hardware.
QEMU's implementation of the architecture might be different than real
hardware, where considerations like power consumption, performance or die
area dictate how a particular feature is implementated. For example, I
don't know how out-of-order TCG is, and bugs like missing barriers are more
easily detected on highly out-of-order implementations.

On the other hand, I'm not opposed to this series if the purpose is just to
add the skeleton code needed to boot under EFI and hardware support comes
later.

> dependencies from gnu-efi and adopt them to work with types and other
> assumptions from kvm-unit-tests. In addition, this series adds support
> for discovering parts of the machine using ACPI.
> 
> The first set of patches moves the existing ACPI code to the common
> lib path. Then, it extends definitions and functions to allow for more
> robust discovery of ACPI tables. In arm64, we add support for setting
> up the PSCI conduit, discovering the UART, timers and cpus via
> ACPI. The code retains existing behavior and gives priority to
> discovery through DT when one has been provided.
> 
> In the second set of patches, we add support for getting the command
> line from the EFI shell. This is a requirement for many of the
> existing arm64 tests.
> 
> In the third set of patches, we import code from gnu-efi, make minor
> changes and add an alternative setup sequence from arm64 systems that
> boot through EFI. Finally, we add support in the build system and a
> run script which is used to run an EFI app.
> 
> After this set of patches one can build arm64 EFI tests:
> 
> $> ./configure --enable-efi
> $> make
> 
> And use the run script to run an EFI tests:
> 
> $> ./arm/efi/run ./arm/selftest.efi -smp 2 -m 256 -append "setup smp=2 mem=256"
> 
> Or all tests:
> 
> $> ./run_tests.sh
> 
> There are a few items that this series does not address but they would
> be useful to have:
> 
> * Support for booting the system from EL2. Currently, we assume that a
> tests starts running at EL1. This the case when we run with EFI, it's
> not always the case in hardware.

I would add to that the fact that the vmalloc area is between 3 and 4 GB.
What happens if real hardware has main memory there? For this point at
least, for testing you can use my kvmtool series that allows the user to
set the memory base address [1].

I think there might be other assumptions that kvm-unit-tests makes which
are not true when running on baremetal. That's why I would prefer that EFI
support is also tested on baremetal.

[1] https://lore.kernel.org/all/20220428155602.29445-1-alexandru.elisei@arm.com/

Thanks,
Alex

> 
> * Support for reading environment variables and populating __envp.
> 
> * Support for discovering the chr-testdev through ACPI.
> 
> PS: Apologies for the mess with v1. Due to a mistake in my git
> send-email configuration some patches didn't make it to the list and
> the right recipients.
> 
> Thanks,
> 
> Nikos
> 
> Andrew Jones (3):
>   arm/arm64: mmu_disable: Clean and invalidate before disabling
>   arm/arm64: Rename etext to _etext
>   arm64: Add a new type of memory type flag MR_F_RESERVED
> 
> Nikos Nikoleris (20):
>   lib: Move acpi header and implementation to lib
>   lib: Ensure all struct definition for ACPI tables are packed
>   lib: Add support for the XSDT ACPI table
>   lib: Extend the definition of the ACPI table FADT
>   arm/arm64: Add support for setting up the PSCI conduit through ACPI
>   arm/arm64: Add support for discovering the UART through ACPI
>   arm/arm64: Add support for timer initialization through ACPI
>   arm/arm64: Add support for cpu initialization through ACPI
>   lib/printf: Support for precision modifier in printing strings
>   lib/printf: Add support for printing wide strings
>   lib/efi: Add support for getting the cmdline
>   lib: Avoid ms_abi for calls related to EFI on arm64
>   arm/arm64: Add a setup sequence for systems that boot through EFI
>   arm64: Copy code from GNU-EFI
>   arm64: Change GNU-EFI imported file to use defined types
>   arm64: Use code from the gnu-efi when booting with EFI
>   lib: Avoid external dependency in libelf
>   x86: Move x86_64-specific EFI CFLAGS to x86_64 Makefile
>   arm64: Add support for efi in Makefile
>   arm64: Add an efi/run script
> 
>  scripts/runtime.bash        |  14 +-
>  arm/efi/run                 |  61 +++++++++
>  arm/run                     |   8 +-
>  configure                   |  15 ++-
>  Makefile                    |   4 -
>  arm/Makefile.arm            |   6 +
>  arm/Makefile.arm64          |  18 ++-
>  arm/Makefile.common         |  48 +++++--
>  x86/Makefile.common         |   2 +-
>  x86/Makefile.x86_64         |   4 +
>  lib/linux/efi.h             |  44 ++++++
>  lib/arm/asm/setup.h         |   3 +
>  lib/arm/asm/timer.h         |   2 +
>  lib/x86/asm/setup.h         |   2 +-
>  lib/acpi.h                  | 260 ++++++++++++++++++++++++++++++++++++
>  lib/stdlib.h                |   1 +
>  lib/x86/acpi.h              | 112 ----------------
>  lib/acpi.c                  | 124 +++++++++++++++++
>  lib/arm/io.c                |  21 ++-
>  lib/arm/mmu.c               |   4 +
>  lib/arm/psci.c              |  25 +++-
>  lib/arm/setup.c             | 247 +++++++++++++++++++++++++++-------
>  lib/arm/timer.c             |  73 ++++++++++
>  lib/devicetree.c            |   2 +-
>  lib/efi.c                   | 123 +++++++++++++++++
>  lib/printf.c                | 183 +++++++++++++++++++++++--
>  lib/string.c                |   2 +-
>  lib/x86/acpi.c              |  82 ------------
>  arm/efi/elf_aarch64_efi.lds |  63 +++++++++
>  arm/flat.lds                |   2 +-
>  arm/cstart.S                |  29 +++-
>  arm/cstart64.S              |  28 +++-
>  arm/efi/crt0-efi-aarch64.S  | 143 ++++++++++++++++++++
>  arm/dummy.c                 |   4 +
>  arm/efi/reloc_aarch64.c     |  93 +++++++++++++
>  x86/s3.c                    |  20 +--
>  x86/vmexit.c                |   4 +-
>  37 files changed, 1556 insertions(+), 320 deletions(-)
>  create mode 100755 arm/efi/run
>  create mode 100644 lib/acpi.h
>  delete mode 100644 lib/x86/acpi.h
>  create mode 100644 lib/acpi.c
>  create mode 100644 lib/arm/timer.c
>  delete mode 100644 lib/x86/acpi.c
>  create mode 100644 arm/efi/elf_aarch64_efi.lds
>  create mode 100644 arm/efi/crt0-efi-aarch64.S
>  create mode 100644 arm/dummy.c
>  create mode 100644 arm/efi/reloc_aarch64.c
> 
> -- 
> 2.25.1
> 

Re: [kvm-unit-tests PATCH v2 00/23] EFI and ACPI support for arm64

[Ricardo Koller]

Hi Nikos,

On Fri, May 06, 2022 at 09:55:42PM +0100, Nikos Nikoleris wrote:
> Hello,
> 
> This patch series adds initial support for building arm64 tests as EFI
> tests and running them under QEMU. Much like x86_64 we import external
> dependencies from gnu-efi and adopt them to work with types and other
> assumptions from kvm-unit-tests. In addition, this series adds support
> for discovering parts of the machine using ACPI.
> 
> The first set of patches moves the existing ACPI code to the common
> lib path. Then, it extends definitions and functions to allow for more
> robust discovery of ACPI tables. In arm64, we add support for setting
> up the PSCI conduit, discovering the UART, timers and cpus via
> ACPI. The code retains existing behavior and gives priority to
> discovery through DT when one has been provided.
> 
> In the second set of patches, we add support for getting the command
> line from the EFI shell. This is a requirement for many of the
> existing arm64 tests.
> 
> In the third set of patches, we import code from gnu-efi, make minor
> changes and add an alternative setup sequence from arm64 systems that
> boot through EFI. Finally, we add support in the build system and a
> run script which is used to run an EFI app.
> 
> After this set of patches one can build arm64 EFI tests:
> 
> $> ./configure --enable-efi
> $> make
> 
> And use the run script to run an EFI tests:
> 
> $> ./arm/efi/run ./arm/selftest.efi -smp 2 -m 256 -append "setup smp=2 mem=256"
>

Thanks very much for this!

I'm having some issues with the other tests. I'm cross-compiling with
gcc-11. But then "selftest setup" passes and others, like the timer
test, fail:

	$ ./configure --arch=arm64 --cross-prefix=aarch64-linux-gnu- \
		--processor=max --enable-efi
	$ make

	passes:
	$ ./arm/efi/run ./arm/selftest.efi -smp 2 -m 256 \
		-append "setup smp=2 mem=256"

	fails:
	$ ./arm/efi/run ./arm/timer.efi -smp 1 -m 256

	Load address: 5833e000
	PC: 5834ad20 PC offset: cd20
	Unhandled exception ec=0x25 (DABT_EL1)
	Vector: 4 (el1h_sync)
	ESR_EL1:         96000006, ec=0x25 (DABT_EL1)
	FAR_EL1: 0000000000000004 (valid)
	Exception frame registers:
	pc : [<000000005834ad20>] lr : [<000000005834cadc>] pstate: 600003c5
	sp : 000000005f70fd80
	x29: 000000005f70ffe0 x28: 0000000000000000 
	x27: 000000005835dc50 x26: 000000005834eb80 
	x25: 000000000000d800 x24: 000000005f70fe50 
	x23: 0000000000000000 x22: 000000005835f000 
	x21: 000000005834eb80 x20: 0000000000000000 
	x19: 00000000ffffffff x18: 0000000000000000 
	x17: 0000000000000009 x16: 000000005bae8c38 
	x15: 0000000000000002 x14: 0000000000000001 
	x13: 0000000058350000 x12: 0000000058350000 
	x11: 000000005833e000 x10: 000000000005833d 
	x9 : 0000000000000000 x8 : 0000000000000000 
	x7 : 0000000000000000 x6 : 0000000000000001 
	x5 : 00000000000000c9 x4 : 000000005f70fe68 
	x3 : 000000005f70fe68 x2 : 000000005834eb80 
	x1 : 00000000ffffffff x0 : 0000000000000000 

Thanks!
Ricardo

> Or all tests:
> 
> $> ./run_tests.sh
> 
> There are a few items that this series does not address but they would
> be useful to have:
> 
> * Support for booting the system from EL2. Currently, we assume that a
> tests starts running at EL1. This the case when we run with EFI, it's
> not always the case in hardware.
> 
> * Support for reading environment variables and populating __envp.
> 
> * Support for discovering the chr-testdev through ACPI.
> 
> PS: Apologies for the mess with v1. Due to a mistake in my git
> send-email configuration some patches didn't make it to the list and
> the right recipients.
> 
> Thanks,
> 
> Nikos
> 
> Andrew Jones (3):
>   arm/arm64: mmu_disable: Clean and invalidate before disabling
>   arm/arm64: Rename etext to _etext
>   arm64: Add a new type of memory type flag MR_F_RESERVED
> 
> Nikos Nikoleris (20):
>   lib: Move acpi header and implementation to lib
>   lib: Ensure all struct definition for ACPI tables are packed
>   lib: Add support for the XSDT ACPI table
>   lib: Extend the definition of the ACPI table FADT
>   arm/arm64: Add support for setting up the PSCI conduit through ACPI
>   arm/arm64: Add support for discovering the UART through ACPI
>   arm/arm64: Add support for timer initialization through ACPI
>   arm/arm64: Add support for cpu initialization through ACPI
>   lib/printf: Support for precision modifier in printing strings
>   lib/printf: Add support for printing wide strings
>   lib/efi: Add support for getting the cmdline
>   lib: Avoid ms_abi for calls related to EFI on arm64
>   arm/arm64: Add a setup sequence for systems that boot through EFI
>   arm64: Copy code from GNU-EFI
>   arm64: Change GNU-EFI imported file to use defined types
>   arm64: Use code from the gnu-efi when booting with EFI
>   lib: Avoid external dependency in libelf
>   x86: Move x86_64-specific EFI CFLAGS to x86_64 Makefile
>   arm64: Add support for efi in Makefile
>   arm64: Add an efi/run script
> 
>  scripts/runtime.bash        |  14 +-
>  arm/efi/run                 |  61 +++++++++
>  arm/run                     |   8 +-
>  configure                   |  15 ++-
>  Makefile                    |   4 -
>  arm/Makefile.arm            |   6 +
>  arm/Makefile.arm64          |  18 ++-
>  arm/Makefile.common         |  48 +++++--
>  x86/Makefile.common         |   2 +-
>  x86/Makefile.x86_64         |   4 +
>  lib/linux/efi.h             |  44 ++++++
>  lib/arm/asm/setup.h         |   3 +
>  lib/arm/asm/timer.h         |   2 +
>  lib/x86/asm/setup.h         |   2 +-
>  lib/acpi.h                  | 260 ++++++++++++++++++++++++++++++++++++
>  lib/stdlib.h                |   1 +
>  lib/x86/acpi.h              | 112 ----------------
>  lib/acpi.c                  | 124 +++++++++++++++++
>  lib/arm/io.c                |  21 ++-
>  lib/arm/mmu.c               |   4 +
>  lib/arm/psci.c              |  25 +++-
>  lib/arm/setup.c             | 247 +++++++++++++++++++++++++++-------
>  lib/arm/timer.c             |  73 ++++++++++
>  lib/devicetree.c            |   2 +-
>  lib/efi.c                   | 123 +++++++++++++++++
>  lib/printf.c                | 183 +++++++++++++++++++++++--
>  lib/string.c                |   2 +-
>  lib/x86/acpi.c              |  82 ------------
>  arm/efi/elf_aarch64_efi.lds |  63 +++++++++
>  arm/flat.lds                |   2 +-
>  arm/cstart.S                |  29 +++-
>  arm/cstart64.S              |  28 +++-
>  arm/efi/crt0-efi-aarch64.S  | 143 ++++++++++++++++++++
>  arm/dummy.c                 |   4 +
>  arm/efi/reloc_aarch64.c     |  93 +++++++++++++
>  x86/s3.c                    |  20 +--
>  x86/vmexit.c                |   4 +-
>  37 files changed, 1556 insertions(+), 320 deletions(-)
>  create mode 100755 arm/efi/run
>  create mode 100644 lib/acpi.h
>  delete mode 100644 lib/x86/acpi.h
>  create mode 100644 lib/acpi.c
>  create mode 100644 lib/arm/timer.c
>  delete mode 100644 lib/x86/acpi.c
>  create mode 100644 arm/efi/elf_aarch64_efi.lds
>  create mode 100644 arm/efi/crt0-efi-aarch64.S
>  create mode 100644 arm/dummy.c
>  create mode 100644 arm/efi/reloc_aarch64.c
> 
> -- 
> 2.25.1
> 

Re: [kvm-unit-tests PATCH v2 00/23] EFI and ACPI support for arm64

[Nikos Nikoleris]

Hi Alex,

On 13/05/2022 15:09, Alexandru Elisei wrote:
> Hi,
> 
> On Fri, May 06, 2022 at 09:55:42PM +0100, Nikos Nikoleris wrote:
>> Hello,
>>
>> This patch series adds initial support for building arm64 tests as EFI
>> tests and running them under QEMU. Much like x86_64 we import external
> 
> I would like to see kvm-unit-tests run as an EFI app on real hardware.
> QEMU's implementation of the architecture might be different than real
> hardware, where considerations like power consumption, performance or die
> area dictate how a particular feature is implementated. For example, I
> don't know how out-of-order TCG is, and bugs like missing barriers are more
> easily detected on highly out-of-order implementations.
> 
> On the other hand, I'm not opposed to this series if the purpose is just to
> add the skeleton code needed to boot under EFI and hardware support comes
> later.

I fully agree with you. This series is just the first step in getting 
EFI apps to run on real hardware. Hopefully, it's in the right direction 
and helps further development and testing for some of the functionality 
that we will need anyway.

If I understand correctly, we can run EFI tests with KVM too, at least 
for as long as we're fine with starting from EL1 so we should be exposed 
to some of the timing issues already.

> 
>> dependencies from gnu-efi and adopt them to work with types and other
>> assumptions from kvm-unit-tests. In addition, this series adds support
>> for discovering parts of the machine using ACPI.
>>
>> The first set of patches moves the existing ACPI code to the common
>> lib path. Then, it extends definitions and functions to allow for more
>> robust discovery of ACPI tables. In arm64, we add support for setting
>> up the PSCI conduit, discovering the UART, timers and cpus via
>> ACPI. The code retains existing behavior and gives priority to
>> discovery through DT when one has been provided.
>>
>> In the second set of patches, we add support for getting the command
>> line from the EFI shell. This is a requirement for many of the
>> existing arm64 tests.
>>
>> In the third set of patches, we import code from gnu-efi, make minor
>> changes and add an alternative setup sequence from arm64 systems that
>> boot through EFI. Finally, we add support in the build system and a
>> run script which is used to run an EFI app.
>>
>> After this set of patches one can build arm64 EFI tests:
>>
>> $> ./configure --enable-efi
>> $> make
>>
>> And use the run script to run an EFI tests:
>>
>> $> ./arm/efi/run ./arm/selftest.efi -smp 2 -m 256 -append "setup smp=2 mem=256"
>>
>> Or all tests:
>>
>> $> ./run_tests.sh
>>
>> There are a few items that this series does not address but they would
>> be useful to have:
>>
>> * Support for booting the system from EL2. Currently, we assume that a
>> tests starts running at EL1. This the case when we run with EFI, it's
>> not always the case in hardware.
> 
> I would add to that the fact that the vmalloc area is between 3 and 4 GB.
> What happens if real hardware has main memory there? For this point at
> least, for testing you can use my kvmtool series that allows the user to
> set the memory base address [1].

I am not sure, I fully understand the problem with this. In 13/26, we 
use the efi memory map to avoid making many assumption about the 
physical memory map, but I will have a look at the functionality we 
implement in vmalloc.c to understand this. On a high level, I agree with 
you. The goal should to have tests discover as much about the underlying 
system as possible.

FWIW, this TODO list is missing many points that I've already discovered 
and I am sure there will be a few more on top of that.

> 
> I think there might be other assumptions that kvm-unit-tests makes which
> are not true when running on baremetal. That's why I would prefer that EFI
> support is also tested on baremetal.
> 
> [1] https://lore.kernel.org/all/20220428155602.29445-1-alexandru.elisei@arm.com/
> 

I agree, hopefully testing with KVM and TCG is helpful but I wouldn't 
expect it to be sufficient.

Thanks,

Nikos

> Thanks,
> Alex
> 
>>
>> * Support for reading environment variables and populating __envp.
>>
>> * Support for discovering the chr-testdev through ACPI.
>>
>> PS: Apologies for the mess with v1. Due to a mistake in my git
>> send-email configuration some patches didn't make it to the list and
>> the right recipients.
>>
>> Thanks,
>>
>> Nikos
>>
>> Andrew Jones (3):
>>    arm/arm64: mmu_disable: Clean and invalidate before disabling
>>    arm/arm64: Rename etext to _etext
>>    arm64: Add a new type of memory type flag MR_F_RESERVED
>>
>> Nikos Nikoleris (20):
>>    lib: Move acpi header and implementation to lib
>>    lib: Ensure all struct definition for ACPI tables are packed
>>    lib: Add support for the XSDT ACPI table
>>    lib: Extend the definition of the ACPI table FADT
>>    arm/arm64: Add support for setting up the PSCI conduit through ACPI
>>    arm/arm64: Add support for discovering the UART through ACPI
>>    arm/arm64: Add support for timer initialization through ACPI
>>    arm/arm64: Add support for cpu initialization through ACPI
>>    lib/printf: Support for precision modifier in printing strings
>>    lib/printf: Add support for printing wide strings
>>    lib/efi: Add support for getting the cmdline
>>    lib: Avoid ms_abi for calls related to EFI on arm64
>>    arm/arm64: Add a setup sequence for systems that boot through EFI
>>    arm64: Copy code from GNU-EFI
>>    arm64: Change GNU-EFI imported file to use defined types
>>    arm64: Use code from the gnu-efi when booting with EFI
>>    lib: Avoid external dependency in libelf
>>    x86: Move x86_64-specific EFI CFLAGS to x86_64 Makefile
>>    arm64: Add support for efi in Makefile
>>    arm64: Add an efi/run script
>>
>>   scripts/runtime.bash        |  14 +-
>>   arm/efi/run                 |  61 +++++++++
>>   arm/run                     |   8 +-
>>   configure                   |  15 ++-
>>   Makefile                    |   4 -
>>   arm/Makefile.arm            |   6 +
>>   arm/Makefile.arm64          |  18 ++-
>>   arm/Makefile.common         |  48 +++++--
>>   x86/Makefile.common         |   2 +-
>>   x86/Makefile.x86_64         |   4 +
>>   lib/linux/efi.h             |  44 ++++++
>>   lib/arm/asm/setup.h         |   3 +
>>   lib/arm/asm/timer.h         |   2 +
>>   lib/x86/asm/setup.h         |   2 +-
>>   lib/acpi.h                  | 260 ++++++++++++++++++++++++++++++++++++
>>   lib/stdlib.h                |   1 +
>>   lib/x86/acpi.h              | 112 ----------------
>>   lib/acpi.c                  | 124 +++++++++++++++++
>>   lib/arm/io.c                |  21 ++-
>>   lib/arm/mmu.c               |   4 +
>>   lib/arm/psci.c              |  25 +++-
>>   lib/arm/setup.c             | 247 +++++++++++++++++++++++++++-------
>>   lib/arm/timer.c             |  73 ++++++++++
>>   lib/devicetree.c            |   2 +-
>>   lib/efi.c                   | 123 +++++++++++++++++
>>   lib/printf.c                | 183 +++++++++++++++++++++++--
>>   lib/string.c                |   2 +-
>>   lib/x86/acpi.c              |  82 ------------
>>   arm/efi/elf_aarch64_efi.lds |  63 +++++++++
>>   arm/flat.lds                |   2 +-
>>   arm/cstart.S                |  29 +++-
>>   arm/cstart64.S              |  28 +++-
>>   arm/efi/crt0-efi-aarch64.S  | 143 ++++++++++++++++++++
>>   arm/dummy.c                 |   4 +
>>   arm/efi/reloc_aarch64.c     |  93 +++++++++++++
>>   x86/s3.c                    |  20 +--
>>   x86/vmexit.c                |   4 +-
>>   37 files changed, 1556 insertions(+), 320 deletions(-)
>>   create mode 100755 arm/efi/run
>>   create mode 100644 lib/acpi.h
>>   delete mode 100644 lib/x86/acpi.h
>>   create mode 100644 lib/acpi.c
>>   create mode 100644 lib/arm/timer.c
>>   delete mode 100644 lib/x86/acpi.c
>>   create mode 100644 arm/efi/elf_aarch64_efi.lds
>>   create mode 100644 arm/efi/crt0-efi-aarch64.S
>>   create mode 100644 arm/dummy.c
>>   create mode 100644 arm/efi/reloc_aarch64.c
>>
>> -- 
>> 2.25.1
>>

Re: [kvm-unit-tests PATCH v2 00/23] EFI and ACPI support for arm64

[Nikos Nikoleris]

Hi Ricardo,

On 17/05/2022 18:56, Ricardo Koller wrote:
> Hi Nikos,
> 
> On Fri, May 06, 2022 at 09:55:42PM +0100, Nikos Nikoleris wrote:
>> Hello,
>>
>> This patch series adds initial support for building arm64 tests as EFI
>> tests and running them under QEMU. Much like x86_64 we import external
>> dependencies from gnu-efi and adopt them to work with types and other
>> assumptions from kvm-unit-tests. In addition, this series adds support
>> for discovering parts of the machine using ACPI.
>>
>> The first set of patches moves the existing ACPI code to the common
>> lib path. Then, it extends definitions and functions to allow for more
>> robust discovery of ACPI tables. In arm64, we add support for setting
>> up the PSCI conduit, discovering the UART, timers and cpus via
>> ACPI. The code retains existing behavior and gives priority to
>> discovery through DT when one has been provided.
>>
>> In the second set of patches, we add support for getting the command
>> line from the EFI shell. This is a requirement for many of the
>> existing arm64 tests.
>>
>> In the third set of patches, we import code from gnu-efi, make minor
>> changes and add an alternative setup sequence from arm64 systems that
>> boot through EFI. Finally, we add support in the build system and a
>> run script which is used to run an EFI app.
>>
>> After this set of patches one can build arm64 EFI tests:
>>
>> $> ./configure --enable-efi
>> $> make
>>
>> And use the run script to run an EFI tests:
>>
>> $> ./arm/efi/run ./arm/selftest.efi -smp 2 -m 256 -append "setup smp=2 mem=256"
>>
> 
> Thanks very much for this!
> 
> I'm having some issues with the other tests. I'm cross-compiling with
> gcc-11. But then "selftest setup" passes and others, like the timer
> test, fail:
> 
> 	$ ./configure --arch=arm64 --cross-prefix=aarch64-linux-gnu- \
> 		--processor=max --enable-efi
> 	$ make
> 
> 	passes:
> 	$ ./arm/efi/run ./arm/selftest.efi -smp 2 -m 256 \
> 		-append "setup smp=2 mem=256"
> 
> 	fails:
> 	$ ./arm/efi/run ./arm/timer.efi -smp 1 -m 256
> 
> 	Load address: 5833e000
> 	PC: 5834ad20 PC offset: cd20
> 	Unhandled exception ec=0x25 (DABT_EL1)
> 	Vector: 4 (el1h_sync)
> 	ESR_EL1:         96000006, ec=0x25 (DABT_EL1)
> 	FAR_EL1: 0000000000000004 (valid)
> 	Exception frame registers:
> 	pc : [<000000005834ad20>] lr : [<000000005834cadc>] pstate: 600003c5
> 	sp : 000000005f70fd80
> 	x29: 000000005f70ffe0 x28: 0000000000000000
> 	x27: 000000005835dc50 x26: 000000005834eb80
> 	x25: 000000000000d800 x24: 000000005f70fe50
> 	x23: 0000000000000000 x22: 000000005835f000
> 	x21: 000000005834eb80 x20: 0000000000000000
> 	x19: 00000000ffffffff x18: 0000000000000000
> 	x17: 0000000000000009 x16: 000000005bae8c38
> 	x15: 0000000000000002 x14: 0000000000000001
> 	x13: 0000000058350000 x12: 0000000058350000
> 	x11: 000000005833e000 x10: 000000000005833d
> 	x9 : 0000000000000000 x8 : 0000000000000000
> 	x7 : 0000000000000000 x6 : 0000000000000001
> 	x5 : 00000000000000c9 x4 : 000000005f70fe68
> 	x3 : 000000005f70fe68 x2 : 000000005834eb80
> 	x1 : 00000000ffffffff x0 : 0000000000000000
> 

Thank you for having a look!

Apologies, I should have been more explicit about this. At this point, 
not all tests run successfully. There are bits and pieces missing, some 
of which I tried to list in this TODO list and more that I missed. For 
example, to get the timer tests to pass we have to add support for GIC 
initialization through ACPI.

I've continued working on this series, and I will be ironing some of 
these issues out and, in the meantime, I wanted some early feedback on 
whether some of these features are even desirable upstream (e.g., ACPI 
support for arm64).

I don't want to spam the list too much and I will wait for comments 
before I send a v3 but I already have a couple of fixes and one more 
patch [1]. With these applied both the timer and most gicv3 tests pass.

[1]: https://github.com/relokin/kvm-unit-tests/tree/target-efi-upstream

Thanks,

Nikos

> Thanks!
> Ricardo
> 
>> Or all tests:
>>
>> $> ./run_tests.sh
>>
>> There are a few items that this series does not address but they would
>> be useful to have:
>>
>> * Support for booting the system from EL2. Currently, we assume that a
>> tests starts running at EL1. This the case when we run with EFI, it's
>> not always the case in hardware.
>>
>> * Support for reading environment variables and populating __envp.
>>
>> * Support for discovering the chr-testdev through ACPI.
>>
>> PS: Apologies for the mess with v1. Due to a mistake in my git
>> send-email configuration some patches didn't make it to the list and
>> the right recipients.
>>
>> Thanks,
>>
>> Nikos
>>
>> Andrew Jones (3):
>>    arm/arm64: mmu_disable: Clean and invalidate before disabling
>>    arm/arm64: Rename etext to _etext
>>    arm64: Add a new type of memory type flag MR_F_RESERVED
>>
>> Nikos Nikoleris (20):
>>    lib: Move acpi header and implementation to lib
>>    lib: Ensure all struct definition for ACPI tables are packed
>>    lib: Add support for the XSDT ACPI table
>>    lib: Extend the definition of the ACPI table FADT
>>    arm/arm64: Add support for setting up the PSCI conduit through ACPI
>>    arm/arm64: Add support for discovering the UART through ACPI
>>    arm/arm64: Add support for timer initialization through ACPI
>>    arm/arm64: Add support for cpu initialization through ACPI
>>    lib/printf: Support for precision modifier in printing strings
>>    lib/printf: Add support for printing wide strings
>>    lib/efi: Add support for getting the cmdline
>>    lib: Avoid ms_abi for calls related to EFI on arm64
>>    arm/arm64: Add a setup sequence for systems that boot through EFI
>>    arm64: Copy code from GNU-EFI
>>    arm64: Change GNU-EFI imported file to use defined types
>>    arm64: Use code from the gnu-efi when booting with EFI
>>    lib: Avoid external dependency in libelf
>>    x86: Move x86_64-specific EFI CFLAGS to x86_64 Makefile
>>    arm64: Add support for efi in Makefile
>>    arm64: Add an efi/run script
>>
>>   scripts/runtime.bash        |  14 +-
>>   arm/efi/run                 |  61 +++++++++
>>   arm/run                     |   8 +-
>>   configure                   |  15 ++-
>>   Makefile                    |   4 -
>>   arm/Makefile.arm            |   6 +
>>   arm/Makefile.arm64          |  18 ++-
>>   arm/Makefile.common         |  48 +++++--
>>   x86/Makefile.common         |   2 +-
>>   x86/Makefile.x86_64         |   4 +
>>   lib/linux/efi.h             |  44 ++++++
>>   lib/arm/asm/setup.h         |   3 +
>>   lib/arm/asm/timer.h         |   2 +
>>   lib/x86/asm/setup.h         |   2 +-
>>   lib/acpi.h                  | 260 ++++++++++++++++++++++++++++++++++++
>>   lib/stdlib.h                |   1 +
>>   lib/x86/acpi.h              | 112 ----------------
>>   lib/acpi.c                  | 124 +++++++++++++++++
>>   lib/arm/io.c                |  21 ++-
>>   lib/arm/mmu.c               |   4 +
>>   lib/arm/psci.c              |  25 +++-
>>   lib/arm/setup.c             | 247 +++++++++++++++++++++++++++-------
>>   lib/arm/timer.c             |  73 ++++++++++
>>   lib/devicetree.c            |   2 +-
>>   lib/efi.c                   | 123 +++++++++++++++++
>>   lib/printf.c                | 183 +++++++++++++++++++++++--
>>   lib/string.c                |   2 +-
>>   lib/x86/acpi.c              |  82 ------------
>>   arm/efi/elf_aarch64_efi.lds |  63 +++++++++
>>   arm/flat.lds                |   2 +-
>>   arm/cstart.S                |  29 +++-
>>   arm/cstart64.S              |  28 +++-
>>   arm/efi/crt0-efi-aarch64.S  | 143 ++++++++++++++++++++
>>   arm/dummy.c                 |   4 +
>>   arm/efi/reloc_aarch64.c     |  93 +++++++++++++
>>   x86/s3.c                    |  20 +--
>>   x86/vmexit.c                |   4 +-
>>   37 files changed, 1556 insertions(+), 320 deletions(-)
>>   create mode 100755 arm/efi/run
>>   create mode 100644 lib/acpi.h
>>   delete mode 100644 lib/x86/acpi.h
>>   create mode 100644 lib/acpi.c
>>   create mode 100644 lib/arm/timer.c
>>   delete mode 100644 lib/x86/acpi.c
>>   create mode 100644 arm/efi/elf_aarch64_efi.lds
>>   create mode 100644 arm/efi/crt0-efi-aarch64.S
>>   create mode 100644 arm/dummy.c
>>   create mode 100644 arm/efi/reloc_aarch64.c
>>
>> -- 
>> 2.25.1
>>

Re: [kvm-unit-tests PATCH v2 00/23] EFI and ACPI support for arm64

[Ricardo Koller]

Hi Nikos,

On Wed, May 18, 2022 at 01:44:20PM +0100, Nikos Nikoleris wrote:
> Hi Ricardo,
> 
> On 17/05/2022 18:56, Ricardo Koller wrote:
> > Hi Nikos,
> > 
> > On Fri, May 06, 2022 at 09:55:42PM +0100, Nikos Nikoleris wrote:
> > > Hello,
> > > 
> > > This patch series adds initial support for building arm64 tests as EFI
> > > tests and running them under QEMU. Much like x86_64 we import external
> > > dependencies from gnu-efi and adopt them to work with types and other
> > > assumptions from kvm-unit-tests. In addition, this series adds support
> > > for discovering parts of the machine using ACPI.
> > > 
> > > The first set of patches moves the existing ACPI code to the common
> > > lib path. Then, it extends definitions and functions to allow for more
> > > robust discovery of ACPI tables. In arm64, we add support for setting
> > > up the PSCI conduit, discovering the UART, timers and cpus via
> > > ACPI. The code retains existing behavior and gives priority to
> > > discovery through DT when one has been provided.
> > > 
> > > In the second set of patches, we add support for getting the command
> > > line from the EFI shell. This is a requirement for many of the
> > > existing arm64 tests.
> > > 
> > > In the third set of patches, we import code from gnu-efi, make minor
> > > changes and add an alternative setup sequence from arm64 systems that
> > > boot through EFI. Finally, we add support in the build system and a
> > > run script which is used to run an EFI app.
> > > 
> > > After this set of patches one can build arm64 EFI tests:
> > > 
> > > $> ./configure --enable-efi
> > > $> make
> > > 
> > > And use the run script to run an EFI tests:
> > > 
> > > $> ./arm/efi/run ./arm/selftest.efi -smp 2 -m 256 -append "setup smp=2 mem=256"
> > > 
> > 
> > Thanks very much for this!
> > 
> > I'm having some issues with the other tests. I'm cross-compiling with
> > gcc-11. But then "selftest setup" passes and others, like the timer
> > test, fail:
> > 
> > 	$ ./configure --arch=arm64 --cross-prefix=aarch64-linux-gnu- \
> > 		--processor=max --enable-efi
> > 	$ make
> > 
> > 	passes:
> > 	$ ./arm/efi/run ./arm/selftest.efi -smp 2 -m 256 \
> > 		-append "setup smp=2 mem=256"
> > 
> > 	fails:
> > 	$ ./arm/efi/run ./arm/timer.efi -smp 1 -m 256
> > 
> > 	Load address: 5833e000
> > 	PC: 5834ad20 PC offset: cd20
> > 	Unhandled exception ec=0x25 (DABT_EL1)
> > 	Vector: 4 (el1h_sync)
> > 	ESR_EL1:         96000006, ec=0x25 (DABT_EL1)
> > 	FAR_EL1: 0000000000000004 (valid)
> > 	Exception frame registers:
> > 	pc : [<000000005834ad20>] lr : [<000000005834cadc>] pstate: 600003c5
> > 	sp : 000000005f70fd80
> > 	x29: 000000005f70ffe0 x28: 0000000000000000
> > 	x27: 000000005835dc50 x26: 000000005834eb80
> > 	x25: 000000000000d800 x24: 000000005f70fe50
> > 	x23: 0000000000000000 x22: 000000005835f000
> > 	x21: 000000005834eb80 x20: 0000000000000000
> > 	x19: 00000000ffffffff x18: 0000000000000000
> > 	x17: 0000000000000009 x16: 000000005bae8c38
> > 	x15: 0000000000000002 x14: 0000000000000001
> > 	x13: 0000000058350000 x12: 0000000058350000
> > 	x11: 000000005833e000 x10: 000000000005833d
> > 	x9 : 0000000000000000 x8 : 0000000000000000
> > 	x7 : 0000000000000000 x6 : 0000000000000001
> > 	x5 : 00000000000000c9 x4 : 000000005f70fe68
> > 	x3 : 000000005f70fe68 x2 : 000000005834eb80
> > 	x1 : 00000000ffffffff x0 : 0000000000000000
> > 
> 
> Thank you for having a look!
> 
> Apologies, I should have been more explicit about this. At this point, not
> all tests run successfully. There are bits and pieces missing, some of which
> I tried to list in this TODO list and more that I missed. For example, to
> get the timer tests to pass we have to add support for GIC initialization
> through ACPI.
> 
> I've continued working on this series, and I will be ironing some of these
> issues out and, in the meantime, I wanted some early feedback on whether

Sounds good, will take a look as well. Thanks!

> some of these features are even desirable upstream (e.g., ACPI support for
> arm64).

It's highly desirable for us. This is great.

Thanks,
Ricardo

> 
> I don't want to spam the list too much and I will wait for comments before I
> send a v3 but I already have a couple of fixes and one more patch [1]. With
> these applied both the timer and most gicv3 tests pass.
> 
> [1]: https://github.com/relokin/kvm-unit-tests/tree/target-efi-upstream
> 
> Thanks,
> 
> Nikos
> 
> > Thanks!
> > Ricardo
> > 
> > > Or all tests:
> > > 
> > > $> ./run_tests.sh
> > > 
> > > There are a few items that this series does not address but they would
> > > be useful to have:
> > > 
> > > * Support for booting the system from EL2. Currently, we assume that a
> > > tests starts running at EL1. This the case when we run with EFI, it's
> > > not always the case in hardware.
> > > 
> > > * Support for reading environment variables and populating __envp.
> > > 
> > > * Support for discovering the chr-testdev through ACPI.
> > > 
> > > PS: Apologies for the mess with v1. Due to a mistake in my git
> > > send-email configuration some patches didn't make it to the list and
> > > the right recipients.
> > > 
> > > Thanks,
> > > 
> > > Nikos
> > > 
> > > Andrew Jones (3):
> > >    arm/arm64: mmu_disable: Clean and invalidate before disabling
> > >    arm/arm64: Rename etext to _etext
> > >    arm64: Add a new type of memory type flag MR_F_RESERVED
> > > 
> > > Nikos Nikoleris (20):
> > >    lib: Move acpi header and implementation to lib
> > >    lib: Ensure all struct definition for ACPI tables are packed
> > >    lib: Add support for the XSDT ACPI table
> > >    lib: Extend the definition of the ACPI table FADT
> > >    arm/arm64: Add support for setting up the PSCI conduit through ACPI
> > >    arm/arm64: Add support for discovering the UART through ACPI
> > >    arm/arm64: Add support for timer initialization through ACPI
> > >    arm/arm64: Add support for cpu initialization through ACPI
> > >    lib/printf: Support for precision modifier in printing strings
> > >    lib/printf: Add support for printing wide strings
> > >    lib/efi: Add support for getting the cmdline
> > >    lib: Avoid ms_abi for calls related to EFI on arm64
> > >    arm/arm64: Add a setup sequence for systems that boot through EFI
> > >    arm64: Copy code from GNU-EFI
> > >    arm64: Change GNU-EFI imported file to use defined types
> > >    arm64: Use code from the gnu-efi when booting with EFI
> > >    lib: Avoid external dependency in libelf
> > >    x86: Move x86_64-specific EFI CFLAGS to x86_64 Makefile
> > >    arm64: Add support for efi in Makefile
> > >    arm64: Add an efi/run script
> > > 
> > >   scripts/runtime.bash        |  14 +-
> > >   arm/efi/run                 |  61 +++++++++
> > >   arm/run                     |   8 +-
> > >   configure                   |  15 ++-
> > >   Makefile                    |   4 -
> > >   arm/Makefile.arm            |   6 +
> > >   arm/Makefile.arm64          |  18 ++-
> > >   arm/Makefile.common         |  48 +++++--
> > >   x86/Makefile.common         |   2 +-
> > >   x86/Makefile.x86_64         |   4 +
> > >   lib/linux/efi.h             |  44 ++++++
> > >   lib/arm/asm/setup.h         |   3 +
> > >   lib/arm/asm/timer.h         |   2 +
> > >   lib/x86/asm/setup.h         |   2 +-
> > >   lib/acpi.h                  | 260 ++++++++++++++++++++++++++++++++++++
> > >   lib/stdlib.h                |   1 +
> > >   lib/x86/acpi.h              | 112 ----------------
> > >   lib/acpi.c                  | 124 +++++++++++++++++
> > >   lib/arm/io.c                |  21 ++-
> > >   lib/arm/mmu.c               |   4 +
> > >   lib/arm/psci.c              |  25 +++-
> > >   lib/arm/setup.c             | 247 +++++++++++++++++++++++++++-------
> > >   lib/arm/timer.c             |  73 ++++++++++
> > >   lib/devicetree.c            |   2 +-
> > >   lib/efi.c                   | 123 +++++++++++++++++
> > >   lib/printf.c                | 183 +++++++++++++++++++++++--
> > >   lib/string.c                |   2 +-
> > >   lib/x86/acpi.c              |  82 ------------
> > >   arm/efi/elf_aarch64_efi.lds |  63 +++++++++
> > >   arm/flat.lds                |   2 +-
> > >   arm/cstart.S                |  29 +++-
> > >   arm/cstart64.S              |  28 +++-
> > >   arm/efi/crt0-efi-aarch64.S  | 143 ++++++++++++++++++++
> > >   arm/dummy.c                 |   4 +
> > >   arm/efi/reloc_aarch64.c     |  93 +++++++++++++
> > >   x86/s3.c                    |  20 +--
> > >   x86/vmexit.c                |   4 +-
> > >   37 files changed, 1556 insertions(+), 320 deletions(-)
> > >   create mode 100755 arm/efi/run
> > >   create mode 100644 lib/acpi.h
> > >   delete mode 100644 lib/x86/acpi.h
> > >   create mode 100644 lib/acpi.c
> > >   create mode 100644 lib/arm/timer.c
> > >   delete mode 100644 lib/x86/acpi.c
> > >   create mode 100644 arm/efi/elf_aarch64_efi.lds
> > >   create mode 100644 arm/efi/crt0-efi-aarch64.S
> > >   create mode 100644 arm/dummy.c
> > >   create mode 100644 arm/efi/reloc_aarch64.c
> > > 
> > > -- 
> > > 2.25.1
> > > 

Re: [kvm-unit-tests PATCH v2 02/23] lib: Ensure all struct definition for ACPI tables are packed

[Andrew Jones]

On Fri, May 06, 2022 at 09:55:44PM +0100, Nikos Nikoleris wrote:
> All ACPI table definitions are provided with precise definitions of
> field sizes and offsets, make sure that no compiler optimization can
> interfere with the memory layout of the corresponding structs.

That seems like a reasonable thing to do. I'm wondering why Linux doesn't
appear to do it. I see u-boot does, but not for all tables, which also
makes me scratch my head... I see this patch packs every struct except
rsdp_descriptor. Is there a reason it was left out?

Another comment below

> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  lib/acpi.h | 11 ++++++++---
>  x86/s3.c   | 16 ++++------------
>  2 files changed, 12 insertions(+), 15 deletions(-)
> 
> diff --git a/lib/acpi.h b/lib/acpi.h
> index 1e89840..42a2c16 100644
> --- a/lib/acpi.h
> +++ b/lib/acpi.h
> @@ -3,6 +3,11 @@
>  
>  #include "libcflat.h"
>  
> +/*
> + * All tables and structures must be byte-packed to match the ACPI
> + * specification, since the tables are provided by the system BIOS
> + */
> +
>  #define ACPI_SIGNATURE(c1, c2, c3, c4) \
>  	((c1) | ((c2) << 8) | ((c3) << 16) | ((c4) << 24))
>  
> @@ -44,12 +49,12 @@ struct rsdp_descriptor {        /* Root System Descriptor Pointer */
>  struct acpi_table {
>      ACPI_TABLE_HEADER_DEF
>      char data[0];
> -};
> +} __attribute__ ((packed));
>  
>  struct rsdt_descriptor_rev1 {
>      ACPI_TABLE_HEADER_DEF
>      u32 table_offset_entry[0];
> -};
> +} __attribute__ ((packed));
>  
>  struct fadt_descriptor_rev1
>  {
> @@ -104,7 +109,7 @@ struct facs_descriptor_rev1
>      u32 S4bios_f        : 1;    /* Indicates if S4BIOS support is present */
>      u32 reserved1       : 31;   /* Must be 0 */
>      u8  reserved3 [40];         /* Reserved - must be zero */
> -};
> +} __attribute__ ((packed));
>  
>  void set_efi_rsdp(struct rsdp_descriptor *rsdp);
>  void* find_acpi_table_addr(u32 sig);
> diff --git a/x86/s3.c b/x86/s3.c

The changes below in this file are unrelated, so they should be in a
separate patch. However, I'm also curious why they're needed. I see
that find_acpi_table_addr() can return NULL, so it doesn't seem like
we should be removing the check, but instead changing the check to
an assert.

> index 378d37a..89d69fc 100644
> --- a/x86/s3.c
> +++ b/x86/s3.c
> @@ -2,15 +2,6 @@
>  #include "acpi.h"
>  #include "asm/io.h"
>  
> -static u32* find_resume_vector_addr(void)
> -{
> -    struct facs_descriptor_rev1 *facs = find_acpi_table_addr(FACS_SIGNATURE);
> -    if (!facs)
> -        return 0;
> -    printf("FACS is at %p\n", facs);
> -    return &facs->firmware_waking_vector;
> -}
> -
>  #define RTC_SECONDS_ALARM       1
>  #define RTC_MINUTES_ALARM       3
>  #define RTC_HOURS_ALARM         5
> @@ -40,12 +31,13 @@ extern char resume_start, resume_end;
>  int main(int argc, char **argv)
>  {
>  	struct fadt_descriptor_rev1 *fadt = find_acpi_table_addr(FACP_SIGNATURE);
> -	volatile u32 *resume_vector_ptr = find_resume_vector_addr();
> +	struct facs_descriptor_rev1 *facs = find_acpi_table_addr(FACS_SIGNATURE);
>  	char *addr, *resume_vec = (void*)0x1000;
>  
> -	*resume_vector_ptr = (u32)(ulong)resume_vec;
> +	facs->firmware_waking_vector = (u32)(ulong)resume_vec;
>  
> -	printf("resume vector addr is %p\n", resume_vector_ptr);
> +	printf("FACS is at %p\n", facs);
> +	printf("resume vector addr is %p\n", &facs->firmware_waking_vector);
>  	for (addr = &resume_start; addr < &resume_end; addr++)
>  		*resume_vec++ = *addr;
>  	printf("copy resume code from %p\n", &resume_start);
> -- 
> 2.25.1
> 

Thanks,
drew

Re: [kvm-unit-tests PATCH v2 01/23] lib: Move acpi header and implementation to lib

[Andrew Jones]

On Fri, May 06, 2022 at 09:55:43PM +0100, Nikos Nikoleris wrote:
> This change is in preparation of using ACPI in arm64 systems booting
> with EFI.
> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  x86/Makefile.common  | 2 +-
>  lib/x86/asm/setup.h  | 2 +-
>  lib/{x86 => }/acpi.h | 4 ++--
>  lib/{x86 => }/acpi.c | 0

I like that diffstat that git-format-patch -M creates when only
moving, but not changing a file. I'd prefer we take the opportunity
to update the file's coding style though and then confirm we didn't
introduce any changes with a whitespace ignoring diff.

>  x86/s3.c             | 2 +-
>  x86/vmexit.c         | 2 +-
>  6 files changed, 6 insertions(+), 6 deletions(-)
>  rename lib/{x86 => }/acpi.h (99%)
>  rename lib/{x86 => }/acpi.c (100%)
> 
> diff --git a/x86/Makefile.common b/x86/Makefile.common
> index b903988..4cdba79 100644
> --- a/x86/Makefile.common
> +++ b/x86/Makefile.common
> @@ -2,6 +2,7 @@
>  
>  all: directories test_cases
>  
> +cflatobjs += lib/acpi.o
>  cflatobjs += lib/pci.o
>  cflatobjs += lib/pci-edu.o
>  cflatobjs += lib/alloc.o
> @@ -18,7 +19,6 @@ cflatobjs += lib/x86/apic.o
>  cflatobjs += lib/x86/atomic.o
>  cflatobjs += lib/x86/desc.o
>  cflatobjs += lib/x86/isr.o
> -cflatobjs += lib/x86/acpi.o
>  cflatobjs += lib/x86/stack.o
>  cflatobjs += lib/x86/fault_test.o
>  cflatobjs += lib/x86/delay.o
> diff --git a/lib/x86/asm/setup.h b/lib/x86/asm/setup.h
> index 24d4fa9..f46462c 100644
> --- a/lib/x86/asm/setup.h
> +++ b/lib/x86/asm/setup.h
> @@ -4,7 +4,7 @@
>  unsigned long setup_tss(u8 *stacktop);
>  
>  #ifdef CONFIG_EFI
> -#include "x86/acpi.h"
> +#include "acpi.h"
>  #include "x86/apic.h"
>  #include "x86/processor.h"
>  #include "x86/smp.h"
> diff --git a/lib/x86/acpi.h b/lib/acpi.h
> similarity index 99%
> rename from lib/x86/acpi.h
> rename to lib/acpi.h
> index 67ba389..1e89840 100644
> --- a/lib/x86/acpi.h
> +++ b/lib/acpi.h
> @@ -1,5 +1,5 @@
> -#ifndef _X86_ACPI_H_
> -#define _X86_ACPI_H_
> +#ifndef _ACPI_H_
> +#define _ACPI_H_
>  
>  #include "libcflat.h"
>  
> diff --git a/lib/x86/acpi.c b/lib/acpi.c
> similarity index 100%
> rename from lib/x86/acpi.c
> rename to lib/acpi.c
> diff --git a/x86/s3.c b/x86/s3.c
> index 6e41d0c..378d37a 100644
> --- a/x86/s3.c
> +++ b/x86/s3.c
> @@ -1,5 +1,5 @@
>  #include "libcflat.h"
> -#include "x86/acpi.h"
> +#include "acpi.h"
>  #include "asm/io.h"
>  
>  static u32* find_resume_vector_addr(void)
> diff --git a/x86/vmexit.c b/x86/vmexit.c
> index 4adec78..2bac049 100644
> --- a/x86/vmexit.c
> +++ b/x86/vmexit.c
> @@ -1,9 +1,9 @@
> +#include "acpi.h"
>  #include "libcflat.h"

I prefer libcflat.h on top.

>  #include "smp.h"
>  #include "pci.h"
>  #include "x86/vm.h"
>  #include "x86/desc.h"
> -#include "x86/acpi.h"
>  #include "x86/apic.h"
>  #include "x86/isr.h"
>  
> -- 
> 2.25.1
> 

Thanks,
drew

Re: [kvm-unit-tests PATCH v2 03/23] lib: Add support for the XSDT ACPI table

[Andrew Jones]

On Fri, May 06, 2022 at 09:55:45PM +0100, Nikos Nikoleris wrote:
> XSDT provides pointers to other ACPI tables much like RSDT. However,
> contrary to RSDT that provides 32-bit addresses, XSDT provides 64-bit
> pointers. ACPI requires that if XSDT is valid then it takes precedence
> over RSDT.
> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  lib/acpi.h |   6 ++++
>  lib/acpi.c | 103 ++++++++++++++++++++++++++++++++---------------------
>  2 files changed, 68 insertions(+), 41 deletions(-)
> 
> diff --git a/lib/acpi.h b/lib/acpi.h
> index 42a2c16..d80b983 100644
> --- a/lib/acpi.h
> +++ b/lib/acpi.h
> @@ -13,6 +13,7 @@
>  
>  #define RSDP_SIGNATURE ACPI_SIGNATURE('R','S','D','P')
>  #define RSDT_SIGNATURE ACPI_SIGNATURE('R','S','D','T')
> +#define XSDT_SIGNATURE ACPI_SIGNATURE('X','S','D','T')
>  #define FACP_SIGNATURE ACPI_SIGNATURE('F','A','C','P')
>  #define FACS_SIGNATURE ACPI_SIGNATURE('F','A','C','S')
>  
> @@ -56,6 +57,11 @@ struct rsdt_descriptor_rev1 {
>      u32 table_offset_entry[0];
>  } __attribute__ ((packed));
>  
> +struct acpi_table_xsdt {
> +    ACPI_TABLE_HEADER_DEF
> +    u64 table_offset_entry[1];
> +} __attribute__ ((packed));
> +
>  struct fadt_descriptor_rev1
>  {
>      ACPI_TABLE_HEADER_DEF     /* ACPI common table header */
> diff --git a/lib/acpi.c b/lib/acpi.c
> index de275ca..9b8700c 100644
> --- a/lib/acpi.c
> +++ b/lib/acpi.c
> @@ -38,45 +38,66 @@ static struct rsdp_descriptor *get_rsdp(void)
>  
>  void* find_acpi_table_addr(u32 sig)
>  {
> -    struct rsdp_descriptor *rsdp;
> -    struct rsdt_descriptor_rev1 *rsdt;
> -    void *end;
> -    int i;
> -
> -    /* FACS is special... */
> -    if (sig == FACS_SIGNATURE) {
> -        struct fadt_descriptor_rev1 *fadt;
> -        fadt = find_acpi_table_addr(FACP_SIGNATURE);
> -        if (!fadt) {
> -            return NULL;
> -        }
> -        return (void*)(ulong)fadt->firmware_ctrl;
> -    }
> -
> -    rsdp = get_rsdp();
> -    if (rsdp == NULL) {
> -        printf("Can't find RSDP\n");
> -        return 0;
> -    }
> -
> -    if (sig == RSDP_SIGNATURE) {
> -        return rsdp;
> -    }
> -
> -    rsdt = (void*)(ulong)rsdp->rsdt_physical_address;
> -    if (!rsdt || rsdt->signature != RSDT_SIGNATURE)
> -        return 0;
> -
> -    if (sig == RSDT_SIGNATURE) {
> -        return rsdt;
> -    }
> -
> -    end = (void*)rsdt + rsdt->length;
> -    for (i=0; (void*)&rsdt->table_offset_entry[i] < end; i++) {
> -        struct acpi_table *t = (void*)(ulong)rsdt->table_offset_entry[i];
> -        if (t && t->signature == sig) {
> -            return t;
> -        }
> -    }
> -   return NULL;

Let's definitely fix the coding style earlier in the series. Either while
moving the file or as another patch right after moving the file. That, or
use the old style for this file when updating it, since we don't want to
mix styles in the same file.

> +	struct rsdp_descriptor *rsdp;
> +	struct rsdt_descriptor_rev1 *rsdt;
> +	struct acpi_table_xsdt *xsdt = NULL;
> +	void *end;
> +	int i;
> +
> +	/* FACS is special... */
> +	if (sig == FACS_SIGNATURE) {
> +		struct fadt_descriptor_rev1 *fadt;
> +
> +		fadt = find_acpi_table_addr(FACP_SIGNATURE);
> +		if (!fadt)
> +			return NULL;
> +
> +		return (void*)(ulong)fadt->firmware_ctrl;
> +	}
> +
> +	rsdp = get_rsdp();
> +	if (rsdp == NULL) {
> +		printf("Can't find RSDP\n");
> +		return 0;
> +	}
> +
> +	if (sig == RSDP_SIGNATURE)
> +		return rsdp;
> +
> +	rsdt = (void *)(ulong)rsdp->rsdt_physical_address;
> +	if (!rsdt || rsdt->signature != RSDT_SIGNATURE)
> +		rsdt = NULL;
> +
> +	if (sig == RSDT_SIGNATURE)
> +		return rsdt;
> +
> +	if (rsdp->revision > 1)
> +		xsdt = (void *)(ulong)rsdp->xsdt_physical_address;
> +	if (!xsdt || xsdt->signature != XSDT_SIGNATURE)
> +		xsdt = NULL;
> +
> +	if (sig == XSDT_SIGNATURE)
> +		return xsdt;
> +
> +	// APCI requires that we first try to use XSDT if it's valid,
> +	//  we use to find other tables, otherwise we use RSDT.

/* ... */ style comments please. And the comment looks like it's missing
something like "When it's valid..."

> +	if (xsdt) {
> +		end = (void *)(ulong)xsdt + xsdt->length;
> +		for (i = 0; (void *)&xsdt->table_offset_entry[i] < end; i++) {
> +			struct acpi_table *t =
> +				(void *)xsdt->table_offset_entry[i];

nit: The kernel's checkpatch allows 100 char line length. Let's use all of them :-)

> +			if (t && t->signature == sig)
> +				return t;
> +		}
> +	} else if (rsdt) {
> +		end = (void *)rsdt + rsdt->length;
> +		for (i = 0; (void *)&rsdt->table_offset_entry[i] < end; i++) {
> +			struct acpi_table *t =
> +				(void *)(ulong)rsdt->table_offset_entry[i];

Same nit as above.

> +			if (t && t->signature == sig)
> +				return t;
> +		}
> +	}
> +
> +	return NULL;
>  }
> -- 
> 2.25.1
>

Thanks,
drew 

Re: [kvm-unit-tests PATCH v2 04/23] lib: Extend the definition of the ACPI table FADT

[Andrew Jones]

On Fri, May 06, 2022 at 09:55:46PM +0100, Nikos Nikoleris wrote:
> This change add more fields in the APCI table FADT to allow for the
> discovery of the PSCI conduit in arm64 systems. The definition for
> FADT is similar to the one in include/acpi/actbl.h in Linux.
> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  lib/acpi.h   | 35 ++++++++++++++++++++++++++++++-----
>  lib/acpi.c   |  2 +-
>  x86/s3.c     |  2 +-
>  x86/vmexit.c |  2 +-
>  4 files changed, 33 insertions(+), 8 deletions(-)

Looks fine to me, so

Reviewed-by: Andrew Jones <drjones@redhat.com>

but we should get sign-off from x86 people since they've been expecting
fadt to be rev1 all this time and now it's changing.

Thanks,
drew

> 
> diff --git a/lib/acpi.h b/lib/acpi.h
> index d80b983..9f27eb1 100644
> --- a/lib/acpi.h
> +++ b/lib/acpi.h
> @@ -62,7 +62,15 @@ struct acpi_table_xsdt {
>      u64 table_offset_entry[1];
>  } __attribute__ ((packed));
>  
> -struct fadt_descriptor_rev1
> +struct acpi_generic_address {
> +    u8 space_id;            /* Address space where struct or register exists */
> +    u8 bit_width;           /* Size in bits of given register */
> +    u8 bit_offset;          /* Bit offset within the register */
> +    u8 access_width;        /* Minimum Access size (ACPI 3.0) */
> +    u64 address;            /* 64-bit address of struct or register */
> +} __attribute__ ((packed));
> +
> +struct acpi_table_fadt
>  {
>      ACPI_TABLE_HEADER_DEF     /* ACPI common table header */
>      u32 firmware_ctrl;          /* Physical address of FACS */
> @@ -100,10 +108,27 @@ struct fadt_descriptor_rev1
>      u8  day_alrm;               /* Index to day-of-month alarm in RTC CMOS RAM */
>      u8  mon_alrm;               /* Index to month-of-year alarm in RTC CMOS RAM */
>      u8  century;                /* Index to century in RTC CMOS RAM */
> -    u8  reserved4;              /* Reserved */
> -    u8  reserved4a;             /* Reserved */
> -    u8  reserved4b;             /* Reserved */
> -};
> +    u16 boot_flags;             /* IA-PC Boot Architecture Flags (see below for individual flags) */
> +    u8 reserved;                /* Reserved, must be zero */
> +    u32 flags;                  /* Miscellaneous flag bits (see below for individual flags) */
> +    struct acpi_generic_address reset_register;     /* 64-bit address of the Reset register */
> +    u8 reset_value;             /* Value to write to the reset_register port to reset the system */
> +    u16 arm_boot_flags;         /* ARM-Specific Boot Flags (see below for individual flags) (ACPI 5.1) */
> +    u8 minor_revision;          /* FADT Minor Revision (ACPI 5.1) */
> +    u64 Xfacs;                  /* 64-bit physical address of FACS */
> +    u64 Xdsdt;                  /* 64-bit physical address of DSDT */
> +    struct acpi_generic_address xpm1a_event_block;  /* 64-bit Extended Power Mgt 1a Event Reg Blk address */
> +    struct acpi_generic_address xpm1b_event_block;  /* 64-bit Extended Power Mgt 1b Event Reg Blk address */
> +    struct acpi_generic_address xpm1a_control_block;        /* 64-bit Extended Power Mgt 1a Control Reg Blk address */
> +    struct acpi_generic_address xpm1b_control_block;        /* 64-bit Extended Power Mgt 1b Control Reg Blk address */
> +    struct acpi_generic_address xpm2_control_block; /* 64-bit Extended Power Mgt 2 Control Reg Blk address */
> +    struct acpi_generic_address xpm_timer_block;    /* 64-bit Extended Power Mgt Timer Ctrl Reg Blk address */
> +    struct acpi_generic_address xgpe0_block;        /* 64-bit Extended General Purpose Event 0 Reg Blk address */
> +    struct acpi_generic_address xgpe1_block;        /* 64-bit Extended General Purpose Event 1 Reg Blk address */
> +    struct acpi_generic_address sleep_control;      /* 64-bit Sleep Control register (ACPI 5.0) */
> +    struct acpi_generic_address sleep_status;       /* 64-bit Sleep Status register (ACPI 5.0) */
> +    u64 hypervisor_id;      /* Hypervisor Vendor ID (ACPI 6.0) */
> +}  __attribute__ ((packed));
>  
>  struct facs_descriptor_rev1
>  {
> diff --git a/lib/acpi.c b/lib/acpi.c
> index 9b8700c..e8440ae 100644
> --- a/lib/acpi.c
> +++ b/lib/acpi.c
> @@ -46,7 +46,7 @@ void* find_acpi_table_addr(u32 sig)
>  
>  	/* FACS is special... */
>  	if (sig == FACS_SIGNATURE) {
> -		struct fadt_descriptor_rev1 *fadt;
> +		struct acpi_table_fadt *fadt;
>  
>  		fadt = find_acpi_table_addr(FACP_SIGNATURE);
>  		if (!fadt)
> diff --git a/x86/s3.c b/x86/s3.c
> index 89d69fc..16e79f8 100644
> --- a/x86/s3.c
> +++ b/x86/s3.c
> @@ -30,7 +30,7 @@ extern char resume_start, resume_end;
>  
>  int main(int argc, char **argv)
>  {
> -	struct fadt_descriptor_rev1 *fadt = find_acpi_table_addr(FACP_SIGNATURE);
> +	struct acpi_table_fadt *fadt = find_acpi_table_addr(FACP_SIGNATURE);
>  	struct facs_descriptor_rev1 *facs = find_acpi_table_addr(FACS_SIGNATURE);
>  	char *addr, *resume_vec = (void*)0x1000;
>  
> diff --git a/x86/vmexit.c b/x86/vmexit.c
> index 2bac049..fcc0760 100644
> --- a/x86/vmexit.c
> +++ b/x86/vmexit.c
> @@ -206,7 +206,7 @@ int pm_tmr_blk;
>  static void inl_pmtimer(void)
>  {
>      if (!pm_tmr_blk) {
> -	struct fadt_descriptor_rev1 *fadt;
> +	struct acpi_table_fadt *fadt;
>  
>  	fadt = find_acpi_table_addr(FACP_SIGNATURE);
>  	pm_tmr_blk = fadt->pm_tmr_blk;
> -- 
> 2.25.1
> 

Re: [kvm-unit-tests PATCH v2 05/23] arm/arm64: Add support for setting up the PSCI conduit through ACPI

[Andrew Jones]

On Fri, May 06, 2022 at 09:55:47PM +0100, Nikos Nikoleris wrote:
> In systems with ACPI support and when a DT is not provided, we can use
> the FADT to discover whether PSCI calls need to use smc or hvc
> calls. This change implements this but retains the default behavior;
> we check if a valid DT is provided, if not, we try to setup the PSCI
> conduit using ACPI.
> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  arm/Makefile.common |  1 +
>  lib/acpi.h          |  5 +++++
>  lib/arm/psci.c      | 23 ++++++++++++++++++++++-
>  lib/devicetree.c    |  2 +-
>  4 files changed, 29 insertions(+), 2 deletions(-)
> 
> diff --git a/arm/Makefile.common b/arm/Makefile.common
> index 38385e0..8e9b3bb 100644
> --- a/arm/Makefile.common
> +++ b/arm/Makefile.common
> @@ -38,6 +38,7 @@ cflatobjs += lib/alloc_page.o
>  cflatobjs += lib/vmalloc.o
>  cflatobjs += lib/alloc.o
>  cflatobjs += lib/devicetree.o
> +cflatobjs += lib/acpi.o
>  cflatobjs += lib/pci.o
>  cflatobjs += lib/pci-host-generic.o
>  cflatobjs += lib/pci-testdev.o
> diff --git a/lib/acpi.h b/lib/acpi.h
> index 9f27eb1..139ccba 100644
> --- a/lib/acpi.h
> +++ b/lib/acpi.h
> @@ -130,6 +130,11 @@ struct acpi_table_fadt
>      u64 hypervisor_id;      /* Hypervisor Vendor ID (ACPI 6.0) */
>  }  __attribute__ ((packed));
>  
> +/* Masks for FADT ARM Boot Architecture Flags (arm_boot_flags) ACPI 5.1 */
> +
> +#define ACPI_FADT_PSCI_COMPLIANT    (1)         /* 00: [V5+] PSCI 0.2+ is implemented */
> +#define ACPI_FADT_PSCI_USE_HVC      (1<<1)      /* 01: [V5+] HVC must be used instead of SMC as the PSCI conduit */
> +
>  struct facs_descriptor_rev1
>  {
>      u32 signature;           /* ACPI Signature */
> diff --git a/lib/arm/psci.c b/lib/arm/psci.c
> index 9c031a1..0e96d19 100644
> --- a/lib/arm/psci.c
> +++ b/lib/arm/psci.c
> @@ -6,6 +6,7 @@
>   *
>   * This work is licensed under the terms of the GNU LGPL, version 2.
>   */
> +#include <acpi.h>
>  #include <devicetree.h>
>  #include <asm/psci.h>
>  #include <asm/setup.h>
> @@ -56,7 +57,7 @@ void psci_system_off(void)
>  	printf("CPU%d unable to do system off (error = %d)\n", smp_processor_id(), err);
>  }
>  
> -void psci_set_conduit(void)
> +static void psci_set_conduit_fdt(void)
>  {
>  	const void *fdt = dt_fdt();
>  	const struct fdt_property *method;
> @@ -75,3 +76,23 @@ void psci_set_conduit(void)
>  	else
>  		assert_msg(false, "Unknown PSCI conduit: %s", method->data);
>  }
> +
> +static void psci_set_conduit_acpi(void)
> +{
> +	struct acpi_table_fadt *fadt = find_acpi_table_addr(FACP_SIGNATURE);
> +	assert_msg(fadt, "Unable to find ACPI FADT");
> +	assert_msg(fadt->arm_boot_flags & ACPI_FADT_PSCI_COMPLIANT,
> +		   "PSCI is not supported in this platfrom");
> +	if (fadt->arm_boot_flags & ACPI_FADT_PSCI_USE_HVC)
> +		psci_invoke = psci_invoke_hvc;
> +	else
> +		psci_invoke = psci_invoke_smc;
> +}
> +
> +void psci_set_conduit(void)
> +{
> +	if (dt_available())
> +		psci_set_conduit_fdt();
> +	else
> +		psci_set_conduit_acpi();
> +}
> diff --git a/lib/devicetree.c b/lib/devicetree.c
> index 78c1f6f..3ff9d16 100644
> --- a/lib/devicetree.c
> +++ b/lib/devicetree.c
> @@ -16,7 +16,7 @@ const void *dt_fdt(void)
>  
>  bool dt_available(void)
>  {
> -	return fdt_check_header(fdt) == 0;
> +	return fdt && fdt_check_header(fdt) == 0;
>  }
>  
>  int dt_get_nr_cells(int fdtnode, u32 *nr_address_cells, u32 *nr_size_cells)
> -- 
> 2.25.1
>

Reviewed-by: Andrew Jones <drjones@redhat.com>

Re: [kvm-unit-tests PATCH v2 06/23] arm/arm64: Add support for discovering the UART through ACPI

[Andrew Jones]

On Fri, May 06, 2022 at 09:55:48PM +0100, Nikos Nikoleris wrote:
> In systems with ACPI support and when a DT is not provided, we can use
> the SPCR to discover the serial port address range. This change
> implements this but retains the default behavior; we check if a valid
> DT is provided, if not, we try to discover the UART using ACPI.
> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  lib/acpi.h     | 25 +++++++++++++++++++++++++
>  lib/arm/io.c   | 21 +++++++++++++++++++--
>  lib/arm/psci.c |  4 +++-
>  3 files changed, 47 insertions(+), 3 deletions(-)
> 
> diff --git a/lib/acpi.h b/lib/acpi.h
> index 139ccba..5213299 100644
> --- a/lib/acpi.h
> +++ b/lib/acpi.h
> @@ -16,6 +16,7 @@
>  #define XSDT_SIGNATURE ACPI_SIGNATURE('X','S','D','T')
>  #define FACP_SIGNATURE ACPI_SIGNATURE('F','A','C','P')
>  #define FACS_SIGNATURE ACPI_SIGNATURE('F','A','C','S')
> +#define SPCR_SIGNATURE ACPI_SIGNATURE('S','P','C','R')
>  
>  
>  #define ACPI_SIGNATURE_8BYTE(c1, c2, c3, c4, c5, c6, c7, c8) \
> @@ -147,6 +148,30 @@ struct facs_descriptor_rev1
>      u8  reserved3 [40];         /* Reserved - must be zero */
>  } __attribute__ ((packed));
>  
> +struct spcr_descriptor {
> +    ACPI_TABLE_HEADER_DEF   /* ACPI common table header */
> +    u8 interface_type;      /* 0=full 16550, 1=subset of 16550 */
> +    u8 reserved[3];
> +    struct acpi_generic_address serial_port;
> +    u8 interrupt_type;
> +    u8 pc_interrupt;
> +    u32 interrupt;
> +    u8 baud_rate;
> +    u8 parity;
> +    u8 stop_bits;
> +    u8 flow_control;
> +    u8 terminal_type;
> +    u8 reserved1;
> +    u16 pci_device_id;
> +    u16 pci_vendor_id;
> +    u8 pci_bus;
> +    u8 pci_device;
> +    u8 pci_function;
> +    u32 pci_flags;
> +    u8 pci_segment;
> +    u32 reserved2;
> +} __attribute__ ((packed));
> +
>  void set_efi_rsdp(struct rsdp_descriptor *rsdp);
>  void* find_acpi_table_addr(u32 sig);
>  
> diff --git a/lib/arm/io.c b/lib/arm/io.c
> index 343e108..893bdfc 100644
> --- a/lib/arm/io.c
> +++ b/lib/arm/io.c
> @@ -8,6 +8,7 @@
>   *
>   * This work is licensed under the terms of the GNU LGPL, version 2.
>   */
> +#include <acpi.h>

nit: below libcflat.h

>  #include <libcflat.h>
>  #include <devicetree.h>
>  #include <chr-testdev.h>
> @@ -29,7 +30,7 @@ static struct spinlock uart_lock;
>  #define UART_EARLY_BASE (u8 *)(unsigned long)CONFIG_UART_EARLY_BASE
>  static volatile u8 *uart0_base = UART_EARLY_BASE;
>  
> -static void uart0_init(void)
> +static void uart0_init_fdt(void)
>  {
>  	/*
>  	 * kvm-unit-tests uses the uart only for output. Both uart models have
> @@ -73,9 +74,25 @@ static void uart0_init(void)
>  	}
>  }
>  
> +static void uart0_init_acpi(void)
> +{
> +	struct spcr_descriptor *spcr = find_acpi_table_addr(SPCR_SIGNATURE);
> +	assert_msg(spcr, "Unable to find ACPI SPCR");
> +	uart0_base = ioremap(spcr->serial_port.address, spcr->serial_port.bit_width);
> +
> +	if (uart0_base != UART_EARLY_BASE) {
> +		printf("WARNING: early print support may not work. "
> +		       "Found uart at %p, but early base is %p.\n",
> +			uart0_base, UART_EARLY_BASE);
> +	}
> +}
> +
>  void io_init(void)
>  {
> -	uart0_init();
> +	if (dt_available())
> +		uart0_init_fdt();
> +	else
> +		uart0_init_acpi();
>  	chr_testdev_init();
>  }
>  
> diff --git a/lib/arm/psci.c b/lib/arm/psci.c
> index 0e96d19..afbc33d 100644
> --- a/lib/arm/psci.c
> +++ b/lib/arm/psci.c
> @@ -80,9 +80,11 @@ static void psci_set_conduit_fdt(void)
>  static void psci_set_conduit_acpi(void)
>  {
>  	struct acpi_table_fadt *fadt = find_acpi_table_addr(FACP_SIGNATURE);
> +
>  	assert_msg(fadt, "Unable to find ACPI FADT");
>  	assert_msg(fadt->arm_boot_flags & ACPI_FADT_PSCI_COMPLIANT,
> -		   "PSCI is not supported in this platfrom");
> +		   "PSCI is not supported in this platform");
> +

These psci_set_conduit_acpi() cleanups belong in the last patch.

>  	if (fadt->arm_boot_flags & ACPI_FADT_PSCI_USE_HVC)
>  		psci_invoke = psci_invoke_hvc;
>  	else
> -- 
> 2.25.1
>

Besides the patch cleanup and the #include nit,

Reviewed-by: Andrew Jones <drjones@redhat.com>

Thanks,
drew

Re: [kvm-unit-tests PATCH v2 07/23] arm/arm64: Add support for timer initialization through ACPI

[Andrew Jones]

On Fri, May 06, 2022 at 09:55:49PM +0100, Nikos Nikoleris wrote:
> For systems with ACPI support, we can discover timers through the ACPI
> GTDT table. This change implements the code to discover timers through
> the GTDT and adds ACPI support in timer_save_state. This change
> retains the default behavior; we check if a valid DT is provided, if
> not, we try to discover timers using ACPI.
> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  arm/Makefile.common |  1 +
>  lib/arm/asm/timer.h |  2 ++
>  lib/acpi.h          | 18 +++++++++++
>  lib/arm/setup.c     | 39 ------------------------
>  lib/arm/timer.c     | 73 +++++++++++++++++++++++++++++++++++++++++++++
>  5 files changed, 94 insertions(+), 39 deletions(-)
>  create mode 100644 lib/arm/timer.c
> 
> diff --git a/arm/Makefile.common b/arm/Makefile.common
> index 8e9b3bb..5be42c0 100644
> --- a/arm/Makefile.common
> +++ b/arm/Makefile.common
> @@ -53,6 +53,7 @@ cflatobjs += lib/arm/psci.o
>  cflatobjs += lib/arm/smp.o
>  cflatobjs += lib/arm/delay.o
>  cflatobjs += lib/arm/gic.o lib/arm/gic-v2.o lib/arm/gic-v3.o
> +cflatobjs += lib/arm/timer.o
>  
>  OBJDIRS += lib/arm
>  
> diff --git a/lib/arm/asm/timer.h b/lib/arm/asm/timer.h
> index f75cc67..aaf839f 100644
> --- a/lib/arm/asm/timer.h
> +++ b/lib/arm/asm/timer.h
> @@ -27,5 +27,7 @@ extern struct timer_state __timer_state;
>  #define TIMER_PTIMER_IRQ (__timer_state.ptimer.irq)
>  #define TIMER_VTIMER_IRQ (__timer_state.vtimer.irq)
>  
> +void timer_save_state(void);
> +
>  #endif /* !__ASSEMBLY__ */
>  #endif /* _ASMARM_TIMER_H_ */
> diff --git a/lib/acpi.h b/lib/acpi.h
> index 5213299..297ad87 100644
> --- a/lib/acpi.h
> +++ b/lib/acpi.h
> @@ -17,6 +17,7 @@
>  #define FACP_SIGNATURE ACPI_SIGNATURE('F','A','C','P')
>  #define FACS_SIGNATURE ACPI_SIGNATURE('F','A','C','S')
>  #define SPCR_SIGNATURE ACPI_SIGNATURE('S','P','C','R')
> +#define GTDT_SIGNATURE ACPI_SIGNATURE('G','T','D','T')
>  
>  
>  #define ACPI_SIGNATURE_8BYTE(c1, c2, c3, c4, c5, c6, c7, c8) \
> @@ -172,6 +173,23 @@ struct spcr_descriptor {
>      u32 reserved2;
>  } __attribute__ ((packed));
>  
> +struct acpi_table_gtdt {
> +    ACPI_TABLE_HEADER_DEF   /* ACPI common table header */
> +    u64 counter_block_addresss;
> +    u32 reserved;
> +    u32 secure_el1_interrupt;
> +    u32 secure_el1_flags;
> +    u32 non_secure_el1_interrupt;
> +    u32 non_secure_el1_flags;
> +    u32 virtual_timer_interrupt;
> +    u32 virtual_timer_flags;
> +    u32 non_secure_el2_interrupt;
> +    u32 non_secure_el2_flags;
> +    u64 counter_read_block_address;
> +    u32 platform_timer_count;
> +    u32 platform_timer_offset;
> +} __attribute__ ((packed));
> +
>  void set_efi_rsdp(struct rsdp_descriptor *rsdp);
>  void* find_acpi_table_addr(u32 sig);
>  
> diff --git a/lib/arm/setup.c b/lib/arm/setup.c
> index bcdf0d7..1572c64 100644
> --- a/lib/arm/setup.c
> +++ b/lib/arm/setup.c
> @@ -35,8 +35,6 @@
>  
>  extern unsigned long etext;
>  
> -struct timer_state __timer_state;
> -
>  char *initrd;
>  u32 initrd_size;
>  
> @@ -199,43 +197,6 @@ static void mem_init(phys_addr_t freemem_start)
>  	page_alloc_ops_enable();
>  }
>  
> -static void timer_save_state(void)
> -{
> -	const struct fdt_property *prop;
> -	const void *fdt = dt_fdt();
> -	int node, len;
> -	u32 *data;
> -
> -	node = fdt_node_offset_by_compatible(fdt, -1, "arm,armv8-timer");
> -	assert(node >= 0 || node == -FDT_ERR_NOTFOUND);
> -
> -	if (node == -FDT_ERR_NOTFOUND) {
> -		__timer_state.ptimer.irq = -1;
> -		__timer_state.vtimer.irq = -1;
> -		return;
> -	}
> -
> -	/*
> -	 * From Linux devicetree timer binding documentation
> -	 *
> -	 * interrupts <type irq flags>:
> -	 *	secure timer irq
> -	 *	non-secure timer irq		(ptimer)
> -	 *	virtual timer irq		(vtimer)
> -	 *	hypervisor timer irq
> -	 */
> -	prop = fdt_get_property(fdt, node, "interrupts", &len);
> -	assert(prop && len == (4 * 3 * sizeof(u32)));
> -
> -	data = (u32 *)prop->data;
> -	assert(fdt32_to_cpu(data[3]) == 1 /* PPI */);
> -	__timer_state.ptimer.irq = fdt32_to_cpu(data[4]);
> -	__timer_state.ptimer.irq_flags = fdt32_to_cpu(data[5]);
> -	assert(fdt32_to_cpu(data[6]) == 1 /* PPI */);
> -	__timer_state.vtimer.irq = fdt32_to_cpu(data[7]);
> -	__timer_state.vtimer.irq_flags = fdt32_to_cpu(data[8]);
> -}
> -
>  void setup(const void *fdt, phys_addr_t freemem_start)
>  {
>  	void *freemem;
> diff --git a/lib/arm/timer.c b/lib/arm/timer.c
> new file mode 100644
> index 0000000..eceabdf
> --- /dev/null
> +++ b/lib/arm/timer.c
> @@ -0,0 +1,73 @@
> +/*
> + * Initialize timers.
> + *
> + * Copyright (C) 2022, Arm Ltd., Nikos Nikoleris <nikos.nikoleris@arm.com>
> + * Copyright (C) 2014, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#include <libcflat.h>
> +#include <asm/timer.h>

nit: I usually prefer an order like this for #includes

#include <lib-include1, usually libcflat.h>  
#include <lib-include2>
...
#include <sublib-include1, libfdt/* and linux/*>
#include <sublib-include2>
...
#include <asm-include1>
#include <asm-include2>

> +
> +#include <acpi.h>
> +#include <devicetree.h>
> +#include <libfdt/libfdt.h>
> +
> +struct timer_state __timer_state;
> +
> +static void timer_save_state_fdt(void)
> +{
> +	const struct fdt_property *prop;
> +	const void *fdt = dt_fdt();
> +	int node, len;
> +	u32 *data;
> +
> +	node = fdt_node_offset_by_compatible(fdt, -1, "arm,armv8-timer");
> +	assert(node >= 0 || node == -FDT_ERR_NOTFOUND);
> +
> +	if (node == -FDT_ERR_NOTFOUND) {
> +		__timer_state.ptimer.irq = -1;
> +		__timer_state.vtimer.irq = -1;
> +		return;
> +	}
> +
> +	/*
> +	 * From Linux devicetree timer binding documentation
> +	 *
> +	 * interrupts <type irq flags>:
> +	 *	secure timer irq
> +	 *	non-secure timer irq		(ptimer)
> +	 *	virtual timer irq		(vtimer)
> +	 *	hypervisor timer irq
> +	 */
> +	prop = fdt_get_property(fdt, node, "interrupts", &len);
> +	assert(prop && len == (4 * 3 * sizeof(u32)));
> +
> +	data = (u32 *)prop->data;
> +	assert(fdt32_to_cpu(data[3]) == 1 /* PPI */);
> +	__timer_state.ptimer.irq = fdt32_to_cpu(data[4]);
> +	__timer_state.ptimer.irq_flags = fdt32_to_cpu(data[5]);
> +	assert(fdt32_to_cpu(data[6]) == 1 /* PPI */);
> +	__timer_state.vtimer.irq = fdt32_to_cpu(data[7]);
> +	__timer_state.vtimer.irq_flags = fdt32_to_cpu(data[8]);
> +}
> +
> +static void timer_save_state_acpi(void)
> +{
> +	struct acpi_table_gtdt *gtdt = find_acpi_table_addr(GTDT_SIGNATURE);
> +
> +	assert_msg(gtdt, "Unable to find ACPI GTDT");
> +	__timer_state.ptimer.irq = gtdt->non_secure_el1_interrupt;
> +	__timer_state.ptimer.irq_flags = gtdt->non_secure_el1_flags;
> +
> +	__timer_state.vtimer.irq = gtdt->virtual_timer_interrupt;
> +	__timer_state.vtimer.irq_flags = gtdt->virtual_timer_flags;
> +}
> +
> +void timer_save_state(void)
> +{
> +	if (dt_available())
> +		timer_save_state_fdt();
> +	else
> +		timer_save_state_acpi();
> +}
> -- 
> 2.25.1
>

Reviewed-by: Andrew Jones <drjones@redhat.com>

Thanks,
drew

Re: [kvm-unit-tests PATCH v2 08/23] arm/arm64: Add support for cpu initialization through ACPI

[Andrew Jones]

On Fri, May 06, 2022 at 09:55:50PM +0100, Nikos Nikoleris wrote:
> In systems with ACPI support and when a DT is not provided, we can use
> the MADTs to discover the number of CPUs and their corresponding MIDR.
> This change implements this but retains the default behavior; we check
> if a valid DT is provided, if not, we try to discover the cores in the
> system using ACPI.
> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  lib/acpi.h      | 64 +++++++++++++++++++++++++++++++++++++++++++++++++
>  lib/acpi.c      | 21 ++++++++++++++++
>  lib/arm/setup.c | 25 ++++++++++++++++---
>  3 files changed, 107 insertions(+), 3 deletions(-)
> 
> diff --git a/lib/acpi.h b/lib/acpi.h
> index 297ad87..55de54a 100644
> --- a/lib/acpi.h
> +++ b/lib/acpi.h
> @@ -16,6 +16,7 @@
>  #define XSDT_SIGNATURE ACPI_SIGNATURE('X','S','D','T')
>  #define FACP_SIGNATURE ACPI_SIGNATURE('F','A','C','P')
>  #define FACS_SIGNATURE ACPI_SIGNATURE('F','A','C','S')
> +#define MADT_SIGNATURE ACPI_SIGNATURE('A','P','I','C')
>  #define SPCR_SIGNATURE ACPI_SIGNATURE('S','P','C','R')
>  #define GTDT_SIGNATURE ACPI_SIGNATURE('G','T','D','T')
>  
> @@ -149,6 +150,67 @@ struct facs_descriptor_rev1
>      u8  reserved3 [40];         /* Reserved - must be zero */
>  } __attribute__ ((packed));
>  
> +struct acpi_table_madt {
> +    ACPI_TABLE_HEADER_DEF     /* ACPI common table header */
> +    u32 address;               /* Physical address of local APIC */

nit: some whitespace issues

> +    u32 flags;
> +} __attribute__ ((packed));
> +
> +struct acpi_subtable_header {
> +    u8 type;
> +    u8 length;
> +}  __attribute__ ((packed));
> +
> +typedef int (*acpi_table_handler)(struct acpi_subtable_header *header);
> +
> +/* 11: Generic interrupt - GICC (ACPI 5.0 + ACPI 6.0 + ACPI 6.3 changes) */
> +
> +struct acpi_madt_generic_interrupt {
> +    u8 type;
> +    u8 length;
> +    u16 reserved;           /* reserved - must be zero */
> +    u32 cpu_interface_number;
> +    u32 uid;
> +    u32 flags;
> +    u32 parking_version;
> +    u32 performance_interrupt;
> +    u64 parked_address;
> +    u64 base_address;
> +    u64 gicv_base_address;
> +    u64 gich_base_address;
> +    u32 vgic_interrupt;
> +    u64 gicr_base_address;
> +    u64 arm_mpidr;
> +    u8 efficiency_class;
> +    u8 reserved2[1];
> +    u16 spe_interrupt;      /* ACPI 6.3 */
> +} __attribute__ ((packed));
> +
> +/* Values for MADT subtable type in struct acpi_subtable_header */
> +
> +enum acpi_madt_type {
> +    ACPI_MADT_TYPE_LOCAL_APIC = 0,
> +    ACPI_MADT_TYPE_IO_APIC = 1,
> +    ACPI_MADT_TYPE_INTERRUPT_OVERRIDE = 2,
> +    ACPI_MADT_TYPE_NMI_SOURCE = 3,
> +    ACPI_MADT_TYPE_LOCAL_APIC_NMI = 4,
> +    ACPI_MADT_TYPE_LOCAL_APIC_OVERRIDE = 5,
> +    ACPI_MADT_TYPE_IO_SAPIC = 6,
> +    ACPI_MADT_TYPE_LOCAL_SAPIC = 7,
> +    ACPI_MADT_TYPE_INTERRUPT_SOURCE = 8,
> +    ACPI_MADT_TYPE_LOCAL_X2APIC = 9,
> +    ACPI_MADT_TYPE_LOCAL_X2APIC_NMI = 10,
> +    ACPI_MADT_TYPE_GENERIC_INTERRUPT = 11,
> +    ACPI_MADT_TYPE_GENERIC_DISTRIBUTOR = 12,
> +    ACPI_MADT_TYPE_GENERIC_MSI_FRAME = 13,
> +    ACPI_MADT_TYPE_GENERIC_REDISTRIBUTOR = 14,
> +    ACPI_MADT_TYPE_GENERIC_TRANSLATOR = 15,
> +    ACPI_MADT_TYPE_RESERVED = 16    /* 16 and greater are reserved */
> +};
> +
> +/* MADT Local APIC flags */
> +#define ACPI_MADT_ENABLED           (1) /* 00: Processor is usable if set */
> +
>  struct spcr_descriptor {
>      ACPI_TABLE_HEADER_DEF   /* ACPI common table header */
>      u8 interface_type;      /* 0=full 16550, 1=subset of 16550 */
> @@ -192,5 +254,7 @@ struct acpi_table_gtdt {
>  
>  void set_efi_rsdp(struct rsdp_descriptor *rsdp);
>  void* find_acpi_table_addr(u32 sig);
> +void acpi_table_parse_madt(enum acpi_madt_type mtype,
> +			   acpi_table_handler handler);

nit: unnecessary line break

>  
>  #endif
> diff --git a/lib/acpi.c b/lib/acpi.c
> index e8440ae..ad3ae8d 100644
> --- a/lib/acpi.c
> +++ b/lib/acpi.c
> @@ -101,3 +101,24 @@ void* find_acpi_table_addr(u32 sig)
>  
>  	return NULL;
>  }
> +
> +void acpi_table_parse_madt(enum acpi_madt_type mtype,
> +			   acpi_table_handler handler)
> +{
> +	struct acpi_table_madt *madt;
> +	void *end;
> +
> +	madt = find_acpi_table_addr(MADT_SIGNATURE);
> +	assert(madt);
> +
> +	struct acpi_subtable_header *header =
> +		(void *)(ulong)madt + sizeof(struct acpi_table_madt);

nit: unnecessary line break

> +	end = (void *)((ulong)madt + madt->length);
> +
> +	while ((void *)header < end) {
> +		if (header->type == mtype)
> +			handler(header);
> +
> +		header = (void *)(ulong)header + header->length;
> +	}
> +}
> diff --git a/lib/arm/setup.c b/lib/arm/setup.c
> index 1572c64..3c24c75 100644
> --- a/lib/arm/setup.c
> +++ b/lib/arm/setup.c
> @@ -13,6 +13,7 @@
>  #include <libcflat.h>
>  #include <libfdt/libfdt.h>
>  #include <devicetree.h>
> +#include <acpi.h>
>  #include <alloc.h>
>  #include <alloc_phys.h>
>  #include <alloc_page.h>
> @@ -55,7 +56,7 @@ int mpidr_to_cpu(uint64_t mpidr)
>  	return -1;
>  }
>  
> -static void cpu_set(int fdtnode __unused, u64 regval, void *info __unused)
> +static void cpu_set_fdt(int fdtnode __unused, u64 regval, void *info __unused)
>  {
>  	int cpu = nr_cpus++;
>  
> @@ -65,13 +66,31 @@ static void cpu_set(int fdtnode __unused, u64 regval, void *info __unused)
>  	set_cpu_present(cpu, true);
>  }
>  
> +static int cpu_set_acpi(struct acpi_subtable_header *header)
> +{
> +	int cpu = nr_cpus++;
> +	struct acpi_madt_generic_interrupt *gicc = (void *)header;
> +
> +	assert_msg(cpu < NR_CPUS, "Number cpus exceeds maximum supported (%d).", NR_CPUS);
> +
> +	cpus[cpu] = gicc->arm_mpidr;
> +	set_cpu_present(cpu, true);
> +
> +	return 0;
> +}
> +
>  static void cpu_init(void)
>  {
>  	int ret;
>  
>  	nr_cpus = 0;
> -	ret = dt_for_each_cpu_node(cpu_set, NULL);
> -	assert(ret == 0);
> +	if (dt_available()) {
> +		ret = dt_for_each_cpu_node(cpu_set_fdt, NULL);
> +		assert(ret == 0);
> +	} else
> +		acpi_table_parse_madt(ACPI_MADT_TYPE_GENERIC_INTERRUPT,
> +				      cpu_set_acpi);

nit: unnecessary line break and please balance the {} on the else

> +
>  	set_cpu_online(0, true);
>  }
>  
> -- 
> 2.25.1
>

I looked ahead and see you make a lot more use of acpi_table_parse_madt()
when adding the gic support.

Reviewed-by: Andrew Jones <drjones@redhat.com>

Thanks,
drew

Re: [kvm-unit-tests PATCH v2 09/23] lib/printf: Support for precision modifier in printing strings

[Andrew Jones]

On Fri, May 06, 2022 at 09:55:51PM +0100, Nikos Nikoleris wrote:
> This follows the typical format of:
> 
> printf("%.Ns", *str);
> 
> Where N might be a decimal digit string or '*'. This feature is used
> by a future change.
> 
> See also: man 3 printf
> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  lib/printf.c | 84 ++++++++++++++++++++++++++++++++++++++++++++--------
>  1 file changed, 71 insertions(+), 13 deletions(-)
> 
> diff --git a/lib/printf.c b/lib/printf.c
> index 1269723..724befa 100644
> --- a/lib/printf.c
> +++ b/lib/printf.c
> @@ -19,6 +19,7 @@ typedef struct strprops {
>      char pad;
>      int npad;
>      bool alternate;
> +    int precision;
>  } strprops_t;
>  
>  static void addchar(pstream_t *p, char c)
> @@ -43,7 +44,7 @@ static void print_str(pstream_t *p, const char *s, strprops_t props)
>  	}
>      }
>  
> -    while (*s)
> +    while (*s && props.precision--)
>  	addchar(p, *s++);
>  
>      if (npad < 0) {
> @@ -147,9 +148,61 @@ static int fmtnum(const char **fmt)
>      return num;
>  }
>  
> +static inline int isdigit(int c)
> +{
> +    return '0' <= c && c <= '9';
> +}

We desperately need to add ctype to our library. We've already got
isblank, isalpha, and isalnum local to argv.c and I see later you
add isspace. I'll post a patch now that introduces ctype.[ch] with
the ones used by argv. Then, when you respin this series you can
add your ctype functions there.

> +
> +/*
> + * Adapted from drivers/firmware/efi/libstub/vsprintf.c
> + */
> +static int skip_atoi(const char **s)
> +{
> +    int i = 0;
> +
> +    do {
> +	i = i*10 + *((*s)++) - '0';
> +    } while (isdigit(**s));
> +
> +    return i;
> +}
> +
> +/*
> + * Adapted from drivers/firmware/efi/libstub/vsprintf.c
> + */
> +static int get_int(const char **fmt, va_list *ap)
> +{
> +    if (isdigit(**fmt)) {
> +	return skip_atoi(fmt);
> +    }
> +    if (**fmt == '*') {
> +	++(*fmt);
> +	/* it's the next argument */
> +	return va_arg(*ap, int);
> +    }
> +    return 0;
> +}
> +
>  int vsnprintf(char *buf, int size, const char *fmt, va_list va)
>  {
>      pstream_t s;
> +    va_list args;
> +
> +    /*
> +     * We want to pass our input va_list to helper functions by reference,
> +     * but there's an annoying edge case. If va_list was originally passed
> +     * to us by value, we could just pass &ap down to the helpers. This is
> +     * the case on, for example, X86_32.
> +     * However, on X86_64 (and possibly others), va_list is actually a
> +     * size-1 array containing a structure. Our function parameter ap has
> +     * decayed from T[1] to T*, and &ap has type T** rather than T(*)[1],
> +     * which is what will be expected by a function taking a va_list *
> +     * parameter.
> +     * One standard way to solve this mess is by creating a copy in a local
> +     * variable of type va_list and then passing a pointer to that local
> +     * copy instead, which is what we do here.
> +     */
> +    va_copy(args, va);
>  
>      s.buffer = buf;
>      s.remain = size - 1;
> @@ -160,6 +213,7 @@ int vsnprintf(char *buf, int size, const char *fmt, va_list va)
>  	strprops_t props;
>  	memset(&props, 0, sizeof(props));
>  	props.pad = ' ';
> +	props.precision = -1;
>  
>  	if (f != '%') {
>  	    addchar(&s, f);
> @@ -172,11 +226,14 @@ int vsnprintf(char *buf, int size, const char *fmt, va_list va)
>  	    addchar(&s, '%');
>  	    break;
>  	case 'c':
> -            addchar(&s, va_arg(va, int));
> +	    addchar(&s, va_arg(args, int));
>  	    break;
>  	case '\0':
>  	    --fmt;
>  	    break;
> +	case '.':
> +	    props.precision = get_int(&fmt, &args);
> +	    goto morefmt;
>  	case '#':
>  	    props.alternate = true;
>  	    goto morefmt;
> @@ -204,54 +261,55 @@ int vsnprintf(char *buf, int size, const char *fmt, va_list va)
>  	case 'd':
>  	    switch (nlong) {
>  	    case 0:
> -		print_int(&s, va_arg(va, int), 10, props);
> +		print_int(&s, va_arg(args, int), 10, props);
>  		break;
>  	    case 1:
> -		print_int(&s, va_arg(va, long), 10, props);
> +		print_int(&s, va_arg(args, long), 10, props);
>  		break;
>  	    default:
> -		print_int(&s, va_arg(va, long long), 10, props);
> +		print_int(&s, va_arg(args, long long), 10, props);
>  		break;
>  	    }
>  	    break;
>  	case 'u':
>  	    switch (nlong) {
>  	    case 0:
> -		print_unsigned(&s, va_arg(va, unsigned), 10, props);
> +		print_unsigned(&s, va_arg(args, unsigned), 10, props);
>  		break;
>  	    case 1:
> -		print_unsigned(&s, va_arg(va, unsigned long), 10, props);
> +		print_unsigned(&s, va_arg(args, unsigned long), 10, props);
>  		break;
>  	    default:
> -		print_unsigned(&s, va_arg(va, unsigned long long), 10, props);
> +		print_unsigned(&s, va_arg(args, unsigned long long), 10, props);
>  		break;
>  	    }
>  	    break;
>  	case 'x':
>  	    switch (nlong) {
>  	    case 0:
> -		print_unsigned(&s, va_arg(va, unsigned), 16, props);
> +		print_unsigned(&s, va_arg(args, unsigned), 16, props);
>  		break;
>  	    case 1:
> -		print_unsigned(&s, va_arg(va, unsigned long), 16, props);
> +		print_unsigned(&s, va_arg(args, unsigned long), 16, props);
>  		break;
>  	    default:
> -		print_unsigned(&s, va_arg(va, unsigned long long), 16, props);
> +		print_unsigned(&s, va_arg(args, unsigned long long), 16, props);
>  		break;
>  	    }
>  	    break;
>  	case 'p':
>  	    props.alternate = true;
> -	    print_unsigned(&s, (unsigned long)va_arg(va, void *), 16, props);
> +	    print_unsigned(&s, (unsigned long)va_arg(args, void *), 16, props);
>  	    break;
>  	case 's':
> -	    print_str(&s, va_arg(va, const char *), props);
> +	    print_str(&s, va_arg(args, const char *), props);
>  	    break;
>  	default:
>  	    addchar(&s, f);
>  	    break;
>  	}
>      }
> +    va_end(args);
>      *s.buffer = 0;
>      return s.added;
>  }
> -- 
> 2.25.1
>

I think I should also post a patches that finally reformat these older
files. The tab+4spaces stuff must go!

When we get ctype we'll want to move out isdigit, but otherwise

Reviewed-by: Andrew Jones <drjones@redhat.com>

Thanks,
drew
 

Re: [kvm-unit-tests PATCH v2 02/23] lib: Ensure all struct definition for ACPI tables are packed

[Nikos Nikoleris]

Hi Drew,

On 19/05/2022 14:17, Andrew Jones wrote:
> On Fri, May 06, 2022 at 09:55:44PM +0100, Nikos Nikoleris wrote:
>> All ACPI table definitions are provided with precise definitions of
>> field sizes and offsets, make sure that no compiler optimization can
>> interfere with the memory layout of the corresponding structs.
> 
> That seems like a reasonable thing to do. I'm wondering why Linux doesn't
> appear to do it. I see u-boot does, but not for all tables, which also
> makes me scratch my head... I see this patch packs every struct except
> rsdp_descriptor. Is there a reason it was left out?
> 

Thanks for the review!

Linux uses the following:

/*
  * All tables must be byte-packed to match the ACPI specification, since
  * the tables are provided by the system BIOS.
  */
#pragma pack(1)


...


/* Reset to default packing */

#pragma pack()


Happy to switch to #pragma, if we prefer this style.

I missed rsdp_descriptor, it should be packed will fix in v3.


> Another comment below
> 
>>
>> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
>> ---
>>   lib/acpi.h | 11 ++++++++---
>>   x86/s3.c   | 16 ++++------------
>>   2 files changed, 12 insertions(+), 15 deletions(-)
>>
>> diff --git a/lib/acpi.h b/lib/acpi.h
>> index 1e89840..42a2c16 100644
>> --- a/lib/acpi.h
>> +++ b/lib/acpi.h
>> @@ -3,6 +3,11 @@
>>   
>>   #include "libcflat.h"
>>   
>> +/*
>> + * All tables and structures must be byte-packed to match the ACPI
>> + * specification, since the tables are provided by the system BIOS
>> + */
>> +
>>   #define ACPI_SIGNATURE(c1, c2, c3, c4) \
>>   	((c1) | ((c2) << 8) | ((c3) << 16) | ((c4) << 24))
>>   
>> @@ -44,12 +49,12 @@ struct rsdp_descriptor {        /* Root System Descriptor Pointer */
>>   struct acpi_table {
>>       ACPI_TABLE_HEADER_DEF
>>       char data[0];
>> -};
>> +} __attribute__ ((packed));
>>   
>>   struct rsdt_descriptor_rev1 {
>>       ACPI_TABLE_HEADER_DEF
>>       u32 table_offset_entry[0];
>> -};
>> +} __attribute__ ((packed));
>>   
>>   struct fadt_descriptor_rev1
>>   {
>> @@ -104,7 +109,7 @@ struct facs_descriptor_rev1
>>       u32 S4bios_f        : 1;    /* Indicates if S4BIOS support is present */
>>       u32 reserved1       : 31;   /* Must be 0 */
>>       u8  reserved3 [40];         /* Reserved - must be zero */
>> -};
>> +} __attribute__ ((packed));
>>   
>>   void set_efi_rsdp(struct rsdp_descriptor *rsdp);
>>   void* find_acpi_table_addr(u32 sig);
>> diff --git a/x86/s3.c b/x86/s3.c
> 
> The changes below in this file are unrelated, so they should be in a
> separate patch. However, I'm also curious why they're needed. I see
> that find_acpi_table_addr() can return NULL, so it doesn't seem like
> we should be removing the check, but instead changing the check to
> an assert.
> 

These changes are necessary to appease gcc after requiring struct 
facs_descriptor_rev1 to be packed.

>> index 378d37a..89d69fc 100644
>> --- a/x86/s3.c
>> +++ b/x86/s3.c
>> @@ -2,15 +2,6 @@
>>   #include "acpi.h"
>>   #include "asm/io.h"
>>   
>> -static u32* find_resume_vector_addr(void)
>> -{
>> -    struct facs_descriptor_rev1 *facs = find_acpi_table_addr(FACS_SIGNATURE);
>> -    if (!facs)
>> -        return 0;
>> -    printf("FACS is at %p\n", facs);
>> -    return &facs->firmware_waking_vector;

This statement in particular results to a gcc warning. We can't get a 
reference to member of a packed struct.

"taking address of packed member of ‘struct facs_descriptor_rev1’ may 
result in an unaligned pointer value"

What I could do is move the x86/* changes in a separate patch in 
preparation of this one that packs all structs in <acpi.h>

Thanks,

Nikos

>> -}
>> -
>>   #define RTC_SECONDS_ALARM       1
>>   #define RTC_MINUTES_ALARM       3
>>   #define RTC_HOURS_ALARM         5
>> @@ -40,12 +31,13 @@ extern char resume_start, resume_end;
>>   int main(int argc, char **argv)
>>   {
>>   	struct fadt_descriptor_rev1 *fadt = find_acpi_table_addr(FACP_SIGNATURE);
>> -	volatile u32 *resume_vector_ptr = find_resume_vector_addr();
>> +	struct facs_descriptor_rev1 *facs = find_acpi_table_addr(FACS_SIGNATURE);
>>   	char *addr, *resume_vec = (void*)0x1000;
>>   
>> -	*resume_vector_ptr = (u32)(ulong)resume_vec;
>> +	facs->firmware_waking_vector = (u32)(ulong)resume_vec;
>>   
>> -	printf("resume vector addr is %p\n", resume_vector_ptr);
>> +	printf("FACS is at %p\n", facs);
>> +	printf("resume vector addr is %p\n", &facs->firmware_waking_vector);
>>   	for (addr = &resume_start; addr < &resume_end; addr++)
>>   		*resume_vec++ = *addr;
>>   	printf("copy resume code from %p\n", &resume_start);
>> -- 
>> 2.25.1
>>
> 
> Thanks,
> drew
> 

Re: [kvm-unit-tests PATCH v2 09/23] lib/printf: Support for precision modifier in printing strings

[Andrew Jones]

On Thu, May 19, 2022 at 04:52:33PM +0200, Andrew Jones wrote:
> On Fri, May 06, 2022 at 09:55:51PM +0100, Nikos Nikoleris wrote:
> > This follows the typical format of:
> > 
> > printf("%.Ns", *str);
> > 
> > Where N might be a decimal digit string or '*'. This feature is used
> > by a future change.
> > 
> > See also: man 3 printf
> > 
> > Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> > ---
> >  lib/printf.c | 84 ++++++++++++++++++++++++++++++++++++++++++++--------
> >  1 file changed, 71 insertions(+), 13 deletions(-)
> > 
> > diff --git a/lib/printf.c b/lib/printf.c
> > index 1269723..724befa 100644
> > --- a/lib/printf.c
> > +++ b/lib/printf.c
> > @@ -19,6 +19,7 @@ typedef struct strprops {
> >      char pad;
> >      int npad;
> >      bool alternate;
> > +    int precision;
> >  } strprops_t;
> >  
> >  static void addchar(pstream_t *p, char c)
> > @@ -43,7 +44,7 @@ static void print_str(pstream_t *p, const char *s, strprops_t props)
> >  	}
> >      }
> >  
> > -    while (*s)
> > +    while (*s && props.precision--)
> >  	addchar(p, *s++);
> >  
> >      if (npad < 0) {
> > @@ -147,9 +148,61 @@ static int fmtnum(const char **fmt)
> >      return num;
> >  }
> >  
> > +static inline int isdigit(int c)
> > +{
> > +    return '0' <= c && c <= '9';
> > +}
> 
> We desperately need to add ctype to our library. We've already got
> isblank, isalpha, and isalnum local to argv.c and I see later you
> add isspace. I'll post a patch now that introduces ctype.[ch] with
> the ones used by argv. Then, when you respin this series you can
> add your ctype functions there.
> 
> > +
> > +/*
> > + * Adapted from drivers/firmware/efi/libstub/vsprintf.c
> > + */
> > +static int skip_atoi(const char **s)
> > +{
> > +    int i = 0;
> > +
> > +    do {
> > +	i = i*10 + *((*s)++) - '0';
> > +    } while (isdigit(**s));
> > +
> > +    return i;
> > +}
> > +
> > +/*
> > + * Adapted from drivers/firmware/efi/libstub/vsprintf.c
> > + */
> > +static int get_int(const char **fmt, va_list *ap)
> > +{
> > +    if (isdigit(**fmt)) {
> > +	return skip_atoi(fmt);
> > +    }
> > +    if (**fmt == '*') {
> > +	++(*fmt);
> > +	/* it's the next argument */
> > +	return va_arg(*ap, int);
> > +    }
> > +    return 0;
> > +}
> > +
> >  int vsnprintf(char *buf, int size, const char *fmt, va_list va)
> >  {
> >      pstream_t s;
> > +    va_list args;
> > +
> > +    /*
> > +     * We want to pass our input va_list to helper functions by reference,
> > +     * but there's an annoying edge case. If va_list was originally passed
> > +     * to us by value, we could just pass &ap down to the helpers. This is
> > +     * the case on, for example, X86_32.
> > +     * However, on X86_64 (and possibly others), va_list is actually a
> > +     * size-1 array containing a structure. Our function parameter ap has
> > +     * decayed from T[1] to T*, and &ap has type T** rather than T(*)[1],
> > +     * which is what will be expected by a function taking a va_list *
> > +     * parameter.
> > +     * One standard way to solve this mess is by creating a copy in a local
> > +     * variable of type va_list and then passing a pointer to that local
> > +     * copy instead, which is what we do here.
> > +     */
> > +    va_copy(args, va);
> >  
> >      s.buffer = buf;
> >      s.remain = size - 1;
> > @@ -160,6 +213,7 @@ int vsnprintf(char *buf, int size, const char *fmt, va_list va)
> >  	strprops_t props;
> >  	memset(&props, 0, sizeof(props));
> >  	props.pad = ' ';
> > +	props.precision = -1;
> >  
> >  	if (f != '%') {
> >  	    addchar(&s, f);
> > @@ -172,11 +226,14 @@ int vsnprintf(char *buf, int size, const char *fmt, va_list va)
> >  	    addchar(&s, '%');
> >  	    break;
> >  	case 'c':
> > -            addchar(&s, va_arg(va, int));
> > +	    addchar(&s, va_arg(args, int));
> >  	    break;
> >  	case '\0':
> >  	    --fmt;
> >  	    break;
> > +	case '.':
> > +	    props.precision = get_int(&fmt, &args);
> > +	    goto morefmt;
> >  	case '#':
> >  	    props.alternate = true;
> >  	    goto morefmt;
> > @@ -204,54 +261,55 @@ int vsnprintf(char *buf, int size, const char *fmt, va_list va)
> >  	case 'd':
> >  	    switch (nlong) {
> >  	    case 0:
> > -		print_int(&s, va_arg(va, int), 10, props);
> > +		print_int(&s, va_arg(args, int), 10, props);
> >  		break;
> >  	    case 1:
> > -		print_int(&s, va_arg(va, long), 10, props);
> > +		print_int(&s, va_arg(args, long), 10, props);
> >  		break;
> >  	    default:
> > -		print_int(&s, va_arg(va, long long), 10, props);
> > +		print_int(&s, va_arg(args, long long), 10, props);
> >  		break;
> >  	    }
> >  	    break;
> >  	case 'u':
> >  	    switch (nlong) {
> >  	    case 0:
> > -		print_unsigned(&s, va_arg(va, unsigned), 10, props);
> > +		print_unsigned(&s, va_arg(args, unsigned), 10, props);
> >  		break;
> >  	    case 1:
> > -		print_unsigned(&s, va_arg(va, unsigned long), 10, props);
> > +		print_unsigned(&s, va_arg(args, unsigned long), 10, props);
> >  		break;
> >  	    default:
> > -		print_unsigned(&s, va_arg(va, unsigned long long), 10, props);
> > +		print_unsigned(&s, va_arg(args, unsigned long long), 10, props);
> >  		break;
> >  	    }
> >  	    break;
> >  	case 'x':
> >  	    switch (nlong) {
> >  	    case 0:
> > -		print_unsigned(&s, va_arg(va, unsigned), 16, props);
> > +		print_unsigned(&s, va_arg(args, unsigned), 16, props);
> >  		break;
> >  	    case 1:
> > -		print_unsigned(&s, va_arg(va, unsigned long), 16, props);
> > +		print_unsigned(&s, va_arg(args, unsigned long), 16, props);
> >  		break;
> >  	    default:
> > -		print_unsigned(&s, va_arg(va, unsigned long long), 16, props);
> > +		print_unsigned(&s, va_arg(args, unsigned long long), 16, props);
> >  		break;
> >  	    }
> >  	    break;
> >  	case 'p':
> >  	    props.alternate = true;
> > -	    print_unsigned(&s, (unsigned long)va_arg(va, void *), 16, props);
> > +	    print_unsigned(&s, (unsigned long)va_arg(args, void *), 16, props);
> >  	    break;
> >  	case 's':
> > -	    print_str(&s, va_arg(va, const char *), props);
> > +	    print_str(&s, va_arg(args, const char *), props);
> >  	    break;
> >  	default:
> >  	    addchar(&s, f);
> >  	    break;
> >  	}
> >      }
> > +    va_end(args);
> >      *s.buffer = 0;
> >      return s.added;
> >  }
> > -- 
> > 2.25.1
> >
> 
> I think I should also post a patches that finally reformat these older
> files. The tab+4spaces stuff must go!
> 
> When we get ctype we'll want to move out isdigit, but otherwise
> 
> Reviewed-by: Andrew Jones <drjones@redhat.com>

Actually, I'd prefer we make the feature work correctly for %d/%x/%#
from the beginning. It probably wouldn't take much to ensure precision
acts as a minimum (with zero padding when necessary) for numbers.

Thanks,
drew

Re: [kvm-unit-tests PATCH v2 02/23] lib: Ensure all struct definition for ACPI tables are packed

[Andrew Jones]

On Thu, May 19, 2022 at 04:52:04PM +0100, Nikos Nikoleris wrote:
> Hi Drew,
> 
> On 19/05/2022 14:17, Andrew Jones wrote:
> > On Fri, May 06, 2022 at 09:55:44PM +0100, Nikos Nikoleris wrote:
> > > All ACPI table definitions are provided with precise definitions of
> > > field sizes and offsets, make sure that no compiler optimization can
> > > interfere with the memory layout of the corresponding structs.
> > 
> > That seems like a reasonable thing to do. I'm wondering why Linux doesn't
> > appear to do it. I see u-boot does, but not for all tables, which also
> > makes me scratch my head... I see this patch packs every struct except
> > rsdp_descriptor. Is there a reason it was left out?
> > 
> 
> Thanks for the review!
> 
> Linux uses the following:
> 
> /*
>  * All tables must be byte-packed to match the ACPI specification, since
>  * the tables are provided by the system BIOS.
>  */
> #pragma pack(1)
> 
> 
> ...
> 
> 
> /* Reset to default packing */
> 
> #pragma pack()

Ah, that makes sense.

> 
> 
> Happy to switch to #pragma, if we prefer this style.
> 
> I missed rsdp_descriptor, it should be packed will fix in v3.

Maybe we should switch to the #pragma to avoid the verbosity and missing
structures? (I don't have a strong opinion...)

> 
> 
> > Another comment below
> > 
> > > 
> > > Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> > > ---
> > >   lib/acpi.h | 11 ++++++++---
> > >   x86/s3.c   | 16 ++++------------
> > >   2 files changed, 12 insertions(+), 15 deletions(-)
> > > 
> > > diff --git a/lib/acpi.h b/lib/acpi.h
> > > index 1e89840..42a2c16 100644
> > > --- a/lib/acpi.h
> > > +++ b/lib/acpi.h
> > > @@ -3,6 +3,11 @@
> > >   #include "libcflat.h"
> > > +/*
> > > + * All tables and structures must be byte-packed to match the ACPI
> > > + * specification, since the tables are provided by the system BIOS
> > > + */
> > > +
> > >   #define ACPI_SIGNATURE(c1, c2, c3, c4) \
> > >   	((c1) | ((c2) << 8) | ((c3) << 16) | ((c4) << 24))
> > > @@ -44,12 +49,12 @@ struct rsdp_descriptor {        /* Root System Descriptor Pointer */
> > >   struct acpi_table {
> > >       ACPI_TABLE_HEADER_DEF
> > >       char data[0];
> > > -};
> > > +} __attribute__ ((packed));
> > >   struct rsdt_descriptor_rev1 {
> > >       ACPI_TABLE_HEADER_DEF
> > >       u32 table_offset_entry[0];
> > > -};
> > > +} __attribute__ ((packed));
> > >   struct fadt_descriptor_rev1
> > >   {
> > > @@ -104,7 +109,7 @@ struct facs_descriptor_rev1
> > >       u32 S4bios_f        : 1;    /* Indicates if S4BIOS support is present */
> > >       u32 reserved1       : 31;   /* Must be 0 */
> > >       u8  reserved3 [40];         /* Reserved - must be zero */
> > > -};
> > > +} __attribute__ ((packed));
> > >   void set_efi_rsdp(struct rsdp_descriptor *rsdp);
> > >   void* find_acpi_table_addr(u32 sig);
> > > diff --git a/x86/s3.c b/x86/s3.c
> > 
> > The changes below in this file are unrelated, so they should be in a
> > separate patch. However, I'm also curious why they're needed. I see
> > that find_acpi_table_addr() can return NULL, so it doesn't seem like
> > we should be removing the check, but instead changing the check to
> > an assert.
> > 
> 
> These changes are necessary to appease gcc after requiring struct
> facs_descriptor_rev1 to be packed.
> 
> > > index 378d37a..89d69fc 100644
> > > --- a/x86/s3.c
> > > +++ b/x86/s3.c
> > > @@ -2,15 +2,6 @@
> > >   #include "acpi.h"
> > >   #include "asm/io.h"
> > > -static u32* find_resume_vector_addr(void)
> > > -{
> > > -    struct facs_descriptor_rev1 *facs = find_acpi_table_addr(FACS_SIGNATURE);
> > > -    if (!facs)
> > > -        return 0;
> > > -    printf("FACS is at %p\n", facs);
> > > -    return &facs->firmware_waking_vector;
> 
> This statement in particular results to a gcc warning. We can't get a
> reference to member of a packed struct.
> 
> "taking address of packed member of ‘struct facs_descriptor_rev1’ may result
> in an unaligned pointer value"
> 
> What I could do is move the x86/* changes in a separate patch in preparation
> of this one that packs all structs in <acpi.h>

Yes, please. Also please add an assert(facs) or 'if (!facs) exit()' to
preserve that NULL check (although it doesn't look like the original code
cared about the return being NULL anyway...)

Thanks,
drew

> 
> Thanks,
> 
> Nikos
> 
> > > -}
> > > -
> > >   #define RTC_SECONDS_ALARM       1
> > >   #define RTC_MINUTES_ALARM       3
> > >   #define RTC_HOURS_ALARM         5
> > > @@ -40,12 +31,13 @@ extern char resume_start, resume_end;
> > >   int main(int argc, char **argv)
> > >   {
> > >   	struct fadt_descriptor_rev1 *fadt = find_acpi_table_addr(FACP_SIGNATURE);
> > > -	volatile u32 *resume_vector_ptr = find_resume_vector_addr();
> > > +	struct facs_descriptor_rev1 *facs = find_acpi_table_addr(FACS_SIGNATURE);
> > >   	char *addr, *resume_vec = (void*)0x1000;
> > > -	*resume_vector_ptr = (u32)(ulong)resume_vec;
> > > +	facs->firmware_waking_vector = (u32)(ulong)resume_vec;
> > > -	printf("resume vector addr is %p\n", resume_vector_ptr);
> > > +	printf("FACS is at %p\n", facs);
> > > +	printf("resume vector addr is %p\n", &facs->firmware_waking_vector);
> > >   	for (addr = &resume_start; addr < &resume_end; addr++)
> > >   		*resume_vec++ = *addr;
> > >   	printf("copy resume code from %p\n", &resume_start);
> > > -- 
> > > 2.25.1
> > > 
> > 
> > Thanks,
> > drew
> > 
> 

Re: [kvm-unit-tests PATCH v2 00/23] EFI and ACPI support for arm64

[Alexandru Elisei]

Hi,

On Wed, May 18, 2022 at 10:00:46AM +0100, Nikos Nikoleris wrote:
> Hi Alex,
> 
> On 13/05/2022 15:09, Alexandru Elisei wrote:
> > Hi,
> > 
> > On Fri, May 06, 2022 at 09:55:42PM +0100, Nikos Nikoleris wrote:
> > > Hello,
> > > 
> > > This patch series adds initial support for building arm64 tests as EFI
> > > tests and running them under QEMU. Much like x86_64 we import external
> > 
> > I would like to see kvm-unit-tests run as an EFI app on real hardware.
> > QEMU's implementation of the architecture might be different than real
> > hardware, where considerations like power consumption, performance or die
> > area dictate how a particular feature is implementated. For example, I
> > don't know how out-of-order TCG is, and bugs like missing barriers are more
> > easily detected on highly out-of-order implementations.
> > 
> > On the other hand, I'm not opposed to this series if the purpose is just to
> > add the skeleton code needed to boot under EFI and hardware support comes
> > later.
> 
> I fully agree with you. This series is just the first step in getting EFI
> apps to run on real hardware. Hopefully, it's in the right direction and
> helps further development and testing for some of the functionality that we
> will need anyway.
> 
> If I understand correctly, we can run EFI tests with KVM too, at least for
> as long as we're fine with starting from EL1 so we should be exposed to some
> of the timing issues already.
> 
> > 
> > > dependencies from gnu-efi and adopt them to work with types and other
> > > assumptions from kvm-unit-tests. In addition, this series adds support
> > > for discovering parts of the machine using ACPI.
> > > 
> > > The first set of patches moves the existing ACPI code to the common
> > > lib path. Then, it extends definitions and functions to allow for more
> > > robust discovery of ACPI tables. In arm64, we add support for setting
> > > up the PSCI conduit, discovering the UART, timers and cpus via
> > > ACPI. The code retains existing behavior and gives priority to
> > > discovery through DT when one has been provided.
> > > 
> > > In the second set of patches, we add support for getting the command
> > > line from the EFI shell. This is a requirement for many of the
> > > existing arm64 tests.
> > > 
> > > In the third set of patches, we import code from gnu-efi, make minor
> > > changes and add an alternative setup sequence from arm64 systems that
> > > boot through EFI. Finally, we add support in the build system and a
> > > run script which is used to run an EFI app.
> > > 
> > > After this set of patches one can build arm64 EFI tests:
> > > 
> > > $> ./configure --enable-efi
> > > $> make
> > > 
> > > And use the run script to run an EFI tests:
> > > 
> > > $> ./arm/efi/run ./arm/selftest.efi -smp 2 -m 256 -append "setup smp=2 mem=256"
> > > 
> > > Or all tests:
> > > 
> > > $> ./run_tests.sh
> > > 
> > > There are a few items that this series does not address but they would
> > > be useful to have:
> > > 
> > > * Support for booting the system from EL2. Currently, we assume that a
> > > tests starts running at EL1. This the case when we run with EFI, it's
> > > not always the case in hardware.
> > 
> > I would add to that the fact that the vmalloc area is between 3 and 4 GB.
> > What happens if real hardware has main memory there? For this point at
> > least, for testing you can use my kvmtool series that allows the user to
> > set the memory base address [1].
> 
> I am not sure, I fully understand the problem with this. In 13/26, we use

setup_efi() calls setup_vm() -> .. -> setup_mmu(). In setup_mmu(), the
vmalloc area is initialized as starting from 4GB, and
malloc/calloc/memalign will allocate memory down from it.

This means that memory allocations made by a test have the potential to
overwrite the test image if enough memory is allocated:

vree_top - size <= image_base_address && RAM size >= size

The second condition is needed because otherwise the assert in
virt_to_phys() -> __virt_to_phys() triggers when the allocator tries to
allocate more memory than the total RAM size.

This also can happen with mainline. Launch a test with at least 3GB of
memory on qemu/2GB of memory on kvmtool (so RAM top is at least 4GB) and
allocate 3GB on qemu/2GB on kvmtool with malloc and you will see the VM
hang because vm_memalign() will write metadata to the text section of the
test image.

I believe this is more likely to happen on baremetal as real machines have
a lot more memory than tests, which use the default memory size for qemu,
128MB (except selftest-setup, which configures 256MB).

One idea would be to reserve a region for vmalloc that doesn't overlap with
any of the detected memory regions. That would involve adding a limit on
the addresses that allocations can return. It would be nice to take into
account the memory layout for kvmtool (kvm-unit-tests adds IO regions only
for the qemu memory layout, not for kvmtool). There might be other/better
ways to solve this.

Just FYI, kvmtool can also run UEFI in a VM, so you can use that too for
testing.

Thanks,
Alex

> the efi memory map to avoid making many assumption about the physical memory
> map, but I will have a look at the functionality we implement in vmalloc.c
> to understand this. On a high level, I agree with you. The goal should to
> have tests discover as much about the underlying system as possible.
> 
> FWIW, this TODO list is missing many points that I've already discovered and
> I am sure there will be a few more on top of that.
> 
> > 
> > I think there might be other assumptions that kvm-unit-tests makes which
> > are not true when running on baremetal. That's why I would prefer that EFI
> > support is also tested on baremetal.
> > 
> > [1] https://lore.kernel.org/all/20220428155602.29445-1-alexandru.elisei@arm.com/
> > 
> 
> I agree, hopefully testing with KVM and TCG is helpful but I wouldn't expect
> it to be sufficient.
> 
> Thanks,
> 
> Nikos
> 
> > Thanks,
> > Alex
> > 
> > > 
> > > * Support for reading environment variables and populating __envp.
> > > 
> > > * Support for discovering the chr-testdev through ACPI.
> > > 
> > > PS: Apologies for the mess with v1. Due to a mistake in my git
> > > send-email configuration some patches didn't make it to the list and
> > > the right recipients.
> > > 
> > > Thanks,
> > > 
> > > Nikos
> > > 
> > > Andrew Jones (3):
> > >    arm/arm64: mmu_disable: Clean and invalidate before disabling
> > >    arm/arm64: Rename etext to _etext
> > >    arm64: Add a new type of memory type flag MR_F_RESERVED
> > > 
> > > Nikos Nikoleris (20):
> > >    lib: Move acpi header and implementation to lib
> > >    lib: Ensure all struct definition for ACPI tables are packed
> > >    lib: Add support for the XSDT ACPI table
> > >    lib: Extend the definition of the ACPI table FADT
> > >    arm/arm64: Add support for setting up the PSCI conduit through ACPI
> > >    arm/arm64: Add support for discovering the UART through ACPI
> > >    arm/arm64: Add support for timer initialization through ACPI
> > >    arm/arm64: Add support for cpu initialization through ACPI
> > >    lib/printf: Support for precision modifier in printing strings
> > >    lib/printf: Add support for printing wide strings
> > >    lib/efi: Add support for getting the cmdline
> > >    lib: Avoid ms_abi for calls related to EFI on arm64
> > >    arm/arm64: Add a setup sequence for systems that boot through EFI
> > >    arm64: Copy code from GNU-EFI
> > >    arm64: Change GNU-EFI imported file to use defined types
> > >    arm64: Use code from the gnu-efi when booting with EFI
> > >    lib: Avoid external dependency in libelf
> > >    x86: Move x86_64-specific EFI CFLAGS to x86_64 Makefile
> > >    arm64: Add support for efi in Makefile
> > >    arm64: Add an efi/run script
> > > 
> > >   scripts/runtime.bash        |  14 +-
> > >   arm/efi/run                 |  61 +++++++++
> > >   arm/run                     |   8 +-
> > >   configure                   |  15 ++-
> > >   Makefile                    |   4 -
> > >   arm/Makefile.arm            |   6 +
> > >   arm/Makefile.arm64          |  18 ++-
> > >   arm/Makefile.common         |  48 +++++--
> > >   x86/Makefile.common         |   2 +-
> > >   x86/Makefile.x86_64         |   4 +
> > >   lib/linux/efi.h             |  44 ++++++
> > >   lib/arm/asm/setup.h         |   3 +
> > >   lib/arm/asm/timer.h         |   2 +
> > >   lib/x86/asm/setup.h         |   2 +-
> > >   lib/acpi.h                  | 260 ++++++++++++++++++++++++++++++++++++
> > >   lib/stdlib.h                |   1 +
> > >   lib/x86/acpi.h              | 112 ----------------
> > >   lib/acpi.c                  | 124 +++++++++++++++++
> > >   lib/arm/io.c                |  21 ++-
> > >   lib/arm/mmu.c               |   4 +
> > >   lib/arm/psci.c              |  25 +++-
> > >   lib/arm/setup.c             | 247 +++++++++++++++++++++++++++-------
> > >   lib/arm/timer.c             |  73 ++++++++++
> > >   lib/devicetree.c            |   2 +-
> > >   lib/efi.c                   | 123 +++++++++++++++++
> > >   lib/printf.c                | 183 +++++++++++++++++++++++--
> > >   lib/string.c                |   2 +-
> > >   lib/x86/acpi.c              |  82 ------------
> > >   arm/efi/elf_aarch64_efi.lds |  63 +++++++++
> > >   arm/flat.lds                |   2 +-
> > >   arm/cstart.S                |  29 +++-
> > >   arm/cstart64.S              |  28 +++-
> > >   arm/efi/crt0-efi-aarch64.S  | 143 ++++++++++++++++++++
> > >   arm/dummy.c                 |   4 +
> > >   arm/efi/reloc_aarch64.c     |  93 +++++++++++++
> > >   x86/s3.c                    |  20 +--
> > >   x86/vmexit.c                |   4 +-
> > >   37 files changed, 1556 insertions(+), 320 deletions(-)
> > >   create mode 100755 arm/efi/run
> > >   create mode 100644 lib/acpi.h
> > >   delete mode 100644 lib/x86/acpi.h
> > >   create mode 100644 lib/acpi.c
> > >   create mode 100644 lib/arm/timer.c
> > >   delete mode 100644 lib/x86/acpi.c
> > >   create mode 100644 arm/efi/elf_aarch64_efi.lds
> > >   create mode 100644 arm/efi/crt0-efi-aarch64.S
> > >   create mode 100644 arm/dummy.c
> > >   create mode 100644 arm/efi/reloc_aarch64.c
> > > 
> > > -- 
> > > 2.25.1
> > > 

Re: [kvm-unit-tests PATCH v2 14/23] lib: Avoid ms_abi for calls related to EFI on arm64

[Andrew Jones]

On Fri, May 06, 2022 at 09:55:56PM +0100, Nikos Nikoleris wrote:
> x86_64 requires that EFI calls use the ms_abi calling convention. For
> arm64 this is unnecessary.
> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  lib/linux/efi.h | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/lib/linux/efi.h b/lib/linux/efi.h
> index e3aba1d..594eaca 100644
> --- a/lib/linux/efi.h
> +++ b/lib/linux/efi.h
> @@ -33,7 +33,11 @@ typedef u16 efi_char16_t;		/* UNICODE character */
>  typedef u64 efi_physical_addr_t;
>  typedef void *efi_handle_t;
>  
> +#ifdef __x86_64__
>  #define __efiapi __attribute__((ms_abi))
> +#else
> +#define __efiapi
> +#endif
>  
>  /*
>   * The UEFI spec and EDK2 reference implementation both define EFI_GUID as
> -- 
> 2.25.1
>

Reviewed-by: Andrew Jones <drjones@redhat.com>

Re: [kvm-unit-tests PATCH v2 03/23] lib: Add support for the XSDT ACPI table

[Ricardo Koller]

Hi Nikos,

On Fri, May 06, 2022 at 09:55:45PM +0100, Nikos Nikoleris wrote:
> XSDT provides pointers to other ACPI tables much like RSDT. However,
> contrary to RSDT that provides 32-bit addresses, XSDT provides 64-bit
> pointers. ACPI requires that if XSDT is valid then it takes precedence
> over RSDT.
> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  lib/acpi.h |   6 ++++
>  lib/acpi.c | 103 ++++++++++++++++++++++++++++++++---------------------
>  2 files changed, 68 insertions(+), 41 deletions(-)
> 
> diff --git a/lib/acpi.h b/lib/acpi.h
> index 42a2c16..d80b983 100644
> --- a/lib/acpi.h
> +++ b/lib/acpi.h
> @@ -13,6 +13,7 @@
>  
>  #define RSDP_SIGNATURE ACPI_SIGNATURE('R','S','D','P')
>  #define RSDT_SIGNATURE ACPI_SIGNATURE('R','S','D','T')
> +#define XSDT_SIGNATURE ACPI_SIGNATURE('X','S','D','T')
>  #define FACP_SIGNATURE ACPI_SIGNATURE('F','A','C','P')
>  #define FACS_SIGNATURE ACPI_SIGNATURE('F','A','C','S')
>  
> @@ -56,6 +57,11 @@ struct rsdt_descriptor_rev1 {
>      u32 table_offset_entry[0];
>  } __attribute__ ((packed));
>  
> +struct acpi_table_xsdt {
> +    ACPI_TABLE_HEADER_DEF
> +    u64 table_offset_entry[1];

nit: This should be "[0]" to match the usage above (in rsdt).

I was about to suggest using an unspecified size "[]", but after reading
what the C standard says about it (below), now I'm not sure. was the
"[1]" needed for something that I'm missing?

	106) The length is unspecified to allow for the fact that
	implementations may give array members different
	alignments according to their lengths.


> +} __attribute__ ((packed));
> +
>  struct fadt_descriptor_rev1
>  {
>      ACPI_TABLE_HEADER_DEF     /* ACPI common table header */
> diff --git a/lib/acpi.c b/lib/acpi.c
> index de275ca..9b8700c 100644
> --- a/lib/acpi.c
> +++ b/lib/acpi.c
> @@ -38,45 +38,66 @@ static struct rsdp_descriptor *get_rsdp(void)
>  
>  void* find_acpi_table_addr(u32 sig)

nit: This one could also be fixed as well: "void *".

>  {
> -    struct rsdp_descriptor *rsdp;
> -    struct rsdt_descriptor_rev1 *rsdt;
> -    void *end;
> -    int i;
> -
> -    /* FACS is special... */
> -    if (sig == FACS_SIGNATURE) {
> -        struct fadt_descriptor_rev1 *fadt;
> -        fadt = find_acpi_table_addr(FACP_SIGNATURE);
> -        if (!fadt) {
> -            return NULL;
> -        }
> -        return (void*)(ulong)fadt->firmware_ctrl;
> -    }
> -
> -    rsdp = get_rsdp();
> -    if (rsdp == NULL) {
> -        printf("Can't find RSDP\n");
> -        return 0;
> -    }
> -
> -    if (sig == RSDP_SIGNATURE) {
> -        return rsdp;
> -    }
> -
> -    rsdt = (void*)(ulong)rsdp->rsdt_physical_address;
> -    if (!rsdt || rsdt->signature != RSDT_SIGNATURE)
> -        return 0;
> -
> -    if (sig == RSDT_SIGNATURE) {
> -        return rsdt;
> -    }
> -
> -    end = (void*)rsdt + rsdt->length;
> -    for (i=0; (void*)&rsdt->table_offset_entry[i] < end; i++) {
> -        struct acpi_table *t = (void*)(ulong)rsdt->table_offset_entry[i];
> -        if (t && t->signature == sig) {
> -            return t;
> -        }
> -    }
> -   return NULL;
> +	struct rsdp_descriptor *rsdp;
> +	struct rsdt_descriptor_rev1 *rsdt;
> +	struct acpi_table_xsdt *xsdt = NULL;
> +	void *end;
> +	int i;
> +
> +	/* FACS is special... */
> +	if (sig == FACS_SIGNATURE) {
> +		struct fadt_descriptor_rev1 *fadt;
> +
> +		fadt = find_acpi_table_addr(FACP_SIGNATURE);
> +		if (!fadt)
> +			return NULL;
> +
> +		return (void*)(ulong)fadt->firmware_ctrl;
> +	}
> +
> +	rsdp = get_rsdp();
> +	if (rsdp == NULL) {
> +		printf("Can't find RSDP\n");
> +		return 0;
> +	}
> +
> +	if (sig == RSDP_SIGNATURE)
> +		return rsdp;
> +
> +	rsdt = (void *)(ulong)rsdp->rsdt_physical_address;
> +	if (!rsdt || rsdt->signature != RSDT_SIGNATURE)
> +		rsdt = NULL;
> +
> +	if (sig == RSDT_SIGNATURE)
> +		return rsdt;
> +
> +	if (rsdp->revision > 1)
> +		xsdt = (void *)(ulong)rsdp->xsdt_physical_address;
> +	if (!xsdt || xsdt->signature != XSDT_SIGNATURE)
> +		xsdt = NULL;
> +

To simplify this function a bit, finding the xsdt could be moved to some
kind of init function.

> +	if (sig == XSDT_SIGNATURE)
> +		return xsdt;
> +
> +	// APCI requires that we first try to use XSDT if it's valid,
> +	//  we use to find other tables, otherwise we use RSDT.
> +	if (xsdt) {
> +		end = (void *)(ulong)xsdt + xsdt->length;
> +		for (i = 0; (void *)&xsdt->table_offset_entry[i] < end; i++) {
> +			struct acpi_table *t =
> +				(void *)xsdt->table_offset_entry[i];
> +			if (t && t->signature == sig)
> +				return t;
> +		}
> +	} else if (rsdt) {
> +		end = (void *)rsdt + rsdt->length;
> +		for (i = 0; (void *)&rsdt->table_offset_entry[i] < end; i++) {
> +			struct acpi_table *t =
> +				(void *)(ulong)rsdt->table_offset_entry[i];
> +			if (t && t->signature == sig)
> +				return t;
> +		}
> +	}

The two for-loops could be moved into some common function, or maybe a
macro to deal with the fact that it deals with two different structures
(the rsdt and the xsdt). This is my attempt at making it a function:

void *scan_system_descriptor_table(void *dt)
{
	int i;
	void *end;
	/* XXX: Not sure if this is the nicest thing to do, but the rsdt
	 * and the xsdt have "length" and "table_offset_entry" at the
	 * same offsets. */
	struct acpi_table_xsdt *xsdt = dt;

	end = (void *)(ulong)xsdt + xsdt->length;
	for (i = 0; &xsdt->table_offset_entry[i] < end; i++) {
		struct acpi_table *t = (void *)xsdt->table_offset_entry[i];
		if (t && t->signature == sig)
			return t;
}

Thanks,
Ricardo

> +
> +	return NULL;
>  }
> -- 
> 2.25.1
> 

Re: [kvm-unit-tests PATCH v2 04/23] lib: Extend the definition of the ACPI table FADT

[Ricardo Koller]

On Fri, May 06, 2022 at 09:55:46PM +0100, Nikos Nikoleris wrote:
> This change add more fields in the APCI table FADT to allow for the
> discovery of the PSCI conduit in arm64 systems. The definition for
> FADT is similar to the one in include/acpi/actbl.h in Linux.
> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  lib/acpi.h   | 35 ++++++++++++++++++++++++++++++-----
>  lib/acpi.c   |  2 +-
>  x86/s3.c     |  2 +-
>  x86/vmexit.c |  2 +-
>  4 files changed, 33 insertions(+), 8 deletions(-)
> 
> diff --git a/lib/acpi.h b/lib/acpi.h
> index d80b983..9f27eb1 100644
> --- a/lib/acpi.h
> +++ b/lib/acpi.h
> @@ -62,7 +62,15 @@ struct acpi_table_xsdt {
>      u64 table_offset_entry[1];
>  } __attribute__ ((packed));
>  
> -struct fadt_descriptor_rev1
> +struct acpi_generic_address {
> +    u8 space_id;            /* Address space where struct or register exists */
> +    u8 bit_width;           /* Size in bits of given register */
> +    u8 bit_offset;          /* Bit offset within the register */
> +    u8 access_width;        /* Minimum Access size (ACPI 3.0) */
> +    u64 address;            /* 64-bit address of struct or register */
> +} __attribute__ ((packed));
> +
> +struct acpi_table_fadt
>  {
>      ACPI_TABLE_HEADER_DEF     /* ACPI common table header */
>      u32 firmware_ctrl;          /* Physical address of FACS */
> @@ -100,10 +108,27 @@ struct fadt_descriptor_rev1
>      u8  day_alrm;               /* Index to day-of-month alarm in RTC CMOS RAM */
>      u8  mon_alrm;               /* Index to month-of-year alarm in RTC CMOS RAM */
>      u8  century;                /* Index to century in RTC CMOS RAM */
> -    u8  reserved4;              /* Reserved */
> -    u8  reserved4a;             /* Reserved */
> -    u8  reserved4b;             /* Reserved */
> -};
> +    u16 boot_flags;             /* IA-PC Boot Architecture Flags (see below for individual flags) */
> +    u8 reserved;                /* Reserved, must be zero */
> +    u32 flags;                  /* Miscellaneous flag bits (see below for individual flags) */
> +    struct acpi_generic_address reset_register;     /* 64-bit address of the Reset register */
> +    u8 reset_value;             /* Value to write to the reset_register port to reset the system */
> +    u16 arm_boot_flags;         /* ARM-Specific Boot Flags (see below for individual flags) (ACPI 5.1) */
> +    u8 minor_revision;          /* FADT Minor Revision (ACPI 5.1) */
> +    u64 Xfacs;                  /* 64-bit physical address of FACS */
> +    u64 Xdsdt;                  /* 64-bit physical address of DSDT */
> +    struct acpi_generic_address xpm1a_event_block;  /* 64-bit Extended Power Mgt 1a Event Reg Blk address */
> +    struct acpi_generic_address xpm1b_event_block;  /* 64-bit Extended Power Mgt 1b Event Reg Blk address */
> +    struct acpi_generic_address xpm1a_control_block;        /* 64-bit Extended Power Mgt 1a Control Reg Blk address */
> +    struct acpi_generic_address xpm1b_control_block;        /* 64-bit Extended Power Mgt 1b Control Reg Blk address */
> +    struct acpi_generic_address xpm2_control_block; /* 64-bit Extended Power Mgt 2 Control Reg Blk address */
> +    struct acpi_generic_address xpm_timer_block;    /* 64-bit Extended Power Mgt Timer Ctrl Reg Blk address */
> +    struct acpi_generic_address xgpe0_block;        /* 64-bit Extended General Purpose Event 0 Reg Blk address */
> +    struct acpi_generic_address xgpe1_block;        /* 64-bit Extended General Purpose Event 1 Reg Blk address */
> +    struct acpi_generic_address sleep_control;      /* 64-bit Sleep Control register (ACPI 5.0) */
> +    struct acpi_generic_address sleep_status;       /* 64-bit Sleep Status register (ACPI 5.0) */
> +    u64 hypervisor_id;      /* Hypervisor Vendor ID (ACPI 6.0) */
> +}  __attribute__ ((packed));
>  
>  struct facs_descriptor_rev1
>  {
> diff --git a/lib/acpi.c b/lib/acpi.c
> index 9b8700c..e8440ae 100644
> --- a/lib/acpi.c
> +++ b/lib/acpi.c
> @@ -46,7 +46,7 @@ void* find_acpi_table_addr(u32 sig)
>  
>  	/* FACS is special... */
>  	if (sig == FACS_SIGNATURE) {
> -		struct fadt_descriptor_rev1 *fadt;
> +		struct acpi_table_fadt *fadt;
>  
>  		fadt = find_acpi_table_addr(FACP_SIGNATURE);
>  		if (!fadt)
> diff --git a/x86/s3.c b/x86/s3.c
> index 89d69fc..16e79f8 100644
> --- a/x86/s3.c
> +++ b/x86/s3.c
> @@ -30,7 +30,7 @@ extern char resume_start, resume_end;
>  
>  int main(int argc, char **argv)
>  {
> -	struct fadt_descriptor_rev1 *fadt = find_acpi_table_addr(FACP_SIGNATURE);
> +	struct acpi_table_fadt *fadt = find_acpi_table_addr(FACP_SIGNATURE);
>  	struct facs_descriptor_rev1 *facs = find_acpi_table_addr(FACS_SIGNATURE);
>  	char *addr, *resume_vec = (void*)0x1000;
>  
> diff --git a/x86/vmexit.c b/x86/vmexit.c
> index 2bac049..fcc0760 100644
> --- a/x86/vmexit.c
> +++ b/x86/vmexit.c
> @@ -206,7 +206,7 @@ int pm_tmr_blk;
>  static void inl_pmtimer(void)
>  {
>      if (!pm_tmr_blk) {
> -	struct fadt_descriptor_rev1 *fadt;
> +	struct acpi_table_fadt *fadt;
>  
>  	fadt = find_acpi_table_addr(FACP_SIGNATURE);
>  	pm_tmr_blk = fadt->pm_tmr_blk;
> -- 
> 2.25.1
>

Reviewed-by: Ricardo Koller <ricarkol@google.com>

Re: [kvm-unit-tests PATCH v2 03/23] lib: Add support for the XSDT ACPI table

[Alexandru Elisei]

Hi,

On Fri, Jun 17, 2022 at 05:38:01PM -0700, Ricardo Koller wrote:
> Hi Nikos,
> 
> On Fri, May 06, 2022 at 09:55:45PM +0100, Nikos Nikoleris wrote:
> > XSDT provides pointers to other ACPI tables much like RSDT. However,
> > contrary to RSDT that provides 32-bit addresses, XSDT provides 64-bit
> > pointers. ACPI requires that if XSDT is valid then it takes precedence
> > over RSDT.
> > 
> > Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> > ---
> >  lib/acpi.h |   6 ++++
> >  lib/acpi.c | 103 ++++++++++++++++++++++++++++++++---------------------
> >  2 files changed, 68 insertions(+), 41 deletions(-)
> > 
> > diff --git a/lib/acpi.h b/lib/acpi.h
> > index 42a2c16..d80b983 100644
> > --- a/lib/acpi.h
> > +++ b/lib/acpi.h
> > @@ -13,6 +13,7 @@
> >  
> >  #define RSDP_SIGNATURE ACPI_SIGNATURE('R','S','D','P')
> >  #define RSDT_SIGNATURE ACPI_SIGNATURE('R','S','D','T')
> > +#define XSDT_SIGNATURE ACPI_SIGNATURE('X','S','D','T')
> >  #define FACP_SIGNATURE ACPI_SIGNATURE('F','A','C','P')
> >  #define FACS_SIGNATURE ACPI_SIGNATURE('F','A','C','S')
> >  
> > @@ -56,6 +57,11 @@ struct rsdt_descriptor_rev1 {
> >      u32 table_offset_entry[0];
> >  } __attribute__ ((packed));
> >  
> > +struct acpi_table_xsdt {
> > +    ACPI_TABLE_HEADER_DEF
> > +    u64 table_offset_entry[1];
> 
> nit: This should be "[0]" to match the usage above (in rsdt).
> 
> I was about to suggest using an unspecified size "[]", but after reading
> what the C standard says about it (below), now I'm not sure. was the
> "[1]" needed for something that I'm missing?
> 
> 	106) The length is unspecified to allow for the fact that
> 	implementations may give array members different
> 	alignments according to their lengths.

GCC prefers "flexible array members" (array[]) [1]. Linux has deprecated
the use of zero-length arrays [2]. The kernel docs do make a pretty good
case for flexible array members.

[1] https://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html
[2] https://elixir.bootlin.com/linux/v5.18/source/Documentation/process/deprecated.rst#L234

Thanks,
Alex

Re: [kvm-unit-tests PATCH v2 03/23] lib: Add support for the XSDT ACPI table

[Nikos Nikoleris]

Hi Alex, Ricardo,

Thank you both for the reviews!

On 20/06/2022 09:53, Alexandru Elisei wrote:
> Hi,
> 
> On Fri, Jun 17, 2022 at 05:38:01PM -0700, Ricardo Koller wrote:
>> Hi Nikos,
>>
>> On Fri, May 06, 2022 at 09:55:45PM +0100, Nikos Nikoleris wrote:
>>> XSDT provides pointers to other ACPI tables much like RSDT. However,
>>> contrary to RSDT that provides 32-bit addresses, XSDT provides 64-bit
>>> pointers. ACPI requires that if XSDT is valid then it takes precedence
>>> over RSDT.
>>>
>>> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
>>> ---
>>>   lib/acpi.h |   6 ++++
>>>   lib/acpi.c | 103 ++++++++++++++++++++++++++++++++---------------------
>>>   2 files changed, 68 insertions(+), 41 deletions(-)
>>>
>>> diff --git a/lib/acpi.h b/lib/acpi.h
>>> index 42a2c16..d80b983 100644
>>> --- a/lib/acpi.h
>>> +++ b/lib/acpi.h
>>> @@ -13,6 +13,7 @@
>>>   
>>>   #define RSDP_SIGNATURE ACPI_SIGNATURE('R','S','D','P')
>>>   #define RSDT_SIGNATURE ACPI_SIGNATURE('R','S','D','T')
>>> +#define XSDT_SIGNATURE ACPI_SIGNATURE('X','S','D','T')
>>>   #define FACP_SIGNATURE ACPI_SIGNATURE('F','A','C','P')
>>>   #define FACS_SIGNATURE ACPI_SIGNATURE('F','A','C','S')
>>>   
>>> @@ -56,6 +57,11 @@ struct rsdt_descriptor_rev1 {
>>>       u32 table_offset_entry[0];
>>>   } __attribute__ ((packed));
>>>   
>>> +struct acpi_table_xsdt {
>>> +    ACPI_TABLE_HEADER_DEF
>>> +    u64 table_offset_entry[1];
>>
>> nit: This should be "[0]" to match the usage above (in rsdt).
>>
>> I was about to suggest using an unspecified size "[]", but after reading
>> what the C standard says about it (below), now I'm not sure. was the
>> "[1]" needed for something that I'm missing?
>>
>> 	106) The length is unspecified to allow for the fact that
>> 	implementations may give array members different
>> 	alignments according to their lengths.
> 
> GCC prefers "flexible array members" (array[]) [1]. Linux has deprecated
> the use of zero-length arrays [2]. The kernel docs do make a pretty good
> case for flexible array members.
> 
> [1] https://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html
> [2] https://elixir.bootlin.com/linux/v5.18/source/Documentation/process/deprecated.rst#L234
> 

Happy to change this, I don't have a strong preference. To be consistent 
with RSDT we would have to declare:

u64 table_offset_entry[0];

But it might be better to change RSDT as well. Linux kernel declares:

u64 table_offset_entry[1];

but it seems, we would rather have:

u64 table_offset_entry[];

For alignment, we shouldn't be relying on the length specifier, all 
structs in <acpi.h> should be packed.

Thanks,

Nikos

Re: [kvm-unit-tests PATCH v2 03/23] lib: Add support for the XSDT ACPI table

[Nikos Nikoleris]

Hi Ricardo,

Thanks for the review!

On 18/06/2022 01:38, Ricardo Koller wrote:
> Hi Nikos,
> 
> On Fri, May 06, 2022 at 09:55:45PM +0100, Nikos Nikoleris wrote:
>> XSDT provides pointers to other ACPI tables much like RSDT. However,
>> contrary to RSDT that provides 32-bit addresses, XSDT provides 64-bit
>> pointers. ACPI requires that if XSDT is valid then it takes precedence
>> over RSDT.
>>
>> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
>> ---
>>   lib/acpi.h |   6 ++++
>>   lib/acpi.c | 103 ++++++++++++++++++++++++++++++++---------------------
>>   2 files changed, 68 insertions(+), 41 deletions(-)
>>
>> diff --git a/lib/acpi.h b/lib/acpi.h
>> index 42a2c16..d80b983 100644
>> --- a/lib/acpi.h
>> +++ b/lib/acpi.h
>> @@ -13,6 +13,7 @@
>>   
>>   #define RSDP_SIGNATURE ACPI_SIGNATURE('R','S','D','P')
>>   #define RSDT_SIGNATURE ACPI_SIGNATURE('R','S','D','T')
>> +#define XSDT_SIGNATURE ACPI_SIGNATURE('X','S','D','T')
>>   #define FACP_SIGNATURE ACPI_SIGNATURE('F','A','C','P')
>>   #define FACS_SIGNATURE ACPI_SIGNATURE('F','A','C','S')
>>   
>> @@ -56,6 +57,11 @@ struct rsdt_descriptor_rev1 {
>>       u32 table_offset_entry[0];
>>   } __attribute__ ((packed));
>>   
>> +struct acpi_table_xsdt {
>> +    ACPI_TABLE_HEADER_DEF
>> +    u64 table_offset_entry[1];
> 
> nit: This should be "[0]" to match the usage above (in rsdt).
> 
> I was about to suggest using an unspecified size "[]", but after reading
> what the C standard says about it (below), now I'm not sure. was the
> "[1]" needed for something that I'm missing?
> 
> 	106) The length is unspecified to allow for the fact that
> 	implementations may give array members different
> 	alignments according to their lengths.
> 
> 
>> +} __attribute__ ((packed));
>> +
>>   struct fadt_descriptor_rev1
>>   {
>>       ACPI_TABLE_HEADER_DEF     /* ACPI common table header */
>> diff --git a/lib/acpi.c b/lib/acpi.c
>> index de275ca..9b8700c 100644
>> --- a/lib/acpi.c
>> +++ b/lib/acpi.c
>> @@ -38,45 +38,66 @@ static struct rsdp_descriptor *get_rsdp(void)
>>   
>>   void* find_acpi_table_addr(u32 sig)
> 
> nit: This one could also be fixed as well: "void *".
> 
>>   {
>> -    struct rsdp_descriptor *rsdp;
>> -    struct rsdt_descriptor_rev1 *rsdt;
>> -    void *end;
>> -    int i;
>> -
>> -    /* FACS is special... */
>> -    if (sig == FACS_SIGNATURE) {
>> -        struct fadt_descriptor_rev1 *fadt;
>> -        fadt = find_acpi_table_addr(FACP_SIGNATURE);
>> -        if (!fadt) {
>> -            return NULL;
>> -        }
>> -        return (void*)(ulong)fadt->firmware_ctrl;
>> -    }
>> -
>> -    rsdp = get_rsdp();
>> -    if (rsdp == NULL) {
>> -        printf("Can't find RSDP\n");
>> -        return 0;
>> -    }
>> -
>> -    if (sig == RSDP_SIGNATURE) {
>> -        return rsdp;
>> -    }
>> -
>> -    rsdt = (void*)(ulong)rsdp->rsdt_physical_address;
>> -    if (!rsdt || rsdt->signature != RSDT_SIGNATURE)
>> -        return 0;
>> -
>> -    if (sig == RSDT_SIGNATURE) {
>> -        return rsdt;
>> -    }
>> -
>> -    end = (void*)rsdt + rsdt->length;
>> -    for (i=0; (void*)&rsdt->table_offset_entry[i] < end; i++) {
>> -        struct acpi_table *t = (void*)(ulong)rsdt->table_offset_entry[i];
>> -        if (t && t->signature == sig) {
>> -            return t;
>> -        }
>> -    }
>> -   return NULL;
>> +	struct rsdp_descriptor *rsdp;
>> +	struct rsdt_descriptor_rev1 *rsdt;
>> +	struct acpi_table_xsdt *xsdt = NULL;
>> +	void *end;
>> +	int i;
>> +
>> +	/* FACS is special... */
>> +	if (sig == FACS_SIGNATURE) {
>> +		struct fadt_descriptor_rev1 *fadt;
>> +
>> +		fadt = find_acpi_table_addr(FACP_SIGNATURE);
>> +		if (!fadt)
>> +			return NULL;
>> +
>> +		return (void*)(ulong)fadt->firmware_ctrl;
>> +	}
>> +
>> +	rsdp = get_rsdp();
>> +	if (rsdp == NULL) {
>> +		printf("Can't find RSDP\n");
>> +		return 0;
>> +	}
>> +
>> +	if (sig == RSDP_SIGNATURE)
>> +		return rsdp;
>> +
>> +	rsdt = (void *)(ulong)rsdp->rsdt_physical_address;
>> +	if (!rsdt || rsdt->signature != RSDT_SIGNATURE)
>> +		rsdt = NULL;
>> +
>> +	if (sig == RSDT_SIGNATURE)
>> +		return rsdt;
>> +
>> +	if (rsdp->revision > 1)
>> +		xsdt = (void *)(ulong)rsdp->xsdt_physical_address;
>> +	if (!xsdt || xsdt->signature != XSDT_SIGNATURE)
>> +		xsdt = NULL;
>> +
> 
> To simplify this function a bit, finding the xsdt could be moved to some
> kind of init function.

That's a good point, I can add xsdt and rsdt as globals (like we do for 
efi_rsdp) and then initialise them in set_efi_rsdp().

On the other hand, if we would like to avoid the global variables there 
  is only a handful of time we will be calling  find_acpi_table_addr()

> 
>> +	if (sig == XSDT_SIGNATURE)
>> +		return xsdt;
>> +
>> +	// APCI requires that we first try to use XSDT if it's valid,
>> +	//  we use to find other tables, otherwise we use RSDT.
>> +	if (xsdt) {
>> +		end = (void *)(ulong)xsdt + xsdt->length;
>> +		for (i = 0; (void *)&xsdt->table_offset_entry[i] < end; i++) {
>> +			struct acpi_table *t =
>> +				(void *)xsdt->table_offset_entry[i];
>> +			if (t && t->signature == sig)
>> +				return t;
>> +		}
>> +	} else if (rsdt) {
>> +		end = (void *)rsdt + rsdt->length;
>> +		for (i = 0; (void *)&rsdt->table_offset_entry[i] < end; i++) {
>> +			struct acpi_table *t =
>> +				(void *)(ulong)rsdt->table_offset_entry[i];
>> +			if (t && t->signature == sig)
>> +				return t;
>> +		}
>> +	}
> 
> The two for-loops could be moved into some common function, or maybe a
> macro to deal with the fact that it deals with two different structures
> (the rsdt and the xsdt). This is my attempt at making it a function:
> 
> void *scan_system_descriptor_table(void *dt)
> {
> 	int i;
> 	void *end;
> 	/* XXX: Not sure if this is the nicest thing to do, but the rsdt
> 	 * and the xsdt have "length" and "table_offset_entry" at the
> 	 * same offsets. */
> 	struct acpi_table_xsdt *xsdt = dt;
> 
> 	end = (void *)(ulong)xsdt + xsdt->length;
> 	for (i = 0; &xsdt->table_offset_entry[i] < end; i++) {
> 		struct acpi_table *t = (void *)xsdt->table_offset_entry[i];
> 		if (t && t->signature == sig)
> 			return t;
> }
> 

I remember trying something similar but it got a bit messy because 
table_offset_entry is u32 in rsdt and u64 in xsdt. I'll check again if 
there is a way we can improve this in v3.

Thanks,

Nikos

> Thanks,
> Ricardo
> 
>> +
>> +	return NULL;
>>   }
>> -- 
>> 2.25.1
>>

Re: [kvm-unit-tests PATCH v2 03/23] lib: Add support for the XSDT ACPI table

[Alexandru Elisei]

Hi,

On Mon, Jun 20, 2022 at 12:06:24PM +0100, Nikos Nikoleris wrote:
> Hi Alex, Ricardo,
> 
> Thank you both for the reviews!
> 
> On 20/06/2022 09:53, Alexandru Elisei wrote:
> > Hi,
> > 
> > On Fri, Jun 17, 2022 at 05:38:01PM -0700, Ricardo Koller wrote:
> > > Hi Nikos,
> > > 
> > > On Fri, May 06, 2022 at 09:55:45PM +0100, Nikos Nikoleris wrote:
> > > > XSDT provides pointers to other ACPI tables much like RSDT. However,
> > > > contrary to RSDT that provides 32-bit addresses, XSDT provides 64-bit
> > > > pointers. ACPI requires that if XSDT is valid then it takes precedence
> > > > over RSDT.
> > > > 
> > > > Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> > > > ---
> > > >   lib/acpi.h |   6 ++++
> > > >   lib/acpi.c | 103 ++++++++++++++++++++++++++++++++---------------------
> > > >   2 files changed, 68 insertions(+), 41 deletions(-)
> > > > 
> > > > diff --git a/lib/acpi.h b/lib/acpi.h
> > > > index 42a2c16..d80b983 100644
> > > > --- a/lib/acpi.h
> > > > +++ b/lib/acpi.h
> > > > @@ -13,6 +13,7 @@
> > > >   #define RSDP_SIGNATURE ACPI_SIGNATURE('R','S','D','P')
> > > >   #define RSDT_SIGNATURE ACPI_SIGNATURE('R','S','D','T')
> > > > +#define XSDT_SIGNATURE ACPI_SIGNATURE('X','S','D','T')
> > > >   #define FACP_SIGNATURE ACPI_SIGNATURE('F','A','C','P')
> > > >   #define FACS_SIGNATURE ACPI_SIGNATURE('F','A','C','S')
> > > > @@ -56,6 +57,11 @@ struct rsdt_descriptor_rev1 {
> > > >       u32 table_offset_entry[0];
> > > >   } __attribute__ ((packed));
> > > > +struct acpi_table_xsdt {
> > > > +    ACPI_TABLE_HEADER_DEF
> > > > +    u64 table_offset_entry[1];
> > > 
> > > nit: This should be "[0]" to match the usage above (in rsdt).
> > > 
> > > I was about to suggest using an unspecified size "[]", but after reading
> > > what the C standard says about it (below), now I'm not sure. was the
> > > "[1]" needed for something that I'm missing?
> > > 
> > > 	106) The length is unspecified to allow for the fact that
> > > 	implementations may give array members different
> > > 	alignments according to their lengths.
> > 
> > GCC prefers "flexible array members" (array[]) [1]. Linux has deprecated
> > the use of zero-length arrays [2]. The kernel docs do make a pretty good
> > case for flexible array members.
> > 
> > [1] https://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html
> > [2] https://elixir.bootlin.com/linux/v5.18/source/Documentation/process/deprecated.rst#L234
> > 
> 
> Happy to change this, I don't have a strong preference. To be consistent
> with RSDT we would have to declare:
> 
> u64 table_offset_entry[0];
> 
> But it might be better to change RSDT as well. Linux kernel declares:
> 
> u64 table_offset_entry[1];
> 
> but it seems, we would rather have:
> 
> u64 table_offset_entry[];

Sorry, I didn't think about Linux when I wrote my reply. In my opinion, we
should keep our definitions aligned with Linux for two reasons:

- Makes making changes to the header file and borrowing code from Linux less
  error prone.

- Linux' implementation, even though it might not be ideal, is proven to be
  working.

It might also be that Linux uses table_offset_entry[1] for compatibility
with older compilers, and supporting the same compilers that Linux does
looks like a safe approach for me; I am not aware of an official policy for
kvm-unit-tests regarding compiler versions (might be one, it just that I
don't know about it!).

Thanks,
Alex

Re: [kvm-unit-tests PATCH v2 05/23] arm/arm64: Add support for setting up the PSCI conduit through ACPI

[Ricardo Koller]

On Fri, May 06, 2022 at 09:55:47PM +0100, Nikos Nikoleris wrote:
> In systems with ACPI support and when a DT is not provided, we can use
> the FADT to discover whether PSCI calls need to use smc or hvc
> calls. This change implements this but retains the default behavior;
> we check if a valid DT is provided, if not, we try to setup the PSCI
> conduit using ACPI.
> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  arm/Makefile.common |  1 +
>  lib/acpi.h          |  5 +++++
>  lib/arm/psci.c      | 23 ++++++++++++++++++++++-
>  lib/devicetree.c    |  2 +-
>  4 files changed, 29 insertions(+), 2 deletions(-)
> 
> diff --git a/arm/Makefile.common b/arm/Makefile.common
> index 38385e0..8e9b3bb 100644
> --- a/arm/Makefile.common
> +++ b/arm/Makefile.common
> @@ -38,6 +38,7 @@ cflatobjs += lib/alloc_page.o
>  cflatobjs += lib/vmalloc.o
>  cflatobjs += lib/alloc.o
>  cflatobjs += lib/devicetree.o
> +cflatobjs += lib/acpi.o
>  cflatobjs += lib/pci.o
>  cflatobjs += lib/pci-host-generic.o
>  cflatobjs += lib/pci-testdev.o
> diff --git a/lib/acpi.h b/lib/acpi.h
> index 9f27eb1..139ccba 100644
> --- a/lib/acpi.h
> +++ b/lib/acpi.h
> @@ -130,6 +130,11 @@ struct acpi_table_fadt
>      u64 hypervisor_id;      /* Hypervisor Vendor ID (ACPI 6.0) */
>  }  __attribute__ ((packed));
>  
> +/* Masks for FADT ARM Boot Architecture Flags (arm_boot_flags) ACPI 5.1 */
> +
> +#define ACPI_FADT_PSCI_COMPLIANT    (1)         /* 00: [V5+] PSCI 0.2+ is implemented */
> +#define ACPI_FADT_PSCI_USE_HVC      (1<<1)      /* 01: [V5+] HVC must be used instead of SMC as the PSCI conduit */
> +
>  struct facs_descriptor_rev1
>  {
>      u32 signature;           /* ACPI Signature */
> diff --git a/lib/arm/psci.c b/lib/arm/psci.c
> index 9c031a1..0e96d19 100644
> --- a/lib/arm/psci.c
> +++ b/lib/arm/psci.c
> @@ -6,6 +6,7 @@
>   *
>   * This work is licensed under the terms of the GNU LGPL, version 2.
>   */
> +#include <acpi.h>
>  #include <devicetree.h>
>  #include <asm/psci.h>
>  #include <asm/setup.h>
> @@ -56,7 +57,7 @@ void psci_system_off(void)
>  	printf("CPU%d unable to do system off (error = %d)\n", smp_processor_id(), err);
>  }
>  
> -void psci_set_conduit(void)
> +static void psci_set_conduit_fdt(void)
>  {
>  	const void *fdt = dt_fdt();
>  	const struct fdt_property *method;
> @@ -75,3 +76,23 @@ void psci_set_conduit(void)
>  	else
>  		assert_msg(false, "Unknown PSCI conduit: %s", method->data);
>  }
> +
> +static void psci_set_conduit_acpi(void)
> +{
> +	struct acpi_table_fadt *fadt = find_acpi_table_addr(FACP_SIGNATURE);
> +	assert_msg(fadt, "Unable to find ACPI FADT");
> +	assert_msg(fadt->arm_boot_flags & ACPI_FADT_PSCI_COMPLIANT,
> +		   "PSCI is not supported in this platfrom");
> +	if (fadt->arm_boot_flags & ACPI_FADT_PSCI_USE_HVC)
> +		psci_invoke = psci_invoke_hvc;
> +	else
> +		psci_invoke = psci_invoke_smc;
> +}
> +
> +void psci_set_conduit(void)
> +{
> +	if (dt_available())
> +		psci_set_conduit_fdt();
> +	else
> +		psci_set_conduit_acpi();
> +}
> diff --git a/lib/devicetree.c b/lib/devicetree.c
> index 78c1f6f..3ff9d16 100644
> --- a/lib/devicetree.c
> +++ b/lib/devicetree.c
> @@ -16,7 +16,7 @@ const void *dt_fdt(void)
>  
>  bool dt_available(void)
>  {
> -	return fdt_check_header(fdt) == 0;
> +	return fdt && fdt_check_header(fdt) == 0;
>  }
>  
>  int dt_get_nr_cells(int fdtnode, u32 *nr_address_cells, u32 *nr_size_cells)
> -- 
>2.25.1
>

Reviewed-by: Ricardo Koller <ricarkol@google.com>

Re: [kvm-unit-tests PATCH v2 06/23] arm/arm64: Add support for discovering the UART through ACPI

[Ricardo Koller]

On Fri, May 06, 2022 at 09:55:48PM +0100, Nikos Nikoleris wrote:
> In systems with ACPI support and when a DT is not provided, we can use
> the SPCR to discover the serial port address range. This change
> implements this but retains the default behavior; we check if a valid
> DT is provided, if not, we try to discover the UART using ACPI.
> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  lib/acpi.h     | 25 +++++++++++++++++++++++++
>  lib/arm/io.c   | 21 +++++++++++++++++++--
>  lib/arm/psci.c |  4 +++-
>  3 files changed, 47 insertions(+), 3 deletions(-)
> 
> diff --git a/lib/acpi.h b/lib/acpi.h
> index 139ccba..5213299 100644
> --- a/lib/acpi.h
> +++ b/lib/acpi.h
> @@ -16,6 +16,7 @@
>  #define XSDT_SIGNATURE ACPI_SIGNATURE('X','S','D','T')
>  #define FACP_SIGNATURE ACPI_SIGNATURE('F','A','C','P')
>  #define FACS_SIGNATURE ACPI_SIGNATURE('F','A','C','S')
> +#define SPCR_SIGNATURE ACPI_SIGNATURE('S','P','C','R')
>  
>  
>  #define ACPI_SIGNATURE_8BYTE(c1, c2, c3, c4, c5, c6, c7, c8) \
> @@ -147,6 +148,30 @@ struct facs_descriptor_rev1
>      u8  reserved3 [40];         /* Reserved - must be zero */
>  } __attribute__ ((packed));
>  
> +struct spcr_descriptor {
> +    ACPI_TABLE_HEADER_DEF   /* ACPI common table header */
> +    u8 interface_type;      /* 0=full 16550, 1=subset of 16550 */
> +    u8 reserved[3];
> +    struct acpi_generic_address serial_port;
> +    u8 interrupt_type;
> +    u8 pc_interrupt;
> +    u32 interrupt;
> +    u8 baud_rate;
> +    u8 parity;
> +    u8 stop_bits;
> +    u8 flow_control;
> +    u8 terminal_type;
> +    u8 reserved1;
> +    u16 pci_device_id;
> +    u16 pci_vendor_id;
> +    u8 pci_bus;
> +    u8 pci_device;
> +    u8 pci_function;
> +    u32 pci_flags;
> +    u8 pci_segment;
> +    u32 reserved2;
> +} __attribute__ ((packed));
> +
>  void set_efi_rsdp(struct rsdp_descriptor *rsdp);
>  void* find_acpi_table_addr(u32 sig);
>  
> diff --git a/lib/arm/io.c b/lib/arm/io.c
> index 343e108..893bdfc 100644
> --- a/lib/arm/io.c
> +++ b/lib/arm/io.c
> @@ -8,6 +8,7 @@
>   *
>   * This work is licensed under the terms of the GNU LGPL, version 2.
>   */
> +#include <acpi.h>
>  #include <libcflat.h>
>  #include <devicetree.h>
>  #include <chr-testdev.h>
> @@ -29,7 +30,7 @@ static struct spinlock uart_lock;
>  #define UART_EARLY_BASE (u8 *)(unsigned long)CONFIG_UART_EARLY_BASE
>  static volatile u8 *uart0_base = UART_EARLY_BASE;
>  
> -static void uart0_init(void)
> +static void uart0_init_fdt(void)
>  {
>  	/*
>  	 * kvm-unit-tests uses the uart only for output. Both uart models have
> @@ -73,9 +74,25 @@ static void uart0_init(void)
>  	}
>  }
>  
> +static void uart0_init_acpi(void)
> +{
> +	struct spcr_descriptor *spcr = find_acpi_table_addr(SPCR_SIGNATURE);
> +	assert_msg(spcr, "Unable to find ACPI SPCR");
> +	uart0_base = ioremap(spcr->serial_port.address, spcr->serial_port.bit_width);
> +
> +	if (uart0_base != UART_EARLY_BASE) {
> +		printf("WARNING: early print support may not work. "
> +		       "Found uart at %p, but early base is %p.\n",
> +			uart0_base, UART_EARLY_BASE);
> +	}
> +}
> +
>  void io_init(void)
>  {
> -	uart0_init();
> +	if (dt_available())
> +		uart0_init_fdt();
> +	else
> +		uart0_init_acpi();
>  	chr_testdev_init();
>  }
>  
> diff --git a/lib/arm/psci.c b/lib/arm/psci.c
> index 0e96d19..afbc33d 100644
> --- a/lib/arm/psci.c
> +++ b/lib/arm/psci.c
> @@ -80,9 +80,11 @@ static void psci_set_conduit_fdt(void)
>  static void psci_set_conduit_acpi(void)
>  {
>  	struct acpi_table_fadt *fadt = find_acpi_table_addr(FACP_SIGNATURE);
> +
>  	assert_msg(fadt, "Unable to find ACPI FADT");
>  	assert_msg(fadt->arm_boot_flags & ACPI_FADT_PSCI_COMPLIANT,
> -		   "PSCI is not supported in this platfrom");
> +		   "PSCI is not supported in this platform");
> +
>  	if (fadt->arm_boot_flags & ACPI_FADT_PSCI_USE_HVC)
>  		psci_invoke = psci_invoke_hvc;
>  	else
> -- 
> 2.25.1
>

Reviewed-by: Ricardo Koller <ricarkol@google.com>

Re: [kvm-unit-tests PATCH v2 10/23] lib/printf: Add support for printing wide strings

[Ricardo Koller]

On Fri, May 06, 2022 at 09:55:52PM +0100, Nikos Nikoleris wrote:
> This change adds support for wide strings (u16*) to printf()
> variants. This feature is used by a future change.
> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  lib/printf.c | 101 ++++++++++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 100 insertions(+), 1 deletion(-)
> 
> diff --git a/lib/printf.c b/lib/printf.c
> index 724befa..14a3fed 100644
> --- a/lib/printf.c
> +++ b/lib/printf.c
> @@ -57,6 +57,102 @@ static void print_str(pstream_t *p, const char *s, strprops_t props)
>      }
>  }
>  
> +/*
> + * Adapted from drivers/firmware/efi/libstub/vsprintf.c
> + */
> +static u32 utf16_to_utf32(const u16 **s16)
> +{
> +    u16 c0, c1;
> +
> +    c0 = *(*s16)++;
> +    /* not a surrogate */
> +    if ((c0 & 0xf800) != 0xd800)
> +	return c0;
> +    /* invalid: low surrogate instead of high */
> +    if (c0 & 0x0400)
> +	return 0xfffd;
> +    c1 = **s16;
> +    /* invalid: missing low surrogate */
> +    if ((c1 & 0xfc00) != 0xdc00)
> +	return 0xfffd;
> +    /* valid surrogate pair */
> +    ++(*s16);
> +    return (0x10000 - (0xd800 << 10) - 0xdc00) + (c0 << 10) + c1;
> +}
> +
> +/*
> + * Adapted from drivers/firmware/efi/libstub/vsprintf.c
> + */
> +static size_t utf16s_utf8nlen(const u16 *s16, size_t maxlen)
> +{
> +    size_t len, clen;
> +
> +    for (len = 0; len < maxlen && *s16; len += clen) {
> +	u16 c0 = *s16++;
> +
> +	/* First, get the length for a BMP character */
> +	clen = 1 + (c0 >= 0x80) + (c0 >= 0x800);
> +	if (len + clen > maxlen)
> +	    break;
> +	/*
> +	 * If this is a high surrogate, and we're already at maxlen, we
> +	 * can't include the character if it's a valid surrogate pair.
> +	 * Avoid accessing one extra word just to check if it's valid
> +	 * or not.
> +	 */
> +	if ((c0 & 0xfc00) == 0xd800) {
> +	    if (len + clen == maxlen)
> +		break;
> +	    if ((*s16 & 0xfc00) == 0xdc00) {
> +		++s16;
> +		++clen;
> +	    }
> +	}
> +    }
> +
> +    return len;
> +}
> +
> +/*
> + * Adapted from drivers/firmware/efi/libstub/vsprintf.c
> + */
> +static void print_wstring(pstream_t *p, const u16 *s, strprops_t props)
> +{
> +    const u16 *ws = (const u16 *)s;
> +    size_t pos = 0, size = p->remain + 1, len = utf16s_utf8nlen(ws, props.precision);
> +
> +    while (len-- > 0) {
> +	u32 c32 = utf16_to_utf32(&ws);
> +	u8 *s8;
> +	size_t clen;
> +
> +	if (c32 < 0x80) {
> +	    addchar(p, c32);
> +	    continue;
> +	}
> +
> +	/* Number of trailing octets */
> +	clen = 1 + (c32 >= 0x800) + (c32 >= 0x10000);
> +
> +	len -= clen;
> +	s8 = (u8 *)(p->buffer - p->added + pos);
> +
> +	/* Avoid writing partial character */
> +	addchar(p, '\0');
> +	pos += clen;
> +	if (pos >= size)
> +	    continue;
> +
> +	/* Set high bits of leading octet */
> +	*s8 = (0xf00 >> 1) >> clen;
> +	/* Write trailing octets in reverse order */
> +	for (s8 += clen; clen; --clen, c32 >>= 6)
> +	    *s8-- = 0x80 | (c32 & 0x3f);
> +	/* Set low bits of leading octet */
> +	*s8 |= c32;
> +    }
> +}
> +
>  static char digits[16] = "0123456789abcdef";
>  
>  static void print_int(pstream_t *ps, long long n, int base, strprops_t props)
> @@ -302,7 +398,10 @@ int vsnprintf(char *buf, int size, const char *fmt, va_list va)
>  	    print_unsigned(&s, (unsigned long)va_arg(args, void *), 16, props);
>  	    break;
>  	case 's':
> -	    print_str(&s, va_arg(args, const char *), props);
> +	    if (nlong)
> +		print_wstring(&s, va_arg(args, const u16 *), props);
> +	    else
> +		print_str(&s, va_arg(args, const char *), props);
>  	    break;
>  	default:
>  	    addchar(&s, f);
> -- 
> 2.25.1
>

Reviewed-by: Ricardo Koller <ricarkol@google.com>

Re: [kvm-unit-tests PATCH v2 11/23] lib/efi: Add support for getting the cmdline

[Ricardo Koller]

On Fri, May 06, 2022 at 09:55:53PM +0100, Nikos Nikoleris wrote:
> This change adds support for discovering the command line arguments,
> as a string. Then, we parse this string to populate __argc and __argv
> for EFI tests.
> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  lib/linux/efi.h |  39 +++++++++++++++
>  lib/stdlib.h    |   1 +
>  lib/efi.c       | 123 ++++++++++++++++++++++++++++++++++++++++++++++++
>  lib/string.c    |   2 +-
>  4 files changed, 164 insertions(+), 1 deletion(-)
> 
> diff --git a/lib/linux/efi.h b/lib/linux/efi.h
> index 455625a..e3aba1d 100644
> --- a/lib/linux/efi.h
> +++ b/lib/linux/efi.h
> @@ -60,6 +60,10 @@ typedef guid_t efi_guid_t;
>  
>  #define ACPI_TABLE_GUID EFI_GUID(0xeb9d2d30, 0x2d88, 0x11d3, 0x9a, 0x16, 0x00, 0x90, 0x27, 0x3f, 0xc1, 0x4d)
>  
> +#define LOADED_IMAGE_PROTOCOL_GUID EFI_GUID(0x5b1b31a1, 0x9562, 0x11d2,  0x8e, 0x3f, 0x00, 0xa0, 0xc9, 0x69, 0x72, 0x3b)
> +
> +#define efi_table_attr(inst, attr) (inst->attr)
> +
>  typedef struct {
>  	efi_guid_t guid;
>  	void *table;
> @@ -416,6 +420,41 @@ struct efi_boot_memmap {
>  	unsigned long           *buff_size;
>  };
>  
> +#define __aligned_u64 u64 __attribute__((aligned(8)))
> +
> +typedef union {
> +	struct {
> +		u32			revision;
> +		efi_handle_t		parent_handle;
> +		efi_system_table_t	*system_table;
> +		efi_handle_t		device_handle;
> +		void			*file_path;
> +		void			*reserved;
> +		u32			load_options_size;
> +		void			*load_options;
> +		void			*image_base;
> +		__aligned_u64		image_size;
> +		unsigned int		image_code_type;
> +		unsigned int		image_data_type;
> +		efi_status_t		(__efiapi *unload)(efi_handle_t image_handle);
> +	};
> +	struct {
> +		u32		revision;
> +		u32		parent_handle;
> +		u32		system_table;
> +		u32		device_handle;
> +		u32		file_path;
> +		u32		reserved;
> +		u32		load_options_size;
> +		u32		load_options;
> +		u32		image_base;
> +		__aligned_u64	image_size;
> +		u32		image_code_type;
> +		u32		image_data_type;
> +		u32		unload;
> +	} mixed_mode;
> +} efi_loaded_image_t;

Is the 32-bit mode used (in later commits)? Why not reuse
efi_loaded_image_64_t only and make sure it's the one expected.

> +
>  #define efi_bs_call(func, ...) efi_system_table->boottime->func(__VA_ARGS__)
>  #define efi_rs_call(func, ...) efi_system_table->runtime->func(__VA_ARGS__)
>  
> diff --git a/lib/stdlib.h b/lib/stdlib.h
> index 28496d7..2c524d7 100644
> --- a/lib/stdlib.h
> +++ b/lib/stdlib.h
> @@ -7,6 +7,7 @@
>  #ifndef _STDLIB_H_
>  #define _STDLIB_H_
>  
> +int isspace(int c);
>  long int strtol(const char *nptr, char **endptr, int base);
>  unsigned long int strtoul(const char *nptr, char **endptr, int base);
>  long long int strtoll(const char *nptr, char **endptr, int base);
> diff --git a/lib/efi.c b/lib/efi.c
> index 64cc978..5341942 100644
> --- a/lib/efi.c
> +++ b/lib/efi.c
> @@ -8,6 +8,7 @@
>   */
>  
>  #include "efi.h"
> +#include <stdlib.h>
>  #include <libcflat.h>
>  #include <asm/setup.h>
>  
> @@ -96,6 +97,97 @@ static void efi_exit(efi_status_t code)
>  	efi_rs_call(reset_system, EFI_RESET_SHUTDOWN, code, 0, NULL);
>  }
>  
> +static void efi_cmdline_to_argv(char *cmdline_ptr)
> +{
> +	char *c = cmdline_ptr;
> +	bool narg = true;
> +	while (*c) {
> +		if (isspace(*c)) {
> +			*c = '\0';
> +			narg = true;
> +		} else if (narg) {
> +			__argv[__argc++] = c;
> +			narg = false;
> +		}
> +		c++;
> +	}
> +}
> +
> +static char *efi_convert_cmdline(efi_loaded_image_t *image, int *cmd_line_len)

Mention that this was adapted from drivers/firmware/efi/libstub/efi-stub.c.

> +{
> +	const u16 *s2;
> +	unsigned long cmdline_addr = 0;
> +	int options_chars = efi_table_attr(image, load_options_size) / 2;
> +	const u16 *options = efi_table_attr(image, load_options);
> +	int options_bytes = 0, safe_options_bytes = 0;  /* UTF-8 bytes */
> +	bool in_quote = false;
> +	efi_status_t status;
> +	const int COMMAND_LINE_SIZE = 2048;
> +
> +	if (options) {
> +		s2 = options;
> +		while (options_bytes < COMMAND_LINE_SIZE && options_chars--) {
> +			u16 c = *s2++;
> +
> +			if (c < 0x80) {
> +				if (c == L'\0' || c == L'\n')
> +					break;
> +				if (c == L'"')
> +					in_quote = !in_quote;
> +				else if (!in_quote && isspace((char)c))
> +					safe_options_bytes = options_bytes;
> +
> +				options_bytes++;
> +				continue;
> +			}
> +
> +			/*
> +			 * Get the number of UTF-8 bytes corresponding to a
> +			 * UTF-16 character.
> +			 * The first part handles everything in the BMP.
> +			 */
> +			options_bytes += 2 + (c >= 0x800);
> +			/*
> +			 * Add one more byte for valid surrogate pairs. Invalid
> +			 * surrogates will be replaced with 0xfffd and take up
> +			 * only 3 bytes.
> +			 */
> +			if ((c & 0xfc00) == 0xd800) {
> +				/*
> +				 * If the very last word is a high surrogate,
> +				 * we must ignore it since we can't access the
> +				 * low surrogate.
> +				 */
> +				if (!options_chars) {
> +					options_bytes -= 3;
> +				} else if ((*s2 & 0xfc00) == 0xdc00) {
> +					options_bytes++;
> +					options_chars--;
> +					s2++;
> +				}
> +			}
> +		}
> +		if (options_bytes >= COMMAND_LINE_SIZE) {
> +			options_bytes = safe_options_bytes;
> +			printf("Command line is too long: truncated to %d bytes\n",
> +			       options_bytes);
> +		}
> +        }
> +
> +	options_bytes++;        /* NUL termination */
> +
> +	status = efi_bs_call(allocate_pool, EFI_LOADER_DATA, options_bytes,
> +			     (void **)&cmdline_addr);
> +	if (status != EFI_SUCCESS)
> +		return NULL;
> +
> +	snprintf((char *)cmdline_addr, options_bytes, "%.*ls",
> +		 options_bytes - 1, options);
> +
> +	*cmd_line_len = options_bytes;
> +	return (char *)cmdline_addr;
> +}
> +
>  efi_status_t efi_main(efi_handle_t handle, efi_system_table_t *sys_tab)
>  {
>  	int ret;
> @@ -109,6 +201,37 @@ efi_status_t efi_main(efi_handle_t handle, efi_system_table_t *sys_tab)
>  	unsigned long map_size = 0, desc_size = 0, key = 0, buff_size = 0;
>  	u32 desc_ver;
>  
> +	/* Helper variables needed to get the cmdline */
> +	efi_loaded_image_t *image;
> +	efi_guid_t loaded_image_proto = LOADED_IMAGE_PROTOCOL_GUID;
> +	char *cmdline_ptr = NULL;
> +	int cmdline_size = 0;
> +
> +	/*
> +	 * Get a handle to the loaded image protocol.  This is used to get
> +	 * information about the running image, such as size and the command
> +	 * line.
> +	 */
> +	status = efi_bs_call(handle_protocol, handle, &loaded_image_proto,
> +			     (void *)&image);
> +	if (status != EFI_SUCCESS) {
> +		printf("Failed to get loaded image protocol\n");
> +		goto efi_main_error;
> +	}
> +
> +	/*
> +	 * Get the command line from EFI, using the LOADED_IMAGE
> +	 * protocol. We are going to copy the command line into the
> +	 * device tree, so this can be allocated anywhere.

Does the "device tree" comment still make sense?

> +	 */
> +	cmdline_ptr = efi_convert_cmdline(image, &cmdline_size);
> +	if (!cmdline_ptr) {
> +		printf("getting command line via LOADED_IMAGE_PROTOCOL\n");
> +		status = EFI_OUT_OF_RESOURCES;
> +		goto efi_main_error;
> +	}
> +	efi_cmdline_to_argv(cmdline_ptr);
> +
>  	/* Set up efi_bootinfo */
>  	efi_bootinfo.mem_map.map = &map;
>  	efi_bootinfo.mem_map.map_size = &map_size;
> diff --git a/lib/string.c b/lib/string.c
> index 27106da..b191ab1 100644
> --- a/lib/string.c
> +++ b/lib/string.c
> @@ -163,7 +163,7 @@ void *memchr(const void *s, int c, size_t n)
>      return NULL;
>  }
>  
> -static int isspace(int c)
> +int isspace(int c)
>  {
>      return c == ' ' || c == '\t' || c == '\r' || c == '\n' || c == '\v' || c == '\f';
>  }
> -- 
> 2.25.1
> 

Re: [kvm-unit-tests PATCH v2 13/23] arm/arm64: Rename etext to _etext

[Ricardo Koller]

On Fri, May 06, 2022 at 09:55:55PM +0100, Nikos Nikoleris wrote:
> From: Andrew Jones <drjones@redhat.com>
> 
> Rename etext to the more popular _etext allowing different linker
> scripts to more easily be used.
> 
> Signed-off-by: Andrew Jones <drjones@redhat.com>
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  lib/arm/setup.c | 4 ++--
>  arm/flat.lds    | 2 +-
>  2 files changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/lib/arm/setup.c b/lib/arm/setup.c
> index 3c24c75..2d67292 100644
> --- a/lib/arm/setup.c
> +++ b/lib/arm/setup.c
> @@ -34,7 +34,7 @@
>  #define NR_EXTRA_MEM_REGIONS	16
>  #define NR_INITIAL_MEM_REGIONS	(MAX_DT_MEM_REGIONS + NR_EXTRA_MEM_REGIONS)
>  
> -extern unsigned long etext;
> +extern unsigned long _etext;
>  
>  char *initrd;
>  u32 initrd_size;
> @@ -140,7 +140,7 @@ unsigned int mem_region_get_flags(phys_addr_t paddr)
>  
>  static void mem_regions_add_assumed(void)
>  {
> -	phys_addr_t code_end = (phys_addr_t)(unsigned long)&etext;
> +	phys_addr_t code_end = (phys_addr_t)(unsigned long)&_etext;
>  	struct mem_region *r;
>  
>  	r = mem_region_find(code_end - 1);
> diff --git a/arm/flat.lds b/arm/flat.lds
> index 47fcb64..9016ac9 100644
> --- a/arm/flat.lds
> +++ b/arm/flat.lds
> @@ -27,7 +27,7 @@ SECTIONS
>      PROVIDE(_text = .);
>      .text : { *(.init) *(.text) *(.text.*) }
>      . = ALIGN(64K);
> -    PROVIDE(etext = .);
> +    PROVIDE(_etext = .);
>  
>      PROVIDE(reloc_start = .);
>      .rela.dyn : { *(.rela.dyn) }
> -- 
> 2.25.1
> 

Reviewed-by: Ricardo Koller <ricarkol@google.com>

Re: [kvm-unit-tests PATCH v2 15/23] arm64: Add a new type of memory type flag MR_F_RESERVED

[Ricardo Koller]

On Fri, May 06, 2022 at 09:55:57PM +0100, Nikos Nikoleris wrote:
> From: Andrew Jones <drjones@redhat.com>
> 
> This will be used by future change to add PTE entries for special EFI
> memory regions.
> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  lib/arm/asm/setup.h | 1 +
>  lib/arm/mmu.c       | 4 ++++
>  2 files changed, 5 insertions(+)
> 
> diff --git a/lib/arm/asm/setup.h b/lib/arm/asm/setup.h
> index f0e70b1..64cd379 100644
> --- a/lib/arm/asm/setup.h
> +++ b/lib/arm/asm/setup.h
> @@ -15,6 +15,7 @@ extern int nr_cpus;
>  
>  #define MR_F_IO			(1U << 0)
>  #define MR_F_CODE		(1U << 1)
> +#define MR_F_RESERVED		(1U << 2)
>  #define MR_F_UNKNOWN		(1U << 31)
>  
>  struct mem_region {
> diff --git a/lib/arm/mmu.c b/lib/arm/mmu.c
> index e1a72fe..931be98 100644
> --- a/lib/arm/mmu.c
> +++ b/lib/arm/mmu.c
> @@ -174,6 +174,10 @@ void *setup_mmu(phys_addr_t phys_end, void *unused)
>  	for (r = mem_regions; r->end; ++r) {
>  		if (r->flags & MR_F_IO) {
>  			continue;
> +		} else if (r->flags & MR_F_RESERVED) {
> +			/* Reserved pages need to be writable for whatever reserved them */
> +			mmu_set_range_ptes(mmu_idmap, r->start, r->start, r->end,
> +					   __pgprot(PTE_WBWA));
>  		} else if (r->flags & MR_F_CODE) {
>  			/* armv8 requires code shared between EL1 and EL0 to be read-only */
>  			mmu_set_range_ptes(mmu_idmap, r->start, r->start, r->end,
> -- 
> 2.25.1
>

Reviewed-by: Ricardo Koller <ricarkol@google.com>

Re: [kvm-unit-tests PATCH v2 17/23] arm64: Copy code from GNU-EFI

[Ricardo Koller]

On Fri, May 06, 2022 at 09:55:59PM +0100, Nikos Nikoleris wrote:
> This change adds unmodified dependencies that we need from GNU-EFI in
> order to build arm64 EFI apps.
> 
> GNU-EFI sources from  https://git.code.sf.net/p/gnu-efi/code v3.0.14
> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  arm/efi/elf_aarch64_efi.lds |  63 +++++++++++++++++
>  arm/efi/crt0-efi-aarch64.S  | 130 ++++++++++++++++++++++++++++++++++++
>  arm/efi/reloc_aarch64.c     |  97 +++++++++++++++++++++++++++
>  3 files changed, 290 insertions(+)
>  create mode 100644 arm/efi/elf_aarch64_efi.lds
>  create mode 100644 arm/efi/crt0-efi-aarch64.S
>  create mode 100644 arm/efi/reloc_aarch64.c
> 
> diff --git a/arm/efi/elf_aarch64_efi.lds b/arm/efi/elf_aarch64_efi.lds
> new file mode 100644
> index 0000000..836d982
> --- /dev/null
> +++ b/arm/efi/elf_aarch64_efi.lds
> @@ -0,0 +1,63 @@
> +OUTPUT_FORMAT("elf64-littleaarch64", "elf64-littleaarch64", "elf64-littleaarch64")
> +OUTPUT_ARCH(aarch64)
> +ENTRY(_start)
> +SECTIONS
> +{
> +  .text 0x0 : {
> +    _text = .;
> +    *(.text.head)
> +    *(.text)
> +    *(.text.*)
> +    *(.gnu.linkonce.t.*)
> +    *(.srodata)
> +    *(.rodata*)
> +    . = ALIGN(16);
> +  }
> +  _etext = .;
> +  _text_size = . - _text;
> +  .dynamic  : { *(.dynamic) }
> +  .data : ALIGN(4096)
> +  {
> +   _data = .;
> +   *(.sdata)
> +   *(.data)
> +   *(.data1)
> +   *(.data.*)
> +   *(.got.plt)
> +   *(.got)
> +
> +   /* the EFI loader doesn't seem to like a .bss section, so we stick
> +      it all into .data: */
> +   . = ALIGN(16);
> +   _bss = .;
> +   *(.sbss)
> +   *(.scommon)
> +   *(.dynbss)
> +   *(.bss)
> +   *(COMMON)
> +   . = ALIGN(16);
> +   _bss_end = .;
> +  }
> +
> +  .rela.dyn : { *(.rela.dyn) }
> +  .rela.plt : { *(.rela.plt) }
> +  .rela.got : { *(.rela.got) }
> +  .rela.data : { *(.rela.data) *(.rela.data*) }
> +  . = ALIGN(512);
> +  _edata = .;
> +  _data_size = . - _data;
> +
> +  . = ALIGN(4096);
> +  .dynsym   : { *(.dynsym) }
> +  . = ALIGN(4096);
> +  .dynstr   : { *(.dynstr) }
> +  . = ALIGN(4096);
> +  .note.gnu.build-id : { *(.note.gnu.build-id) }
> +  /DISCARD/ :
> +  {
> +    *(.rel.reloc)
> +    *(.eh_frame)
> +    *(.note.GNU-stack)
> +  }
> +  .comment 0 : { *(.comment) }
> +}
> diff --git a/arm/efi/crt0-efi-aarch64.S b/arm/efi/crt0-efi-aarch64.S
> new file mode 100644
> index 0000000..d50e78d
> --- /dev/null
> +++ b/arm/efi/crt0-efi-aarch64.S
> @@ -0,0 +1,130 @@
> +/*
> + * crt0-efi-aarch64.S - PE/COFF header for AArch64 EFI applications
> + *
> + * Copright (C) 2014 Linaro Ltd. <ard.biesheuvel@linaro.org>
> + *
> + * Redistribution and use in source and binary forms, with or without
> + * modification, are permitted provided that the following conditions
> + * are met:
> + * 1. Redistributions of source code must retain the above copyright
> + *    notice and this list of conditions, without modification.
> + * 2. The name of the author may not be used to endorse or promote products
> + *    derived from this software without specific prior written permission.
> + *
> + * Alternatively, this software may be distributed under the terms of the
> + * GNU General Public License as published by the Free Software Foundation;
> + * either version 2 of the License, or (at your option) any later version.
> + */
> +
> +	.section	.text.head
> +
> +	/*
> +	 * Magic "MZ" signature for PE/COFF
> +	 */
> +	.globl	ImageBase
> +ImageBase:
> +	.ascii	"MZ"
> +	.skip	58				// 'MZ' + pad + offset == 64
> +	.long	pe_header - ImageBase		// Offset to the PE header.
> +pe_header:
> +	.ascii	"PE"
> +	.short 	0
> +coff_header:
> +	.short	0xaa64				// AArch64
> +	.short	2				// nr_sections
> +	.long	0 				// TimeDateStamp
> +	.long	0				// PointerToSymbolTable
> +	.long	0				// NumberOfSymbols
> +	.short	section_table - optional_header	// SizeOfOptionalHeader
> +	.short	0x206				// Characteristics.
> +						// IMAGE_FILE_DEBUG_STRIPPED |
> +						// IMAGE_FILE_EXECUTABLE_IMAGE |
> +						// IMAGE_FILE_LINE_NUMS_STRIPPED
> +optional_header:
> +	.short	0x20b				// PE32+ format
> +	.byte	0x02				// MajorLinkerVersion
> +	.byte	0x14				// MinorLinkerVersion
> +	.long	_data - _start			// SizeOfCode
> +	.long	_data_size			// SizeOfInitializedData
> +	.long	0				// SizeOfUninitializedData
> +	.long	_start - ImageBase		// AddressOfEntryPoint
> +	.long	_start - ImageBase		// BaseOfCode
> +
> +extra_header_fields:
> +	.quad	0				// ImageBase
> +	.long	0x1000				// SectionAlignment
> +	.long	0x200				// FileAlignment
> +	.short	0				// MajorOperatingSystemVersion
> +	.short	0				// MinorOperatingSystemVersion
> +	.short	0				// MajorImageVersion
> +	.short	0				// MinorImageVersion
> +	.short	0				// MajorSubsystemVersion
> +	.short	0				// MinorSubsystemVersion
> +	.long	0				// Win32VersionValue
> +
> +	.long	_edata - ImageBase		// SizeOfImage
> +
> +	// Everything before the kernel image is considered part of the header
> +	.long	_start - ImageBase		// SizeOfHeaders
> +	.long	0				// CheckSum
> +	.short	EFI_SUBSYSTEM			// Subsystem
> +	.short	0				// DllCharacteristics
> +	.quad	0				// SizeOfStackReserve
> +	.quad	0				// SizeOfStackCommit
> +	.quad	0				// SizeOfHeapReserve
> +	.quad	0				// SizeOfHeapCommit
> +	.long	0				// LoaderFlags
> +	.long	0x6				// NumberOfRvaAndSizes
> +
> +	.quad	0				// ExportTable
> +	.quad	0				// ImportTable
> +	.quad	0				// ResourceTable
> +	.quad	0				// ExceptionTable
> +	.quad	0				// CertificationTable
> +	.quad	0				// BaseRelocationTable
> +
> +	// Section table
> +section_table:
> +	.ascii	".text\0\0\0"
> +	.long	_data - _start		// VirtualSize
> +	.long	_start - ImageBase	// VirtualAddress
> +	.long	_data - _start		// SizeOfRawData
> +	.long	_start - ImageBase	// PointerToRawData
> +
> +	.long	0		// PointerToRelocations (0 for executables)
> +	.long	0		// PointerToLineNumbers (0 for executables)
> +	.short	0		// NumberOfRelocations  (0 for executables)
> +	.short	0		// NumberOfLineNumbers  (0 for executables)
> +	.long	0x60000020	// Characteristics (section flags)
> +
> +	.ascii	".data\0\0\0"
> +	.long	_data_size		// VirtualSize
> +	.long	_data - ImageBase	// VirtualAddress
> +	.long	_data_size		// SizeOfRawData
> +	.long	_data - ImageBase	// PointerToRawData
> +
> +	.long	0		// PointerToRelocations (0 for executables)
> +	.long	0		// PointerToLineNumbers (0 for executables)
> +	.short	0		// NumberOfRelocations  (0 for executables)
> +	.short	0		// NumberOfLineNumbers  (0 for executables)
> +	.long	0xc0000040	// Characteristics (section flags)
> +
> +	.align		12
> +_start:
> +	stp		x29, x30, [sp, #-32]!
> +	mov		x29, sp
> +
> +	stp		x0, x1, [sp, #16]
> +	mov		x2, x0
> +	mov		x3, x1
> +	adr		x0, ImageBase
> +	adrp		x1, _DYNAMIC
> +	add		x1, x1, #:lo12:_DYNAMIC
> +	bl		_relocate
> +	cbnz		x0, 0f
> +
> +	ldp		x0, x1, [sp, #16]
> +	bl		efi_main
> +
> +0:	ldp		x29, x30, [sp], #32
> +	ret
> diff --git a/arm/efi/reloc_aarch64.c b/arm/efi/reloc_aarch64.c
> new file mode 100644
> index 0000000..0867279
> --- /dev/null
> +++ b/arm/efi/reloc_aarch64.c
> @@ -0,0 +1,97 @@
> +/* reloc_aarch64.c - position independent x86 ELF shared object relocator
> +   Copyright (C) 2014 Linaro Ltd. <ard.biesheuvel@linaro.org>
> +   Copyright (C) 1999 Hewlett-Packard Co.
> +	Contributed by David Mosberger <davidm@hpl.hp.com>.
> +
> +    All rights reserved.
> +
> +    Redistribution and use in source and binary forms, with or without
> +    modification, are permitted provided that the following conditions
> +    are met:
> +
> +    * Redistributions of source code must retain the above copyright
> +      notice, this list of conditions and the following disclaimer.
> +    * Redistributions in binary form must reproduce the above
> +      copyright notice, this list of conditions and the following
> +      disclaimer in the documentation and/or other materials
> +      provided with the distribution.
> +    * Neither the name of Hewlett-Packard Co. nor the names of its
> +      contributors may be used to endorse or promote products derived
> +      from this software without specific prior written permission.
> +
> +    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
> +    CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
> +    INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
> +    MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
> +    DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS
> +    BE LIABLE FOR ANYDIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY,
> +    OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
> +    PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
> +    PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> +    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR
> +    TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
> +    THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
> +    SUCH DAMAGE.
> +*/
> +
> +#include <efi.h>
> +#include <efilib.h>
> +
> +#include <elf.h>
> +
> +EFI_STATUS _relocate (long ldbase, Elf64_Dyn *dyn,
> +		      EFI_HANDLE image EFI_UNUSED,
> +		      EFI_SYSTEM_TABLE *systab EFI_UNUSED)
> +{
> +	long relsz = 0, relent = 0;
> +	Elf64_Rela *rel = 0;
> +	unsigned long *addr;
> +	int i;
> +
> +	for (i = 0; dyn[i].d_tag != DT_NULL; ++i) {
> +		switch (dyn[i].d_tag) {
> +			case DT_RELA:
> +				rel = (Elf64_Rela*)
> +					((unsigned long)dyn[i].d_un.d_ptr
> +					 + ldbase);
> +				break;
> +
> +			case DT_RELASZ:
> +				relsz = dyn[i].d_un.d_val;
> +				break;
> +
> +			case DT_RELAENT:
> +				relent = dyn[i].d_un.d_val;
> +				break;
> +
> +			default:
> +				break;
> +		}
> +	}
> +
> +	if (!rel && relent == 0)
> +		return EFI_SUCCESS;
> +
> +	if (!rel || relent == 0)
> +		return EFI_LOAD_ERROR;
> +
> +	while (relsz > 0) {
> +		/* apply the relocs */
> +		switch (ELF64_R_TYPE (rel->r_info)) {
> +			case R_AARCH64_NONE:
> +				break;
> +
> +			case R_AARCH64_RELATIVE:
> +				addr = (unsigned long *)
> +					(ldbase + rel->r_offset);
> +				*addr = ldbase + rel->r_addend;
> +				break;
> +
> +			default:
> +				break;
> +		}
> +		rel = (Elf64_Rela*) ((char *) rel + relent);
> +		relsz -= relent;
> +	}
> +	return EFI_SUCCESS;
> +}
> -- 
> 2.25.1
> 

Reviewed-by: Ricardo Koller <ricarkol@google.com>

Re: [kvm-unit-tests PATCH v2 19/23] arm64: Use code from the gnu-efi when booting with EFI

[Ricardo Koller]

On Fri, May 06, 2022 at 09:56:01PM +0100, Nikos Nikoleris wrote:
> arm/efi/crt0-efi-aarch64.S defines the header and the handover
> sequence from EFI to a efi_main. This change includes the whole file
> in arm/cstart64.S when we compile with EFI support.
> 
> In addition, we change the handover code in arm/efi/crt0-efi-aarch64.S
> to align the stack pointer. This alignment is necessary because we
> make assumptions about cpu0's stack alignment and most importantly we
> place its thread_info at the bottom of this stack.
> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  arm/cstart64.S             |  6 ++++++
>  arm/efi/crt0-efi-aarch64.S | 17 +++++++++++++++--
>  2 files changed, 21 insertions(+), 2 deletions(-)
> 
> diff --git a/arm/cstart64.S b/arm/cstart64.S
> index 55b41ea..08cf02f 100644
> --- a/arm/cstart64.S
> +++ b/arm/cstart64.S
> @@ -15,6 +15,10 @@
>  #include <asm/thread_info.h>
>  #include <asm/sysreg.h>
>  
> +#ifdef CONFIG_EFI
> +#include "efi/crt0-efi-aarch64.S"
> +#else
> +
>  .macro zero_range, tmp1, tmp2
>  9998:	cmp	\tmp1, \tmp2
>  	b.eq	9997f
> @@ -107,6 +111,8 @@ start:
>  	bl	exit
>  	b	halt
>  
> +#endif
> +
>  .text
>  
>  /*
> diff --git a/arm/efi/crt0-efi-aarch64.S b/arm/efi/crt0-efi-aarch64.S
> index d50e78d..11a062d 100644
> --- a/arm/efi/crt0-efi-aarch64.S
> +++ b/arm/efi/crt0-efi-aarch64.S
> @@ -111,10 +111,19 @@ section_table:
>  
>  	.align		12
>  _start:
> -	stp		x29, x30, [sp, #-32]!
> +	stp		x29, x30, [sp, #-16]!

Is this and the "ldp x29, x30, [sp], #16" change below needed?
why is #-32 not good?

> +
> +	// Align sp; this is necessary due to way we store cpu0's thread_info

/* */ comment style

>  	mov		x29, sp
> +	and		x29, x29, #THREAD_MASK
> +	mov		x30, sp
> +	mov		sp, x29
> +	str		x30, [sp, #-32]!
> +
> +	mov             x29, sp
>  
>  	stp		x0, x1, [sp, #16]
> +
>  	mov		x2, x0
>  	mov		x3, x1
>  	adr		x0, ImageBase
> @@ -126,5 +135,9 @@ _start:
>  	ldp		x0, x1, [sp, #16]
>  	bl		efi_main
>  
> -0:	ldp		x29, x30, [sp], #32
> +	// Restore sp

/* */ comment style

> +	ldr		x30, [sp]

I'm not able to understand this. Is this ldr restoring the value pushed
with "str x30, [sp, #-32]!" above? in that case, shouldn't this be at
[sp - 32]? But, given that this code is unreachable when efi_main is
called, do you even need to restore the sp?

> +	mov             sp, x30
> +
> +0:	ldp		x29, x30, [sp], #16
>  	ret
> -- 
> 2.25.1
> 

Re: [kvm-unit-tests PATCH v2 20/23] lib: Avoid external dependency in libelf

[Ricardo Koller]

On Fri, May 06, 2022 at 09:56:02PM +0100, Nikos Nikoleris wrote:
> There is just a small number of definitions we need from
> uapi/linux/elf.h and asm/elf.h and the relocation code is
> self-contained.
> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  arm/efi/reloc_aarch64.c | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/arm/efi/reloc_aarch64.c b/arm/efi/reloc_aarch64.c
> index fa0cd6b..3f6d9a6 100644
> --- a/arm/efi/reloc_aarch64.c
> +++ b/arm/efi/reloc_aarch64.c
> @@ -34,8 +34,7 @@
>      SUCH DAMAGE.
>  */
>  
> -#include "efi.h"
> -#include <elf.h>
> +#include <efi.h>
>  
>  efi_status_t _relocate(long ldbase, Elf64_Dyn *dyn, efi_handle_t image,
>  		       efi_system_table_t *sys_tab)
> -- 
> 2.25.1
>

Reviewed-by: Ricardo Koller <ricarkol@google.com>

Re: [kvm-unit-tests PATCH v2 21/23] x86: Move x86_64-specific EFI CFLAGS to x86_64 Makefile

[Ricardo Koller]

On Fri, May 06, 2022 at 09:56:03PM +0100, Nikos Nikoleris wrote:
> Compiler flag -macculate-outgoing-args is only needed by the x86_64
> ABI. Move it to the relevant Makefile.
> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  Makefile            | 4 ----
>  x86/Makefile.x86_64 | 4 ++++
>  2 files changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 6ed5dea..307bc29 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -40,14 +40,10 @@ OBJDIRS += $(LIBFDT_objdir)
>  
>  # EFI App
>  ifeq ($(CONFIG_EFI),y)
> -EFI_ARCH = x86_64

Should this be moved to the x86 Makefile as well? (just in case)

>  EFI_CFLAGS := -DCONFIG_EFI
>  # The following CFLAGS and LDFLAGS come from:
>  #   - GNU-EFI/Makefile.defaults
>  #   - GNU-EFI/apps/Makefile
> -# Function calls must include the number of arguments passed to the functions
> -# More details: https://wiki.osdev.org/GNU-EFI
> -EFI_CFLAGS += -maccumulate-outgoing-args
>  # GCC defines wchar to be 32 bits, but EFI expects 16 bits
>  EFI_CFLAGS += -fshort-wchar
>  # EFI applications use PIC as they are loaded to dynamic addresses, not a fixed
> diff --git a/x86/Makefile.x86_64 b/x86/Makefile.x86_64
> index f18c1e2..ac588ed 100644
> --- a/x86/Makefile.x86_64
> +++ b/x86/Makefile.x86_64
> @@ -2,6 +2,10 @@ cstart.o = $(TEST_DIR)/cstart64.o
>  bits = 64
>  ldarch = elf64-x86-64
>  ifeq ($(CONFIG_EFI),y)
> +# Function calls must include the number of arguments passed to the functions
> +# More details: https://wiki.osdev.org/GNU-EFI
> +CFLAGS += -maccumulate-outgoing-args
> +
>  exe = efi
>  bin = so
>  FORMAT = efi-app-x86_64
> -- 
> 2.25.1
> 

Reviewed-by: Ricardo Koller <ricarkol@google.com>

Re: [kvm-unit-tests PATCH v2 22/23] arm64: Add support for efi in Makefile

[Ricardo Koller]

On Fri, May 06, 2022 at 09:56:04PM +0100, Nikos Nikoleris wrote:
> Users can now build kvm-unit-tests as efi apps by supplying an extra
> argument when invoking configure:
> 
> $> ./configure --enable-efi
> 
> This patch is based on an earlier version by
> Andrew Jones <drjones@redhat.com>
> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  configure           | 15 ++++++++++++---
>  arm/Makefile.arm    |  6 ++++++
>  arm/Makefile.arm64  | 18 ++++++++++++++----
>  arm/Makefile.common | 45 ++++++++++++++++++++++++++++++++++-----------
>  4 files changed, 66 insertions(+), 18 deletions(-)
> 
> diff --git a/configure b/configure
> index 86c3095..beef655 100755
> --- a/configure
> +++ b/configure
> @@ -195,14 +195,19 @@ else
>      fi
>  fi
>  
> -if [ "$efi" ] && [ "$arch" != "x86_64" ]; then
> +if [ "$efi" ] && [ "$arch" != "x86_64" ] && [ "$arch" != "arm64" ]; then
>      echo "--[enable|disable]-efi is not supported for $arch"
>      usage
>  fi
>  
>  if [ -z "$page_size" ]; then
> -    [ "$arch" = "arm64" ] && page_size="65536"
> -    [ "$arch" = "arm" ] && page_size="4096"
> +    if [ "$efi" = 'y' ] && [ "$arch" = "arm64" ]; then
> +        page_size="4096"
> +    elif [ "$arch" = "arm64" ]; then
> +        page_size="65536"
> +    elif [ "$arch" = "arm" ]; then
> +        page_size="4096"
> +    fi
>  else
>      if [ "$arch" != "arm64" ]; then
>          echo "--page-size is not supported for $arch"
> @@ -217,6 +222,10 @@ else
>          echo "arm64 doesn't support page size of $page_size"
>          usage
>      fi
> +    if [ "$efi" = 'y' ] && [ "$page_size" != "4096" ]; then
> +        echo "efi must use 4K pages"
> +        exit 1
> +    fi

nit: not really needed as x86 doesn't allow other page sizes, arm64
already forces 4096 above, and arm32 doesn't support efi.

>  fi
>  
>  [ -z "$processor" ] && processor="$arch"
> diff --git a/arm/Makefile.arm b/arm/Makefile.arm
> index 01fd4c7..2ce00f5 100644
> --- a/arm/Makefile.arm
> +++ b/arm/Makefile.arm
> @@ -7,6 +7,10 @@ bits = 32
>  ldarch = elf32-littlearm
>  machine = -marm -mfpu=vfp
>  
> +ifeq ($(CONFIG_EFI),y)
> +$(error Cannot build arm32 tests as EFI apps)
> +endif
> +
>  # stack.o relies on frame pointers.
>  KEEP_FRAME_POINTER := y
>  
> @@ -32,6 +36,8 @@ cflatobjs += lib/arm/stack.o
>  cflatobjs += lib/ldiv32.o
>  cflatobjs += lib/arm/ldivmod.o
>  
> +exe = flat
> +
>  # arm specific tests
>  tests =
>  
> diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
> index 6feac76..550e1b2 100644
> --- a/arm/Makefile.arm64
> +++ b/arm/Makefile.arm64
> @@ -27,11 +27,21 @@ cflatobjs += lib/arm64/gic-v3-its.o lib/arm64/gic-v3-its-cmd.o
>  
>  OBJDIRS += lib/arm64
>  
> +ifeq ($(CONFIG_EFI),y)
> +# avoid jump tables before all relocations have been processed
> +arm/efi/reloc_aarch64.o: CFLAGS += -fno-jump-tables
> +cflatobjs += arm/efi/reloc_aarch64.o
> +
> +exe = efi
> +else
> +exe = flat
> +endif
> +
>  # arm64 specific tests
> -tests = $(TEST_DIR)/timer.flat
> -tests += $(TEST_DIR)/micro-bench.flat
> -tests += $(TEST_DIR)/cache.flat
> -tests += $(TEST_DIR)/debug.flat
> +tests = $(TEST_DIR)/timer.$(exe)
> +tests += $(TEST_DIR)/micro-bench.$(exe)
> +tests += $(TEST_DIR)/cache.$(exe)
> +tests += $(TEST_DIR)/debug.$(exe)
>  
>  include $(SRCDIR)/$(TEST_DIR)/Makefile.common
>  
> diff --git a/arm/Makefile.common b/arm/Makefile.common
> index 5be42c0..a8007f4 100644
> --- a/arm/Makefile.common
> +++ b/arm/Makefile.common
> @@ -4,14 +4,14 @@
>  # Authors: Andrew Jones <drjones@redhat.com>
>  #
>  
> -tests-common  = $(TEST_DIR)/selftest.flat
> -tests-common += $(TEST_DIR)/spinlock-test.flat
> -tests-common += $(TEST_DIR)/pci-test.flat
> -tests-common += $(TEST_DIR)/pmu.flat
> -tests-common += $(TEST_DIR)/gic.flat
> -tests-common += $(TEST_DIR)/psci.flat
> -tests-common += $(TEST_DIR)/sieve.flat
> -tests-common += $(TEST_DIR)/pl031.flat
> +tests-common  = $(TEST_DIR)/selftest.$(exe)
> +tests-common += $(TEST_DIR)/spinlock-test.$(exe)
> +tests-common += $(TEST_DIR)/pci-test.$(exe)
> +tests-common += $(TEST_DIR)/pmu.$(exe)
> +tests-common += $(TEST_DIR)/gic.$(exe)
> +tests-common += $(TEST_DIR)/psci.$(exe)
> +tests-common += $(TEST_DIR)/sieve.$(exe)
> +tests-common += $(TEST_DIR)/pl031.$(exe)
>  
>  tests-all = $(tests-common) $(tests)
>  all: directories $(tests-all)
> @@ -54,6 +54,9 @@ cflatobjs += lib/arm/smp.o
>  cflatobjs += lib/arm/delay.o
>  cflatobjs += lib/arm/gic.o lib/arm/gic-v2.o lib/arm/gic-v3.o
>  cflatobjs += lib/arm/timer.o
> +ifeq ($(CONFIG_EFI),y)
> +cflatobjs += lib/efi.o
> +endif
>  
>  OBJDIRS += lib/arm
>  
> @@ -61,6 +64,25 @@ libeabi = lib/arm/libeabi.a
>  eabiobjs = lib/arm/eabi_compat.o
>  
>  FLATLIBS = $(libcflat) $(LIBFDT_archive) $(libeabi)
> +
> +ifeq ($(CONFIG_EFI),y)
> +%.so: EFI_LDFLAGS += -defsym=EFI_SUBSYSTEM=0xa --no-undefined
> +%.so: %.o $(FLATLIBS) $(SRCDIR)/arm/efi/elf_aarch64_efi.lds $(cstart.o)
> +	$(CC) $(CFLAGS) -c -o $(@:.so=.aux.o) $(SRCDIR)/lib/auxinfo.c \
> +		-DPROGNAME=\"$(@:.so=.efi)\" -DAUXFLAGS=$(AUXFLAGS)
> +	$(LD) $(EFI_LDFLAGS) -o $@ -T $(SRCDIR)/arm/efi/elf_aarch64_efi.lds \
> +		$(filter %.o, $^) $(FLATLIBS) $(@:.so=.aux.o) \
> +		$(EFI_LIBS)
> +	$(RM) $(@:.so=.aux.o)
> +
> +%.efi: %.so
> +	$(call arch_elf_check, $^)
> +	$(OBJCOPY) \
> +		-j .text -j .sdata -j .data -j .dynamic -j .dynsym \
> +		-j .rel -j .rela -j .rel.* -j .rela.* -j .rel* -j .rela* \
> +		-j .reloc \
> +		-O binary $^ $@
> +else
>  %.elf: LDFLAGS = -nostdlib $(arch_LDFLAGS)
>  %.elf: %.o $(FLATLIBS) $(SRCDIR)/arm/flat.lds $(cstart.o)
>  	$(CC) $(CFLAGS) -c -o $(@:.elf=.aux.o) $(SRCDIR)/lib/auxinfo.c \
> @@ -74,13 +96,14 @@ FLATLIBS = $(libcflat) $(LIBFDT_archive) $(libeabi)
>  	$(call arch_elf_check, $^)
>  	$(OBJCOPY) -O binary $^ $@
>  	@chmod a-x $@
> +endif
>  
>  $(libeabi): $(eabiobjs)
>  	$(AR) rcs $@ $^
>  
>  arm_clean: asm_offsets_clean
> -	$(RM) $(TEST_DIR)/*.{o,flat,elf} $(libeabi) $(eabiobjs) \
> -	      $(TEST_DIR)/.*.d lib/arm/.*.d
> +	$(RM) $(TEST_DIR)/*.{o,flat,elf,so,efi} $(libeabi) $(eabiobjs) \
> +	      $(TEST_DIR)/.*.d $(TEST_DIR)/efi/.*.d lib/arm/.*.d
>  
>  generated-files = $(asm-offsets)
> -$(tests-all:.flat=.o) $(cstart.o) $(cflatobjs): $(generated-files)
> +$(tests-all:.$(exe)=.o) $(cstart.o) $(cflatobjs): $(generated-files)
> -- 
> 2.25.1
> 

Re: [kvm-unit-tests PATCH v2 22/23] arm64: Add support for efi in Makefile

[Ricardo Koller]

On Fri, May 06, 2022 at 09:56:04PM +0100, Nikos Nikoleris wrote:
> Users can now build kvm-unit-tests as efi apps by supplying an extra
> argument when invoking configure:
> 
> $> ./configure --enable-efi
> 
> This patch is based on an earlier version by
> Andrew Jones <drjones@redhat.com>
> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  configure           | 15 ++++++++++++---
>  arm/Makefile.arm    |  6 ++++++
>  arm/Makefile.arm64  | 18 ++++++++++++++----
>  arm/Makefile.common | 45 ++++++++++++++++++++++++++++++++++-----------
>  4 files changed, 66 insertions(+), 18 deletions(-)
> 
> diff --git a/configure b/configure
> index 86c3095..beef655 100755
> --- a/configure
> +++ b/configure
> @@ -195,14 +195,19 @@ else
>      fi
>  fi
>  
> -if [ "$efi" ] && [ "$arch" != "x86_64" ]; then
> +if [ "$efi" ] && [ "$arch" != "x86_64" ] && [ "$arch" != "arm64" ]; then
>      echo "--[enable|disable]-efi is not supported for $arch"
>      usage
>  fi
>  
>  if [ -z "$page_size" ]; then
> -    [ "$arch" = "arm64" ] && page_size="65536"
> -    [ "$arch" = "arm" ] && page_size="4096"
> +    if [ "$efi" = 'y' ] && [ "$arch" = "arm64" ]; then
> +        page_size="4096"
> +    elif [ "$arch" = "arm64" ]; then
> +        page_size="65536"
> +    elif [ "$arch" = "arm" ]; then
> +        page_size="4096"
> +    fi
>  else
>      if [ "$arch" != "arm64" ]; then
>          echo "--page-size is not supported for $arch"
> @@ -217,6 +222,10 @@ else
>          echo "arm64 doesn't support page size of $page_size"
>          usage
>      fi
> +    if [ "$efi" = 'y' ] && [ "$page_size" != "4096" ]; then
> +        echo "efi must use 4K pages"
> +        exit 1
> +    fi
>  fi
>  
>  [ -z "$processor" ] && processor="$arch"
> diff --git a/arm/Makefile.arm b/arm/Makefile.arm
> index 01fd4c7..2ce00f5 100644
> --- a/arm/Makefile.arm
> +++ b/arm/Makefile.arm
> @@ -7,6 +7,10 @@ bits = 32
>  ldarch = elf32-littlearm
>  machine = -marm -mfpu=vfp
>  
> +ifeq ($(CONFIG_EFI),y)
> +$(error Cannot build arm32 tests as EFI apps)
> +endif
> +
>  # stack.o relies on frame pointers.
>  KEEP_FRAME_POINTER := y
>  
> @@ -32,6 +36,8 @@ cflatobjs += lib/arm/stack.o
>  cflatobjs += lib/ldiv32.o
>  cflatobjs += lib/arm/ldivmod.o
>  
> +exe = flat
> +
>  # arm specific tests
>  tests =
>  
> diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
> index 6feac76..550e1b2 100644
> --- a/arm/Makefile.arm64
> +++ b/arm/Makefile.arm64
> @@ -27,11 +27,21 @@ cflatobjs += lib/arm64/gic-v3-its.o lib/arm64/gic-v3-its-cmd.o
>  
>  OBJDIRS += lib/arm64
>  
> +ifeq ($(CONFIG_EFI),y)
> +# avoid jump tables before all relocations have been processed
> +arm/efi/reloc_aarch64.o: CFLAGS += -fno-jump-tables
> +cflatobjs += arm/efi/reloc_aarch64.o
> +
> +exe = efi
> +else
> +exe = flat
> +endif
> +
>  # arm64 specific tests
> -tests = $(TEST_DIR)/timer.flat
> -tests += $(TEST_DIR)/micro-bench.flat
> -tests += $(TEST_DIR)/cache.flat
> -tests += $(TEST_DIR)/debug.flat
> +tests = $(TEST_DIR)/timer.$(exe)
> +tests += $(TEST_DIR)/micro-bench.$(exe)
> +tests += $(TEST_DIR)/cache.$(exe)
> +tests += $(TEST_DIR)/debug.$(exe)
>  
>  include $(SRCDIR)/$(TEST_DIR)/Makefile.common
>  
> diff --git a/arm/Makefile.common b/arm/Makefile.common
> index 5be42c0..a8007f4 100644
> --- a/arm/Makefile.common
> +++ b/arm/Makefile.common
> @@ -4,14 +4,14 @@
>  # Authors: Andrew Jones <drjones@redhat.com>
>  #
>  
> -tests-common  = $(TEST_DIR)/selftest.flat
> -tests-common += $(TEST_DIR)/spinlock-test.flat
> -tests-common += $(TEST_DIR)/pci-test.flat
> -tests-common += $(TEST_DIR)/pmu.flat
> -tests-common += $(TEST_DIR)/gic.flat
> -tests-common += $(TEST_DIR)/psci.flat
> -tests-common += $(TEST_DIR)/sieve.flat
> -tests-common += $(TEST_DIR)/pl031.flat
> +tests-common  = $(TEST_DIR)/selftest.$(exe)
> +tests-common += $(TEST_DIR)/spinlock-test.$(exe)
> +tests-common += $(TEST_DIR)/pci-test.$(exe)
> +tests-common += $(TEST_DIR)/pmu.$(exe)
> +tests-common += $(TEST_DIR)/gic.$(exe)
> +tests-common += $(TEST_DIR)/psci.$(exe)
> +tests-common += $(TEST_DIR)/sieve.$(exe)
> +tests-common += $(TEST_DIR)/pl031.$(exe)
>  
>  tests-all = $(tests-common) $(tests)
>  all: directories $(tests-all)
> @@ -54,6 +54,9 @@ cflatobjs += lib/arm/smp.o
>  cflatobjs += lib/arm/delay.o
>  cflatobjs += lib/arm/gic.o lib/arm/gic-v2.o lib/arm/gic-v3.o
>  cflatobjs += lib/arm/timer.o
> +ifeq ($(CONFIG_EFI),y)
> +cflatobjs += lib/efi.o
> +endif
>  
>  OBJDIRS += lib/arm
>  
> @@ -61,6 +64,25 @@ libeabi = lib/arm/libeabi.a
>  eabiobjs = lib/arm/eabi_compat.o
>  
>  FLATLIBS = $(libcflat) $(LIBFDT_archive) $(libeabi)
> +
> +ifeq ($(CONFIG_EFI),y)
> +%.so: EFI_LDFLAGS += -defsym=EFI_SUBSYSTEM=0xa --no-undefined
> +%.so: %.o $(FLATLIBS) $(SRCDIR)/arm/efi/elf_aarch64_efi.lds $(cstart.o)
> +	$(CC) $(CFLAGS) -c -o $(@:.so=.aux.o) $(SRCDIR)/lib/auxinfo.c \
> +		-DPROGNAME=\"$(@:.so=.efi)\" -DAUXFLAGS=$(AUXFLAGS)
> +	$(LD) $(EFI_LDFLAGS) -o $@ -T $(SRCDIR)/arm/efi/elf_aarch64_efi.lds \
> +		$(filter %.o, $^) $(FLATLIBS) $(@:.so=.aux.o) \
> +		$(EFI_LIBS)
> +	$(RM) $(@:.so=.aux.o)
> +
> +%.efi: %.so
> +	$(call arch_elf_check, $^)
> +	$(OBJCOPY) \
> +		-j .text -j .sdata -j .data -j .dynamic -j .dynsym \
> +		-j .rel -j .rela -j .rel.* -j .rela.* -j .rel* -j .rela* \
> +		-j .reloc \
> +		-O binary $^ $@
> +else
>  %.elf: LDFLAGS = -nostdlib $(arch_LDFLAGS)
>  %.elf: %.o $(FLATLIBS) $(SRCDIR)/arm/flat.lds $(cstart.o)
>  	$(CC) $(CFLAGS) -c -o $(@:.elf=.aux.o) $(SRCDIR)/lib/auxinfo.c \
> @@ -74,13 +96,14 @@ FLATLIBS = $(libcflat) $(LIBFDT_archive) $(libeabi)
>  	$(call arch_elf_check, $^)
>  	$(OBJCOPY) -O binary $^ $@
>  	@chmod a-x $@
> +endif
>  
>  $(libeabi): $(eabiobjs)
>  	$(AR) rcs $@ $^
>  
>  arm_clean: asm_offsets_clean
> -	$(RM) $(TEST_DIR)/*.{o,flat,elf} $(libeabi) $(eabiobjs) \
> -	      $(TEST_DIR)/.*.d lib/arm/.*.d
> +	$(RM) $(TEST_DIR)/*.{o,flat,elf,so,efi} $(libeabi) $(eabiobjs) \
> +	      $(TEST_DIR)/.*.d $(TEST_DIR)/efi/.*.d lib/arm/.*.d
>  
>  generated-files = $(asm-offsets)
> -$(tests-all:.flat=.o) $(cstart.o) $(cflatobjs): $(generated-files)
> +$(tests-all:.$(exe)=.o) $(cstart.o) $(cflatobjs): $(generated-files)
> -- 
> 2.25.1
>

Reviewed-by: Ricardo Koller <ricarkol@google.com>

Re: [kvm-unit-tests PATCH v2 23/23] arm64: Add an efi/run script

[Ricardo Koller]

On Fri, May 06, 2022 at 09:56:05PM +0100, Nikos Nikoleris wrote:
> This change adds a efi/run script inspired by the one in x86. This
> script will setup a folder with the test compiled as an EFI app and a
> startup.nsh script. The script launches QEMU providing an image with
> EDKII and the path to the folder with the test which is executed
> automatically.
> 
> For example:
> 
> $> ./arm/efi/run ./arm/selftest.efi setup smp=2 mem=256
> 
> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> ---
>  scripts/runtime.bash | 14 +++++-----
>  arm/efi/run          | 61 ++++++++++++++++++++++++++++++++++++++++++++
>  arm/run              |  8 ++++--
>  arm/Makefile.common  |  1 +
>  arm/dummy.c          |  4 +++
>  5 files changed, 78 insertions(+), 10 deletions(-)
>  create mode 100755 arm/efi/run
>  create mode 100644 arm/dummy.c
> 
> diff --git a/scripts/runtime.bash b/scripts/runtime.bash
> index 7d0180b..dc28f24 100644
> --- a/scripts/runtime.bash
> +++ b/scripts/runtime.bash
> @@ -131,14 +131,12 @@ function run()
>      fi
>  
>      last_line=$(premature_failure > >(tail -1)) && {
> -        skip=true
> -        if [ "${CONFIG_EFI}" == "y" ] && [[ "${last_line}" =~ "enabling apic" ]]; then
> -            skip=false
> -        fi
> -        if [ ${skip} == true ]; then
> -            print_result "SKIP" $testname "" "$last_line"
> -            return 77
> -        fi
> +        if [ "${CONFIG_EFI}" == "y" ] && [ "${ARCH}" = x86_64 ]; then
> +		if ! [[ "${last_line}" =~ "enabling apic" ]]; then
> +			print_result "SKIP" $testname "" "$last_line"
> +			return 77
> +		fi
> +	fi
>      }
>  
>      cmdline=$(get_cmdline $kernel)
> diff --git a/arm/efi/run b/arm/efi/run
> new file mode 100755
> index 0000000..dfff717
> --- /dev/null
> +++ b/arm/efi/run
> @@ -0,0 +1,61 @@
> +#!/bin/bash
> +
> +set -e
> +
> +if [ $# -eq 0 ]; then
> +	echo "Usage $0 TEST_CASE [QEMU_ARGS]"
> +	exit 2
> +fi
> +
> +if [ ! -f config.mak ]; then
> +	echo "run './configure --enable-efi && make' first. See ./configure -h"
> +	exit 2
> +fi
> +source config.mak
> +source scripts/arch-run.bash
> +source scripts/common.bash
> +
> +: "${EFI_SRC:=$(realpath "$(dirname "$0")/../")}"
> +: "${EFI_UEFI:=/usr/share/qemu-efi-aarch64/QEMU_EFI.fd}"
> +: "${EFI_TEST:=efi-tests}"
> +: "${EFI_CASE:=$(basename $1 .efi)}"
> +
> +if [ ! -f "$EFI_UEFI" ]; then
> +	echo "UEFI firmware not found: $EFI_UEFI"
> +	echo "Please install the UEFI firmware to this path"
> +	echo "Or specify the correct path with the env variable EFI_UEFI"
> +	exit 2
> +fi
> +
> +# Remove the TEST_CASE from $@
> +shift 1
> +
> +# Fish out the arguments for the test, they should be the next string
> +# after the "-append" option
> +qemu_args=()
> +cmd_args=()
> +while (( "$#" )); do
> +	if [ "$1" = "-append" ]; then
> +		cmd_args=$2
> +		shift 2

Does this work with params like this (2 words)?

	[pmu-cycle-counter]
	file = pmu.flat
	groups = pmu
	extra_params = -append 'cycle-counter 0'

> +	else
> +		qemu_args+=("$1")
> +		shift 1
> +	fi
> +done
> +
> +if [ "$EFI_CASE" = "_NO_FILE_4Uhere_" ]; then
> +	EFI_CASE=dummy
> +fi
> +
> +: "${EFI_CASE_DIR:="$EFI_TEST/$EFI_CASE"}"
> +mkdir -p "$EFI_CASE_DIR"
> +
> +cp "$EFI_SRC/$EFI_CASE.efi" "$EFI_TEST/$EFI_CASE/"
> +echo "@echo -off" > "$EFI_TEST/$EFI_CASE/startup.nsh"
> +echo "$EFI_CASE.efi" "${cmd_args[@]}" >> "$EFI_TEST/$EFI_CASE/startup.nsh"
> +
> +EFI_RUN=y $TEST_DIR/run \
> +       -bios "$EFI_UEFI" \
> +       -drive file.dir="$EFI_TEST/$EFI_CASE/",file.driver=vvfat,file.rw=on,format=raw,if=virtio \
> +       "${qemu_args[@]}"
> diff --git a/arm/run b/arm/run
> index 28a0b4a..e96875e 100755
> --- a/arm/run
> +++ b/arm/run
> @@ -67,7 +67,11 @@ fi
>  
>  A="-accel $ACCEL"
>  command="$qemu -nodefaults $M $A -cpu $processor $chr_testdev $pci_testdev"
> -command+=" -display none -serial stdio -kernel"
> +command+=" -display none -serial stdio"
>  command="$(migration_cmd) $(timeout_cmd) $command"
>  
> -run_qemu $command "$@"
> +if [ "$EFI_RUN" = "y" ]; then
> +	ENVIRON_DEFAULT=n run_qemu $command "$@"
> +else
> +	run_qemu $command -kernel "$@"
> +fi
> diff --git a/arm/Makefile.common b/arm/Makefile.common
> index a8007f4..aabd335 100644
> --- a/arm/Makefile.common
> +++ b/arm/Makefile.common
> @@ -12,6 +12,7 @@ tests-common += $(TEST_DIR)/gic.$(exe)
>  tests-common += $(TEST_DIR)/psci.$(exe)
>  tests-common += $(TEST_DIR)/sieve.$(exe)
>  tests-common += $(TEST_DIR)/pl031.$(exe)
> +tests-common += $(TEST_DIR)/dummy.$(exe)
>  
>  tests-all = $(tests-common) $(tests)
>  all: directories $(tests-all)
> diff --git a/arm/dummy.c b/arm/dummy.c
> new file mode 100644
> index 0000000..5019e79
> --- /dev/null
> +++ b/arm/dummy.c
> @@ -0,0 +1,4 @@
> +int main(int argc, char **argv)
> +{
> +	return 0;
> +}
> -- 
> 2.25.1
> 

Re: [kvm-unit-tests PATCH v2 21/23] x86: Move x86_64-specific EFI CFLAGS to x86_64 Makefile

[Nikos Nikoleris]

Hi Ricardo,

On 21/06/2022 23:45, Ricardo Koller wrote:
> On Fri, May 06, 2022 at 09:56:03PM +0100, Nikos Nikoleris wrote:
>> Compiler flag -macculate-outgoing-args is only needed by the x86_64
>> ABI. Move it to the relevant Makefile.
>>
>> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
>> ---
>>   Makefile            | 4 ----
>>   x86/Makefile.x86_64 | 4 ++++
>>   2 files changed, 4 insertions(+), 4 deletions(-)
>>
>> diff --git a/Makefile b/Makefile
>> index 6ed5dea..307bc29 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -40,14 +40,10 @@ OBJDIRS += $(LIBFDT_objdir)
>>   
>>   # EFI App
>>   ifeq ($(CONFIG_EFI),y)
>> -EFI_ARCH = x86_64
> 
> Should this be moved to the x86 Makefile as well? (just in case)

I searched for uses of EFI_ARCH and I couldn't find any. It would be 
good if people more familiar with x86 would approve this though.

Thanks,

Nikos

> 
>>   EFI_CFLAGS := -DCONFIG_EFI
>>   # The following CFLAGS and LDFLAGS come from:
>>   #   - GNU-EFI/Makefile.defaults
>>   #   - GNU-EFI/apps/Makefile
>> -# Function calls must include the number of arguments passed to the functions
>> -# More details: https://wiki.osdev.org/GNU-EFI
>> -EFI_CFLAGS += -maccumulate-outgoing-args
>>   # GCC defines wchar to be 32 bits, but EFI expects 16 bits
>>   EFI_CFLAGS += -fshort-wchar
>>   # EFI applications use PIC as they are loaded to dynamic addresses, not a fixed
>> diff --git a/x86/Makefile.x86_64 b/x86/Makefile.x86_64
>> index f18c1e2..ac588ed 100644
>> --- a/x86/Makefile.x86_64
>> +++ b/x86/Makefile.x86_64
>> @@ -2,6 +2,10 @@ cstart.o = $(TEST_DIR)/cstart64.o
>>   bits = 64
>>   ldarch = elf64-x86-64
>>   ifeq ($(CONFIG_EFI),y)
>> +# Function calls must include the number of arguments passed to the functions
>> +# More details: https://wiki.osdev.org/GNU-EFI
>> +CFLAGS += -maccumulate-outgoing-args
>> +
>>   exe = efi
>>   bin = so
>>   FORMAT = efi-app-x86_64
>> -- 
>> 2.25.1
>>
> 
> Reviewed-by: Ricardo Koller <ricarkol@google.com>

Re: [kvm-unit-tests PATCH v2 22/23] arm64: Add support for efi in Makefile

[Nikos Nikoleris]

On 21/06/2022 23:51, Ricardo Koller wrote:
> On Fri, May 06, 2022 at 09:56:04PM +0100, Nikos Nikoleris wrote:
>> Users can now build kvm-unit-tests as efi apps by supplying an extra
>> argument when invoking configure:
>>
>> $> ./configure --enable-efi
>>
>> This patch is based on an earlier version by
>> Andrew Jones <drjones@redhat.com>
>>
>> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
>> ---
>>   configure           | 15 ++++++++++++---
>>   arm/Makefile.arm    |  6 ++++++
>>   arm/Makefile.arm64  | 18 ++++++++++++++----
>>   arm/Makefile.common | 45 ++++++++++++++++++++++++++++++++++-----------
>>   4 files changed, 66 insertions(+), 18 deletions(-)
>>
>> diff --git a/configure b/configure
>> index 86c3095..beef655 100755
>> --- a/configure
>> +++ b/configure
>> @@ -195,14 +195,19 @@ else
>>       fi
>>   fi
>>   
>> -if [ "$efi" ] && [ "$arch" != "x86_64" ]; then
>> +if [ "$efi" ] && [ "$arch" != "x86_64" ] && [ "$arch" != "arm64" ]; then
>>       echo "--[enable|disable]-efi is not supported for $arch"
>>       usage
>>   fi
>>   
>>   if [ -z "$page_size" ]; then
>> -    [ "$arch" = "arm64" ] && page_size="65536"
>> -    [ "$arch" = "arm" ] && page_size="4096"
>> +    if [ "$efi" = 'y' ] && [ "$arch" = "arm64" ]; then
>> +        page_size="4096"
>> +    elif [ "$arch" = "arm64" ]; then
>> +        page_size="65536"
>> +    elif [ "$arch" = "arm" ]; then
>> +        page_size="4096"
>> +    fi
>>   else
>>       if [ "$arch" != "arm64" ]; then
>>           echo "--page-size is not supported for $arch"
>> @@ -217,6 +222,10 @@ else
>>           echo "arm64 doesn't support page size of $page_size"
>>           usage
>>       fi
>> +    if [ "$efi" = 'y' ] && [ "$page_size" != "4096" ]; then
>> +        echo "efi must use 4K pages"
>> +        exit 1
>> +    fi
> 
> nit: not really needed as x86 doesn't allow other page sizes, arm64
> already forces 4096 above, and arm32 doesn't support efi.
> 

I might be missing something but without this nothing prevents me from 
doing:

$> ./configure --enable-efi --page-size=16k

Which we need to catch.

Thanks,

Nikos

>>   fi
>>   
>>   [ -z "$processor" ] && processor="$arch"
>> diff --git a/arm/Makefile.arm b/arm/Makefile.arm
>> index 01fd4c7..2ce00f5 100644
>> --- a/arm/Makefile.arm
>> +++ b/arm/Makefile.arm
>> @@ -7,6 +7,10 @@ bits = 32
>>   ldarch = elf32-littlearm
>>   machine = -marm -mfpu=vfp
>>   
>> +ifeq ($(CONFIG_EFI),y)
>> +$(error Cannot build arm32 tests as EFI apps)
>> +endif
>> +
>>   # stack.o relies on frame pointers.
>>   KEEP_FRAME_POINTER := y
>>   
>> @@ -32,6 +36,8 @@ cflatobjs += lib/arm/stack.o
>>   cflatobjs += lib/ldiv32.o
>>   cflatobjs += lib/arm/ldivmod.o
>>   
>> +exe = flat
>> +
>>   # arm specific tests
>>   tests =
>>   
>> diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
>> index 6feac76..550e1b2 100644
>> --- a/arm/Makefile.arm64
>> +++ b/arm/Makefile.arm64
>> @@ -27,11 +27,21 @@ cflatobjs += lib/arm64/gic-v3-its.o lib/arm64/gic-v3-its-cmd.o
>>   
>>   OBJDIRS += lib/arm64
>>   
>> +ifeq ($(CONFIG_EFI),y)
>> +# avoid jump tables before all relocations have been processed
>> +arm/efi/reloc_aarch64.o: CFLAGS += -fno-jump-tables
>> +cflatobjs += arm/efi/reloc_aarch64.o
>> +
>> +exe = efi
>> +else
>> +exe = flat
>> +endif
>> +
>>   # arm64 specific tests
>> -tests = $(TEST_DIR)/timer.flat
>> -tests += $(TEST_DIR)/micro-bench.flat
>> -tests += $(TEST_DIR)/cache.flat
>> -tests += $(TEST_DIR)/debug.flat
>> +tests = $(TEST_DIR)/timer.$(exe)
>> +tests += $(TEST_DIR)/micro-bench.$(exe)
>> +tests += $(TEST_DIR)/cache.$(exe)
>> +tests += $(TEST_DIR)/debug.$(exe)
>>   
>>   include $(SRCDIR)/$(TEST_DIR)/Makefile.common
>>   
>> diff --git a/arm/Makefile.common b/arm/Makefile.common
>> index 5be42c0..a8007f4 100644
>> --- a/arm/Makefile.common
>> +++ b/arm/Makefile.common
>> @@ -4,14 +4,14 @@
>>   # Authors: Andrew Jones <drjones@redhat.com>
>>   #
>>   
>> -tests-common  = $(TEST_DIR)/selftest.flat
>> -tests-common += $(TEST_DIR)/spinlock-test.flat
>> -tests-common += $(TEST_DIR)/pci-test.flat
>> -tests-common += $(TEST_DIR)/pmu.flat
>> -tests-common += $(TEST_DIR)/gic.flat
>> -tests-common += $(TEST_DIR)/psci.flat
>> -tests-common += $(TEST_DIR)/sieve.flat
>> -tests-common += $(TEST_DIR)/pl031.flat
>> +tests-common  = $(TEST_DIR)/selftest.$(exe)
>> +tests-common += $(TEST_DIR)/spinlock-test.$(exe)
>> +tests-common += $(TEST_DIR)/pci-test.$(exe)
>> +tests-common += $(TEST_DIR)/pmu.$(exe)
>> +tests-common += $(TEST_DIR)/gic.$(exe)
>> +tests-common += $(TEST_DIR)/psci.$(exe)
>> +tests-common += $(TEST_DIR)/sieve.$(exe)
>> +tests-common += $(TEST_DIR)/pl031.$(exe)
>>   
>>   tests-all = $(tests-common) $(tests)
>>   all: directories $(tests-all)
>> @@ -54,6 +54,9 @@ cflatobjs += lib/arm/smp.o
>>   cflatobjs += lib/arm/delay.o
>>   cflatobjs += lib/arm/gic.o lib/arm/gic-v2.o lib/arm/gic-v3.o
>>   cflatobjs += lib/arm/timer.o
>> +ifeq ($(CONFIG_EFI),y)
>> +cflatobjs += lib/efi.o
>> +endif
>>   
>>   OBJDIRS += lib/arm
>>   
>> @@ -61,6 +64,25 @@ libeabi = lib/arm/libeabi.a
>>   eabiobjs = lib/arm/eabi_compat.o
>>   
>>   FLATLIBS = $(libcflat) $(LIBFDT_archive) $(libeabi)
>> +
>> +ifeq ($(CONFIG_EFI),y)
>> +%.so: EFI_LDFLAGS += -defsym=EFI_SUBSYSTEM=0xa --no-undefined
>> +%.so: %.o $(FLATLIBS) $(SRCDIR)/arm/efi/elf_aarch64_efi.lds $(cstart.o)
>> +	$(CC) $(CFLAGS) -c -o $(@:.so=.aux.o) $(SRCDIR)/lib/auxinfo.c \
>> +		-DPROGNAME=\"$(@:.so=.efi)\" -DAUXFLAGS=$(AUXFLAGS)
>> +	$(LD) $(EFI_LDFLAGS) -o $@ -T $(SRCDIR)/arm/efi/elf_aarch64_efi.lds \
>> +		$(filter %.o, $^) $(FLATLIBS) $(@:.so=.aux.o) \
>> +		$(EFI_LIBS)
>> +	$(RM) $(@:.so=.aux.o)
>> +
>> +%.efi: %.so
>> +	$(call arch_elf_check, $^)
>> +	$(OBJCOPY) \
>> +		-j .text -j .sdata -j .data -j .dynamic -j .dynsym \
>> +		-j .rel -j .rela -j .rel.* -j .rela.* -j .rel* -j .rela* \
>> +		-j .reloc \
>> +		-O binary $^ $@
>> +else
>>   %.elf: LDFLAGS = -nostdlib $(arch_LDFLAGS)
>>   %.elf: %.o $(FLATLIBS) $(SRCDIR)/arm/flat.lds $(cstart.o)
>>   	$(CC) $(CFLAGS) -c -o $(@:.elf=.aux.o) $(SRCDIR)/lib/auxinfo.c \
>> @@ -74,13 +96,14 @@ FLATLIBS = $(libcflat) $(LIBFDT_archive) $(libeabi)
>>   	$(call arch_elf_check, $^)
>>   	$(OBJCOPY) -O binary $^ $@
>>   	@chmod a-x $@
>> +endif
>>   
>>   $(libeabi): $(eabiobjs)
>>   	$(AR) rcs $@ $^
>>   
>>   arm_clean: asm_offsets_clean
>> -	$(RM) $(TEST_DIR)/*.{o,flat,elf} $(libeabi) $(eabiobjs) \
>> -	      $(TEST_DIR)/.*.d lib/arm/.*.d
>> +	$(RM) $(TEST_DIR)/*.{o,flat,elf,so,efi} $(libeabi) $(eabiobjs) \
>> +	      $(TEST_DIR)/.*.d $(TEST_DIR)/efi/.*.d lib/arm/.*.d
>>   
>>   generated-files = $(asm-offsets)
>> -$(tests-all:.flat=.o) $(cstart.o) $(cflatobjs): $(generated-files)
>> +$(tests-all:.$(exe)=.o) $(cstart.o) $(cflatobjs): $(generated-files)
>> -- 
>> 2.25.1
>>

Re: [kvm-unit-tests PATCH v2 23/23] arm64: Add an efi/run script

[Nikos Nikoleris]

On 22/06/2022 00:09, Ricardo Koller wrote:
> On Fri, May 06, 2022 at 09:56:05PM +0100, Nikos Nikoleris wrote:
>> This change adds a efi/run script inspired by the one in x86. This
>> script will setup a folder with the test compiled as an EFI app and a
>> startup.nsh script. The script launches QEMU providing an image with
>> EDKII and the path to the folder with the test which is executed
>> automatically.
>>
>> For example:
>>
>> $> ./arm/efi/run ./arm/selftest.efi setup smp=2 mem=256
>>
>> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
>> ---
>>   scripts/runtime.bash | 14 +++++-----
>>   arm/efi/run          | 61 ++++++++++++++++++++++++++++++++++++++++++++
>>   arm/run              |  8 ++++--
>>   arm/Makefile.common  |  1 +
>>   arm/dummy.c          |  4 +++
>>   5 files changed, 78 insertions(+), 10 deletions(-)
>>   create mode 100755 arm/efi/run
>>   create mode 100644 arm/dummy.c
>>
>> diff --git a/scripts/runtime.bash b/scripts/runtime.bash
>> index 7d0180b..dc28f24 100644
>> --- a/scripts/runtime.bash
>> +++ b/scripts/runtime.bash
>> @@ -131,14 +131,12 @@ function run()
>>       fi
>>
>>       last_line=$(premature_failure > >(tail -1)) && {
>> -        skip=true
>> -        if [ "${CONFIG_EFI}" == "y" ] && [[ "${last_line}" =~ "enabling apic" ]]; then
>> -            skip=false
>> -        fi
>> -        if [ ${skip} == true ]; then
>> -            print_result "SKIP" $testname "" "$last_line"
>> -            return 77
>> -        fi
>> +        if [ "${CONFIG_EFI}" == "y" ] && [ "${ARCH}" = x86_64 ]; then
>> +            if ! [[ "${last_line}" =~ "enabling apic" ]]; then
>> +                    print_result "SKIP" $testname "" "$last_line"
>> +                    return 77
>> +            fi
>> +    fi
>>       }
>>
>>       cmdline=$(get_cmdline $kernel)
>> diff --git a/arm/efi/run b/arm/efi/run
>> new file mode 100755
>> index 0000000..dfff717
>> --- /dev/null
>> +++ b/arm/efi/run
>> @@ -0,0 +1,61 @@
>> +#!/bin/bash
>> +
>> +set -e
>> +
>> +if [ $# -eq 0 ]; then
>> +    echo "Usage $0 TEST_CASE [QEMU_ARGS]"
>> +    exit 2
>> +fi
>> +
>> +if [ ! -f config.mak ]; then
>> +    echo "run './configure --enable-efi && make' first. See ./configure -h"
>> +    exit 2
>> +fi
>> +source config.mak
>> +source scripts/arch-run.bash
>> +source scripts/common.bash
>> +
>> +: "${EFI_SRC:=$(realpath "$(dirname "$0")/../")}"
>> +: "${EFI_UEFI:=/usr/share/qemu-efi-aarch64/QEMU_EFI.fd}"
>> +: "${EFI_TEST:=efi-tests}"
>> +: "${EFI_CASE:=$(basename $1 .efi)}"
>> +
>> +if [ ! -f "$EFI_UEFI" ]; then
>> +    echo "UEFI firmware not found: $EFI_UEFI"
>> +    echo "Please install the UEFI firmware to this path"
>> +    echo "Or specify the correct path with the env variable EFI_UEFI"
>> +    exit 2
>> +fi
>> +
>> +# Remove the TEST_CASE from $@
>> +shift 1
>> +
>> +# Fish out the arguments for the test, they should be the next string
>> +# after the "-append" option
>> +qemu_args=()
>> +cmd_args=()
>> +while (( "$#" )); do
>> +    if [ "$1" = "-append" ]; then
>> +            cmd_args=$2
>> +            shift 2
>
> Does this work with params like this (2 words)?
>
>       [pmu-cycle-counter]
>       file = pmu.flat
>       groups = pmu
>       extra_params = -append 'cycle-counter 0'
>

I think it does. cmd_args will take the value of the string that follows
append and in this case whatever is between the single/double quotes.

Thanks,

Nikos

>> +    else
>> +            qemu_args+=("$1")
>> +            shift 1
>> +    fi
>> +done
>> +
>> +if [ "$EFI_CASE" = "_NO_FILE_4Uhere_" ]; then
>> +    EFI_CASE=dummy
>> +fi
>> +
>> +: "${EFI_CASE_DIR:="$EFI_TEST/$EFI_CASE"}"
>> +mkdir -p "$EFI_CASE_DIR"
>> +
>> +cp "$EFI_SRC/$EFI_CASE.efi" "$EFI_TEST/$EFI_CASE/"
>> +echo "@echo -off" > "$EFI_TEST/$EFI_CASE/startup.nsh"
>> +echo "$EFI_CASE.efi" "${cmd_args[@]}" >> "$EFI_TEST/$EFI_CASE/startup.nsh"
>> +
>> +EFI_RUN=y $TEST_DIR/run \
>> +       -bios "$EFI_UEFI" \
>> +       -drive file.dir="$EFI_TEST/$EFI_CASE/",file.driver=vvfat,file.rw=on,format=raw,if=virtio \
>> +       "${qemu_args[@]}"
>> diff --git a/arm/run b/arm/run
>> index 28a0b4a..e96875e 100755
>> --- a/arm/run
>> +++ b/arm/run
>> @@ -67,7 +67,11 @@ fi
>>
>>   A="-accel $ACCEL"
>>   command="$qemu -nodefaults $M $A -cpu $processor $chr_testdev $pci_testdev"
>> -command+=" -display none -serial stdio -kernel"
>> +command+=" -display none -serial stdio"
>>   command="$(migration_cmd) $(timeout_cmd) $command"
>>
>> -run_qemu $command "$@"
>> +if [ "$EFI_RUN" = "y" ]; then
>> +    ENVIRON_DEFAULT=n run_qemu $command "$@"
>> +else
>> +    run_qemu $command -kernel "$@"
>> +fi
>> diff --git a/arm/Makefile.common b/arm/Makefile.common
>> index a8007f4..aabd335 100644
>> --- a/arm/Makefile.common
>> +++ b/arm/Makefile.common
>> @@ -12,6 +12,7 @@ tests-common += $(TEST_DIR)/gic.$(exe)
>>   tests-common += $(TEST_DIR)/psci.$(exe)
>>   tests-common += $(TEST_DIR)/sieve.$(exe)
>>   tests-common += $(TEST_DIR)/pl031.$(exe)
>> +tests-common += $(TEST_DIR)/dummy.$(exe)
>>
>>   tests-all = $(tests-common) $(tests)
>>   all: directories $(tests-all)
>> diff --git a/arm/dummy.c b/arm/dummy.c
>> new file mode 100644
>> index 0000000..5019e79
>> --- /dev/null
>> +++ b/arm/dummy.c
>> @@ -0,0 +1,4 @@
>> +int main(int argc, char **argv)
>> +{
>> +    return 0;
>> +}
>> --
>> 2.25.1
>>
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

Re: [kvm-unit-tests PATCH v2 11/23] lib/efi: Add support for getting the cmdline

[Nikos Nikoleris]

Hi Ricardo,

On 21/06/2022 17:33, Ricardo Koller wrote:
> On Fri, May 06, 2022 at 09:55:53PM +0100, Nikos Nikoleris wrote:
>> This change adds support for discovering the command line arguments,
>> as a string. Then, we parse this string to populate __argc and __argv
>> for EFI tests.
>>
>> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
>> ---
>>   lib/linux/efi.h |  39 +++++++++++++++
>>   lib/stdlib.h    |   1 +
>>   lib/efi.c       | 123 ++++++++++++++++++++++++++++++++++++++++++++++++
>>   lib/string.c    |   2 +-
>>   4 files changed, 164 insertions(+), 1 deletion(-)
>>
>> diff --git a/lib/linux/efi.h b/lib/linux/efi.h
>> index 455625a..e3aba1d 100644
>> --- a/lib/linux/efi.h
>> +++ b/lib/linux/efi.h
>> @@ -60,6 +60,10 @@ typedef guid_t efi_guid_t;
>>   
>>   #define ACPI_TABLE_GUID EFI_GUID(0xeb9d2d30, 0x2d88, 0x11d3, 0x9a, 0x16, 0x00, 0x90, 0x27, 0x3f, 0xc1, 0x4d)
>>   
>> +#define LOADED_IMAGE_PROTOCOL_GUID EFI_GUID(0x5b1b31a1, 0x9562, 0x11d2,  0x8e, 0x3f, 0x00, 0xa0, 0xc9, 0x69, 0x72, 0x3b)
>> +
>> +#define efi_table_attr(inst, attr) (inst->attr)
>> +
>>   typedef struct {
>>   	efi_guid_t guid;
>>   	void *table;
>> @@ -416,6 +420,41 @@ struct efi_boot_memmap {
>>   	unsigned long           *buff_size;
>>   };
>>   
>> +#define __aligned_u64 u64 __attribute__((aligned(8)))
>> +
>> +typedef union {
>> +	struct {
>> +		u32			revision;
>> +		efi_handle_t		parent_handle;
>> +		efi_system_table_t	*system_table;
>> +		efi_handle_t		device_handle;
>> +		void			*file_path;
>> +		void			*reserved;
>> +		u32			load_options_size;
>> +		void			*load_options;
>> +		void			*image_base;
>> +		__aligned_u64		image_size;
>> +		unsigned int		image_code_type;
>> +		unsigned int		image_data_type;
>> +		efi_status_t		(__efiapi *unload)(efi_handle_t image_handle);
>> +	};
>> +	struct {
>> +		u32		revision;
>> +		u32		parent_handle;
>> +		u32		system_table;
>> +		u32		device_handle;
>> +		u32		file_path;
>> +		u32		reserved;
>> +		u32		load_options_size;
>> +		u32		load_options;
>> +		u32		image_base;
>> +		__aligned_u64	image_size;
>> +		u32		image_code_type;
>> +		u32		image_data_type;
>> +		u32		unload;
>> +	} mixed_mode;
>> +} efi_loaded_image_t;
> 
> Is the 32-bit mode used (in later commits)? Why not reuse
> efi_loaded_image_64_t only and make sure it's the one expected.
> 

Good point.

I think it's only x86 that might make use of mixed_mode and I am not 
sure there is a use for it in KUT, I will remove it and if anyone ever 
needs it they can add it then.

Thanks,

Nikos


>> +
>>   #define efi_bs_call(func, ...) efi_system_table->boottime->func(__VA_ARGS__)
>>   #define efi_rs_call(func, ...) efi_system_table->runtime->func(__VA_ARGS__)
>>   
>> diff --git a/lib/stdlib.h b/lib/stdlib.h
>> index 28496d7..2c524d7 100644
>> --- a/lib/stdlib.h
>> +++ b/lib/stdlib.h
>> @@ -7,6 +7,7 @@
>>   #ifndef _STDLIB_H_
>>   #define _STDLIB_H_
>>   
>> +int isspace(int c);
>>   long int strtol(const char *nptr, char **endptr, int base);
>>   unsigned long int strtoul(const char *nptr, char **endptr, int base);
>>   long long int strtoll(const char *nptr, char **endptr, int base);
>> diff --git a/lib/efi.c b/lib/efi.c
>> index 64cc978..5341942 100644
>> --- a/lib/efi.c
>> +++ b/lib/efi.c
>> @@ -8,6 +8,7 @@
>>    */
>>   
>>   #include "efi.h"
>> +#include <stdlib.h>
>>   #include <libcflat.h>
>>   #include <asm/setup.h>
>>   
>> @@ -96,6 +97,97 @@ static void efi_exit(efi_status_t code)
>>   	efi_rs_call(reset_system, EFI_RESET_SHUTDOWN, code, 0, NULL);
>>   }
>>   
>> +static void efi_cmdline_to_argv(char *cmdline_ptr)
>> +{
>> +	char *c = cmdline_ptr;
>> +	bool narg = true;
>> +	while (*c) {
>> +		if (isspace(*c)) {
>> +			*c = '\0';
>> +			narg = true;
>> +		} else if (narg) {
>> +			__argv[__argc++] = c;
>> +			narg = false;
>> +		}
>> +		c++;
>> +	}
>> +}
>> +
>> +static char *efi_convert_cmdline(efi_loaded_image_t *image, int *cmd_line_len)
> 
> Mention that this was adapted from drivers/firmware/efi/libstub/efi-stub.c.
> 
>> +{
>> +	const u16 *s2;
>> +	unsigned long cmdline_addr = 0;
>> +	int options_chars = efi_table_attr(image, load_options_size) / 2;
>> +	const u16 *options = efi_table_attr(image, load_options);
>> +	int options_bytes = 0, safe_options_bytes = 0;  /* UTF-8 bytes */
>> +	bool in_quote = false;
>> +	efi_status_t status;
>> +	const int COMMAND_LINE_SIZE = 2048;
>> +
>> +	if (options) {
>> +		s2 = options;
>> +		while (options_bytes < COMMAND_LINE_SIZE && options_chars--) {
>> +			u16 c = *s2++;
>> +
>> +			if (c < 0x80) {
>> +				if (c == L'\0' || c == L'\n')
>> +					break;
>> +				if (c == L'"')
>> +					in_quote = !in_quote;
>> +				else if (!in_quote && isspace((char)c))
>> +					safe_options_bytes = options_bytes;
>> +
>> +				options_bytes++;
>> +				continue;
>> +			}
>> +
>> +			/*
>> +			 * Get the number of UTF-8 bytes corresponding to a
>> +			 * UTF-16 character.
>> +			 * The first part handles everything in the BMP.
>> +			 */
>> +			options_bytes += 2 + (c >= 0x800);
>> +			/*
>> +			 * Add one more byte for valid surrogate pairs. Invalid
>> +			 * surrogates will be replaced with 0xfffd and take up
>> +			 * only 3 bytes.
>> +			 */
>> +			if ((c & 0xfc00) == 0xd800) {
>> +				/*
>> +				 * If the very last word is a high surrogate,
>> +				 * we must ignore it since we can't access the
>> +				 * low surrogate.
>> +				 */
>> +				if (!options_chars) {
>> +					options_bytes -= 3;
>> +				} else if ((*s2 & 0xfc00) == 0xdc00) {
>> +					options_bytes++;
>> +					options_chars--;
>> +					s2++;
>> +				}
>> +			}
>> +		}
>> +		if (options_bytes >= COMMAND_LINE_SIZE) {
>> +			options_bytes = safe_options_bytes;
>> +			printf("Command line is too long: truncated to %d bytes\n",
>> +			       options_bytes);
>> +		}
>> +        }
>> +
>> +	options_bytes++;        /* NUL termination */
>> +
>> +	status = efi_bs_call(allocate_pool, EFI_LOADER_DATA, options_bytes,
>> +			     (void **)&cmdline_addr);
>> +	if (status != EFI_SUCCESS)
>> +		return NULL;
>> +
>> +	snprintf((char *)cmdline_addr, options_bytes, "%.*ls",
>> +		 options_bytes - 1, options);
>> +
>> +	*cmd_line_len = options_bytes;
>> +	return (char *)cmdline_addr;
>> +}
>> +
>>   efi_status_t efi_main(efi_handle_t handle, efi_system_table_t *sys_tab)
>>   {
>>   	int ret;
>> @@ -109,6 +201,37 @@ efi_status_t efi_main(efi_handle_t handle, efi_system_table_t *sys_tab)
>>   	unsigned long map_size = 0, desc_size = 0, key = 0, buff_size = 0;
>>   	u32 desc_ver;
>>   
>> +	/* Helper variables needed to get the cmdline */
>> +	efi_loaded_image_t *image;
>> +	efi_guid_t loaded_image_proto = LOADED_IMAGE_PROTOCOL_GUID;
>> +	char *cmdline_ptr = NULL;
>> +	int cmdline_size = 0;
>> +
>> +	/*
>> +	 * Get a handle to the loaded image protocol.  This is used to get
>> +	 * information about the running image, such as size and the command
>> +	 * line.
>> +	 */
>> +	status = efi_bs_call(handle_protocol, handle, &loaded_image_proto,
>> +			     (void *)&image);
>> +	if (status != EFI_SUCCESS) {
>> +		printf("Failed to get loaded image protocol\n");
>> +		goto efi_main_error;
>> +	}
>> +
>> +	/*
>> +	 * Get the command line from EFI, using the LOADED_IMAGE
>> +	 * protocol. We are going to copy the command line into the
>> +	 * device tree, so this can be allocated anywhere.
> 
> Does the "device tree" comment still make sense?
> 
>> +	 */
>> +	cmdline_ptr = efi_convert_cmdline(image, &cmdline_size);
>> +	if (!cmdline_ptr) {
>> +		printf("getting command line via LOADED_IMAGE_PROTOCOL\n");
>> +		status = EFI_OUT_OF_RESOURCES;
>> +		goto efi_main_error;
>> +	}
>> +	efi_cmdline_to_argv(cmdline_ptr);
>> +
>>   	/* Set up efi_bootinfo */
>>   	efi_bootinfo.mem_map.map = &map;
>>   	efi_bootinfo.mem_map.map_size = &map_size;
>> diff --git a/lib/string.c b/lib/string.c
>> index 27106da..b191ab1 100644
>> --- a/lib/string.c
>> +++ b/lib/string.c
>> @@ -163,7 +163,7 @@ void *memchr(const void *s, int c, size_t n)
>>       return NULL;
>>   }
>>   
>> -static int isspace(int c)
>> +int isspace(int c)
>>   {
>>       return c == ' ' || c == '\t' || c == '\r' || c == '\n' || c == '\v' || c == '\f';
>>   }
>> -- 
>> 2.25.1
>>

Re: [kvm-unit-tests PATCH v2 16/23] arm/arm64: Add a setup sequence for systems that boot through EFI

[Nikos Nikoleris]

Hi Alex,

On 13/05/2022 14:31, Alexandru Elisei wrote:
> Hi,
> 
> On Fri, May 06, 2022 at 09:55:58PM +0100, Nikos Nikoleris wrote:
>> This change implements an alternative setup sequence for the system
>> when we are booting through EFI. The memory map is discovered through
>> EFI boot services and devices through ACPI.
>>
>> This change is based on a change initially proposed by
>> Andrew Jones <drjones@redhat.com>
>>
>> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
>> ---
>>   lib/linux/efi.h     |   1 +
>>   lib/arm/asm/setup.h |   2 +
>>   lib/arm/setup.c     | 181 +++++++++++++++++++++++++++++++++++++++++++-
>>   arm/cstart.S        |   1 +
>>   arm/cstart64.S      |   1 +
>>   5 files changed, 184 insertions(+), 2 deletions(-)
>>
>> diff --git a/lib/linux/efi.h b/lib/linux/efi.h
>> index 594eaca..9b77c39 100644
>> --- a/lib/linux/efi.h
>> +++ b/lib/linux/efi.h
>> @@ -63,6 +63,7 @@ typedef guid_t efi_guid_t;
>>   	(c) & 0xff, ((c) >> 8) & 0xff, d } }
>>   
>>   #define ACPI_TABLE_GUID EFI_GUID(0xeb9d2d30, 0x2d88, 0x11d3, 0x9a, 0x16, 0x00, 0x90, 0x27, 0x3f, 0xc1, 0x4d)
>> +#define ACPI_20_TABLE_GUID EFI_GUID(0x8868e871, 0xe4f1, 0x11d3,  0xbc, 0x22, 0x00, 0x80, 0xc7, 0x3c, 0x88, 0x81)
>>   
>>   #define LOADED_IMAGE_PROTOCOL_GUID EFI_GUID(0x5b1b31a1, 0x9562, 0x11d2,  0x8e, 0x3f, 0x00, 0xa0, 0xc9, 0x69, 0x72, 0x3b)
>>   
>> diff --git a/lib/arm/asm/setup.h b/lib/arm/asm/setup.h
>> index 64cd379..1a7e734 100644
>> --- a/lib/arm/asm/setup.h
>> +++ b/lib/arm/asm/setup.h
>> @@ -5,6 +5,7 @@
>>    *
>>    * This work is licensed under the terms of the GNU LGPL, version 2.
>>    */
>> +#include <efi.h>
>>   #include <libcflat.h>
>>   #include <asm/page.h>
>>   #include <asm/pgtable-hwdef.h>
>> @@ -37,5 +38,6 @@ extern unsigned int mem_region_get_flags(phys_addr_t paddr);
>>   #define SMP_CACHE_BYTES		L1_CACHE_BYTES
>>   
>>   void setup(const void *fdt, phys_addr_t freemem_start);
>> +efi_status_t setup_efi(efi_bootinfo_t *efi_bootinfo);
>>   
>>   #endif /* _ASMARM_SETUP_H_ */
>> diff --git a/lib/arm/setup.c b/lib/arm/setup.c
>> index 2d67292..542e2ff 100644
>> --- a/lib/arm/setup.c
>> +++ b/lib/arm/setup.c
>> @@ -34,7 +34,7 @@
>>   #define NR_EXTRA_MEM_REGIONS	16
>>   #define NR_INITIAL_MEM_REGIONS	(MAX_DT_MEM_REGIONS + NR_EXTRA_MEM_REGIONS)
>>   
>> -extern unsigned long _etext;
>> +extern unsigned long _text, _etext, _data, _edata;
>>   
>>   char *initrd;
>>   u32 initrd_size;
>> @@ -44,7 +44,10 @@ int nr_cpus;
>>   
>>   static struct mem_region __initial_mem_regions[NR_INITIAL_MEM_REGIONS + 1];
>>   struct mem_region *mem_regions = __initial_mem_regions;
>> -phys_addr_t __phys_offset, __phys_end;
>> +phys_addr_t __phys_offset = (phys_addr_t)-1, __phys_end = 0;
>> +
>> +extern void exceptions_init(void);
>> +extern void asm_mmu_disable(void);
>>   
>>   int mpidr_to_cpu(uint64_t mpidr)
>>   {
>> @@ -272,3 +275,177 @@ void setup(const void *fdt, phys_addr_t freemem_start)
>>   	if (!(auxinfo.flags & AUXINFO_MMU_OFF))
>>   		setup_vm();
>>   }
>> +
>> +#ifdef CONFIG_EFI
>> +
>> +#include <efi.h>
>> +
>> +static efi_status_t setup_rsdp(efi_bootinfo_t *efi_bootinfo)
>> +{
>> +	efi_status_t status;
>> +	struct rsdp_descriptor *rsdp;
>> +
>> +	/*
>> +	 * RSDP resides in an EFI_ACPI_RECLAIM_MEMORY region, which is not used
>> +	 * by kvm-unit-tests arm64 memory allocator. So it is not necessary to
>> +	 * copy the data structure to another memory region to prevent
>> +	 * unintentional overwrite.
>> +	 */
>> +	status = efi_get_system_config_table(ACPI_20_TABLE_GUID, (void **)&rsdp);
>> +	if (status != EFI_SUCCESS)
>> +		return status;
>> +
>> +	set_efi_rsdp(rsdp);
>> +
>> +	return EFI_SUCCESS;
>> +}
>> +
>> +static efi_status_t efi_mem_init(efi_bootinfo_t *efi_bootinfo)
>> +{
>> +	int i;
>> +	unsigned long free_mem_pages = 0;
>> +	unsigned long free_mem_start = 0;
>> +	struct efi_boot_memmap *map = &(efi_bootinfo->mem_map);
>> +	efi_memory_desc_t *buffer = *map->map;
>> +	efi_memory_desc_t *d = NULL;
>> +	phys_addr_t base, top;
>> +	struct mem_region *r;
>> +	uintptr_t text = (uintptr_t)&_text, etext = __ALIGN((uintptr_t)&_etext, 4096);
>> +	uintptr_t data = (uintptr_t)&_data, edata = __ALIGN((uintptr_t)&_edata, 4096);
>> +
>> +	/*
>> +	 * Record the largest free EFI_CONVENTIONAL_MEMORY region
>> +	 * which will be used to set up the memory allocator, so that
>> +	 * the memory allocator can work in the largest free
>> +	 * continuous memory region.
>> +	 */
>> +	for (i = 0, r = &mem_regions[0]; i < *(map->map_size); i += *(map->desc_size), ++r) {
>> +		d = (efi_memory_desc_t *)(&((u8 *)buffer)[i]);
>> +
>> +		r->start = d->phys_addr;
>> +		r->end = d->phys_addr + d->num_pages * EFI_PAGE_SIZE;
>> +
>> +		switch (d->type) {
>> +		case EFI_RESERVED_TYPE:
>> +		case EFI_LOADER_DATA:
>> +		case EFI_BOOT_SERVICES_CODE:
>> +		case EFI_BOOT_SERVICES_DATA:
>> +		case EFI_RUNTIME_SERVICES_CODE:
>> +		case EFI_RUNTIME_SERVICES_DATA:
>> +		case EFI_UNUSABLE_MEMORY:
>> +		case EFI_ACPI_RECLAIM_MEMORY:
>> +		case EFI_ACPI_MEMORY_NVS:
>> +		case EFI_PAL_CODE:
>> +			r->flags = MR_F_RESERVED;
>> +			break;
>> +		case EFI_MEMORY_MAPPED_IO:
>> +		case EFI_MEMORY_MAPPED_IO_PORT_SPACE:
>> +			r->flags = MR_F_IO;
>> +			break;
>> +		case EFI_LOADER_CODE:
>> +			if (r->start <= text && r->end > text) {
>> +				/* This is the unit test region. Flag the code separately. */
>> +				phys_addr_t tmp = r->end;
>> +
>> +				assert(etext <= data);
>> +				assert(edata <= r->end);
>> +				r->flags = MR_F_CODE;
>> +				r->end = data;
>> +				++r;
>> +				r->start = data;
>> +				r->end = tmp;
>> +			} else {
>> +				r->flags = MR_F_RESERVED;
>> +			}
>> +			break;
>> +		case EFI_CONVENTIONAL_MEMORY:
>> +			if (free_mem_pages < d->num_pages) {
>> +				free_mem_pages = d->num_pages;
>> +				free_mem_start = d->phys_addr;
>> +			}
>> +			break;
>> +		}
>> +
>> +		if (!(r->flags & MR_F_IO)) {
>> +			if (r->start < __phys_offset)
>> +				__phys_offset = r->start;
>> +			if (r->end > __phys_end)
>> +				__phys_end = r->end;
>> +		}
>> +	}
>> +	__phys_end &= PHYS_MASK;
>> +	asm_mmu_disable();
> 
> And here's why moving the dcache clean + invalidate *before* turning the MMU off
> is fixing something: __phys_offset and __phys_end are written with MMU on and
> are still in the cache; when the CPU reads them with the MMU off it gets back
> stale values and the CMO doesn't execute correctly.
> 
> Doing a dcache clean for just those two variables should be enough and you can
> drop patch #12 ("arm/arm64: mmu_disable: Clean and invalidate before
> disabling").
> 

I think, there is much more that we need to clean, we're still using the 
same stack, we've loaded the code with the MMU on and we're using some 
memory up until the point we got here. I don't think, it would be safe 
to clean only __phys_offset and __phys_end and move on. Unless what 
you're suggesting is to clean __phys_offset and __phys_end after we 
switch the MMU off. But then I have two questions:
* If the CPU can still speculatively load __phys_offset and __phys_end 
at least the invalidate operation is still questionable.
* If we switch off the MMU and then clean the cache, are we causing 
coherence issues by issuing the CMOs with different memory attributes 
(Device-nGnRnE vs Normal Write-back).

Thanks,

Nikos

Re: [kvm-unit-tests PATCH v2 19/23] arm64: Use code from the gnu-efi when booting with EFI

[Nikos Nikoleris]

Hi Ricardo,

Thanks for this, let me go through the idea I had. Please let me know if 
I am missing something.

On 21/06/2022 23:32, Ricardo Koller wrote:
> On Fri, May 06, 2022 at 09:56:01PM +0100, Nikos Nikoleris wrote:
>> arm/efi/crt0-efi-aarch64.S defines the header and the handover
>> sequence from EFI to a efi_main. This change includes the whole file
>> in arm/cstart64.S when we compile with EFI support.
>>
>> In addition, we change the handover code in arm/efi/crt0-efi-aarch64.S
>> to align the stack pointer. This alignment is necessary because we
>> make assumptions about cpu0's stack alignment and most importantly we
>> place its thread_info at the bottom of this stack.
>>
>> Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
>> ---
>>   arm/cstart64.S             |  6 ++++++
>>   arm/efi/crt0-efi-aarch64.S | 17 +++++++++++++++--
>>   2 files changed, 21 insertions(+), 2 deletions(-)
>>
>> diff --git a/arm/cstart64.S b/arm/cstart64.S
>> index 55b41ea..08cf02f 100644
>> --- a/arm/cstart64.S
>> +++ b/arm/cstart64.S
>> @@ -15,6 +15,10 @@
>>   #include <asm/thread_info.h>
>>   #include <asm/sysreg.h>
>>   
>> +#ifdef CONFIG_EFI
>> +#include "efi/crt0-efi-aarch64.S"
>> +#else
>> +
>>   .macro zero_range, tmp1, tmp2
>>   9998:	cmp	\tmp1, \tmp2
>>   	b.eq	9997f
>> @@ -107,6 +111,8 @@ start:
>>   	bl	exit
>>   	b	halt
>>   
>> +#endif
>> +
>>   .text
>>   
>>   /*
>> diff --git a/arm/efi/crt0-efi-aarch64.S b/arm/efi/crt0-efi-aarch64.S
>> index d50e78d..11a062d 100644
>> --- a/arm/efi/crt0-efi-aarch64.S
>> +++ b/arm/efi/crt0-efi-aarch64.S
>> @@ -111,10 +111,19 @@ section_table:
>>   
>>   	.align		12
>>   _start:
>> -	stp		x29, x30, [sp, #-32]!
>> +	stp		x29, x30, [sp, #-16]!
> 
> Is this and the "ldp x29, x30, [sp], #16" change below needed?
> why is #-32 not good?
> 

The stack is full-descending. Here we make space for x29 and x30 in the 
stack (16bytes) and save the two registers

>> +
>> +	// Align sp; this is necessary due to way we store cpu0's thread_info
> 
> /* */ comment style
> 

ack

>>   	mov		x29, sp
>> +	and		x29, x29, #THREAD_MASK
>> +	mov		x30, sp
>> +	mov		sp, x29
>> +	str		x30, [sp, #-32]!
>> +

Here we're making space in the stack for the old sp (x30), x0 and x1 but 
we have to also ensure that the sp is aligned (32bytes). The we store x30.

(As a side note, I could also change this to

+	str		x30, [sp, #-16]!

and change the next stp to do pre-incrementing mode. This might make 
things simpler.)

>> +	mov             x29, sp
>>   
>>   	stp		x0, x1, [sp, #16]
>> +

Here, we use the space we made before to store x0 and x1.

I think, the stack now should look like:

        |   ...  |
        |   x30  |
        |   x29  |
        |   x1   |
        |   x0   |
        |   pad  |
sp ->  | old_sp |


>>   	mov		x2, x0
>>   	mov		x3, x1
>>   	adr		x0, ImageBase
>> @@ -126,5 +135,9 @@ _start:
>>   	ldp		x0, x1, [sp, #16]
>>   	bl		efi_main
>>   
>> -0:	ldp		x29, x30, [sp], #32
>> +	// Restore sp
> 
> /* */ comment style

ack

> 
>> +	ldr		x30, [sp]

I think this should have been:

+	ldr		x30, [sp], #32

Restore x30 from the current sp and free up space in the stack (all 
32bytes).

> 
> I'm not able to understand this. Is this ldr restoring the value pushed
> with "str x30, [sp, #-32]!" above? in that case, shouldn't this be at
> [sp - 32]? But, given that this code is unreachable when efi_main is
> called, do you even need to restore the sp?
> 
>> +	mov             sp, x30
>> +
>> +0:	ldp		x29, x30, [sp], #16

Then, this restores x29 and x30 and frees up the the corresponding space 
in the stack.


I am not sure we shouldn't get to this point and I wanted to properly 
save and restore the register state. I haven't really found what's the 
right/best way to exit from an EFI app and I wanted to allow for 
graceful return from this point. But I am happy to change all this.

Thanks,

Nikos

>>   	ret
>> -- 
>> 2.25.1
>>

Re: [kvm-unit-tests PATCH v2 19/23] arm64: Use code from the gnu-efi when booting with EFI

[Ricardo Koller]

Ni Nikos,

On Mon, Jun 27, 2022 at 06:10:20PM +0100, Nikos Nikoleris wrote:
> Hi Ricardo,
> 
> Thanks for this, let me go through the idea I had. Please let me know if I
> am missing something.
> 
> On 21/06/2022 23:32, Ricardo Koller wrote:
> > On Fri, May 06, 2022 at 09:56:01PM +0100, Nikos Nikoleris wrote:
> > > arm/efi/crt0-efi-aarch64.S defines the header and the handover
> > > sequence from EFI to a efi_main. This change includes the whole file
> > > in arm/cstart64.S when we compile with EFI support.
> > > 
> > > In addition, we change the handover code in arm/efi/crt0-efi-aarch64.S
> > > to align the stack pointer. This alignment is necessary because we
> > > make assumptions about cpu0's stack alignment and most importantly we
> > > place its thread_info at the bottom of this stack.
> > > 
> > > Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> > > ---
> > >   arm/cstart64.S             |  6 ++++++
> > >   arm/efi/crt0-efi-aarch64.S | 17 +++++++++++++++--
> > >   2 files changed, 21 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/arm/cstart64.S b/arm/cstart64.S
> > > index 55b41ea..08cf02f 100644
> > > --- a/arm/cstart64.S
> > > +++ b/arm/cstart64.S
> > > @@ -15,6 +15,10 @@
> > >   #include <asm/thread_info.h>
> > >   #include <asm/sysreg.h>
> > > +#ifdef CONFIG_EFI
> > > +#include "efi/crt0-efi-aarch64.S"
> > > +#else
> > > +
> > >   .macro zero_range, tmp1, tmp2
> > >   9998:	cmp	\tmp1, \tmp2
> > >   	b.eq	9997f
> > > @@ -107,6 +111,8 @@ start:
> > >   	bl	exit
> > >   	b	halt
> > > +#endif
> > > +
> > >   .text
> > >   /*
> > > diff --git a/arm/efi/crt0-efi-aarch64.S b/arm/efi/crt0-efi-aarch64.S
> > > index d50e78d..11a062d 100644
> > > --- a/arm/efi/crt0-efi-aarch64.S
> > > +++ b/arm/efi/crt0-efi-aarch64.S
> > > @@ -111,10 +111,19 @@ section_table:
> > >   	.align		12
> > >   _start:
> > > -	stp		x29, x30, [sp, #-32]!
> > > +	stp		x29, x30, [sp, #-16]!
> > 
> > Is this and the "ldp x29, x30, [sp], #16" change below needed?
> > why is #-32 not good?
> > 
> 
> The stack is full-descending. Here we make space for x29 and x30 in the
> stack (16bytes) and save the two registers
> 

Got it, was thinking mainly in terms of optimizing for less changes.
But I see your point; plus it makes the restoring simpler.

> > > +
> > > +	// Align sp; this is necessary due to way we store cpu0's thread_info
> > 
> > /* */ comment style
> > 
> 
> ack
> 
> > >   	mov		x29, sp
> > > +	and		x29, x29, #THREAD_MASK
> > > +	mov		x30, sp
> > > +	mov		sp, x29
> > > +	str		x30, [sp, #-32]!
> > > +
> 
> Here we're making space in the stack for the old sp (x30), x0 and x1 but we
> have to also ensure that the sp is aligned (32bytes). The we store x30.
> 
> (As a side note, I could also change this to
> 
> +	str		x30, [sp, #-16]!
> 
> and change the next stp to do pre-incrementing mode. This might make things
> simpler.)

Definitely, it would look like two regular pushes.

> 
> > > +	mov             x29, sp
> > >   	stp		x0, x1, [sp, #16]
> > > +
> 
> Here, we use the space we made before to store x0 and x1.
> 
> I think, the stack now should look like:
> 
>        |   ...  |
>        |   x30  |
>        |   x29  |

possibly some more extra space due to #THREAD_MASK

>        |   x1   |
>        |   x0   |
>        |   pad  |
> sp ->  | old_sp |

Thanks for this!

> 
> 
> > >   	mov		x2, x0
> > >   	mov		x3, x1
> > >   	adr		x0, ImageBase
> > > @@ -126,5 +135,9 @@ _start:
> > >   	ldp		x0, x1, [sp, #16]
> > >   	bl		efi_main
> > > -0:	ldp		x29, x30, [sp], #32
> > > +	// Restore sp
> > 
> > /* */ comment style
> 
> ack
> 
> > 
> > > +	ldr		x30, [sp]
> 
> I think this should have been:
> 
> +	ldr		x30, [sp], #32
> 
> Restore x30 from the current sp and free up space in the stack (all
> 32bytes).
> 

Now I get it. That's not needed actually, as you are restoring sp from
old_sp below to this address:

          |   ...  |
          |   x30  |
sp ->     |   x29  |

(old_sp was the sp after pushing x29,x30)

> > 
> > I'm not able to understand this. Is this ldr restoring the value pushed
> > with "str x30, [sp, #-32]!" above? in that case, shouldn't this be at
> > [sp - 32]? But, given that this code is unreachable when efi_main is
> > called, do you even need to restore the sp?
> > 
> > > +	mov             sp, x30
> > > +
> > > +0:	ldp		x29, x30, [sp], #16
> 
> Then, this restores x29 and x30 and frees up the the corresponding space in
> the stack.
> 
> 
> I am not sure we shouldn't get to this point and I wanted to properly save
> and restore the register state. I haven't really found what's the right/best
> way to exit from an EFI app and I wanted to allow for graceful return from
> this point. But I am happy to change all this.

Yes, let's keep it just in case.

> 
> Thanks,
> 
> Nikos
> 
> > >   	ret
> > > -- 
> > > 2.25.1
> > > 

Thanks!
Ricardo

Re: [kvm-unit-tests PATCH v2 23/23] arm64: Add an efi/run script

[Ricardo Koller]

On Wed, Jun 22, 2022 at 03:13:56PM +0100, Nikos Nikoleris wrote:
> On 22/06/2022 00:09, Ricardo Koller wrote:
> > On Fri, May 06, 2022 at 09:56:05PM +0100, Nikos Nikoleris wrote:
> > > This change adds a efi/run script inspired by the one in x86. This
> > > script will setup a folder with the test compiled as an EFI app and a
> > > startup.nsh script. The script launches QEMU providing an image with
> > > EDKII and the path to the folder with the test which is executed
> > > automatically.
> > > 
> > > For example:
> > > 
> > > $> ./arm/efi/run ./arm/selftest.efi setup smp=2 mem=256
> > > 
> > > Signed-off-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
> > > ---
> > >   scripts/runtime.bash | 14 +++++-----
> > >   arm/efi/run          | 61 ++++++++++++++++++++++++++++++++++++++++++++
> > >   arm/run              |  8 ++++--
> > >   arm/Makefile.common  |  1 +
> > >   arm/dummy.c          |  4 +++
> > >   5 files changed, 78 insertions(+), 10 deletions(-)
> > >   create mode 100755 arm/efi/run
> > >   create mode 100644 arm/dummy.c
> > > 
> > > diff --git a/scripts/runtime.bash b/scripts/runtime.bash
> > > index 7d0180b..dc28f24 100644
> > > --- a/scripts/runtime.bash
> > > +++ b/scripts/runtime.bash
> > > @@ -131,14 +131,12 @@ function run()
> > >       fi
> > > 
> > >       last_line=$(premature_failure > >(tail -1)) && {
> > > -        skip=true
> > > -        if [ "${CONFIG_EFI}" == "y" ] && [[ "${last_line}" =~ "enabling apic" ]]; then
> > > -            skip=false
> > > -        fi
> > > -        if [ ${skip} == true ]; then
> > > -            print_result "SKIP" $testname "" "$last_line"
> > > -            return 77
> > > -        fi
> > > +        if [ "${CONFIG_EFI}" == "y" ] && [ "${ARCH}" = x86_64 ]; then
> > > +            if ! [[ "${last_line}" =~ "enabling apic" ]]; then
> > > +                    print_result "SKIP" $testname "" "$last_line"
> > > +                    return 77
> > > +            fi
> > > +    fi
> > >       }
> > > 
> > >       cmdline=$(get_cmdline $kernel)
> > > diff --git a/arm/efi/run b/arm/efi/run
> > > new file mode 100755
> > > index 0000000..dfff717
> > > --- /dev/null
> > > +++ b/arm/efi/run
> > > @@ -0,0 +1,61 @@
> > > +#!/bin/bash
> > > +
> > > +set -e
> > > +
> > > +if [ $# -eq 0 ]; then
> > > +    echo "Usage $0 TEST_CASE [QEMU_ARGS]"
> > > +    exit 2
> > > +fi
> > > +
> > > +if [ ! -f config.mak ]; then
> > > +    echo "run './configure --enable-efi && make' first. See ./configure -h"
> > > +    exit 2
> > > +fi
> > > +source config.mak
> > > +source scripts/arch-run.bash
> > > +source scripts/common.bash
> > > +
> > > +: "${EFI_SRC:=$(realpath "$(dirname "$0")/../")}"
> > > +: "${EFI_UEFI:=/usr/share/qemu-efi-aarch64/QEMU_EFI.fd}"
> > > +: "${EFI_TEST:=efi-tests}"
> > > +: "${EFI_CASE:=$(basename $1 .efi)}"
> > > +
> > > +if [ ! -f "$EFI_UEFI" ]; then
> > > +    echo "UEFI firmware not found: $EFI_UEFI"
> > > +    echo "Please install the UEFI firmware to this path"
> > > +    echo "Or specify the correct path with the env variable EFI_UEFI"
> > > +    exit 2
> > > +fi
> > > +
> > > +# Remove the TEST_CASE from $@
> > > +shift 1
> > > +
> > > +# Fish out the arguments for the test, they should be the next string
> > > +# after the "-append" option
> > > +qemu_args=()
> > > +cmd_args=()
> > > +while (( "$#" )); do
> > > +    if [ "$1" = "-append" ]; then
> > > +            cmd_args=$2
> > > +            shift 2
> > 
> > Does this work with params like this (2 words)?
> > 
> >       [pmu-cycle-counter]
> >       file = pmu.flat
> >       groups = pmu
> >       extra_params = -append 'cycle-counter 0'
> > 
> 
> I think it does. cmd_args will take the value of the string that follows
> append and in this case whatever is between the single/double quotes.

Just in case I tested it on bash and it works as expected.

So, with that:

Reviewed-by: Ricardo Koller <ricarkol@google.com>