[PATCH v4 01/23] xen: allow console_io hypercalls from certain DomUs

[PATCH v4 01/23] xen: allow console_io hypercalls from certain DomUs

[Stefano Stabellini]

Introduce an is_console option to allow certain classes of domUs to use
the Xen console. Specifically, it will be used to give console access to
all domUs started from Xen from information on device tree.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
Acked-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>
CC: andrew.cooper3@citrix.com
CC: George.Dunlap@eu.citrix.com
CC: ian.jackson@eu.citrix.com
CC: jbeulich@suse.com
CC: konrad.wilk@oracle.com
CC: tim@xen.org
CC: wei.liu2@citrix.com
CC: dgdegra@tycho.nsa.gov
---
Changes in v3:
- remove changes to hooks.c

Changes in v2:
- introduce is_console
- remove #ifdefs
---
 xen/include/xen/sched.h | 2 ++
 xen/include/xsm/dummy.h | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 0ba80cb..abcc74e 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -379,6 +379,8 @@ struct domain
     bool             auto_node_affinity;
     /* Is this guest fully privileged (aka dom0)? */
     bool             is_privileged;
+    /* Can this guest access the Xen console? */
+    bool             is_console;
     /* Is this a xenstore domain (not dom0)? */
     bool             is_xenstore;
     /* Domain's VCPUs are pinned 1:1 to physical CPUs? */
diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
index b0ac1f6..29d7b50 100644
--- a/xen/include/xsm/dummy.h
+++ b/xen/include/xsm/dummy.h
@@ -230,6 +230,8 @@ static XSM_INLINE int xsm_memory_stat_reservation(XSM_DEFAULT_ARG struct domain
 static XSM_INLINE int xsm_console_io(XSM_DEFAULT_ARG struct domain *d, int cmd)
 {
     XSM_ASSERT_ACTION(XSM_OTHER);
+    if ( d->is_console )
+        return xsm_default_action(XSM_HOOK, d, NULL);
 #ifdef CONFIG_VERBOSE_DEBUG
     if ( cmd == CONSOLEIO_write )
         return xsm_default_action(XSM_HOOK, d, NULL);
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v4 02/23] xen/arm: extend device tree based multiboot protocol

[Stefano Stabellini]

Extend the existing device tree based multiboot protocol to include
information regarding multiple domains to boot.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>

---
Changes in v4:
- memory is 64bit

Changes in v3:
- remove "xen,initial-domain" for now
- make vpl011 an empty property
- memory in KBs

Changes in v2:
- lower case kernel
- rename mem to memory
- mandate cpus and memory
- replace domU-kernel with kernel and domU-ramdisk with ramdisk
- rename xen,domU with xen,domain
- add info about dom0
- switch memory and cpus to integers
- remove defaults
- add vpl011
---
 docs/misc/arm/device-tree/booting.txt | 107 ++++++++++++++++++++++++++++++++++
 1 file changed, 107 insertions(+)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index ce2d0dc..317a9e9 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -119,3 +119,110 @@ For those you would hardcode the Xen commandline in the DTB under
 line by writing bootargs (as for native Linux).
 A Xen-aware bootloader would set xen,xen-bootargs for Xen, xen,dom0-bootargs
 for Dom0 and bootargs for native Linux.
+
+
+Creating Multiple Domains directly from Xen
+===========================================
+
+It is possible to have Xen create other domains, in addition to dom0,
+out of the information provided via device tree. A kernel and initrd
+(optional) need to be specified for each guest.
+
+For each domain to be created there needs to be one node under /chosen
+with the following properties:
+
+- compatible
+
+    For domUs: "xen,domain"
+
+- memory
+
+	A 64-bit integer specifying the amount of kilobytes of RAM to
+    allocate to the guest.
+
+- cpus
+
+    An integer specifying the number of vcpus to allocate to the guest.
+
+- vpl011
+
+    An empty property to enable/disable a virtual pl011 for the guest to use.
+
+- #address-cells and #size-cells
+
+    Both #address-cells and #size-cells need to be specified because
+    both sub-nodes (described shortly) have reg properties.
+
+Under the "xen,domain" compatible node, one or more sub-nodes are present
+for the DomU kernel and ramdisk.
+
+The kernel sub-node has the following properties:
+
+- compatible
+
+    "multiboot,kernel"
+
+- reg
+
+    Specifies the physical address of the kernel in RAM and its
+    length.
+
+- bootargs (optional)
+
+    Command line parameters for the guest kernel.
+
+The ramdisk sub-node has the following properties:
+
+- compatible
+
+    "multiboot,ramdisk"
+
+- reg
+
+    Specifies the physical address of the ramdisk in RAM and its
+    length.
+
+
+Example
+=======
+
+chosen {
+    domU1 {
+        compatible = "xen,domain";
+        #address-cells = <0x2>;
+        #size-cells = <0x1>;
+        memory = <0 131072>;
+        cpus = <2>;
+        vpl011;
+
+        module@0x4a000000 {
+            compatible = "multiboot,kernel";
+            reg = <0x0 0x4a000000 0xffffff>;
+            bootargs = "console=ttyAMA0 init=/bin/sh";
+        };
+
+        module@0x4b000000 {
+            compatible = "multiboot,ramdisk";
+            reg = <0x0 0x4b000000 0xffffff>;
+        };
+    };
+
+    domU2 {
+        compatible = "xen,domain";
+        #address-cells = <0x2>;
+        #size-cells = <0x1>;
+        memory = <0 65536>;
+        cpus = <1>;
+
+        module@0x4c000000 {
+            compatible = "multiboot,kernel";
+            reg = <0x0 0x4c000000 0xffffff>;
+            bootargs = "console=ttyAMA0 init=/bin/sh";
+        };
+
+        module@0x4d000000 {
+            compatible = "multiboot,ramdisk";
+            reg = <0x0 0x4d000000 0xffffff>;
+        };
+    };
+};
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v4 03/23] xen/arm: document dom0less

[Stefano Stabellini]

Add a new document to provide information on how to use dom0less related
features and their current limitations.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>

---
Changes in v4:
- rename to .txt
- improve wording

Changes in v3:
- add patch
---
 docs/misc/arm/dom0less.txt | 47 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)
 create mode 100644 docs/misc/arm/dom0less.txt

diff --git a/docs/misc/arm/dom0less.txt b/docs/misc/arm/dom0less.txt
new file mode 100644
index 0000000..df96b41
--- /dev/null
+++ b/docs/misc/arm/dom0less.txt
@@ -0,0 +1,47 @@
+Dom0less
+========
+
+"Dom0less" is a set of Xen features that enable the deployment of a Xen
+system without an hardware domain (often referred to as "dom0"). Each
+feature can be used independently from the others, unless otherwise
+stated.
+
+Booting Multiple Domains from Device Tree
+=========================================
+
+This feature enables Xen to create a set of DomUs alongside Dom0 at boot
+time. Information about the DomUs to be created by Xen is passed to the
+hypervisor via Device Tree. Specifically, the existing Device Tree based
+Multiboot specification has been extended to allow for multiple domains
+to be passed to Xen. See docs/misc/arm/device-tree/booting.txt for more
+information about the Multiboot specification and how to use it.
+
+Instead of waiting for Dom0 to be fully booted and the Xen tools to
+become available, domains created by Xen this way are started in
+parallel to Dom0. Hence, their boot time is typically much shorter.
+
+Domains started by Xen at boot time currently have the following
+limitations:
+
+- they cannot be properly shutdown or rebooted using xl
+If one of them crashes, the whole platform should be rebooted.
+
+- some xl operations might not work as expected
+xl is meant to be used with domains that have been created by it. Using
+xl with domains started by Xen at boot might not work as expected.
+
+- the GIC version is the native version
+In absence of other information, the GIC version exposed to the domains
+started by Xen at boot is the same as the native GIC version.
+
+- no PV drivers
+There is no support for PV devices at the moment. All devices need to be
+statically assigned to guests.
+
+- vCPU pinning
+Pinning vCPUs of domains started by Xen at boot can be done from dom0,
+using `xl vcpu-pin' as usual. It is not currently possible to configure
+vCPU pinning for domains other than dom0 without dom0. However, the NULL
+scheduler can be selected by passing `sched=null' to the Xen command
+line. The NULL scheduler automatically assigns and pins vCPUs to pCPUs,
+but the vCPU-pCPU assignments cannot be configured.
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v4 04/23] xen/arm: increase MAX_MODULES

[Stefano Stabellini]

Xen boot modules need to account not just for Dom0 but also for a few
potential DomUs, each of them coming with their own kernel and initrd.
Increase MAX_MODULES to 32 to allow for more DomUs.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
Reviewed-by: Doug Goldstein <cardoe@cardoe.com>
---
 xen/include/asm-arm/setup.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
index 0cc3330..f1e4a3f 100644
--- a/xen/include/asm-arm/setup.h
+++ b/xen/include/asm-arm/setup.h
@@ -8,7 +8,7 @@
 
 #define NR_MEM_BANKS 128
 
-#define MAX_MODULES 5 /* Current maximum useful modules */
+#define MAX_MODULES 32 /* Current maximum useful modules */
 
 typedef enum {
     BOOTMOD_XEN,
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v4 05/23] xen/arm: introduce bootcmdlines

[Stefano Stabellini]

Introduce a new array to store the cmdline of each boot module. It is
separate from struct bootmodules. Remove the cmdline field from struct
boot_module. This way, kernels and initrds with the same address in
memory can share struct bootmodule (important because we want them to be
free'd only once), but they can still have their separate bootcmdline
entries.

Add a dt_name field to struct bootcmdline to make it easier to find the
correct entry. Store the name of the "xen,domain" compatible node (for
example "Dom1"). This is a better choice compared to the name of the
"multiboot,kernel" compatible node, because their names are not unique.
For instance there can be more than one "module@0x4c000000" in the
system, but there can only be one "/chosen/Dom1".

Add a pointer to struct kernel_info to point to the cmdline for a given
kernel.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>

---

Changes in v4:
- check that the multiboot node is under /chosen
- use cmd and cmds as variable names for struct bootcmdline and struct
  bootcmdline*
- fix printk
- use ASSERT instea of panic
- do not add empty cmdline entries
- add more debug printks to early_print_info
- code style fixes
- add comment on DT_MAX_NAME
- increase DT_MAX_NAME to 41
- make nr_mods unsigned int

Changes in v3:
- introduce bootcmdlines
- do not modify boot_fdt_cmdline
- add comments

Changes in v2:
- new patch
---
 xen/arch/arm/bootfdt.c      | 82 +++++++++++++++++++++++++++++++++------------
 xen/arch/arm/domain_build.c |  8 +++--
 xen/arch/arm/kernel.h       |  1 +
 xen/arch/arm/setup.c        | 24 +++++++++----
 xen/include/asm-arm/setup.h | 17 ++++++++--
 5 files changed, 99 insertions(+), 33 deletions(-)

diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
index 8eba42c..273032b 100644
--- a/xen/arch/arm/bootfdt.c
+++ b/xen/arch/arm/bootfdt.c
@@ -163,6 +163,37 @@ static void __init process_memory_node(const void *fdt, int node,
     }
 }
 
+static void __init add_boot_cmdline(const void *fdt, int node,
+                                    const char *name, bootmodule_kind kind)
+{
+    struct bootcmdlines *cmds = &bootinfo.cmdlines;
+    struct bootcmdline *cmd;
+    const struct fdt_property *prop;
+    int len;
+    const char *cmdline;
+
+    if ( cmds->nr_mods == MAX_MODULES )
+    {
+        printk("Ignoring %s cmdline (too many)\n", name);
+        return;
+    }
+
+    prop = fdt_get_property(fdt, node, "bootargs", &len);
+    if ( !prop )
+        return;
+
+    cmd = &cmds->cmdline[cmds->nr_mods++];
+    cmd->kind = kind;
+
+    ASSERT(strlen(name) <= DT_MAX_NAME);
+    safe_strcpy(cmd->dt_name, name);
+
+    if ( len > BOOTMOD_MAX_CMDLINE )
+        panic("module %s command line too long\n", name);
+    cmdline = prop->data;
+    safe_strcpy(cmd->cmdline, cmdline);
+}
+
 static void __init process_multiboot_node(const void *fdt, int node,
                                           const char *name,
                                           u32 address_cells, u32 size_cells)
@@ -172,8 +203,20 @@ static void __init process_multiboot_node(const void *fdt, int node,
     const __be32 *cell;
     bootmodule_kind kind;
     paddr_t start, size;
-    const char *cmdline;
-    int len;
+    int len = sizeof("/chosen");
+    char path[8]; /* sizeof "/chosen" */
+    int parent_node;
+
+    parent_node = fdt_parent_offset(fdt, node);
+    ASSERT(parent_node >= 0);
+
+    /* Check that the node is under "chosen" */
+    fdt_get_path(fdt, node, path, len);
+    if ( strncmp(path, "/chosen", len - 1) )
+    {
+        printk("DEBUG %s %s\n",name,path);
+        return;
+    }
 
     prop = fdt_get_property(fdt, node, "reg", &len);
     if ( !prop )
@@ -220,17 +263,8 @@ static void __init process_multiboot_node(const void *fdt, int node,
             kind = BOOTMOD_XSM;
     }
 
-    prop = fdt_get_property(fdt, node, "bootargs", &len);
-    if ( prop )
-    {
-        if ( len > BOOTMOD_MAX_CMDLINE )
-            panic("module %s command line too long\n", name);
-        cmdline = prop->data;
-    }
-    else
-        cmdline = NULL;
-
-    add_boot_module(kind, start, size, cmdline);
+    add_boot_module(kind, start, size);
+    add_boot_cmdline(fdt, node, fdt_get_name(fdt, parent_node, &len), kind);
 }
 
 static void __init process_chosen_node(const void *fdt, int node,
@@ -276,7 +310,7 @@ static void __init process_chosen_node(const void *fdt, int node,
 
     printk("Initrd %"PRIpaddr"-%"PRIpaddr"\n", start, end);
 
-    add_boot_module(BOOTMOD_RAMDISK, start, end-start, NULL);
+    add_boot_module(BOOTMOD_RAMDISK, start, end-start);
 }
 
 static int __init early_scan_node(const void *fdt,
@@ -299,6 +333,7 @@ static void __init early_print_info(void)
 {
     struct meminfo *mi = &bootinfo.mem;
     struct bootmodules *mods = &bootinfo.modules;
+    struct bootcmdlines *cmds = &bootinfo.cmdlines;
     int i, nr_rsvd;
 
     for ( i = 0; i < mi->nr_banks; i++ )
@@ -307,12 +342,12 @@ static void __init early_print_info(void)
                      mi->bank[i].start + mi->bank[i].size - 1);
     printk("\n");
     for ( i = 0 ; i < mods->nr_mods; i++ )
-        printk("MODULE[%d]: %"PRIpaddr" - %"PRIpaddr" %-12s %s\n",
+        printk("MODULE[%d]: %"PRIpaddr" - %"PRIpaddr" %-12s\n",
                      i,
                      mods->module[i].start,
                      mods->module[i].start + mods->module[i].size,
-                     boot_module_kind_as_string(mods->module[i].kind),
-                     mods->module[i].cmdline);
+                     boot_module_kind_as_string(mods->module[i].kind));
+
     nr_rsvd = fdt_num_mem_rsv(device_tree_flattened);
     for ( i = 0; i < nr_rsvd; i++ )
     {
@@ -325,6 +360,11 @@ static void __init early_print_info(void)
                      i, s, e);
     }
     printk("\n");
+    for ( i = 0 ; i < cmds->nr_mods; i++ )
+        printk("CMDLINE[%d]:%s %s\n", i,
+               cmds->cmdline[i].dt_name,
+               &cmds->cmdline[i].cmdline[0]);
+    printk("\n");
 }
 
 /**
@@ -341,7 +381,7 @@ size_t __init boot_fdt_info(const void *fdt, paddr_t paddr)
     if ( ret < 0 )
         panic("No valid device tree\n");
 
-    add_boot_module(BOOTMOD_FDT, paddr, fdt_totalsize(fdt), NULL);
+    add_boot_module(BOOTMOD_FDT, paddr, fdt_totalsize(fdt));
 
     device_tree_for_each_node((void *)fdt, early_scan_node, NULL);
     early_print_info();
@@ -361,11 +401,11 @@ const char *boot_fdt_cmdline(const void *fdt)
     prop = fdt_get_property(fdt, node, "xen,xen-bootargs", NULL);
     if ( prop == NULL )
     {
-        struct bootmodule *dom0_mod =
-            boot_module_find_by_kind(BOOTMOD_KERNEL);
+        struct bootcmdline *dom0_cmdline =
+            boot_cmdline_find_by_kind(BOOTMOD_KERNEL);
 
         if (fdt_get_property(fdt, node, "xen,dom0-bootargs", NULL) ||
-            ( dom0_mod && dom0_mod->cmdline[0] ) )
+            ( dom0_cmdline && dom0_cmdline->cmdline[0] ) )
             prop = fdt_get_property(fdt, node, "bootargs", NULL);
     }
     if ( prop == NULL )
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index f552154..64f8d6b 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -375,7 +375,7 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
     int res = 0;
     int had_dom0_bootargs = 0;
 
-    const struct bootmodule *kernel = kinfo->kernel_bootmodule;
+    const struct bootcmdline *kernel = boot_cmdline_find_by_kind(BOOTMOD_KERNEL);
 
     if ( kernel && kernel->cmdline[0] )
         bootargs = &kernel->cmdline[0];
@@ -952,9 +952,9 @@ static int __init make_chosen_node(const struct kernel_info *kinfo)
     if ( res )
         return res;
 
-    if ( mod && mod->cmdline[0] )
+    if ( kinfo->cmdline && kinfo->cmdline[0] )
     {
-        bootargs = &mod->cmdline[0];
+        bootargs = &kinfo->cmdline[0];
         res = fdt_property(fdt, "bootargs", bootargs, strlen(bootargs) + 1);
         if ( res )
            return res;
@@ -2109,6 +2109,7 @@ static void __init find_gnttab_region(struct domain *d,
 
 int __init construct_dom0(struct domain *d)
 {
+    const struct bootcmdline *kernel = boot_cmdline_find_by_kind(BOOTMOD_KERNEL);
     struct kernel_info kinfo = {};
     struct vcpu *saved_current;
     int rc, i, cpu;
@@ -2154,6 +2155,7 @@ int __init construct_dom0(struct domain *d)
 
 #endif
 
+    kinfo.cmdline = kernel != NULL ? &kernel->cmdline[0] : NULL;
     allocate_memory(d, &kinfo);
     find_gnttab_region(d, &kinfo);
 
diff --git a/xen/arch/arm/kernel.h b/xen/arch/arm/kernel.h
index 47eacb5..39b7828 100644
--- a/xen/arch/arm/kernel.h
+++ b/xen/arch/arm/kernel.h
@@ -29,6 +29,7 @@ struct kernel_info {
 
     /* boot blob load addresses */
     const struct bootmodule *kernel_bootmodule, *initrd_bootmodule;
+    const char* cmdline;
     paddr_t dtb_paddr;
     paddr_t initrd_paddr;
 
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index ea2495a..bc7dd97 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -201,8 +201,7 @@ void __init dt_unreserved_regions(paddr_t s, paddr_t e,
 }
 
 struct bootmodule __init *add_boot_module(bootmodule_kind kind,
-                                          paddr_t start, paddr_t size,
-                                          const char *cmdline)
+                                          paddr_t start, paddr_t size)
 {
     struct bootmodules *mods = &bootinfo.modules;
     struct bootmodule *mod;
@@ -218,10 +217,6 @@ struct bootmodule __init *add_boot_module(bootmodule_kind kind,
     mod->kind = kind;
     mod->start = start;
     mod->size = size;
-    if ( cmdline )
-        safe_strcpy(mod->cmdline, cmdline);
-    else
-        mod->cmdline[0] = 0;
 
     return mod;
 }
@@ -240,6 +235,21 @@ struct bootmodule * __init boot_module_find_by_kind(bootmodule_kind kind)
     return NULL;
 }
 
+struct bootcmdline * __init boot_cmdline_find_by_kind(bootmodule_kind kind)
+{
+    struct bootcmdlines *cmds = &bootinfo.cmdlines;
+    struct bootcmdline *cmd;
+    int i;
+
+    for ( i = 0 ; i < cmds->nr_mods ; i++ )
+    {
+        cmd = &cmds->cmdline[i];
+        if ( cmd->kind == kind )
+            return cmd;
+    }
+    return NULL;
+}
+
 const char * __init boot_module_kind_as_string(bootmodule_kind kind)
 {
     switch ( kind )
@@ -728,7 +738,7 @@ void __init start_xen(unsigned long boot_phys_offset,
     /* Register Xen's load address as a boot module. */
     xen_bootmodule = add_boot_module(BOOTMOD_XEN,
                              (paddr_t)(uintptr_t)(_start + boot_phys_offset),
-                             (paddr_t)(uintptr_t)(_end - _start + 1), NULL);
+                             (paddr_t)(uintptr_t)(_end - _start + 1));
     BUG_ON(!xen_bootmodule);
 
     xen_paddr = get_xen_paddr();
diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
index f1e4a3f..c2ed5cc 100644
--- a/xen/include/asm-arm/setup.h
+++ b/xen/include/asm-arm/setup.h
@@ -35,6 +35,13 @@ struct bootmodule {
     bootmodule_kind kind;
     paddr_t start;
     paddr_t size;
+};
+
+/* DT_MAX_NAME is the node name max length according the DT spec */
+#define DT_MAX_NAME 41
+struct bootcmdline {
+    bootmodule_kind kind;
+    char dt_name[DT_MAX_NAME];
     char cmdline[BOOTMOD_MAX_CMDLINE];
 };
 
@@ -43,9 +50,15 @@ struct bootmodules {
     struct bootmodule module[MAX_MODULES];
 };
 
+struct bootcmdlines {
+    unsigned int nr_mods;
+    struct bootcmdline cmdline[MAX_MODULES];
+};
+
 struct bootinfo {
     struct meminfo mem;
     struct bootmodules modules;
+    struct bootcmdlines cmdlines;
 #ifdef CONFIG_ACPI
     struct meminfo acpi;
 #endif
@@ -78,9 +91,9 @@ size_t __init boot_fdt_info(const void *fdt, paddr_t paddr);
 const char __init *boot_fdt_cmdline(const void *fdt);
 
 struct bootmodule *add_boot_module(bootmodule_kind kind,
-                                   paddr_t start, paddr_t size,
-                                   const char *cmdline);
+                                   paddr_t start, paddr_t size);
 struct bootmodule *boot_module_find_by_kind(bootmodule_kind kind);
+struct bootcmdline *boot_cmdline_find_by_kind(bootmodule_kind kind);
 const char * __init boot_module_kind_as_string(bootmodule_kind kind);
 
 #endif
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v4 06/23] xen/arm: don't add duplicate boot modules, introduce domU flag

[Stefano Stabellini]

Don't add duplicate boot modules (same kind and same start address),
they are freed later, we don't want to introduce double-free errors.

Introduce a domU flag in struct bootmodule and struct bootcmdline. Set
it for kernels and ramdisks of "xen,domain" nodes to avoid getting
confused in kernel_probe, where we try to guess which is the dom0 kernel
and initrd to be compatible with older versions of the multiboot spec.

boot_module_find_by_kind and boot_cmdline_find_by_kind automatically
check for !domU entries (they are only used for non-domU modules).

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>

---
Changes in v4:
- use unsigned int
- better commit message
- introduce domU flag and usage

Changes in v2:
- new patch
---
 xen/arch/arm/bootfdt.c      | 14 +++++++++-----
 xen/arch/arm/setup.c        | 21 +++++++++++++++++----
 xen/include/asm-arm/setup.h |  4 +++-
 3 files changed, 29 insertions(+), 10 deletions(-)

diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
index 273032b..349aa9d 100644
--- a/xen/arch/arm/bootfdt.c
+++ b/xen/arch/arm/bootfdt.c
@@ -164,7 +164,8 @@ static void __init process_memory_node(const void *fdt, int node,
 }
 
 static void __init add_boot_cmdline(const void *fdt, int node,
-                                    const char *name, bootmodule_kind kind)
+                                    const char *name, bootmodule_kind kind,
+                                    bool domU)
 {
     struct bootcmdlines *cmds = &bootinfo.cmdlines;
     struct bootcmdline *cmd;
@@ -184,6 +185,7 @@ static void __init add_boot_cmdline(const void *fdt, int node,
 
     cmd = &cmds->cmdline[cmds->nr_mods++];
     cmd->kind = kind;
+    cmd->domU = domU;
 
     ASSERT(strlen(name) <= DT_MAX_NAME);
     safe_strcpy(cmd->dt_name, name);
@@ -206,6 +208,7 @@ static void __init process_multiboot_node(const void *fdt, int node,
     int len = sizeof("/chosen");
     char path[8]; /* sizeof "/chosen" */
     int parent_node;
+    bool domU;
 
     parent_node = fdt_parent_offset(fdt, node);
     ASSERT(parent_node >= 0);
@@ -263,8 +266,9 @@ static void __init process_multiboot_node(const void *fdt, int node,
             kind = BOOTMOD_XSM;
     }
 
-    add_boot_module(kind, start, size);
-    add_boot_cmdline(fdt, node, fdt_get_name(fdt, parent_node, &len), kind);
+    domU = fdt_node_check_compatible(fdt, parent_node, "xen,domain") == 0;
+    add_boot_module(kind, start, size, domU);
+    add_boot_cmdline(fdt, node, fdt_get_name(fdt, parent_node, &len), kind, domU);
 }
 
 static void __init process_chosen_node(const void *fdt, int node,
@@ -310,7 +314,7 @@ static void __init process_chosen_node(const void *fdt, int node,
 
     printk("Initrd %"PRIpaddr"-%"PRIpaddr"\n", start, end);
 
-    add_boot_module(BOOTMOD_RAMDISK, start, end-start);
+    add_boot_module(BOOTMOD_RAMDISK, start, end-start, false);
 }
 
 static int __init early_scan_node(const void *fdt,
@@ -381,7 +385,7 @@ size_t __init boot_fdt_info(const void *fdt, paddr_t paddr)
     if ( ret < 0 )
         panic("No valid device tree\n");
 
-    add_boot_module(BOOTMOD_FDT, paddr, fdt_totalsize(fdt));
+    add_boot_module(BOOTMOD_FDT, paddr, fdt_totalsize(fdt), false);
 
     device_tree_for_each_node((void *)fdt, early_scan_node, NULL);
     early_print_info();
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index bc7dd97..dbab232 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -201,10 +201,12 @@ void __init dt_unreserved_regions(paddr_t s, paddr_t e,
 }
 
 struct bootmodule __init *add_boot_module(bootmodule_kind kind,
-                                          paddr_t start, paddr_t size)
+                                          paddr_t start, paddr_t size,
+                                          bool domU)
 {
     struct bootmodules *mods = &bootinfo.modules;
     struct bootmodule *mod;
+    unsigned int i;
 
     if ( mods->nr_mods == MAX_MODULES )
     {
@@ -212,11 +214,22 @@ struct bootmodule __init *add_boot_module(bootmodule_kind kind,
                boot_module_kind_as_string(kind), start, start + size);
         return NULL;
     }
+    for ( i = 0 ; i < mods->nr_mods ; i++ )
+    {
+        mod = &mods->module[i];
+        if ( mod->kind == kind && mod->start == start )
+        {
+            if ( !domU )
+                mod->domU = false;
+            return mod;
+        }
+    }
 
     mod = &mods->module[mods->nr_mods++];
     mod->kind = kind;
     mod->start = start;
     mod->size = size;
+    mod->domU = domU;
 
     return mod;
 }
@@ -229,7 +242,7 @@ struct bootmodule * __init boot_module_find_by_kind(bootmodule_kind kind)
     for (i = 0 ; i < mods->nr_mods ; i++ )
     {
         mod = &mods->module[i];
-        if ( mod->kind == kind )
+        if ( mod->kind == kind && !mod->domU )
             return mod;
     }
     return NULL;
@@ -244,7 +257,7 @@ struct bootcmdline * __init boot_cmdline_find_by_kind(bootmodule_kind kind)
     for ( i = 0 ; i < cmds->nr_mods ; i++ )
     {
         cmd = &cmds->cmdline[i];
-        if ( cmd->kind == kind )
+        if ( cmd->kind == kind && !cmd->domU )
             return cmd;
     }
     return NULL;
@@ -738,7 +751,7 @@ void __init start_xen(unsigned long boot_phys_offset,
     /* Register Xen's load address as a boot module. */
     xen_bootmodule = add_boot_module(BOOTMOD_XEN,
                              (paddr_t)(uintptr_t)(_start + boot_phys_offset),
-                             (paddr_t)(uintptr_t)(_end - _start + 1));
+                             (paddr_t)(uintptr_t)(_end - _start + 1), false);
     BUG_ON(!xen_bootmodule);
 
     xen_paddr = get_xen_paddr();
diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
index c2ed5cc..711b4a2 100644
--- a/xen/include/asm-arm/setup.h
+++ b/xen/include/asm-arm/setup.h
@@ -33,6 +33,7 @@ struct meminfo {
 #define BOOTMOD_MAX_CMDLINE 1024
 struct bootmodule {
     bootmodule_kind kind;
+    bool domU;
     paddr_t start;
     paddr_t size;
 };
@@ -41,6 +42,7 @@ struct bootmodule {
 #define DT_MAX_NAME 41
 struct bootcmdline {
     bootmodule_kind kind;
+    bool domU;
     char dt_name[DT_MAX_NAME];
     char cmdline[BOOTMOD_MAX_CMDLINE];
 };
@@ -91,7 +93,7 @@ size_t __init boot_fdt_info(const void *fdt, paddr_t paddr);
 const char __init *boot_fdt_cmdline(const void *fdt);
 
 struct bootmodule *add_boot_module(bootmodule_kind kind,
-                                   paddr_t start, paddr_t size);
+                                   paddr_t start, paddr_t size, bool domU);
 struct bootmodule *boot_module_find_by_kind(bootmodule_kind kind);
 struct bootcmdline *boot_cmdline_find_by_kind(bootmodule_kind kind);
 const char * __init boot_module_kind_as_string(bootmodule_kind kind);
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v4 07/23] xen/arm: probe domU kernels and initrds

[Stefano Stabellini]

Find addresses, sizes on device tree from kernel_probe.
Find the cmdline from the bootcmdlines array.

Introduce a new boot_module_find_by_addr_and_kind function to match not
just on boot module kind, but also by address so that we can support
multiple domains.

Introduce a boot_cmdline_find_by_name function to find the right struct
cmdline based on the device tree node name of the "xen,domain"
compatible node.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
---
Changes in v3:
- retrieve cmdline from bootcmdlines array

Changes in v2:
- fix indentation
- unify kernel_probe with kernel_probe_domU
- find cmdline on device_tree from kernel_probe
---
 xen/arch/arm/domain_build.c |  2 +-
 xen/arch/arm/kernel.c       | 52 +++++++++++++++++++++++++++++++++++++++------
 xen/arch/arm/kernel.h       |  2 +-
 xen/arch/arm/setup.c        | 29 +++++++++++++++++++++++++
 xen/include/asm-arm/setup.h |  3 +++
 5 files changed, 79 insertions(+), 9 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 64f8d6b..f073e6d 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2137,7 +2137,7 @@ int __init construct_dom0(struct domain *d)
     kinfo.unassigned_mem = dom0_mem;
     kinfo.d = d;
 
-    rc = kernel_probe(&kinfo);
+    rc = kernel_probe(&kinfo, NULL);
     if ( rc < 0 )
         return rc;
 
diff --git a/xen/arch/arm/kernel.c b/xen/arch/arm/kernel.c
index da8410e..e5b8213 100644
--- a/xen/arch/arm/kernel.c
+++ b/xen/arch/arm/kernel.c
@@ -421,22 +421,60 @@ static int __init kernel_zimage32_probe(struct kernel_info *info,
     return 0;
 }
 
-int __init kernel_probe(struct kernel_info *info)
+int __init kernel_probe(struct kernel_info *info, struct dt_device_node *domain) 
 {
-    struct bootmodule *mod = boot_module_find_by_kind(BOOTMOD_KERNEL);
+    struct bootmodule *mod = NULL;
+    struct bootcmdline *cmd = NULL;
+    struct dt_device_node *node;
+    u64 kernel_addr, initrd_addr, size;
+    const char *name = NULL;
     int rc;
 
+    if ( domain == NULL )
+    {
+        mod = boot_module_find_by_kind(BOOTMOD_KERNEL);
+
+        info->kernel_bootmodule = mod;
+        info->initrd_bootmodule = boot_module_find_by_kind(BOOTMOD_RAMDISK);
+    } else {
+        dt_for_each_child_node(domain, node)
+        {
+            if ( dt_device_is_compatible(node, "multiboot,kernel") )
+            {
+                u32 len;
+                const __be32 *val;
+                val = dt_get_property(node, "reg", &len);
+                dt_get_range(&val, node, &kernel_addr, &size);
+            }
+            else if ( dt_device_is_compatible(node, "multiboot,ramdisk") )
+            {
+                u32 len;
+                const __be32 *val;
+                val = dt_get_property(node, "reg", &len);
+                dt_get_range(&val, node, &initrd_addr, &size);
+            }
+            else
+                continue;
+        }
+        if ( kernel_addr )
+            info->kernel_bootmodule = mod = boot_module_find_by_addr_and_kind(
+                                      BOOTMOD_KERNEL, kernel_addr);
+        if ( initrd_addr )
+            info->initrd_bootmodule = boot_module_find_by_addr_and_kind(
+                                      BOOTMOD_RAMDISK, initrd_addr);
+        name = dt_node_name(domain);
+        cmd = boot_cmdline_find_by_name(name);
+        if ( cmd )
+            info->cmdline = &cmd->cmdline[0];
+    }
     if ( !mod || !mod->size )
     {
         printk(XENLOG_ERR "Missing kernel boot module?\n");
         return -ENOENT;
     }
 
-    info->kernel_bootmodule = mod;
-
-    printk("Loading kernel from boot module @ %"PRIpaddr"\n", mod->start);
-
-    info->initrd_bootmodule = boot_module_find_by_kind(BOOTMOD_RAMDISK);
+    printk("Loading DomU kernel from boot module @ %"PRIpaddr"\n",
+           info->kernel_bootmodule->start);
     if ( info->initrd_bootmodule )
         printk("Loading ramdisk from boot module @ %"PRIpaddr"\n",
                info->initrd_bootmodule->start);
diff --git a/xen/arch/arm/kernel.h b/xen/arch/arm/kernel.h
index 39b7828..4a65289 100644
--- a/xen/arch/arm/kernel.h
+++ b/xen/arch/arm/kernel.h
@@ -55,7 +55,7 @@ struct kernel_info {
  *  ->type
  *  ->load hook, and sets loader specific variables ->zimage
  */
-int kernel_probe(struct kernel_info *info);
+int kernel_probe(struct kernel_info *info, struct dt_device_node *domain);
 
 /*
  * Loads the kernel into guest RAM.
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index dbab232..d6d1546 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -263,6 +263,35 @@ struct bootcmdline * __init boot_cmdline_find_by_kind(bootmodule_kind kind)
     return NULL;
 }
 
+struct bootcmdline * __init boot_cmdline_find_by_name(const char *name)
+{
+    struct bootcmdlines *mods = &bootinfo.cmdlines;
+    struct bootcmdline *mod;
+    int i;
+    for (i = 0 ; i < mods->nr_mods ; i++ )
+    {
+        mod = &mods->cmdline[i];
+        if ( strcmp(mod->dt_name, name) == 0 )
+            return mod;
+    }
+    return NULL;
+}
+
+struct bootmodule * __init boot_module_find_by_addr_and_kind(bootmodule_kind kind,
+                                                             paddr_t start)
+{
+    struct bootmodules *mods = &bootinfo.modules;
+    struct bootmodule *mod;
+    int i;
+    for (i = 0 ; i < mods->nr_mods ; i++ )
+    {
+        mod = &mods->module[i];
+        if ( mod->kind == kind && mod->start == start )
+            return mod;
+    }
+    return NULL;
+}
+
 const char * __init boot_module_kind_as_string(bootmodule_kind kind)
 {
     switch ( kind )
diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
index 711b4a2..177e8db 100644
--- a/xen/include/asm-arm/setup.h
+++ b/xen/include/asm-arm/setup.h
@@ -95,7 +95,10 @@ const char __init *boot_fdt_cmdline(const void *fdt);
 struct bootmodule *add_boot_module(bootmodule_kind kind,
                                    paddr_t start, paddr_t size, bool domU);
 struct bootmodule *boot_module_find_by_kind(bootmodule_kind kind);
+struct bootmodule * __init boot_module_find_by_addr_and_kind(bootmodule_kind kind,
+                                                             paddr_t start);
 struct bootcmdline *boot_cmdline_find_by_kind(bootmodule_kind kind);
+struct bootcmdline * __init boot_cmdline_find_by_name(const char *name);
 const char * __init boot_module_kind_as_string(bootmodule_kind kind);
 
 #endif
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v4 08/23] xen/arm: rename get_11_allocation_size to get_allocation_size

[Stefano Stabellini]

No functional changes.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>

---

Changes in v3:
- no change in print messages
- do not remove BUG_ON

Changes in v2:
- new patch
---
 xen/arch/arm/domain_build.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index f073e6d..d0aff35 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -77,7 +77,7 @@ struct vcpu *__init alloc_dom0_vcpu0(struct domain *dom0)
     return vcpu_create(dom0, 0, 0);
 }
 
-static unsigned int __init get_11_allocation_size(paddr_t size)
+static unsigned int __init get_allocation_size(paddr_t size)
 {
     /*
      * get_order_from_bytes returns the order greater than or equal to
@@ -249,7 +249,7 @@ static void __init allocate_memory(struct domain *d, struct kernel_info *kinfo)
         get_order_from_bytes(min_t(paddr_t, dom0_mem, MB(128)));
     const unsigned int min_order = get_order_from_bytes(MB(4));
     struct page_info *pg;
-    unsigned int order = get_11_allocation_size(kinfo->unassigned_mem);
+    unsigned int order = get_allocation_size(kinfo->unassigned_mem);
     int i;
 
     bool lowmem = true;
@@ -301,7 +301,7 @@ static void __init allocate_memory(struct domain *d, struct kernel_info *kinfo)
      * If we failed to allocate bank0 under 4GB, continue allocating
      * memory from above 4GB and fill in banks.
      */
-    order = get_11_allocation_size(kinfo->unassigned_mem);
+    order = get_allocation_size(kinfo->unassigned_mem);
     while ( kinfo->unassigned_mem && kinfo->mem.nr_banks < NR_MEM_BANKS )
     {
         pg = alloc_domheap_pages(d, order, lowmem ? MEMF_bits(32) : 0);
@@ -312,7 +312,7 @@ static void __init allocate_memory(struct domain *d, struct kernel_info *kinfo)
             if ( lowmem && order < min_low_order)
             {
                 D11PRINT("Failed at min_low_order, allow high allocations\n");
-                order = get_11_allocation_size(kinfo->unassigned_mem);
+                order = get_allocation_size(kinfo->unassigned_mem);
                 lowmem = false;
                 continue;
             }
@@ -332,7 +332,7 @@ static void __init allocate_memory(struct domain *d, struct kernel_info *kinfo)
             if ( lowmem )
             {
                 D11PRINT("Allocation below bank 0, allow high allocations\n");
-                order = get_11_allocation_size(kinfo->unassigned_mem);
+                order = get_allocation_size(kinfo->unassigned_mem);
                 lowmem = false;
                 continue;
             }
@@ -347,7 +347,7 @@ static void __init allocate_memory(struct domain *d, struct kernel_info *kinfo)
          * Success, next time around try again to get the largest order
          * allocation possible.
          */
-        order = get_11_allocation_size(kinfo->unassigned_mem);
+        order = get_allocation_size(kinfo->unassigned_mem);
     }
 
     if ( kinfo->unassigned_mem )
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v4 09/23] xen/arm: rename allocate_memory to allocate_memory_11

[Stefano Stabellini]

allocate_memory only deals with directly mapped memory. Rename it to
allocate_memory_11.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>

---
Changes in v3:
- add patch
---
 xen/arch/arm/domain_build.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index d0aff35..41f2f27 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -243,7 +243,8 @@ fail:
  * (as described above) we allow higher allocations and continue until
  * that runs out (or we have allocated sufficient dom0 memory).
  */
-static void __init allocate_memory(struct domain *d, struct kernel_info *kinfo)
+static void __init allocate_memory_11(struct domain *d,
+                                      struct kernel_info *kinfo)
 {
     const unsigned int min_low_order =
         get_order_from_bytes(min_t(paddr_t, dom0_mem, MB(128)));
@@ -2156,7 +2157,7 @@ int __init construct_dom0(struct domain *d)
 #endif
 
     kinfo.cmdline = kernel != NULL ? &kernel->cmdline[0] : NULL;
-    allocate_memory(d, &kinfo);
+    allocate_memory_11(d, &kinfo);
     find_gnttab_region(d, &kinfo);
 
     /* Map extra GIC MMIO, irqs and other hw stuffs to dom0. */
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v4 10/23] xen/arm: introduce allocate_memory

[Stefano Stabellini]

Introduce an allocate_memory function able to allocate memory for DomUs
and map it at the right guest addresses, according to the guest memory
map: GUEST_RAM0_BASE and GUEST_RAM1_BASE.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
---
Changes in v4:
- move earlier, add #if 0
- introduce allocate_bank_memory, remove insert_bank
- allocate_bank_memory allocate memory and inserts the bank, while
  allocate_memory specifies where to do that

Changes in v3:
- new patch
---
 xen/arch/arm/domain_build.c | 83 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 83 insertions(+)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 41f2f27..fddfd82 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -368,6 +368,89 @@ static void __init allocate_memory_11(struct domain *d,
     }
 }
 
+#if 0
+static bool __init allocate_bank_memory(struct domain *d,
+                                        struct kernel_info *kinfo,
+                                        gfn_t sgfn,
+                                        unsigned int order)
+{
+    int res;
+    struct page_info *pg;
+    struct membank *bank;
+    paddr_t gaddr = gfn_to_gaddr(sgfn), size = pfn_to_paddr(1UL << order);
+
+    pg = alloc_domheap_pages(d, order, 0);
+    if ( !pg )
+        return false;
+
+    res = guest_physmap_add_page(d, sgfn, page_to_mfn(pg), order);
+    if ( res )
+    {
+        dprintk(XENLOG_ERR, "Failed map pages to DOMU: %d", res);
+        goto fail;
+    }
+
+    bank = &kinfo->mem.bank[kinfo->mem.nr_banks];
+    bank->start = gaddr;
+    bank->size = size;
+    kinfo->mem.nr_banks++;
+    kinfo->unassigned_mem -= size;
+    return true;
+
+fail:
+    free_domheap_pages(pg, order);
+    return false;
+}
+
+static void __init allocate_memory(struct domain *d, struct kernel_info *kinfo)
+{
+    unsigned int order = get_allocation_size(kinfo->unassigned_mem);
+    unsigned int order_req;
+    int i;
+
+    dprintk(XENLOG_INFO, "Allocating mappings totalling %ldMB for dom%d:\n",
+            /* Don't want format this as PRIpaddr (16 digit hex) */
+            (unsigned long)(kinfo->unassigned_mem >> 20), d->domain_id);
+
+    kinfo->mem.nr_banks = 0;
+
+    order = get_allocation_size(kinfo->unassigned_mem);
+    if ( order > get_order_from_bytes(GUEST_RAM0_SIZE) )
+        order_req = get_order_from_bytes(GUEST_RAM0_SIZE);
+    else
+        order_req = order;
+    if ( !allocate_bank_memory(d, kinfo, gaddr_to_gfn(GUEST_RAM0_BASE), order_req) )
+        goto fail;
+
+    order -= order_req;
+    if ( order > 0 )
+    {
+        if ( !allocate_bank_memory(d, kinfo, gaddr_to_gfn(GUEST_RAM1_BASE), order) )
+            goto fail;
+    }
+
+    for( i = 0; i < kinfo->mem.nr_banks; i++ )
+    {
+        dprintk(XENLOG_INFO, "Dom%d BANK[%d] %#"PRIpaddr"-%#"PRIpaddr" (%ldMB)\n",
+                d->domain_id,
+                i,
+                kinfo->mem.bank[i].start,
+                kinfo->mem.bank[i].start + kinfo->mem.bank[i].size,
+                /* Don't want format this as PRIpaddr (16 digit hex) */
+                (unsigned long)(kinfo->mem.bank[i].size >> 20));
+    }
+
+    return;
+
+fail:
+    dprintk(XENLOG_ERR, "Failed to allocate requested domain memory."
+            /* Don't want format this as PRIpaddr (16 digit hex) */
+            " %ldKB unallocated. Fix the VMs configurations.\n",
+            (unsigned long)kinfo->unassigned_mem >> 10);
+    BUG();
+}
+#endif
+
 static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
                                    const struct dt_device_node *node)
 {
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v4 11/23] xen/arm: refactor construct_dom0

[Stefano Stabellini]

Move generic initializations out of construct_dom0 so that they can be
reused.

Rename prepare_dtb to prepare_dtb_hwdom to avoid confusion.

No functional changes in this patch.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>

---
Changes in v4:
- newline and style changes

Changes in v3:
- move setting type before allocate_memory
- add ifdef around it and a comment

Changes in v2:
- move discard_initial_modules() after __construct_domain()
- remove useless blank line
- leave safety BUG_ONs in __construct_domain
- rename prepare_dtb to prepare_dtb_hwdom
---
 xen/arch/arm/domain_build.c | 126 ++++++++++++++++++++++++--------------------
 1 file changed, 68 insertions(+), 58 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index fddfd82..ba3dad1 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1456,7 +1456,7 @@ static int __init handle_node(struct domain *d, struct kernel_info *kinfo,
     return res;
 }
 
-static int __init prepare_dtb(struct domain *d, struct kernel_info *kinfo)
+static int __init prepare_dtb_hwdom(struct domain *d, struct kernel_info *kinfo)
 {
     const p2m_type_t default_p2mt = p2m_mmio_direct_c;
     const void *fdt;
@@ -2191,75 +2191,29 @@ static void __init find_gnttab_region(struct domain *d,
            kinfo->gnttab_start, kinfo->gnttab_start + kinfo->gnttab_size);
 }
 
-int __init construct_dom0(struct domain *d)
+static int __init __construct_domain(struct domain *d, struct kernel_info *kinfo)
 {
-    const struct bootcmdline *kernel = boot_cmdline_find_by_kind(BOOTMOD_KERNEL);
-    struct kernel_info kinfo = {};
     struct vcpu *saved_current;
-    int rc, i, cpu;
-
+    int i, cpu;
     struct vcpu *v = d->vcpu[0];
     struct cpu_user_regs *regs = &v->arch.cpu_info->guest_cpu_user_regs;
 
-    /* Sanity! */
-    BUG_ON(d->domain_id != 0);
     BUG_ON(d->vcpu[0] == NULL);
     BUG_ON(v->is_initialised);
 
-    printk("*** LOADING DOMAIN 0 ***\n");
-    if ( dom0_mem <= 0 )
-    {
-        warning_add("PLEASE SPECIFY dom0_mem PARAMETER - USING 512M FOR NOW\n");
-        dom0_mem = MB(512);
-    }
-
-
-    iommu_hwdom_init(d);
-
-    d->max_pages = ~0U;
-
-    kinfo.unassigned_mem = dom0_mem;
-    kinfo.d = d;
-
-    rc = kernel_probe(&kinfo, NULL);
-    if ( rc < 0 )
-        return rc;
-
 #ifdef CONFIG_ARM_64
     /* if aarch32 mode is not supported at EL1 do not allow 32-bit domain */
-    if ( !(cpu_has_el1_32) && kinfo.type == DOMAIN_32BIT )
+    if ( !(cpu_has_el1_32) && kinfo->type == DOMAIN_32BIT )
     {
         printk("Platform does not support 32-bit domain\n");
         return -EINVAL;
     }
-    d->arch.type = kinfo.type;
 
     if ( is_64bit_domain(d) )
         vcpu_switch_to_aarch64_mode(v);
 
 #endif
 
-    kinfo.cmdline = kernel != NULL ? &kernel->cmdline[0] : NULL;
-    allocate_memory_11(d, &kinfo);
-    find_gnttab_region(d, &kinfo);
-
-    /* Map extra GIC MMIO, irqs and other hw stuffs to dom0. */
-    rc = gic_map_hwdom_extra_mappings(d);
-    if ( rc < 0 )
-        return rc;
-
-    rc = platform_specific_mapping(d);
-    if ( rc < 0 )
-        return rc;
-
-    if ( acpi_disabled )
-        rc = prepare_dtb(d, &kinfo);
-    else
-        rc = prepare_acpi(d, &kinfo);
-
-    if ( rc < 0 )
-        return rc;
-
     /*
      * The following loads use the domain's p2m and require current to
      * be a vcpu of the domain, temporarily switch
@@ -2272,20 +2226,18 @@ int __init construct_dom0(struct domain *d)
      * kernel_load will determine the placement of the kernel as well
      * as the initrd & fdt in RAM, so call it first.
      */
-    kernel_load(&kinfo);
+    kernel_load(kinfo);
     /* initrd_load will fix up the fdt, so call it before dtb_load */
-    initrd_load(&kinfo);
-    dtb_load(&kinfo);
+    initrd_load(kinfo);
+    dtb_load(kinfo);
 
     /* Now that we are done restore the original p2m and current. */
     set_current(saved_current);
     p2m_restore_state(saved_current);
 
-    discard_initial_modules();
-
     memset(regs, 0, sizeof(*regs));
 
-    regs->pc = (register_t)kinfo.entry;
+    regs->pc = (register_t)kinfo->entry;
 
     if ( is_32bit_domain(d) )
     {
@@ -2303,14 +2255,14 @@ int __init construct_dom0(struct domain *d)
          */
         regs->r0 = 0; /* SBZ */
         regs->r1 = 0xffffffff; /* We use DTB therefore no machine id */
-        regs->r2 = kinfo.dtb_paddr;
+        regs->r2 = kinfo->dtb_paddr;
     }
 #ifdef CONFIG_ARM_64
     else
     {
         regs->cpsr = PSR_GUEST64_INIT;
         /* From linux/Documentation/arm64/booting.txt */
-        regs->x0 = kinfo.dtb_paddr;
+        regs->x0 = kinfo->dtb_paddr;
         regs->x1 = 0; /* Reserved for future use */
         regs->x2 = 0; /* Reserved for future use */
         regs->x3 = 0; /* Reserved for future use */
@@ -2338,6 +2290,64 @@ int __init construct_dom0(struct domain *d)
     return 0;
 }
 
+int __init construct_dom0(struct domain *d)
+{
+    const struct bootcmdline *kernel = boot_cmdline_find_by_kind(BOOTMOD_KERNEL);
+    struct kernel_info kinfo = {};
+    int rc;
+
+    /* Sanity! */
+    BUG_ON(d->domain_id != 0);
+
+    printk("*** LOADING DOMAIN 0 ***\n");
+    if ( dom0_mem <= 0 )
+    {
+        warning_add("PLEASE SPECIFY dom0_mem PARAMETER - USING 512M FOR NOW\n");
+        dom0_mem = MB(512);
+    }
+
+    iommu_hwdom_init(d);
+
+    d->max_pages = ~0U;
+
+    kinfo.unassigned_mem = dom0_mem;
+    kinfo.d = d;
+
+    rc = kernel_probe(&kinfo, NULL);
+    if ( rc < 0 )
+        return rc;
+
+    kinfo.cmdline = kernel != NULL ? &kernel->cmdline[0] : NULL;
+#ifdef CONFIG_ARM_64
+    /* type must be set before allocate_memory */
+    d->arch.type = kinfo.type;
+#endif
+    allocate_memory_11(d, &kinfo);
+    find_gnttab_region(d, &kinfo);
+
+    /* Map extra GIC MMIO, irqs and other hw stuffs to dom0. */
+    rc = gic_map_hwdom_extra_mappings(d);
+    if ( rc < 0 )
+        return rc;
+
+    rc = platform_specific_mapping(d);
+    if ( rc < 0 )
+        return rc;
+
+    if ( acpi_disabled )
+        rc = prepare_dtb_hwdom(d, &kinfo);
+    else
+        rc = prepare_acpi(d, &kinfo);
+
+    if ( rc < 0 )
+        return rc;
+
+    rc = __construct_domain(d, &kinfo);
+    discard_initial_modules();
+
+    return rc;
+}
+
 /*
  * Local variables:
  * mode: C
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v4 12/23] xen/arm: introduce create_domUs

[Stefano Stabellini]

Call a new function, "create_domUs", from setup_xen to start DomU VMs.

Introduce support for the "xen,domU" compatible node on device tree.
Create new DomU VMs based on the information found on device tree under
"xen,domU". Calls construct_domU for each domain.

Introduce a simple global variable named max_init_domid to keep track of
the initial allocated domids. It holds the max domid among the initial
domains.

Move the discard_initial_modules after DomUs have been built.

First create domUs, then start dom0 -- no point in trying to start dom0
when the cpu is busy.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
CC: andrew.cooper3@citrix.com
CC: jbeulich@suse.com
---
Changes in v4:
- constify parameters
- nr_spis to 0 or  GUEST_VPL011_SPI - 32 + 1 depending on vpl011
- remove pointless initializer
- remove change to domain_create for dom0 (useless)
- make construct_domU return error

Changes in v3:
- move patch earlier and introduce empty construct_domU to fix bisection
  builds
- fix max_init_domid to actually hold the max domid among initial
  domains (instead of max_domid + 1)
- move domain_unpause_by_systemcontroller(dom0) after creating domUs

Changes in v2:
- coding style
- set nr_spis to 32
- introduce create_domUs
---
 xen/arch/arm/domain_build.c | 51 +++++++++++++++++++++++++++++++++++++++++----
 xen/arch/arm/setup.c        |  5 +++++
 xen/include/asm-arm/setup.h |  3 +++
 xen/include/asm-x86/setup.h |  2 ++
 4 files changed, 57 insertions(+), 4 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index ba3dad1..547b624 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -7,6 +7,7 @@
 #include <asm/irq.h>
 #include <asm/regs.h>
 #include <xen/errno.h>
+#include <xen/err.h>
 #include <xen/device_tree.h>
 #include <xen/libfdt/libfdt.h>
 #include <xen/guest_access.h>
@@ -2290,6 +2291,51 @@ static int __init __construct_domain(struct domain *d, struct kernel_info *kinfo
     return 0;
 }
 
+static int __init construct_domU(struct domain *d,
+                                 const struct dt_device_node *node)
+{
+    return -ENOSYS;
+}
+
+void __init create_domUs(void)
+{
+    struct dt_device_node *node;
+    const struct dt_device_node *chosen = dt_find_node_by_name(dt_host,
+                                                               "chosen");
+    BUG_ON(chosen == NULL);
+    dt_for_each_child_node(chosen, node)
+    {
+        u32 len;
+        struct domain *d;
+        struct xen_domctl_createdomain d_cfg = {
+            .arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE,
+            .arch.nr_spis = 0,
+            .max_vcpus = 1,
+            .max_evtchn_port = -1,
+            .max_grant_frames = 64,
+            .max_maptrack_frames = 1024,
+        };
+
+        if ( !dt_device_is_compatible(node, "xen,domain") )
+            continue;
+
+        if ( dt_get_property(node, "vpl011", &len) != NULL )
+            d_cfg.arch.nr_spis = GUEST_VPL011_SPI - 32 + 1;
+        dt_property_read_u32(node, "cpus", &d_cfg.max_vcpus);
+
+        d = domain_create(++max_init_domid, &d_cfg, false);
+        if ( IS_ERR(d) )
+            panic("Error creating domain %s", dt_node_name(node));
+
+        d->is_console = 1;
+
+        if ( construct_domU(d, node) != 0 )
+            panic("Could not set up domain %s", dt_node_name(node));
+
+        domain_unpause_by_systemcontroller(d);
+    }
+}
+
 int __init construct_dom0(struct domain *d)
 {
     const struct bootcmdline *kernel = boot_cmdline_find_by_kind(BOOTMOD_KERNEL);
@@ -2342,10 +2388,7 @@ int __init construct_dom0(struct domain *d)
     if ( rc < 0 )
         return rc;
 
-    rc = __construct_domain(d, &kinfo);
-    discard_initial_modules();
-
-    return rc;
+    return __construct_domain(d, &kinfo);
 }
 
 /*
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index d6d1546..8d8f180 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -64,8 +64,11 @@ static unsigned long opt_xenheap_megabytes __initdata;
 integer_param("xenheap_megabytes", opt_xenheap_megabytes);
 #endif
 
+domid_t __read_mostly max_init_domid;
+
 static __used void init_done(void)
 {
+    discard_initial_modules();
     free_init_memory();
     startup_cpu_idle_loop();
 }
@@ -926,6 +929,8 @@ void __init start_xen(unsigned long boot_phys_offset,
     /* Must be done past setting system_state. */
     unregister_init_virtual_region();
 
+    create_domUs();
+
     domain_unpause_by_systemcontroller(dom0);
 
     /* Switch on to the dynamically allocated stack for the idle vcpu
diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
index 177e8db..5620fe7 100644
--- a/xen/include/asm-arm/setup.h
+++ b/xen/include/asm-arm/setup.h
@@ -68,6 +68,8 @@ struct bootinfo {
 
 extern struct bootinfo bootinfo;
 
+extern domid_t max_init_domid;
+
 void arch_init_memory(void);
 
 void copy_from_paddr(void *dst, paddr_t paddr, unsigned long len);
@@ -84,6 +86,7 @@ void acpi_create_efi_mmap_table(struct domain *d,
 int acpi_make_efi_nodes(void *fdt, struct membank tbl_add[]);
 
 int construct_dom0(struct domain *d);
+void __init create_domUs(void);
 
 void discard_initial_modules(void);
 void dt_unreserved_regions(paddr_t s, paddr_t e,
diff --git a/xen/include/asm-x86/setup.h b/xen/include/asm-x86/setup.h
index 42fddeb..1c80783 100644
--- a/xen/include/asm-x86/setup.h
+++ b/xen/include/asm-x86/setup.h
@@ -66,4 +66,6 @@ extern bool opt_dom0_shadow;
 #endif
 extern bool dom0_pvh;
 
+#define max_init_domid (0)
+
 #endif
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v4 13/23] xen/arm: implement construct_domU

[Stefano Stabellini]

Similar to construct_dom0, construct_domU creates a barebone DomU guest.

The device tree node passed as argument is compatible "xen,domain", see
docs/misc/arm/device-tree/booting.txt.

Add const to kernel_probe dt_device_node parameter.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>

---
Changes in v4:
- constify kernel_probe
- change title
- better error messages and printed info
- 64bit memory

Changes in v3:
- move setting type before allocate_memory
- add ifdef around it and a comment

Changes in v2:
- rename mem to memory
- make cpus and memory mandatory
- remove wront comment from commit message
- cpus and memory are read as integers
- read the vpl011 option
---
 xen/arch/arm/domain_build.c | 37 ++++++++++++++++++++++++++++++++++---
 xen/arch/arm/kernel.c       |  3 ++-
 xen/arch/arm/kernel.h       |  2 +-
 3 files changed, 37 insertions(+), 5 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 547b624..efb530a 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -369,7 +369,6 @@ static void __init allocate_memory_11(struct domain *d,
     }
 }
 
-#if 0
 static bool __init allocate_bank_memory(struct domain *d,
                                         struct kernel_info *kinfo,
                                         gfn_t sgfn,
@@ -450,7 +449,6 @@ fail:
             (unsigned long)kinfo->unassigned_mem >> 10);
     BUG();
 }
-#endif
 
 static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
                                    const struct dt_device_node *node)
@@ -2294,7 +2292,40 @@ static int __init __construct_domain(struct domain *d, struct kernel_info *kinfo
 static int __init construct_domU(struct domain *d,
                                  const struct dt_device_node *node)
 {
-    return -ENOSYS;
+    struct kernel_info kinfo = {};
+    int rc;
+    u64 mem;
+
+    rc = dt_property_read_u64(node, "memory", &mem);
+    if ( !rc )
+    {
+        printk("Error building DomU: cannot read \"memory\" property\n");
+        return -EINVAL;
+    }
+    kinfo.unassigned_mem = (paddr_t)mem << 10;
+
+    printk("*** LOADING DOMU cpus=%u memory=%luKB ***\n", d->max_vcpus, mem);
+
+    d->vcpu = xzalloc_array(struct vcpu *, d->max_vcpus);
+    if ( !d->vcpu )
+        return -ENOMEM;;
+    if ( vcpu_create(d, 0, 0) == NULL )
+        return -ENOMEM;
+    d->max_pages = ~0U;
+
+    kinfo.d = d;
+
+    rc = kernel_probe(&kinfo, node);
+    if ( rc < 0 )
+        return rc;
+
+#ifdef CONFIG_ARM_64
+    /* type must be set before allocate memory */
+    d->arch.type = kinfo.type;
+#endif
+    allocate_memory(d, &kinfo);
+
+    return __construct_domain(d, &kinfo);
 }
 
 void __init create_domUs(void)
diff --git a/xen/arch/arm/kernel.c b/xen/arch/arm/kernel.c
index e5b8213..2239a07 100644
--- a/xen/arch/arm/kernel.c
+++ b/xen/arch/arm/kernel.c
@@ -421,7 +421,8 @@ static int __init kernel_zimage32_probe(struct kernel_info *info,
     return 0;
 }
 
-int __init kernel_probe(struct kernel_info *info, struct dt_device_node *domain) 
+int __init kernel_probe(struct kernel_info *info,
+                        const struct dt_device_node *domain)
 {
     struct bootmodule *mod = NULL;
     struct bootcmdline *cmd = NULL;
diff --git a/xen/arch/arm/kernel.h b/xen/arch/arm/kernel.h
index 4a65289..4320f72 100644
--- a/xen/arch/arm/kernel.h
+++ b/xen/arch/arm/kernel.h
@@ -55,7 +55,7 @@ struct kernel_info {
  *  ->type
  *  ->load hook, and sets loader specific variables ->zimage
  */
-int kernel_probe(struct kernel_info *info, struct dt_device_node *domain);
+int kernel_probe(struct kernel_info *info, const struct dt_device_node *domain);
 
 /*
  * Loads the kernel into guest RAM.
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v4 00/23] dom0less step1: boot multiple domains from device tree

[Stefano Stabellini]

Hi all,

This is first step toward "dom0less" as discussed in the various
certifications related threads and discussions.

The goal of this series is to enable Xen to boot multiple domains in
parallel, in addition to dom0, out of information found on device tree.

The device tree based boot protocol is extended to carry information
about DomUs. Based on that information, Xen creates and starts one or
more DomUs. DomUs created this way don't have access to xenstore for the
moment. This is actually OK, because this is meant for mission critical
applications that typically only access directly assigned devices. They
cannot tolerate interference or increased IRQ latency due to PV
protocols. Device assignment is not actually covered by this series, it
will be added later.

DomUs can print to the Xen serial using the CONSOLEIO hypercalls. A
virtual PL011 is also emulated for them so that they can use their
regular PL011 driver. This allows unmodified guests to run as Xen on ARM
guests -- no Xen support needed at all. Console input also comes from
the Xen serial: the Ctrl-AAA switching mechanism is extended to switch
among domUs, dom0, and Xen.

In this version of the series, I reordered the patches to make sure they
are all bisectable.


Cheers,

Stefano


The following changes since commit 359970fd8b781fac2ddcbc84dd5b890075fa08ef:

  tools/libxl: Switch Arm guest type to PVH (2018-10-03 15:58:02 +0100)

are available in the git repository at:

  http://xenbits.xenproject.org/git-http/people/sstabellini/xen-unstable.git dom0less-v4

for you to fetch changes up to ad670c5672597613f4153f356e44e55e8b37a0cd:

  xen/arm: split domain_build.c (2018-10-05 11:40:26 -0700)

----------------------------------------------------------------
Stefano Stabellini (23):
      xen: allow console_io hypercalls from certain DomUs
      xen/arm: extend device tree based multiboot protocol
      xen/arm: document dom0less
      xen/arm: increase MAX_MODULES
      xen/arm: introduce bootcmdlines
      xen/arm: don't add duplicate boot modules, introduce domU flag
      xen/arm: probe domU kernels and initrds
      xen/arm: rename get_11_allocation_size to get_allocation_size
      xen/arm: rename allocate_memory to allocate_memory_11
      xen/arm: introduce allocate_memory
      xen/arm: refactor construct_dom0
      xen/arm: introduce create_domUs
      xen/arm: implement construct_domU
      xen/arm: generate a simple device tree for domUs
      xen/arm: make set_interrupt_ppi able to handle non-PPI
      xen/arm: generate vpl011 node on device tree for domU
      xen/arm: introduce a union in vpl011
      xen/arm: refactor vpl011_data_avail
      xen/arm: Allow vpl011 to be used by DomU
      xen: support console_switching between Dom0 and DomUs on ARM
      xen/vpl011: buffer out chars when the backend is xen
      xen/arm: move kernel.h to asm-arm/
      xen/arm: split domain_build.c

 docs/misc/arm/device-tree/booting.txt      |  107 +++
 docs/misc/arm/dom0less.txt                 |   47 ++
 xen/arch/arm/acpi/Makefile                 |    1 +
 xen/arch/arm/acpi/domain_build.c           |  592 +++++++++++++++
 xen/arch/arm/bootfdt.c                     |   86 ++-
 xen/arch/arm/domain_build.c                | 1073 +++++++++++++---------------
 xen/arch/arm/kernel.c                      |   56 +-
 xen/arch/arm/setup.c                       |   71 +-
 xen/arch/arm/vpl011.c                      |  295 ++++++--
 xen/drivers/char/console.c                 |   80 ++-
 xen/include/asm-arm/domain_build.h         |   31 +
 xen/{arch/arm => include/asm-arm}/kernel.h |    6 +-
 xen/include/asm-arm/setup.h                |   27 +-
 xen/include/asm-arm/vpl011.h               |   20 +-
 xen/include/asm-x86/setup.h                |    2 +
 xen/include/xen/console.h                  |    2 +
 xen/include/xen/sched.h                    |    2 +
 xen/include/xsm/dummy.h                    |    2 +
 18 files changed, 1802 insertions(+), 698 deletions(-)
 create mode 100644 docs/misc/arm/dom0less.txt
 create mode 100644 xen/arch/arm/acpi/domain_build.c
 create mode 100644 xen/include/asm-arm/domain_build.h
 rename xen/{arch/arm => include/asm-arm}/kernel.h (91%)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v4 14/23] xen/arm: generate a simple device tree for domUs

[Stefano Stabellini]

Introduce functions to generate a basic domU device tree, similar to the
existing functions in tools/libxl/libxl_arm.c.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
---
Changes in v4:
- code style
- two separate functions for gicv2 and gicv3
- remove useless local variables
- fix typos
- do not use host address and size cells for the guest DT
- use #define for addrcells and sizecells

Changes in v3:
- remove CONFIG_ACPI for make_chosen_node
- remove make_hypervisor_node until all Xen related functionalities
  (evtchns, grant table, etc.) work correctly

Changes in v2:
- move prepare_dtb rename to previous patch
- use switch for the gic version
- use arm,gic-400 instead of arm,cortex-a15-gic
- add @unit-address in the gic node name
- add comment on DOMU_DTB_SIZE
---
 xen/arch/arm/domain_build.c | 235 +++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 233 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index efb530a..bf8aeca 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1017,7 +1017,6 @@ static int __init make_timer_node(const struct domain *d, void *fdt,
     return res;
 }
 
-#ifdef CONFIG_ACPI
 /*
  * This function is used as part of the device tree generation for Dom0
  * on ACPI systems, and DomUs started directly from Xen based on device
@@ -1063,7 +1062,6 @@ static int __init make_chosen_node(const struct kernel_info *kinfo)
 
     return res;
 }
-#endif
 
 static int __init map_irq_to_domain(struct domain *d, unsigned int irq,
                                     bool need_mapping, const char *devname)
@@ -1455,6 +1453,235 @@ static int __init handle_node(struct domain *d, struct kernel_info *kinfo,
     return res;
 }
 
+static int __init make_gicv2_domU_node(const struct domain *d, void *fdt)
+{
+    int res = 0;
+    __be32 reg[(GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS) * 2];
+    __be32 *cells;
+
+    res = fdt_begin_node(fdt, "interrupt-controller@"__stringify(GUEST_GICD_BASE));
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "#address-cells", 0);
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "#interrupt-cells", 3);
+    if ( res )
+        return res;
+
+    res = fdt_property(fdt, "interrupt-controller", NULL, 0);
+    if ( res )
+        return res;
+
+    res = fdt_property_string(fdt, "compatible", "arm,gic-400");
+    if ( res )
+        return res;
+
+    cells = &reg[0];
+    dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
+                       GUEST_GICD_BASE, GUEST_GICD_SIZE);
+    dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
+                       GUEST_GICC_BASE, GUEST_GICC_SIZE);
+
+    res = fdt_property(fdt, "reg", reg, sizeof(reg));
+    if (res)
+        return res;
+
+    res = fdt_property_cell(fdt, "linux,phandle", GUEST_PHANDLE_GIC);
+    if (res)
+        return res;
+
+    res = fdt_property_cell(fdt, "phandle", GUEST_PHANDLE_GIC);
+    if (res)
+        return res;
+
+    res = fdt_end_node(fdt);
+
+    return res;
+}
+
+static int __init make_gicv3_domU_node(const struct domain *d, void *fdt)
+{
+    int res = 0;
+    __be32 reg[(GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS) * 2];
+    __be32 *cells;
+
+    res = fdt_begin_node(fdt, "interrupt-controller@"__stringify(GUEST_GICV3_GICD_BASE));
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "#address-cells", 0);
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "#interrupt-cells", 3);
+    if ( res )
+        return res;
+
+    res = fdt_property(fdt, "interrupt-controller", NULL, 0);
+    if ( res )
+        return res;
+
+    res = fdt_property_string(fdt, "compatible", "arm,gic-v3");
+    if ( res )
+        return res;
+
+    cells = &reg[0];
+    dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
+                       GUEST_GICV3_GICD_BASE, GUEST_GICV3_GICD_SIZE);
+    dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
+                       GUEST_GICV3_GICR0_BASE, GUEST_GICV3_GICR0_SIZE);
+
+    res = fdt_property(fdt, "reg", reg, sizeof(reg));
+    if (res)
+        return res;
+
+    res = fdt_property_cell(fdt, "linux,phandle", GUEST_PHANDLE_GIC);
+    if (res)
+        return res;
+
+    res = fdt_property_cell(fdt, "phandle", GUEST_PHANDLE_GIC);
+    if (res)
+        return res;
+
+    res = fdt_end_node(fdt);
+
+    return res;
+}
+
+static int __init make_gic_domU_node(const struct domain *d, void *fdt)
+{
+    switch ( gic_hw_version() )
+    {
+    case GIC_V3:
+        return make_gicv3_domU_node(d, fdt);
+    case GIC_V2:
+        return make_gicv2_domU_node(d, fdt);
+    default:
+        panic("Unsupported GIC version");
+    }
+}
+
+static int __init make_timer_domU_node(const struct domain *d, void *fdt)
+{
+    int res;
+    gic_interrupt_t intrs[3];
+
+    res = fdt_begin_node(fdt, "timer");
+    if ( res )
+        return res;
+
+    if ( !is_64bit_domain(d) )
+    {
+        res = fdt_property_string(fdt, "compatible", "arm,armv7-timer");
+        if ( res )
+            return res;
+    }
+    else
+    {
+        res = fdt_property_string(fdt, "compatible", "arm,armv8-timer");
+        if ( res )
+            return res;
+    }
+
+    set_interrupt_ppi(intrs[0], GUEST_TIMER_PHYS_S_PPI, 0xf, DT_IRQ_TYPE_LEVEL_LOW);
+    set_interrupt_ppi(intrs[1], GUEST_TIMER_PHYS_NS_PPI, 0xf, DT_IRQ_TYPE_LEVEL_LOW);
+    set_interrupt_ppi(intrs[2], GUEST_TIMER_VIRT_PPI, 0xf, DT_IRQ_TYPE_LEVEL_LOW);
+
+    res = fdt_property(fdt, "interrupts", intrs, sizeof (intrs[0]) * 3);
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "interrupt-parent",
+                            GUEST_PHANDLE_GIC);
+    if (res)
+        return res;
+
+    res = fdt_end_node(fdt);
+
+    return res;
+}
+
+/*
+ * The max size for DT is 2MB. However, the generated DT is small, 4KB
+ * are enough for now, but we might have to increase it in the future.
+ */
+#define DOMU_DTB_SIZE 4096
+static int __init prepare_dtb_domU(struct domain *d, struct kernel_info *kinfo)
+{
+    int addrcells, sizecells;
+    int ret;
+
+    addrcells = GUEST_ROOT_ADDRESS_CELLS;
+    sizecells = GUEST_ROOT_SIZE_CELLS;
+
+    kinfo->fdt = xmalloc_bytes(DOMU_DTB_SIZE);
+    if ( kinfo->fdt == NULL )
+        return -ENOMEM;
+
+    ret = fdt_create(kinfo->fdt, DOMU_DTB_SIZE);
+    if ( ret < 0 )
+        goto err;
+
+    ret = fdt_finish_reservemap(kinfo->fdt);
+    if ( ret < 0 )
+        goto err;
+
+    ret = fdt_begin_node(kinfo->fdt, "/");
+    if ( ret < 0 )
+        goto err;
+
+    ret = fdt_property_cell(kinfo->fdt, "#address-cells", addrcells);
+    if ( ret )
+        goto err;
+
+    ret = fdt_property_cell(kinfo->fdt, "#size-cells", sizecells);
+    if ( ret )
+        goto err;
+
+    ret = make_chosen_node(kinfo);
+    if ( ret )
+        goto err;
+
+    ret = make_psci_node(kinfo->fdt, NULL);
+    if ( ret )
+        goto err;
+
+    ret = make_cpus_node(d, kinfo->fdt, NULL);
+    if ( ret )
+        goto err;
+
+    ret = make_memory_node(d, kinfo->fdt, addrcells, sizecells, kinfo);
+    if ( ret )
+        goto err;
+
+    ret = make_gic_domU_node(d, kinfo->fdt);
+    if ( ret )
+        goto err;
+
+    ret = make_timer_domU_node(d, kinfo->fdt);
+    if ( ret )
+        goto err;
+
+    ret = fdt_end_node(kinfo->fdt);
+    if ( ret < 0 )
+        goto err;
+
+    ret = fdt_finish(kinfo->fdt);
+    if ( ret < 0 )
+        goto err;
+
+    return 0;
+
+  err:
+    printk("Device tree generation failed (%d).\n", ret);
+    xfree(kinfo->fdt);
+
+    return -EINVAL;
+}
+
 static int __init prepare_dtb_hwdom(struct domain *d, struct kernel_info *kinfo)
 {
     const p2m_type_t default_p2mt = p2m_mmio_direct_c;
@@ -2325,6 +2552,10 @@ static int __init construct_domU(struct domain *d,
 #endif
     allocate_memory(d, &kinfo);
 
+    rc = prepare_dtb_domU(d, &kinfo);
+    if ( rc < 0 )
+        return rc;
+
     return __construct_domain(d, &kinfo);
 }
 
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v4 15/23] xen/arm: make set_interrupt_ppi able to handle non-PPI

[Stefano Stabellini]

also rename it to set_interrupt.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
---
 xen/arch/arm/domain_build.c | 29 +++++++++++++++--------------
 1 file changed, 15 insertions(+), 14 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index bf8aeca..760ebf8 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -577,19 +577,20 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
 
 typedef __be32 gic_interrupt_t[3];
 
-static void __init set_interrupt_ppi(gic_interrupt_t interrupt,
-                                     unsigned int irq,
-                                     unsigned int cpumask,
-                                     unsigned int level)
+static void __init set_interrupt(gic_interrupt_t interrupt,
+                                 unsigned int irq,
+                                 unsigned int cpumask,
+                                 unsigned int level)
 {
     __be32 *cells = interrupt;
+    bool is_ppi = !!(irq < 32);
 
     BUG_ON(irq < 16);
-    BUG_ON(irq >= 32);
+    irq -= (is_ppi) ? 16: 32; /* PPIs start at 16, SPIs at 32 */
 
     /* See linux Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt */
-    dt_set_cell(&cells, 1, 1); /* is a PPI */
-    dt_set_cell(&cells, 1, irq - 16); /* PPIs start at 16 */
+    dt_set_cell(&cells, 1, is_ppi); /* is a PPI? */
+    dt_set_cell(&cells, 1, irq);
     dt_set_cell(&cells, 1, (cpumask << 8) | level);
 }
 
@@ -712,7 +713,7 @@ static int __init make_hypervisor_node(struct domain *d,
      *  - All CPUs
      *  TODO: Handle properly the cpumask;
      */
-    set_interrupt_ppi(intr, d->arch.evtchn_irq, 0xf, DT_IRQ_TYPE_LEVEL_LOW);
+    set_interrupt(intr, d->arch.evtchn_irq, 0xf, DT_IRQ_TYPE_LEVEL_LOW);
     res = fdt_property_interrupts(fdt, &intr, 1);
     if ( res )
         return res;
@@ -989,15 +990,15 @@ static int __init make_timer_node(const struct domain *d, void *fdt,
 
     irq = timer_get_irq(TIMER_PHYS_SECURE_PPI);
     dt_dprintk("  Secure interrupt %u\n", irq);
-    set_interrupt_ppi(intrs[0], irq, 0xf, DT_IRQ_TYPE_LEVEL_LOW);
+    set_interrupt(intrs[0], irq, 0xf, DT_IRQ_TYPE_LEVEL_LOW);
 
     irq = timer_get_irq(TIMER_PHYS_NONSECURE_PPI);
     dt_dprintk("  Non secure interrupt %u\n", irq);
-    set_interrupt_ppi(intrs[1], irq, 0xf, DT_IRQ_TYPE_LEVEL_LOW);
+    set_interrupt(intrs[1], irq, 0xf, DT_IRQ_TYPE_LEVEL_LOW);
 
     irq = timer_get_irq(TIMER_VIRT_PPI);
     dt_dprintk("  Virt interrupt %u\n", irq);
-    set_interrupt_ppi(intrs[2], irq, 0xf, DT_IRQ_TYPE_LEVEL_LOW);
+    set_interrupt(intrs[2], irq, 0xf, DT_IRQ_TYPE_LEVEL_LOW);
 
     res = fdt_property_interrupts(fdt, intrs, 3);
     if ( res )
@@ -1586,9 +1587,9 @@ static int __init make_timer_domU_node(const struct domain *d, void *fdt)
             return res;
     }
 
-    set_interrupt_ppi(intrs[0], GUEST_TIMER_PHYS_S_PPI, 0xf, DT_IRQ_TYPE_LEVEL_LOW);
-    set_interrupt_ppi(intrs[1], GUEST_TIMER_PHYS_NS_PPI, 0xf, DT_IRQ_TYPE_LEVEL_LOW);
-    set_interrupt_ppi(intrs[2], GUEST_TIMER_VIRT_PPI, 0xf, DT_IRQ_TYPE_LEVEL_LOW);
+    set_interrupt(intrs[0], GUEST_TIMER_PHYS_S_PPI, 0xf, DT_IRQ_TYPE_LEVEL_LOW);
+    set_interrupt(intrs[1], GUEST_TIMER_PHYS_NS_PPI, 0xf, DT_IRQ_TYPE_LEVEL_LOW);
+    set_interrupt(intrs[2], GUEST_TIMER_VIRT_PPI, 0xf, DT_IRQ_TYPE_LEVEL_LOW);
 
     res = fdt_property(fdt, "interrupts", intrs, sizeof (intrs[0]) * 3);
     if ( res )
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v4 16/23] xen/arm: generate vpl011 node on device tree for domU

[Stefano Stabellini]

Introduce vpl011 support to guests started from Xen: it provides a
simple way to print output from a guest, as most guests come with a
pl011 driver. It is also able to provide a working console with
interrupt support.

The UART exposed to the guest is a SBSA compatible UART and not a PL011.
SBSA UART is a subset of PL011 r1p5. A full PL011 implementation in Xen
would just be too difficult, so guests may require some drivers changes.

Enable vpl011 conditionally if the user requested it.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
---
Changes in v4:
- move rename set_interrupt_ppi and making set_interrupt_ppi generic to
  a separate patch
- properly name the vpl011 device node name
- code style
- use #define for addrcells and sizecells

Changes in v3:
- use bool
- retain BUG_ON(irq < 16)
- add vpl011 bool to kinfo
- return error of vpl011 is required but CONFIG_SBSA_VUART_CONSOLE is
  missing

Changes in v2:
- code style fixes
- make set_interrupt_ppi generic
- rename set_interrupt_ppi to set_interrupt
- only make the vpl011 node if the option was enabled
---
 xen/arch/arm/domain_build.c | 61 +++++++++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/kernel.h       |  3 +++
 2 files changed, 64 insertions(+)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 760ebf8..049ab84 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1605,6 +1605,54 @@ static int __init make_timer_domU_node(const struct domain *d, void *fdt)
     return res;
 }
 
+#ifdef CONFIG_SBSA_VUART_CONSOLE
+static int __init make_vpl011_uart_node(const struct domain *d, void *fdt)
+{
+    int res;
+    gic_interrupt_t intr;
+    __be32 reg[GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS];
+    __be32 *cells;
+
+    res = fdt_begin_node(fdt, "sbsa-uart@"__stringify(GUEST_PL011_BASE));
+    if ( res )
+        return res;
+
+    res = fdt_property_string(fdt, "compatible", "arm,sbsa-uart");
+    if ( res )
+        return res;
+
+    cells = &reg[0];
+    dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS,
+                       GUEST_ROOT_SIZE_CELLS, GUEST_PL011_BASE,
+                       GUEST_PL011_SIZE);
+    if ( res )
+        return res;
+    res = fdt_property(fdt, "reg", reg, sizeof(reg));
+    if ( res )
+        return res;
+
+    set_interrupt(intr, GUEST_VPL011_SPI, 0xf, DT_IRQ_TYPE_LEVEL_HIGH);
+
+    res = fdt_property(fdt, "interrupts", intr, sizeof (intr));
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "interrupt-parent",
+                            GUEST_PHANDLE_GIC);
+    if ( res )
+        return res;
+
+    /* Use a default baud rate of 115200. */
+    fdt_property_u32(fdt, "current-speed", 115200);
+
+    res = fdt_end_node(fdt);
+    if ( res )
+        return res;
+
+    return 0;
+}
+#endif
+
 /*
  * The max size for DT is 2MB. However, the generated DT is small, 4KB
  * are enough for now, but we might have to increase it in the future.
@@ -1666,6 +1714,16 @@ static int __init prepare_dtb_domU(struct domain *d, struct kernel_info *kinfo)
     if ( ret )
         goto err;
 
+    if ( kinfo->vpl011 )
+    {
+        ret = -EINVAL;
+#ifdef CONFIG_SBSA_VUART_CONSOLE
+        ret = make_vpl011_uart_node(d, kinfo->fdt);
+#endif
+        if ( ret )
+            goto err;
+    }
+
     ret = fdt_end_node(kinfo->fdt);
     if ( ret < 0 )
         goto err;
@@ -2523,6 +2581,7 @@ static int __init construct_domU(struct domain *d,
     struct kernel_info kinfo = {};
     int rc;
     u64 mem;
+    u32 len;
 
     rc = dt_property_read_u64(node, "memory", &mem);
     if ( !rc )
@@ -2534,6 +2593,8 @@ static int __init construct_domU(struct domain *d,
 
     printk("*** LOADING DOMU cpus=%u memory=%luKB ***\n", d->max_vcpus, mem);
 
+    kinfo.vpl011 = dt_get_property(node, "vpl011", &len) != NULL;
+
     d->vcpu = xzalloc_array(struct vcpu *, d->max_vcpus);
     if ( !d->vcpu )
         return -ENOMEM;;
diff --git a/xen/arch/arm/kernel.h b/xen/arch/arm/kernel.h
index 4320f72..33f3e72 100644
--- a/xen/arch/arm/kernel.h
+++ b/xen/arch/arm/kernel.h
@@ -33,6 +33,9 @@ struct kernel_info {
     paddr_t dtb_paddr;
     paddr_t initrd_paddr;
 
+    /* Enable pl011 emulation */
+    bool vpl011;
+
     /* loader to use for this kernel */
     void (*load)(struct kernel_info *info);
     /* loader specific state */
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v4 17/23] xen/arm: introduce a union in vpl011

[Stefano Stabellini]

Introduce a union in struct vpl011 to contain the console ring members.
A later patch will add another member of the union for the case where
the backend is in Xen.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
---
Changes in v4:
- name union "backend"

Changes in v3:
- rename ring field to dom

Changes in v2:
- new patch
---
 xen/arch/arm/vpl011.c        | 22 ++++++++++++----------
 xen/include/asm-arm/vpl011.h |  8 ++++++--
 2 files changed, 18 insertions(+), 12 deletions(-)

diff --git a/xen/arch/arm/vpl011.c b/xen/arch/arm/vpl011.c
index a281eab..ebc045e 100644
--- a/xen/arch/arm/vpl011.c
+++ b/xen/arch/arm/vpl011.c
@@ -82,7 +82,7 @@ static uint8_t vpl011_read_data(struct domain *d)
     unsigned long flags;
     uint8_t data = 0;
     struct vpl011 *vpl011 = &d->arch.vpl011;
-    struct xencons_interface *intf = vpl011->ring_buf;
+    struct xencons_interface *intf = vpl011->backend.dom.ring_buf;
     XENCONS_RING_IDX in_cons, in_prod;
 
     VPL011_LOCK(d, flags);
@@ -145,7 +145,7 @@ static uint8_t vpl011_read_data(struct domain *d)
 static void vpl011_update_tx_fifo_status(struct vpl011 *vpl011,
                                          unsigned int fifo_level)
 {
-    struct xencons_interface *intf = vpl011->ring_buf;
+    struct xencons_interface *intf = vpl011->backend.dom.ring_buf;
     unsigned int fifo_threshold = sizeof(intf->out) - SBSA_UART_FIFO_LEVEL;
 
     BUILD_BUG_ON(sizeof(intf->out) < SBSA_UART_FIFO_SIZE);
@@ -164,7 +164,7 @@ static void vpl011_write_data(struct domain *d, uint8_t data)
 {
     unsigned long flags;
     struct vpl011 *vpl011 = &d->arch.vpl011;
-    struct xencons_interface *intf = vpl011->ring_buf;
+    struct xencons_interface *intf = vpl011->backend.dom.ring_buf;
     XENCONS_RING_IDX out_cons, out_prod;
 
     VPL011_LOCK(d, flags);
@@ -382,7 +382,7 @@ static void vpl011_data_avail(struct domain *d)
 {
     unsigned long flags;
     struct vpl011 *vpl011 = &d->arch.vpl011;
-    struct xencons_interface *intf = vpl011->ring_buf;
+    struct xencons_interface *intf = vpl011->backend.dom.ring_buf;
     XENCONS_RING_IDX in_cons, in_prod, out_cons, out_prod;
     XENCONS_RING_IDX in_fifo_level, out_fifo_level;
 
@@ -459,14 +459,14 @@ int domain_vpl011_init(struct domain *d, struct vpl011_init_info *info)
     int rc;
     struct vpl011 *vpl011 = &d->arch.vpl011;
 
-    if ( vpl011->ring_buf )
+    if ( vpl011->backend.dom.ring_buf )
         return -EINVAL;
 
     /* Map the guest PFN to Xen address space. */
     rc =  prepare_ring_for_helper(d,
                                   gfn_x(info->gfn),
-                                  &vpl011->ring_page,
-                                  &vpl011->ring_buf);
+                                  &vpl011->backend.dom.ring_page,
+                                  &vpl011->backend.dom.ring_buf);
     if ( rc < 0 )
         goto out;
 
@@ -495,7 +495,8 @@ out2:
     vgic_free_virq(d, GUEST_VPL011_SPI);
 
 out1:
-    destroy_ring_for_helper(&vpl011->ring_buf, vpl011->ring_page);
+    destroy_ring_for_helper(&vpl011->backend.dom.ring_buf,
+			                vpl011->backend.dom.ring_page);
 
 out:
     return rc;
@@ -505,11 +506,12 @@ void domain_vpl011_deinit(struct domain *d)
 {
     struct vpl011 *vpl011 = &d->arch.vpl011;
 
-    if ( !vpl011->ring_buf )
+    if ( !vpl011->backend.dom.ring_buf )
         return;
 
     free_xen_event_channel(d, vpl011->evtchn);
-    destroy_ring_for_helper(&vpl011->ring_buf, vpl011->ring_page);
+    destroy_ring_for_helper(&vpl011->backend.dom.ring_buf,
+			                vpl011->backend.dom.ring_page);
 }
 
 /*
diff --git a/xen/include/asm-arm/vpl011.h b/xen/include/asm-arm/vpl011.h
index db95ff8..42d7a24 100644
--- a/xen/include/asm-arm/vpl011.h
+++ b/xen/include/asm-arm/vpl011.h
@@ -31,8 +31,12 @@
 #define SBSA_UART_FIFO_SIZE 32
 
 struct vpl011 {
-    void *ring_buf;
-    struct page_info *ring_page;
+    union {
+        struct {
+            void *ring_buf;
+            struct page_info *ring_page;
+        } dom;
+    } backend;
     uint32_t    uartfr;         /* Flag register */
     uint32_t    uartcr;         /* Control register */
     uint32_t    uartimsc;       /* Interrupt mask register*/
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v4 18/23] xen/arm: refactor vpl011_data_avail

[Stefano Stabellini]

Move the code to calculate in_fifo_level and out_fifo_level out of
vpl011_data_avail, to the caller.
This change will make it possible to reuse vpl011_data_avail with
different ring structures in a later patch.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
Acked-by: Julien Grall <julien.grall@arm.com>

---
Changes in v3:
- remove forward declaration of vpl011_data_avail

Changes in v2:
- new patch
---
 xen/arch/arm/vpl011.c | 64 +++++++++++++++++++++++++++++----------------------
 1 file changed, 36 insertions(+), 28 deletions(-)

diff --git a/xen/arch/arm/vpl011.c b/xen/arch/arm/vpl011.c
index ebc045e..68452a8 100644
--- a/xen/arch/arm/vpl011.c
+++ b/xen/arch/arm/vpl011.c
@@ -378,30 +378,13 @@ static const struct mmio_handler_ops vpl011_mmio_handler = {
     .write = vpl011_mmio_write,
 };
 
-static void vpl011_data_avail(struct domain *d)
+static void vpl011_data_avail(struct domain *d,
+                              XENCONS_RING_IDX in_fifo_level,
+                              XENCONS_RING_IDX in_size,
+                              XENCONS_RING_IDX out_fifo_level,
+                              XENCONS_RING_IDX out_size)
 {
-    unsigned long flags;
     struct vpl011 *vpl011 = &d->arch.vpl011;
-    struct xencons_interface *intf = vpl011->backend.dom.ring_buf;
-    XENCONS_RING_IDX in_cons, in_prod, out_cons, out_prod;
-    XENCONS_RING_IDX in_fifo_level, out_fifo_level;
-
-    VPL011_LOCK(d, flags);
-
-    in_cons = intf->in_cons;
-    in_prod = intf->in_prod;
-    out_cons = intf->out_cons;
-    out_prod = intf->out_prod;
-
-    smp_rmb();
-
-    in_fifo_level = xencons_queued(in_prod,
-                                   in_cons,
-                                   sizeof(intf->in));
-
-    out_fifo_level = xencons_queued(out_prod,
-                                    out_cons,
-                                    sizeof(intf->out));
 
     /**** Update the UART RX state ****/
 
@@ -410,11 +393,11 @@ static void vpl011_data_avail(struct domain *d)
         vpl011->uartfr &= ~RXFE;
 
     /* Set the FIFO_FULL bit if the Xen buffer is full. */
-    if ( in_fifo_level == sizeof(intf->in) )
+    if ( in_fifo_level == in_size )
         vpl011->uartfr |= RXFF;
 
     /* Assert the RX interrupt if the FIFO is more than half way filled. */
-    if ( in_fifo_level >= sizeof(intf->in) - SBSA_UART_FIFO_LEVEL )
+    if ( in_fifo_level >= in_size - SBSA_UART_FIFO_LEVEL )
         vpl011->uartris |= RXI;
 
     /*
@@ -427,7 +410,7 @@ static void vpl011_data_avail(struct domain *d)
 
     /**** Update the UART TX state ****/
 
-    if ( out_fifo_level != sizeof(intf->out) )
+    if ( out_fifo_level != out_size )
     {
         vpl011->uartfr &= ~TXFF;
 
@@ -445,13 +428,38 @@ static void vpl011_data_avail(struct domain *d)
 
     if ( out_fifo_level == 0 )
         vpl011->uartfr |= TXFE;
-
-    VPL011_UNLOCK(d, flags);
 }
 
 static void vpl011_notification(struct vcpu *v, unsigned int port)
 {
-    vpl011_data_avail(v->domain);
+    unsigned long flags;
+    struct domain *d = v->domain;
+    struct vpl011 *vpl011 = &d->arch.vpl011;
+    struct xencons_interface *intf = vpl011->backend.dom.ring_buf;
+    XENCONS_RING_IDX in_cons, in_prod, out_cons, out_prod;
+    XENCONS_RING_IDX in_fifo_level, out_fifo_level;
+
+    VPL011_LOCK(d, flags);
+
+    in_cons = intf->in_cons;
+    in_prod = intf->in_prod;
+    out_cons = intf->out_cons;
+    out_prod = intf->out_prod;
+
+    smp_rmb();
+
+    in_fifo_level = xencons_queued(in_prod,
+                                   in_cons,
+                                   sizeof(intf->in));
+
+    out_fifo_level = xencons_queued(out_prod,
+                                    out_cons,
+                                    sizeof(intf->out));
+
+    vpl011_data_avail(v->domain, in_fifo_level, sizeof(intf->in),
+                      out_fifo_level, sizeof(intf->out));
+
+    VPL011_UNLOCK(d, flags);
 }
 
 int domain_vpl011_init(struct domain *d, struct vpl011_init_info *info)
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v4 19/23] xen/arm: Allow vpl011 to be used by DomU

[Stefano Stabellini]

Make vpl011 being able to be used without a userspace component in Dom0.
In that case, output is printed to the Xen serial and input is received
from the Xen serial one character at a time.

Call domain_vpl011_init during construct_domU if vpl011 is enabled.

Introduce a new ring struct with only the ring array to avoid a waste of
memory. Introduce separate read_data and write_data functions for
initial domains: vpl011_write_data_xen is very simple and just writes
to the console, while vpl011_read_data_xen is a duplicate of
vpl011_read_data. Although textually almost identical, we are forced to
duplicate the functions because the struct layout is different.

Output characters are printed one by one, potentially leading to
intermixed output of different domains on the console. A follow-up patch
will solve the issue by introducing buffering.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
---
Changes in v4:
- code style

Changes in v3:
- add in-code comments
- improve existing comments
- remove ifdef around domain_vpl011_init in construct_domU
- add ASSERT
- use SBSA_UART_FIFO_SIZE for in buffer size
- rename ring_enable to backend_in_domain
- rename struct xencons_in to struct vpl011_xen_backend
- rename inring field to xen
- rename helper functions accordingly
- remove unnecessary stub implementation of vpl011_rx_char
- move vpl011_rx_char_xen within the file to avoid the need of a forward
  declaration of vpl011_data_avail
- fix small bug in vpl011_rx_char_xen: increment in_prod before using it
  to check xencons_queued.

Changes in v2:
- only init if vpl011
- rename vpl011_read_char to vpl011_rx_char
- remove spurious change
- fix coding style
- use different ring struct
- move the write_data changes to their own function
  (vpl011_write_data_noring)
- duplicate vpl011_read_data
---
 xen/arch/arm/domain_build.c  |   9 +-
 xen/arch/arm/vpl011.c        | 200 +++++++++++++++++++++++++++++++++++++------
 xen/include/asm-arm/vpl011.h |   8 ++
 3 files changed, 192 insertions(+), 25 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 049ab84..4379538 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2618,7 +2618,14 @@ static int __init construct_domU(struct domain *d,
     if ( rc < 0 )
         return rc;
 
-    return __construct_domain(d, &kinfo);
+    rc = __construct_domain(d, &kinfo);
+    if ( rc < 0 )
+        return rc;
+
+    if ( kinfo.vpl011 )
+        rc = domain_vpl011_init(d, NULL);
+
+    return rc;
 }
 
 void __init create_domUs(void)
diff --git a/xen/arch/arm/vpl011.c b/xen/arch/arm/vpl011.c
index 68452a8..131507e 100644
--- a/xen/arch/arm/vpl011.c
+++ b/xen/arch/arm/vpl011.c
@@ -77,6 +77,91 @@ static void vpl011_update_interrupt_status(struct domain *d)
 #endif
 }
 
+/*
+ * vpl011_write_data_xen writes chars from the vpl011 out buffer to the
+ * console. Only to be used when the backend is Xen.
+ */
+static void vpl011_write_data_xen(struct domain *d, uint8_t data)
+{
+    unsigned long flags;
+    struct vpl011 *vpl011 = &d->arch.vpl011;
+
+    VPL011_LOCK(d, flags);
+
+    printk("%c", data);
+    if (data == '\n')
+        printk("DOM%u: ", d->domain_id);
+
+    vpl011->uartris |= TXI;
+    vpl011->uartfr &= ~TXFE;
+    vpl011_update_interrupt_status(d);
+
+    VPL011_UNLOCK(d, flags);
+}
+
+/*
+ * vpl011_read_data_xen reads data when the backend is xen. Characters
+ * are added to the vpl011 receive buffer by vpl011_rx_char_xen.
+ */
+static uint8_t vpl011_read_data_xen(struct domain *d)
+{
+    unsigned long flags;
+    uint8_t data = 0;
+    struct vpl011 *vpl011 = &d->arch.vpl011;
+    struct vpl011_xen_backend *intf = vpl011->backend.xen;
+    XENCONS_RING_IDX in_cons, in_prod;
+
+    VPL011_LOCK(d, flags);
+
+    in_cons = intf->in_cons;
+    in_prod = intf->in_prod;
+
+    smp_rmb();
+
+    /*
+     * It is expected that there will be data in the ring buffer when this
+     * function is called since the guest is expected to read the data register
+     * only if the TXFE flag is not set.
+     * If the guest still does read when TXFE bit is set then 0 will be returned.
+     */
+    if ( xencons_queued(in_prod, in_cons, sizeof(intf->in)) > 0 )
+    {
+        unsigned int fifo_level;
+
+        data = intf->in[xencons_mask(in_cons, sizeof(intf->in))];
+        in_cons += 1;
+        smp_mb();
+        intf->in_cons = in_cons;
+
+        fifo_level = xencons_queued(in_prod, in_cons, sizeof(intf->in));
+
+        /* If the FIFO is now empty, we clear the receive timeout interrupt. */
+        if ( fifo_level == 0 )
+        {
+            vpl011->uartfr |= RXFE;
+            vpl011->uartris &= ~RTI;
+        }
+
+        /* If the FIFO is more than half empty, we clear the RX interrupt. */
+        if ( fifo_level < sizeof(intf->in) - SBSA_UART_FIFO_LEVEL )
+            vpl011->uartris &= ~RXI;
+
+        vpl011_update_interrupt_status(d);
+    }
+    else
+        gprintk(XENLOG_ERR, "vpl011: Unexpected IN ring buffer empty\n");
+
+    /*
+     * We have consumed a character or the FIFO was empty, so clear the
+     * "FIFO full" bit.
+     */
+    vpl011->uartfr &= ~RXFF;
+
+    VPL011_UNLOCK(d, flags);
+
+    return data;
+}
+
 static uint8_t vpl011_read_data(struct domain *d)
 {
     unsigned long flags;
@@ -240,7 +325,10 @@ static int vpl011_mmio_read(struct vcpu *v,
     case DR:
         if ( !vpl011_reg32_check_access(dabt) ) goto bad_width;
 
-        *r = vreg_reg32_extract(vpl011_read_data(d), info);
+        if ( vpl011->backend_in_domain )
+            *r = vreg_reg32_extract(vpl011_read_data(d), info);
+        else
+            *r = vreg_reg32_extract(vpl011_read_data_xen(d), info);
         return 1;
 
     case RSR:
@@ -325,7 +413,10 @@ static int vpl011_mmio_write(struct vcpu *v,
 
         vreg_reg32_update(&data, r, info);
         data &= 0xFF;
-        vpl011_write_data(v->domain, data);
+        if ( vpl011->backend_in_domain )
+            vpl011_write_data(v->domain, data);
+        else
+            vpl011_write_data_xen(v->domain, data);
         return 1;
     }
 
@@ -430,6 +521,39 @@ static void vpl011_data_avail(struct domain *d,
         vpl011->uartfr |= TXFE;
 }
 
+/*
+ * vpl011_rx_char_xen adds a char to a domain's vpl011 receive buffer.
+ * It is only used when the vpl011 backend is in Xen.
+ */
+void vpl011_rx_char_xen(struct domain *d, char c)
+{
+    unsigned long flags;
+    struct vpl011 *vpl011 = &d->arch.vpl011;
+    struct vpl011_xen_backend *intf = vpl011->backend.xen;
+    XENCONS_RING_IDX in_cons, in_prod, in_fifo_level;
+
+    ASSERT(!vpl011->backend_in_domain);
+    VPL011_LOCK(d, flags);
+
+    in_cons = intf->in_cons;
+    in_prod = intf->in_prod;
+    if ( xencons_queued(in_prod, in_cons, sizeof(intf->in)) == sizeof(intf->in) )
+    {
+        VPL011_UNLOCK(d, flags);
+        return;
+    }
+
+    intf->in[xencons_mask(in_prod, sizeof(intf->in))] = c;
+    intf->in_prod = ++in_prod;
+
+    in_fifo_level = xencons_queued(in_prod,
+                                   in_cons,
+                                   sizeof(intf->in));
+
+    vpl011_data_avail(d, in_fifo_level, sizeof(intf->in), 0, SBSA_UART_FIFO_SIZE);
+    VPL011_UNLOCK(d, flags);
+}
+
 static void vpl011_notification(struct vcpu *v, unsigned int port)
 {
     unsigned long flags;
@@ -470,27 +594,47 @@ int domain_vpl011_init(struct domain *d, struct vpl011_init_info *info)
     if ( vpl011->backend.dom.ring_buf )
         return -EINVAL;
 
-    /* Map the guest PFN to Xen address space. */
-    rc =  prepare_ring_for_helper(d,
-                                  gfn_x(info->gfn),
-                                  &vpl011->backend.dom.ring_page,
-                                  &vpl011->backend.dom.ring_buf);
-    if ( rc < 0 )
-        goto out;
+    /*
+     * info is NULL when the backend is in Xen.
+     * info is != NULL when the backend is in a domain.
+     */
+    if ( info != NULL )
+    {
+        vpl011->backend_in_domain = true;
+
+        /* Map the guest PFN to Xen address space. */
+        rc =  prepare_ring_for_helper(d,
+                                      gfn_x(info->gfn),
+                                      &vpl011->backend.dom.ring_page,
+                                      &vpl011->backend.dom.ring_buf);
+        if ( rc < 0 )
+            goto out;
+
+        rc = alloc_unbound_xen_event_channel(d, 0, info->console_domid,
+                                             vpl011_notification);
+        if ( rc < 0 )
+            goto out1;
+
+        vpl011->evtchn = info->evtchn = rc;
+    }
+    else
+    {
+        vpl011->backend_in_domain = false;
+
+        vpl011->backend.xen = xzalloc(struct vpl011_xen_backend);
+        if ( vpl011->backend.xen == NULL )
+        {
+            rc = -EINVAL;
+            goto out1;
+        }
+    }
 
     rc = vgic_reserve_virq(d, GUEST_VPL011_SPI);
     if ( !rc )
     {
         rc = -EINVAL;
-        goto out1;
-    }
-
-    rc = alloc_unbound_xen_event_channel(d, 0, info->console_domid,
-                                         vpl011_notification);
-    if ( rc < 0 )
         goto out2;
-
-    vpl011->evtchn = info->evtchn = rc;
+    }
 
     spin_lock_init(&vpl011->lock);
 
@@ -503,8 +647,11 @@ out2:
     vgic_free_virq(d, GUEST_VPL011_SPI);
 
 out1:
-    destroy_ring_for_helper(&vpl011->backend.dom.ring_buf,
-			                vpl011->backend.dom.ring_page);
+    if ( vpl011->backend_in_domain )
+        destroy_ring_for_helper(&vpl011->backend.dom.ring_buf,
+                                vpl011->backend.dom.ring_page);
+    else
+        xfree(vpl011->backend.xen);
 
 out:
     return rc;
@@ -514,12 +661,17 @@ void domain_vpl011_deinit(struct domain *d)
 {
     struct vpl011 *vpl011 = &d->arch.vpl011;
 
-    if ( !vpl011->backend.dom.ring_buf )
-        return;
+    if ( vpl011->backend_in_domain )
+    {
+        if ( !vpl011->backend.dom.ring_buf )
+            return;
 
-    free_xen_event_channel(d, vpl011->evtchn);
-    destroy_ring_for_helper(&vpl011->backend.dom.ring_buf,
-			                vpl011->backend.dom.ring_page);
+        free_xen_event_channel(d, vpl011->evtchn);
+        destroy_ring_for_helper(&vpl011->backend.dom.ring_buf,
+                                vpl011->backend.dom.ring_page);
+    }
+    else
+        xfree(vpl011->backend.xen);
 }
 
 /*
diff --git a/xen/include/asm-arm/vpl011.h b/xen/include/asm-arm/vpl011.h
index 42d7a24..5eb6d25 100644
--- a/xen/include/asm-arm/vpl011.h
+++ b/xen/include/asm-arm/vpl011.h
@@ -21,6 +21,7 @@
 
 #include <public/domctl.h>
 #include <public/io/ring.h>
+#include <public/io/console.h>
 #include <asm/vreg.h>
 #include <xen/mm.h>
 
@@ -29,13 +30,19 @@
 #define VPL011_UNLOCK(d,flags) spin_unlock_irqrestore(&(d)->arch.vpl011.lock, flags)
 
 #define SBSA_UART_FIFO_SIZE 32
+struct vpl011_xen_backend {
+    char in[SBSA_UART_FIFO_SIZE];
+    XENCONS_RING_IDX in_cons, in_prod;
+};
 
 struct vpl011 {
+    bool backend_in_domain;
     union {
         struct {
             void *ring_buf;
             struct page_info *ring_page;
         } dom;
+        struct vpl011_xen_backend *xen;
     } backend;
     uint32_t    uartfr;         /* Flag register */
     uint32_t    uartcr;         /* Control register */
@@ -57,6 +64,7 @@ struct vpl011_init_info {
 int domain_vpl011_init(struct domain *d,
                        struct vpl011_init_info *info);
 void domain_vpl011_deinit(struct domain *d);
+void vpl011_rx_char_xen(struct domain *d, char c);
 #else
 static inline int domain_vpl011_init(struct domain *d,
                                      struct vpl011_init_info *info)
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v4 20/23] xen: support console_switching between Dom0 and DomUs on ARM

[Stefano Stabellini]

Today Ctrl-AAA is used to switch between Xen and Dom0. Extend the
mechanism to allow for switching between Xen, Dom0, and any of the
initial DomU created from Xen alongside Dom0 out of information provided
via device tree.

Rename xen_rx to console_rx to match the new behavior.

Clarify existing comment about "notify the guest", making it clear that
it is only about the hardware domain.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
CC: andrew.cooper3@citrix.com
CC: George.Dunlap@eu.citrix.com
CC: ian.jackson@eu.citrix.com
CC: jbeulich@suse.com
CC: konrad.wilk@oracle.com
CC: tim@xen.org
CC: wei.liu2@citrix.com
---
Changes in v4:
- handle console_rx == 0 in console_input_domain
- use rcu_lock_by_domain instead of get_domain_by_id
- handle the case where the returned domain is NULL
- send_global_virq(VIRQ_CONSOLE) only when chars are for Dom0
- make console_rx unsigned int
- fix comment
- code readability improvement
- fix opt_conswitch[1] == 'x' case
- move console_input_domain to next patch

Changes in v3:
- only call vpl011 functions ifdef CONFIG_SBSA_VUART_CONSOLE
- add blank line and spaces
- remove xen_rx from print messages
- rename xen_rx to console_rx
- keep switch_serial_input() at boot
- add better comments
- fix existing comment
- add warning if no guests console/uart is available
- add console_input_domain function

Changes in v2:
- only call vpl011_rx_char if the vpl011 has been initialized
---
 xen/drivers/char/console.c | 73 +++++++++++++++++++++++++++++++++++++---------
 1 file changed, 59 insertions(+), 14 deletions(-)

diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
index 3b75f7a..0808bac 100644
--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -31,10 +31,13 @@
 #include <xen/early_printk.h>
 #include <xen/warning.h>
 #include <xen/pv_console.h>
+#include <asm/setup.h>
 
 #ifdef CONFIG_X86
 #include <xen/consoled.h>
 #include <asm/guest.h>
+#else
+#include <asm/vpl011.h>
 #endif
 
 /* console: comma-separated list of console outputs. */
@@ -391,31 +394,73 @@ static void dump_console_ring_key(unsigned char key)
     free_xenheap_pages(buf, order);
 }
 
-/* CTRL-<switch_char> switches input direction between Xen and DOM0. */
+/*
+ * CTRL-<switch_char> switches input direction between Xen, Dom0 and
+ * DomUs.
+ */
 #define switch_code (opt_conswitch[0]-'a'+1)
-static int __read_mostly xen_rx = 1; /* FALSE => input passed to domain 0. */
+/*
+ * console_rx=0 => input to xen
+ * console_rx=1 => input to dom0
+ * console_rx=N => input dom(N-1)
+ */
+static unsigned int __read_mostly console_rx = 0;
 
 static void switch_serial_input(void)
 {
-    static char *input_str[2] = { "DOM0", "Xen" };
-    xen_rx = !xen_rx;
-    printk("*** Serial input -> %s", input_str[xen_rx]);
+    if ( console_rx++ == max_init_domid + 1 )
+        console_rx = 0;
+
+    if ( !console_rx )
+        printk("*** Serial input to Xen");
+    else
+        printk("*** Serial input to DOM%d", console_rx - 1);
+
     if ( switch_code )
-        printk(" (type 'CTRL-%c' three times to switch input to %s)",
-               opt_conswitch[0], input_str[!xen_rx]);
+        printk(" (type 'CTRL-%c' three times to switch input)",
+               opt_conswitch[0]);
     printk("\n");
 }
 
 static void __serial_rx(char c, struct cpu_user_regs *regs)
 {
-    if ( xen_rx )
+    if ( console_rx == 0 )
         return handle_keypress(c, regs);
 
-    /* Deliver input to guest buffer, unless it is already full. */
-    if ( (serial_rx_prod-serial_rx_cons) != SERIAL_RX_SIZE )
-        serial_rx_ring[SERIAL_RX_MASK(serial_rx_prod++)] = c;
-    /* Always notify the guest: prevents receive path from getting stuck. */
-    send_global_virq(VIRQ_CONSOLE);
+    if ( console_rx == 1 )
+    {
+        /* Deliver input to hardware domain, unless it is already full. */
+        if ( (serial_rx_prod - serial_rx_cons) != SERIAL_RX_SIZE )
+            serial_rx_ring[SERIAL_RX_MASK(serial_rx_prod++)] = c;
+
+        /*
+         * Always notify the hardware domain: prevents receive path from
+         * getting stuck.
+         */
+        send_global_virq(VIRQ_CONSOLE);
+    }
+#ifdef CONFIG_SBSA_VUART_CONSOLE
+    else
+    {
+        struct domain *d = rcu_lock_domain_by_any_id(console_rx - 1);
+
+        /*
+         * If we have a properly initialized vpl011 console for the
+         * domain, without a full PV ring to Dom0 (in that case input
+         * comes from the PV ring), then send the character to it.
+         */
+        if ( d != NULL &&
+             !d->arch.vpl011.backend_in_domain &&
+             d->arch.vpl011.backend.xen != NULL )
+            vpl011_rx_char_xen(d, c);
+        else
+            printk("Cannot send chars to Dom%d: no UART available\n",
+                   console_rx - 1);
+
+        if ( d != NULL )
+            rcu_unlock_domain(d);
+    }
+#endif
 
 #ifdef CONFIG_X86
     if ( pv_shim && pv_console )
@@ -943,7 +988,7 @@ void __init console_endboot(void)
      * a useful 'how to switch' message.
      */
     if ( opt_conswitch[1] == 'x' )
-        xen_rx = !xen_rx;
+        console_rx = max_init_domid + 1;
 
     register_keyhandler('w', dump_console_ring_key,
                         "synchronously dump console ring buffer (dmesg)", 0);
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v4 21/23] xen/vpl011: buffer out chars when the backend is xen

[Stefano Stabellini]

To avoid mixing the output of different domains on the console, buffer
the output chars and print line by line. Unless the domain has input
from the serial, in which case we want to print char by char for a
smooth user experience.

The size of SBSA_UART_OUT_BUF_SIZE is arbitrary, choose the same size
as VUART_BUT_SIZE used in vuart.c.

Export a function named console_input_domain() to allow others to know
which domains has input at a given time.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
CC: andrew.cooper3@citrix.com
CC: George.Dunlap@eu.citrix.com
CC: ian.jackson@eu.citrix.com
CC: jbeulich@suse.com
CC: konrad.wilk@oracle.com
CC: tim@xen.org
CC: wei.liu2@citrix.com
---
XXX: merge this patch with "xen/arm: Allow vpl011 to be used by DomU" on
     commit

Changes in v4:
- make SBSA_UART_OUT_BUF_SIZE the same size of VUART_BUT_SIZE
- rearrange the code to be clearer input and != input cases
- code style
- add \n when printing the out buffer because is full
- don't print prefix for input domain
---
 xen/arch/arm/vpl011.c        | 35 ++++++++++++++++++++++++++++++++---
 xen/drivers/char/console.c   |  7 +++++++
 xen/include/asm-arm/vpl011.h |  4 ++++
 xen/include/xen/console.h    |  2 ++
 4 files changed, 45 insertions(+), 3 deletions(-)

diff --git a/xen/arch/arm/vpl011.c b/xen/arch/arm/vpl011.c
index 131507e..5e57ada 100644
--- a/xen/arch/arm/vpl011.c
+++ b/xen/arch/arm/vpl011.c
@@ -28,6 +28,7 @@
 #include <xen/lib.h>
 #include <xen/mm.h>
 #include <xen/sched.h>
+#include <xen/console.h>
 #include <public/domctl.h>
 #include <public/io/console.h>
 #include <asm/pl011-uart.h>
@@ -85,12 +86,40 @@ static void vpl011_write_data_xen(struct domain *d, uint8_t data)
 {
     unsigned long flags;
     struct vpl011 *vpl011 = &d->arch.vpl011;
+    struct vpl011_xen_backend *intf = vpl011->backend.xen;
+    struct domain *input = console_input_domain();
 
     VPL011_LOCK(d, flags);
 
-    printk("%c", data);
-    if (data == '\n')
-        printk("DOM%u: ", d->domain_id);
+    intf->out[intf->out_prod++] = data;
+    if ( d == input )
+    {
+        if ( intf->out_prod == 1 )
+        {
+            printk("%c", data);
+            intf->out_prod = 0;
+        }
+        else
+        {
+            if ( data != '\n' )
+                intf->out[intf->out_prod++] = '\n';
+            intf->out[intf->out_prod++] = '\0';
+            printk("%s", intf->out);
+            intf->out_prod = 0;
+        }
+    }
+    else
+    {
+        if ( intf->out_prod == SBSA_UART_OUT_BUF_SIZE - 2 ||
+             data == '\n' )
+        {
+            if ( data != '\n' )
+                intf->out[intf->out_prod++] = '\n';
+            intf->out[intf->out_prod++] = '\0';
+            printk("DOM%u: %s", d->domain_id, intf->out);
+            intf->out_prod = 0;
+        }
+    }
 
     vpl011->uartris |= TXI;
     vpl011->uartfr &= ~TXFE;
diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
index 0808bac..9a2b29a 100644
--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -406,6 +406,13 @@ static void dump_console_ring_key(unsigned char key)
  */
 static unsigned int __read_mostly console_rx = 0;
 
+struct domain *console_input_domain(void)
+{
+    if ( console_rx == 0 )
+            return NULL;
+    return get_domain_by_id(console_rx - 1);
+}
+
 static void switch_serial_input(void)
 {
     if ( console_rx++ == max_init_domid + 1 )
diff --git a/xen/include/asm-arm/vpl011.h b/xen/include/asm-arm/vpl011.h
index 5eb6d25..ab6fd79 100644
--- a/xen/include/asm-arm/vpl011.h
+++ b/xen/include/asm-arm/vpl011.h
@@ -30,9 +30,13 @@
 #define VPL011_UNLOCK(d,flags) spin_unlock_irqrestore(&(d)->arch.vpl011.lock, flags)
 
 #define SBSA_UART_FIFO_SIZE 32
+/* Same size as VUART_BUT_SIZE, used in vuart.c */
+#define SBSA_UART_OUT_BUF_SIZE 128
 struct vpl011_xen_backend {
     char in[SBSA_UART_FIFO_SIZE];
+    char out[SBSA_UART_OUT_BUF_SIZE];
     XENCONS_RING_IDX in_cons, in_prod;
+    XENCONS_RING_IDX out_prod;
 };
 
 struct vpl011 {
diff --git a/xen/include/xen/console.h b/xen/include/xen/console.h
index 70c9911..c5a85c8 100644
--- a/xen/include/xen/console.h
+++ b/xen/include/xen/console.h
@@ -31,6 +31,8 @@ void console_end_sync(void);
 void console_start_log_everything(void);
 void console_end_log_everything(void);
 
+struct domain *console_input_domain(void);
+
 /*
  * Steal output from the console. Returns +ve identifier, else -ve error.
  * Takes the handle of the serial line to steal, and steal callback function.
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v4 22/23] xen/arm: move kernel.h to asm-arm/

[Stefano Stabellini]

It will be #included by a file in a xen/arch/arm subdirectory.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
---
 xen/arch/arm/domain_build.c  |  2 +-
 xen/arch/arm/kernel.c        |  3 +-
 xen/arch/arm/kernel.h        | 86 --------------------------------------------
 xen/include/asm-arm/kernel.h | 86 ++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 88 insertions(+), 89 deletions(-)
 delete mode 100644 xen/arch/arm/kernel.h
 create mode 100644 xen/include/asm-arm/kernel.h

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 4379538..dc9b46e 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -16,6 +16,7 @@
 #include <xen/warning.h>
 #include <acpi/actables.h>
 #include <asm/device.h>
+#include <asm/kernel.h>
 #include <asm/setup.h>
 #include <asm/platform.h>
 #include <asm/psci.h>
@@ -24,7 +25,6 @@
 
 #include <xen/irq.h>
 #include <xen/grant_table.h>
-#include "kernel.h"
 
 static unsigned int __initdata opt_dom0_max_vcpus;
 integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
diff --git a/xen/arch/arm/kernel.c b/xen/arch/arm/kernel.c
index 2239a07..b56aa79 100644
--- a/xen/arch/arm/kernel.c
+++ b/xen/arch/arm/kernel.c
@@ -16,8 +16,7 @@
 #include <xen/vmap.h>
 
 #include <asm/guest_access.h>
-
-#include "kernel.h"
+#include <asm/kernel.h>
 
 #define UIMAGE_MAGIC          0x27051956
 #define UIMAGE_NMLEN          32
diff --git a/xen/arch/arm/kernel.h b/xen/arch/arm/kernel.h
deleted file mode 100644
index 33f3e72..0000000
--- a/xen/arch/arm/kernel.h
+++ /dev/null
@@ -1,86 +0,0 @@
-/*
- * Kernel image loading.
- *
- * Copyright (C) 2011 Citrix Systems, Inc.
- */
-#ifndef __ARCH_ARM_KERNEL_H__
-#define __ARCH_ARM_KERNEL_H__
-
-#include <xen/device_tree.h>
-#include <asm/setup.h>
-
-struct kernel_info {
-#ifdef CONFIG_ARM_64
-    enum domain_type type;
-#endif
-
-    struct domain *d;
-
-    void *fdt; /* flat device tree */
-    paddr_t unassigned_mem; /* RAM not (yet) assigned to a bank */
-    struct meminfo mem;
-
-    /* kernel entry point */
-    paddr_t entry;
-
-    /* grant table region */
-    paddr_t gnttab_start;
-    paddr_t gnttab_size;
-
-    /* boot blob load addresses */
-    const struct bootmodule *kernel_bootmodule, *initrd_bootmodule;
-    const char* cmdline;
-    paddr_t dtb_paddr;
-    paddr_t initrd_paddr;
-
-    /* Enable pl011 emulation */
-    bool vpl011;
-
-    /* loader to use for this kernel */
-    void (*load)(struct kernel_info *info);
-    /* loader specific state */
-    union {
-        struct {
-            paddr_t kernel_addr;
-            paddr_t len;
-#ifdef CONFIG_ARM_64
-            paddr_t text_offset; /* 64-bit Image only */
-#endif
-            paddr_t start; /* 32-bit zImage only */
-        } zimage;
-    };
-};
-
-/*
- * Probe the kernel to detemine its type and select a loader.
- *
- * Sets in info:
- *  ->type
- *  ->load hook, and sets loader specific variables ->zimage
- */
-int kernel_probe(struct kernel_info *info, const struct dt_device_node *domain);
-
-/*
- * Loads the kernel into guest RAM.
- *
- * Expects to be set in info when called:
- *  ->mem
- *  ->fdt
- *
- * Sets in info:
- *  ->entry
- *  ->dtb_paddr
- *  ->initrd_paddr
- */
-void kernel_load(struct kernel_info *info);
-
-#endif /* #ifdef __ARCH_ARM_KERNEL_H__ */
-
-/*
- * Local variables:
- * mode: C
- * c-file-style: "BSD"
- * c-basic-offset: 4
- * indent-tabs-mode: nil
- * End:
- */
diff --git a/xen/include/asm-arm/kernel.h b/xen/include/asm-arm/kernel.h
new file mode 100644
index 0000000..33f3e72
--- /dev/null
+++ b/xen/include/asm-arm/kernel.h
@@ -0,0 +1,86 @@
+/*
+ * Kernel image loading.
+ *
+ * Copyright (C) 2011 Citrix Systems, Inc.
+ */
+#ifndef __ARCH_ARM_KERNEL_H__
+#define __ARCH_ARM_KERNEL_H__
+
+#include <xen/device_tree.h>
+#include <asm/setup.h>
+
+struct kernel_info {
+#ifdef CONFIG_ARM_64
+    enum domain_type type;
+#endif
+
+    struct domain *d;
+
+    void *fdt; /* flat device tree */
+    paddr_t unassigned_mem; /* RAM not (yet) assigned to a bank */
+    struct meminfo mem;
+
+    /* kernel entry point */
+    paddr_t entry;
+
+    /* grant table region */
+    paddr_t gnttab_start;
+    paddr_t gnttab_size;
+
+    /* boot blob load addresses */
+    const struct bootmodule *kernel_bootmodule, *initrd_bootmodule;
+    const char* cmdline;
+    paddr_t dtb_paddr;
+    paddr_t initrd_paddr;
+
+    /* Enable pl011 emulation */
+    bool vpl011;
+
+    /* loader to use for this kernel */
+    void (*load)(struct kernel_info *info);
+    /* loader specific state */
+    union {
+        struct {
+            paddr_t kernel_addr;
+            paddr_t len;
+#ifdef CONFIG_ARM_64
+            paddr_t text_offset; /* 64-bit Image only */
+#endif
+            paddr_t start; /* 32-bit zImage only */
+        } zimage;
+    };
+};
+
+/*
+ * Probe the kernel to detemine its type and select a loader.
+ *
+ * Sets in info:
+ *  ->type
+ *  ->load hook, and sets loader specific variables ->zimage
+ */
+int kernel_probe(struct kernel_info *info, const struct dt_device_node *domain);
+
+/*
+ * Loads the kernel into guest RAM.
+ *
+ * Expects to be set in info when called:
+ *  ->mem
+ *  ->fdt
+ *
+ * Sets in info:
+ *  ->entry
+ *  ->dtb_paddr
+ *  ->initrd_paddr
+ */
+void kernel_load(struct kernel_info *info);
+
+#endif /* #ifdef __ARCH_ARM_KERNEL_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v4 23/23] xen/arm: split domain_build.c

[Stefano Stabellini]

domain_build.c is too large.

Move all the ACPI specific device tree generating functions from
domain_build.c to acpi/domain_build.c.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>

---

Changes in v4:
- rename acpi_dt_build to domain_build.c
- add copyright header
- remove useless #include
- remove acpi_dt_build.h, add domain_build.h
---
 xen/arch/arm/acpi/Makefile         |   1 +
 xen/arch/arm/acpi/domain_build.c   | 592 +++++++++++++++++++++++++++++++++++++
 xen/arch/arm/domain_build.c        | 585 +-----------------------------------
 xen/include/asm-arm/domain_build.h |  31 ++
 4 files changed, 629 insertions(+), 580 deletions(-)
 create mode 100644 xen/arch/arm/acpi/domain_build.c
 create mode 100644 xen/include/asm-arm/domain_build.h

diff --git a/xen/arch/arm/acpi/Makefile b/xen/arch/arm/acpi/Makefile
index 23963f8..94ae249 100644
--- a/xen/arch/arm/acpi/Makefile
+++ b/xen/arch/arm/acpi/Makefile
@@ -1,2 +1,3 @@
 obj-y += lib.o
+obj-y += domain_build.o
 obj-y += boot.init.o
diff --git a/xen/arch/arm/acpi/domain_build.c b/xen/arch/arm/acpi/domain_build.c
new file mode 100644
index 0000000..44d3ad1
--- /dev/null
+++ b/xen/arch/arm/acpi/domain_build.c
@@ -0,0 +1,592 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <xen/mm.h>
+#include <xen/sched.h>
+#include <xen/acpi.h>
+#include <xen/event.h>
+#include <xen/iocap.h>
+#include <xen/device_tree.h>
+#include <xen/libfdt/libfdt.h>
+#include <acpi/actables.h>
+#include <asm/kernel.h>
+#include <asm/domain_build.h>
+
+/* Override macros from asm/page.h to make them work with mfn_t */
+#undef virt_to_mfn
+#define virt_to_mfn(va) _mfn(__virt_to_mfn(va))
+
+#define ACPI_DOM0_FDT_MIN_SIZE 4096
+
+static int __init acpi_iomem_deny_access(struct domain *d)
+{
+    acpi_status status;
+    struct acpi_table_spcr *spcr = NULL;
+    unsigned long mfn;
+    int rc;
+
+    /* Firstly permit full MMIO capabilities. */
+    rc = iomem_permit_access(d, 0UL, ~0UL);
+    if ( rc )
+        return rc;
+
+    /* TODO: Deny MMIO access for SMMU, GIC ITS */
+    status = acpi_get_table(ACPI_SIG_SPCR, 0,
+                            (struct acpi_table_header **)&spcr);
+
+    if ( ACPI_FAILURE(status) )
+    {
+        printk("Failed to get SPCR table\n");
+        return -EINVAL;
+    }
+
+    mfn = spcr->serial_port.address >> PAGE_SHIFT;
+    /* Deny MMIO access for UART */
+    rc = iomem_deny_access(d, mfn, mfn + 1);
+    if ( rc )
+        return rc;
+
+    /* Deny MMIO access for GIC regions */
+    return gic_iomem_deny_access(d);
+}
+
+static int __init acpi_route_spis(struct domain *d)
+{
+    int i, res;
+    struct irq_desc *desc;
+
+    /*
+     * Route the IRQ to hardware domain and permit the access.
+     * The interrupt type will be set by set by the hardware domain.
+     */
+    for( i = NR_LOCAL_IRQS; i < vgic_num_irqs(d); i++ )
+    {
+        /*
+         * TODO: Exclude the SPIs SMMU uses which should not be routed to
+         * the hardware domain.
+         */
+        desc = irq_to_desc(i);
+        if ( desc->action != NULL)
+            continue;
+
+        /* XXX: Shall we use a proper devname? */
+        res = map_irq_to_domain(d, i, true, "ACPI");
+        if ( res )
+            return res;
+    }
+
+    return 0;
+}
+
+static int __init acpi_make_hypervisor_node(const struct kernel_info *kinfo,
+                                            struct membank tbl_add[])
+{
+    const char compat[] =
+        "xen,xen-"__stringify(XEN_VERSION)"."__stringify(XEN_SUBVERSION)"\0"
+        "xen,xen";
+    int res;
+    /* Convenience alias */
+    void *fdt = kinfo->fdt;
+
+    dt_dprintk("Create hypervisor node\n");
+
+    /* See linux Documentation/devicetree/bindings/arm/xen.txt */
+    res = fdt_begin_node(fdt, "hypervisor");
+    if ( res )
+        return res;
+
+    /* Cannot use fdt_property_string due to embedded nulls */
+    res = fdt_property(fdt, "compatible", compat, sizeof(compat));
+    if ( res )
+        return res;
+
+    res = acpi_make_efi_nodes(fdt, tbl_add);
+    if ( res )
+        return res;
+
+    res = fdt_end_node(fdt);
+
+    return res;
+}
+
+/*
+ * Prepare a minimal DTB for Dom0 which contains bootargs, initrd, memory
+ * information, EFI table.
+ */
+static int __init create_acpi_dtb(struct kernel_info *kinfo,
+                                  struct membank tbl_add[])
+{
+    int new_size;
+    int ret;
+
+    dt_dprintk("Prepare a min DTB for DOM0\n");
+
+    /* Allocate min size for DT */
+    new_size = ACPI_DOM0_FDT_MIN_SIZE;
+    kinfo->fdt = xmalloc_bytes(new_size);
+
+    if ( kinfo->fdt == NULL )
+        return -ENOMEM;
+
+    /* Create a new empty DT for DOM0 */
+    ret = fdt_create(kinfo->fdt, new_size);
+    if ( ret < 0 )
+        goto err;
+
+    ret = fdt_finish_reservemap(kinfo->fdt);
+    if ( ret < 0 )
+        goto err;
+
+    ret = fdt_begin_node(kinfo->fdt, "/");
+    if ( ret < 0 )
+        goto err;
+
+    ret = fdt_property_cell(kinfo->fdt, "#address-cells", 2);
+    if ( ret )
+        return ret;
+
+    ret = fdt_property_cell(kinfo->fdt, "#size-cells", 1);
+    if ( ret )
+        return ret;
+
+    /* Create a chosen node for DOM0 */
+    ret = make_chosen_node(kinfo);
+    if ( ret )
+        goto err;
+
+    ret = acpi_make_hypervisor_node(kinfo, tbl_add);
+    if ( ret )
+        goto err;
+
+    ret = fdt_end_node(kinfo->fdt);
+    if ( ret < 0 )
+        goto err;
+
+    ret = fdt_finish(kinfo->fdt);
+    if ( ret < 0 )
+        goto err;
+
+    return 0;
+
+  err:
+    printk("Device tree generation failed (%d).\n", ret);
+    xfree(kinfo->fdt);
+    return -EINVAL;
+}
+
+static void __init acpi_map_other_tables(struct domain *d)
+{
+    int i;
+    unsigned long res;
+    u64 addr, size;
+
+    /* Map all ACPI tables to Dom0 using 1:1 mappings. */
+    for( i = 0; i < acpi_gbl_root_table_list.count; i++ )
+    {
+        addr = acpi_gbl_root_table_list.tables[i].address;
+        size = acpi_gbl_root_table_list.tables[i].length;
+        res = map_regions_p2mt(d,
+                               gaddr_to_gfn(addr),
+                               PFN_UP(size),
+                               maddr_to_mfn(addr),
+                               p2m_mmio_direct_c);
+        if ( res )
+        {
+             panic(XENLOG_ERR "Unable to map ACPI region 0x%"PRIx64
+                   " - 0x%"PRIx64" in domain\n",
+                   addr & PAGE_MASK, PAGE_ALIGN(addr + size) - 1);
+        }
+    }
+}
+
+static int __init acpi_create_rsdp(struct domain *d, struct membank tbl_add[])
+{
+
+    struct acpi_table_rsdp *rsdp = NULL;
+    u64 addr;
+    u64 table_size = sizeof(struct acpi_table_rsdp);
+    u8 *base_ptr;
+    u8 checksum;
+
+    addr = acpi_os_get_root_pointer();
+    if ( !addr  )
+    {
+        printk("Unable to get acpi root pointer\n");
+        return -EINVAL;
+    }
+    rsdp = acpi_os_map_memory(addr, table_size);
+    base_ptr = d->arch.efi_acpi_table
+               + acpi_get_table_offset(tbl_add, TBL_RSDP);
+    memcpy(base_ptr, rsdp, table_size);
+    acpi_os_unmap_memory(rsdp, table_size);
+
+    rsdp = (struct acpi_table_rsdp *)base_ptr;
+    /* Replace xsdt_physical_address */
+    rsdp->xsdt_physical_address = tbl_add[TBL_XSDT].start;
+    checksum = acpi_tb_checksum(ACPI_CAST_PTR(u8, rsdp), table_size);
+    rsdp->checksum = rsdp->checksum - checksum;
+
+    tbl_add[TBL_RSDP].start = d->arch.efi_acpi_gpa
+                              + acpi_get_table_offset(tbl_add, TBL_RSDP);
+    tbl_add[TBL_RSDP].size = table_size;
+
+    return 0;
+}
+
+static void __init acpi_xsdt_modify_entry(u64 entry[],
+                                          unsigned long entry_count,
+                                          char *signature, u64 addr)
+{
+    int i;
+    struct acpi_table_header *table;
+    u64 size = sizeof(struct acpi_table_header);
+
+    for( i = 0; i < entry_count; i++ )
+    {
+        table = acpi_os_map_memory(entry[i], size);
+        if ( ACPI_COMPARE_NAME(table->signature, signature) )
+        {
+            entry[i] = addr;
+            acpi_os_unmap_memory(table, size);
+            break;
+        }
+        acpi_os_unmap_memory(table, size);
+    }
+}
+
+static int __init acpi_create_xsdt(struct domain *d, struct membank tbl_add[])
+{
+    struct acpi_table_header *table = NULL;
+    struct acpi_table_rsdp *rsdp_tbl;
+    struct acpi_table_xsdt *xsdt = NULL;
+    u64 table_size, addr;
+    unsigned long entry_count;
+    u8 *base_ptr;
+    u8 checksum;
+
+    addr = acpi_os_get_root_pointer();
+    if ( !addr )
+    {
+        printk("Unable to get acpi root pointer\n");
+        return -EINVAL;
+    }
+    rsdp_tbl = acpi_os_map_memory(addr, sizeof(struct acpi_table_rsdp));
+    table = acpi_os_map_memory(rsdp_tbl->xsdt_physical_address,
+                               sizeof(struct acpi_table_header));
+
+    /* Add place for STAO table in XSDT table */
+    table_size = table->length + sizeof(u64);
+    entry_count = (table->length - sizeof(struct acpi_table_header))
+                  / sizeof(u64);
+    base_ptr = d->arch.efi_acpi_table
+               + acpi_get_table_offset(tbl_add, TBL_XSDT);
+    memcpy(base_ptr, table, table->length);
+    acpi_os_unmap_memory(table, sizeof(struct acpi_table_header));
+    acpi_os_unmap_memory(rsdp_tbl, sizeof(struct acpi_table_rsdp));
+
+    xsdt = (struct acpi_table_xsdt *)base_ptr;
+    acpi_xsdt_modify_entry(xsdt->table_offset_entry, entry_count,
+                           ACPI_SIG_FADT, tbl_add[TBL_FADT].start);
+    acpi_xsdt_modify_entry(xsdt->table_offset_entry, entry_count,
+                           ACPI_SIG_MADT, tbl_add[TBL_MADT].start);
+    xsdt->table_offset_entry[entry_count] = tbl_add[TBL_STAO].start;
+
+    xsdt->header.length = table_size;
+    checksum = acpi_tb_checksum(ACPI_CAST_PTR(u8, xsdt), table_size);
+    xsdt->header.checksum -= checksum;
+
+    tbl_add[TBL_XSDT].start = d->arch.efi_acpi_gpa
+                              + acpi_get_table_offset(tbl_add, TBL_XSDT);
+    tbl_add[TBL_XSDT].size = table_size;
+
+    return 0;
+}
+
+static int __init acpi_create_stao(struct domain *d, struct membank tbl_add[])
+{
+    struct acpi_table_header *table = NULL;
+    struct acpi_table_stao *stao = NULL;
+    u32 table_size = sizeof(struct acpi_table_stao);
+    u32 offset = acpi_get_table_offset(tbl_add, TBL_STAO);
+    acpi_status status;
+    u8 *base_ptr, checksum;
+
+    /* Copy OEM and ASL compiler fields from another table, use MADT */
+    status = acpi_get_table(ACPI_SIG_MADT, 0, &table);
+
+    if ( ACPI_FAILURE(status) )
+    {
+        const char *msg = acpi_format_exception(status);
+
+        printk("STAO: Failed to get MADT table, %s\n", msg);
+        return -EINVAL;
+    }
+
+    base_ptr = d->arch.efi_acpi_table + offset;
+    memcpy(base_ptr, table, sizeof(struct acpi_table_header));
+
+    stao = (struct acpi_table_stao *)base_ptr;
+    memcpy(stao->header.signature, ACPI_SIG_STAO, 4);
+    stao->header.revision = 1;
+    stao->header.length = table_size;
+    stao->ignore_uart = 1;
+    checksum = acpi_tb_checksum(ACPI_CAST_PTR(u8, stao), table_size);
+    stao->header.checksum -= checksum;
+
+    tbl_add[TBL_STAO].start = d->arch.efi_acpi_gpa + offset;
+    tbl_add[TBL_STAO].size = table_size;
+
+    return 0;
+}
+
+static int __init acpi_create_madt(struct domain *d, struct membank tbl_add[])
+{
+    struct acpi_table_header *table = NULL;
+    struct acpi_table_madt *madt = NULL;
+    struct acpi_subtable_header *header;
+    struct acpi_madt_generic_distributor *gicd;
+    u32 table_size = sizeof(struct acpi_table_madt);
+    u32 offset = acpi_get_table_offset(tbl_add, TBL_MADT);
+    int ret;
+    acpi_status status;
+    u8 *base_ptr, checksum;
+
+    status = acpi_get_table(ACPI_SIG_MADT, 0, &table);
+
+    if ( ACPI_FAILURE(status) )
+    {
+        const char *msg = acpi_format_exception(status);
+
+        printk("Failed to get MADT table, %s\n", msg);
+        return -EINVAL;
+    }
+
+    base_ptr = d->arch.efi_acpi_table + offset;
+    memcpy(base_ptr, table, table_size);
+
+    /* Add Generic Distributor. */
+    header = acpi_table_get_entry_madt(ACPI_MADT_TYPE_GENERIC_DISTRIBUTOR, 0);
+    if ( !header )
+    {
+        printk("Can't get GICD entry\n");
+        return -EINVAL;
+    }
+    gicd = container_of(header, struct acpi_madt_generic_distributor, header);
+    memcpy(base_ptr + table_size, gicd,
+                sizeof(struct acpi_madt_generic_distributor));
+    table_size += sizeof(struct acpi_madt_generic_distributor);
+
+    /* Add other subtables. */
+    ret = gic_make_hwdom_madt(d, offset + table_size);
+    if ( ret < 0 )
+    {
+        printk("Failed to get other subtables\n");
+        return -EINVAL;
+    }
+    table_size += ret;
+
+    madt = (struct acpi_table_madt *)base_ptr;
+    madt->header.length = table_size;
+    checksum = acpi_tb_checksum(ACPI_CAST_PTR(u8, madt), table_size);
+    madt->header.checksum -= checksum;
+
+    tbl_add[TBL_MADT].start = d->arch.efi_acpi_gpa + offset;
+    tbl_add[TBL_MADT].size = table_size;
+
+    return 0;
+}
+
+static int __init acpi_create_fadt(struct domain *d, struct membank tbl_add[])
+{
+    struct acpi_table_header *table = NULL;
+    struct acpi_table_fadt *fadt = NULL;
+    u64 table_size;
+    acpi_status status;
+    u8 *base_ptr;
+    u8 checksum;
+
+    status = acpi_get_table(ACPI_SIG_FADT, 0, &table);
+
+    if ( ACPI_FAILURE(status) )
+    {
+        const char *msg = acpi_format_exception(status);
+
+        printk("Failed to get FADT table, %s\n", msg);
+        return -EINVAL;
+    }
+
+    table_size = table->length;
+    base_ptr = d->arch.efi_acpi_table
+               + acpi_get_table_offset(tbl_add, TBL_FADT);
+    memcpy(base_ptr, table, table_size);
+    fadt = (struct acpi_table_fadt *)base_ptr;
+
+    /* Set PSCI_COMPLIANT and PSCI_USE_HVC */
+    fadt->arm_boot_flags |= (ACPI_FADT_PSCI_COMPLIANT | ACPI_FADT_PSCI_USE_HVC);
+    checksum = acpi_tb_checksum(ACPI_CAST_PTR(u8, fadt), table_size);
+    fadt->header.checksum -= checksum;
+
+    tbl_add[TBL_FADT].start = d->arch.efi_acpi_gpa
+                              + acpi_get_table_offset(tbl_add, TBL_FADT);
+    tbl_add[TBL_FADT].size = table_size;
+
+    return 0;
+}
+
+static int __init estimate_acpi_efi_size(struct domain *d,
+                                         struct kernel_info *kinfo)
+{
+    size_t efi_size, acpi_size, madt_size;
+    u64 addr;
+    struct acpi_table_rsdp *rsdp_tbl;
+    struct acpi_table_header *table;
+
+    efi_size = estimate_efi_size(kinfo->mem.nr_banks);
+
+    acpi_size = ROUNDUP(sizeof(struct acpi_table_fadt), 8);
+    acpi_size += ROUNDUP(sizeof(struct acpi_table_stao), 8);
+
+    madt_size = gic_get_hwdom_madt_size(d);
+    acpi_size += ROUNDUP(madt_size, 8);
+
+    addr = acpi_os_get_root_pointer();
+    if ( !addr )
+    {
+        printk("Unable to get acpi root pointer\n");
+        return -EINVAL;
+    }
+
+    rsdp_tbl = acpi_os_map_memory(addr, sizeof(struct acpi_table_rsdp));
+    if ( !rsdp_tbl )
+    {
+        printk("Unable to map RSDP table\n");
+        return -EINVAL;
+    }
+
+    table = acpi_os_map_memory(rsdp_tbl->xsdt_physical_address,
+                               sizeof(struct acpi_table_header));
+    acpi_os_unmap_memory(rsdp_tbl, sizeof(struct acpi_table_rsdp));
+    if ( !table )
+    {
+        printk("Unable to map XSDT table\n");
+        return -EINVAL;
+    }
+
+    /* Add place for STAO table in XSDT table */
+    acpi_size += ROUNDUP(table->length + sizeof(u64), 8);
+    acpi_os_unmap_memory(table, sizeof(struct acpi_table_header));
+
+    acpi_size += ROUNDUP(sizeof(struct acpi_table_rsdp), 8);
+    d->arch.efi_acpi_len = PAGE_ALIGN(ROUNDUP(efi_size, 8)
+                                      + ROUNDUP(acpi_size, 8));
+
+    return 0;
+}
+
+static int __init prepare_acpi(struct domain *d, struct kernel_info *kinfo)
+{
+    int rc = 0;
+    int order;
+    struct membank tbl_add[TBL_MMAX] = {};
+
+    rc = estimate_acpi_efi_size(d, kinfo);
+    if ( rc != 0 )
+        return rc;
+
+    order = get_order_from_bytes(d->arch.efi_acpi_len);
+    d->arch.efi_acpi_table = alloc_xenheap_pages(order, 0);
+    if ( d->arch.efi_acpi_table == NULL )
+    {
+        printk("unable to allocate memory!\n");
+        return -ENOMEM;
+    }
+    memset(d->arch.efi_acpi_table, 0, d->arch.efi_acpi_len);
+
+    /*
+     * For ACPI, Dom0 doesn't use kinfo->gnttab_start to get the grant table
+     * region. So we use it as the ACPI table mapped address. Also it needs to
+     * check if the size of grant table region is enough for those ACPI tables.
+     */
+    d->arch.efi_acpi_gpa = kinfo->gnttab_start;
+    if ( kinfo->gnttab_size < d->arch.efi_acpi_len )
+    {
+        printk("The grant table region is not enough to fit the ACPI tables!\n");
+        return -EINVAL;
+    }
+
+    rc = acpi_create_fadt(d, tbl_add);
+    if ( rc != 0 )
+        return rc;
+
+    rc = acpi_create_madt(d, tbl_add);
+    if ( rc != 0 )
+        return rc;
+
+    rc = acpi_create_stao(d, tbl_add);
+    if ( rc != 0 )
+        return rc;
+
+    rc = acpi_create_xsdt(d, tbl_add);
+    if ( rc != 0 )
+        return rc;
+
+    rc = acpi_create_rsdp(d, tbl_add);
+    if ( rc != 0 )
+        return rc;
+
+    acpi_map_other_tables(d);
+    acpi_create_efi_system_table(d, tbl_add);
+    acpi_create_efi_mmap_table(d, &kinfo->mem, tbl_add);
+
+    /* Map the EFI and ACPI tables to Dom0 */
+    rc = map_regions_p2mt(d,
+                          gaddr_to_gfn(d->arch.efi_acpi_gpa),
+                          PFN_UP(d->arch.efi_acpi_len),
+                          virt_to_mfn(d->arch.efi_acpi_table),
+                          p2m_mmio_direct_c);
+    if ( rc != 0 )
+    {
+        printk(XENLOG_ERR "Unable to map EFI/ACPI table 0x%"PRIx64
+               " - 0x%"PRIx64" in domain %d\n",
+               d->arch.efi_acpi_gpa & PAGE_MASK,
+               PAGE_ALIGN(d->arch.efi_acpi_gpa + d->arch.efi_acpi_len) - 1,
+               d->domain_id);
+        return rc;
+    }
+
+    /*
+     * Flush the cache for this region, otherwise DOM0 may read wrong data when
+     * the cache is disabled.
+     */
+    clean_and_invalidate_dcache_va_range(d->arch.efi_acpi_table,
+                                         d->arch.efi_acpi_len);
+
+    rc = create_acpi_dtb(kinfo, tbl_add);
+    if ( rc != 0 )
+        return rc;
+
+    rc = acpi_route_spis(d);
+    if ( rc != 0 )
+        return rc;
+
+    rc = acpi_iomem_deny_access(d);
+    if ( rc != 0 )
+        return rc;
+
+    /*
+     * All PPIs have been registered, allocate the event channel
+     * interrupts.
+     */
+    evtchn_allocate(d);
+
+    return 0;
+}
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index dc9b46e..3ed14a4 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -22,6 +22,7 @@
 #include <asm/psci.h>
 #include <asm/setup.h>
 #include <asm/cpufeature.h>
+#include <asm/domain_build.h>
 
 #include <xen/irq.h>
 #include <xen/grant_table.h>
@@ -661,8 +662,6 @@ static int __init make_memory_node(const struct domain *d,
     return res;
 }
 
-static void evtchn_allocate(struct domain *d);
-
 static int __init make_hypervisor_node(struct domain *d,
                                        const struct kernel_info *kinfo,
                                        int addrcells, int sizecells)
@@ -1023,7 +1022,7 @@ static int __init make_timer_node(const struct domain *d, void *fdt,
  * on ACPI systems, and DomUs started directly from Xen based on device
  * tree information.
  */
-static int __init make_chosen_node(const struct kernel_info *kinfo)
+int __init make_chosen_node(const struct kernel_info *kinfo)
 {
     int res;
     const char *bootargs = NULL;
@@ -1064,8 +1063,8 @@ static int __init make_chosen_node(const struct kernel_info *kinfo)
     return res;
 }
 
-static int __init map_irq_to_domain(struct domain *d, unsigned int irq,
-                                    bool need_mapping, const char *devname)
+int __init map_irq_to_domain(struct domain *d, unsigned int irq,
+                             bool need_mapping, const char *devname)
 {
     int res;
 
@@ -1779,580 +1778,6 @@ static int __init prepare_dtb_hwdom(struct domain *d, struct kernel_info *kinfo)
     return -EINVAL;
 }
 
-#ifdef CONFIG_ACPI
-#define ACPI_DOM0_FDT_MIN_SIZE 4096
-
-static int __init acpi_iomem_deny_access(struct domain *d)
-{
-    acpi_status status;
-    struct acpi_table_spcr *spcr = NULL;
-    unsigned long mfn;
-    int rc;
-
-    /* Firstly permit full MMIO capabilities. */
-    rc = iomem_permit_access(d, 0UL, ~0UL);
-    if ( rc )
-        return rc;
-
-    /* TODO: Deny MMIO access for SMMU, GIC ITS */
-    status = acpi_get_table(ACPI_SIG_SPCR, 0,
-                            (struct acpi_table_header **)&spcr);
-
-    if ( ACPI_FAILURE(status) )
-    {
-        printk("Failed to get SPCR table\n");
-        return -EINVAL;
-    }
-
-    mfn = spcr->serial_port.address >> PAGE_SHIFT;
-    /* Deny MMIO access for UART */
-    rc = iomem_deny_access(d, mfn, mfn + 1);
-    if ( rc )
-        return rc;
-
-    /* Deny MMIO access for GIC regions */
-    return gic_iomem_deny_access(d);
-}
-
-static int __init acpi_route_spis(struct domain *d)
-{
-    int i, res;
-    struct irq_desc *desc;
-
-    /*
-     * Route the IRQ to hardware domain and permit the access.
-     * The interrupt type will be set by set by the hardware domain.
-     */
-    for( i = NR_LOCAL_IRQS; i < vgic_num_irqs(d); i++ )
-    {
-        /*
-         * TODO: Exclude the SPIs SMMU uses which should not be routed to
-         * the hardware domain.
-         */
-        desc = irq_to_desc(i);
-        if ( desc->action != NULL)
-            continue;
-
-        /* XXX: Shall we use a proper devname? */
-        res = map_irq_to_domain(d, i, true, "ACPI");
-        if ( res )
-            return res;
-    }
-
-    return 0;
-}
-
-static int __init acpi_make_hypervisor_node(const struct kernel_info *kinfo,
-                                            struct membank tbl_add[])
-{
-    const char compat[] =
-        "xen,xen-"__stringify(XEN_VERSION)"."__stringify(XEN_SUBVERSION)"\0"
-        "xen,xen";
-    int res;
-    /* Convenience alias */
-    void *fdt = kinfo->fdt;
-
-    dt_dprintk("Create hypervisor node\n");
-
-    /* See linux Documentation/devicetree/bindings/arm/xen.txt */
-    res = fdt_begin_node(fdt, "hypervisor");
-    if ( res )
-        return res;
-
-    /* Cannot use fdt_property_string due to embedded nulls */
-    res = fdt_property(fdt, "compatible", compat, sizeof(compat));
-    if ( res )
-        return res;
-
-    res = acpi_make_efi_nodes(fdt, tbl_add);
-    if ( res )
-        return res;
-
-    res = fdt_end_node(fdt);
-
-    return res;
-}
-
-/*
- * Prepare a minimal DTB for Dom0 which contains bootargs, initrd, memory
- * information, EFI table.
- */
-static int __init create_acpi_dtb(struct kernel_info *kinfo,
-                                  struct membank tbl_add[])
-{
-    int new_size;
-    int ret;
-
-    dt_dprintk("Prepare a min DTB for DOM0\n");
-
-    /* Allocate min size for DT */
-    new_size = ACPI_DOM0_FDT_MIN_SIZE;
-    kinfo->fdt = xmalloc_bytes(new_size);
-
-    if ( kinfo->fdt == NULL )
-        return -ENOMEM;
-
-    /* Create a new empty DT for DOM0 */
-    ret = fdt_create(kinfo->fdt, new_size);
-    if ( ret < 0 )
-        goto err;
-
-    ret = fdt_finish_reservemap(kinfo->fdt);
-    if ( ret < 0 )
-        goto err;
-
-    ret = fdt_begin_node(kinfo->fdt, "/");
-    if ( ret < 0 )
-        goto err;
-
-    ret = fdt_property_cell(kinfo->fdt, "#address-cells", 2);
-    if ( ret )
-        return ret;
-
-    ret = fdt_property_cell(kinfo->fdt, "#size-cells", 1);
-    if ( ret )
-        return ret;
-
-    /* Create a chosen node for DOM0 */
-    ret = make_chosen_node(kinfo);
-    if ( ret )
-        goto err;
-
-    ret = acpi_make_hypervisor_node(kinfo, tbl_add);
-    if ( ret )
-        goto err;
-
-    ret = fdt_end_node(kinfo->fdt);
-    if ( ret < 0 )
-        goto err;
-
-    ret = fdt_finish(kinfo->fdt);
-    if ( ret < 0 )
-        goto err;
-
-    return 0;
-
-  err:
-    printk("Device tree generation failed (%d).\n", ret);
-    xfree(kinfo->fdt);
-    return -EINVAL;
-}
-
-static void __init acpi_map_other_tables(struct domain *d)
-{
-    int i;
-    unsigned long res;
-    u64 addr, size;
-
-    /* Map all ACPI tables to Dom0 using 1:1 mappings. */
-    for( i = 0; i < acpi_gbl_root_table_list.count; i++ )
-    {
-        addr = acpi_gbl_root_table_list.tables[i].address;
-        size = acpi_gbl_root_table_list.tables[i].length;
-        res = map_regions_p2mt(d,
-                               gaddr_to_gfn(addr),
-                               PFN_UP(size),
-                               maddr_to_mfn(addr),
-                               p2m_mmio_direct_c);
-        if ( res )
-        {
-             panic(XENLOG_ERR "Unable to map ACPI region 0x%"PRIx64
-                   " - 0x%"PRIx64" in domain\n",
-                   addr & PAGE_MASK, PAGE_ALIGN(addr + size) - 1);
-        }
-    }
-}
-
-static int __init acpi_create_rsdp(struct domain *d, struct membank tbl_add[])
-{
-
-    struct acpi_table_rsdp *rsdp = NULL;
-    u64 addr;
-    u64 table_size = sizeof(struct acpi_table_rsdp);
-    u8 *base_ptr;
-    u8 checksum;
-
-    addr = acpi_os_get_root_pointer();
-    if ( !addr  )
-    {
-        printk("Unable to get acpi root pointer\n");
-        return -EINVAL;
-    }
-    rsdp = acpi_os_map_memory(addr, table_size);
-    base_ptr = d->arch.efi_acpi_table
-               + acpi_get_table_offset(tbl_add, TBL_RSDP);
-    memcpy(base_ptr, rsdp, table_size);
-    acpi_os_unmap_memory(rsdp, table_size);
-
-    rsdp = (struct acpi_table_rsdp *)base_ptr;
-    /* Replace xsdt_physical_address */
-    rsdp->xsdt_physical_address = tbl_add[TBL_XSDT].start;
-    checksum = acpi_tb_checksum(ACPI_CAST_PTR(u8, rsdp), table_size);
-    rsdp->checksum = rsdp->checksum - checksum;
-
-    tbl_add[TBL_RSDP].start = d->arch.efi_acpi_gpa
-                              + acpi_get_table_offset(tbl_add, TBL_RSDP);
-    tbl_add[TBL_RSDP].size = table_size;
-
-    return 0;
-}
-
-static void __init acpi_xsdt_modify_entry(u64 entry[],
-                                          unsigned long entry_count,
-                                          char *signature, u64 addr)
-{
-    int i;
-    struct acpi_table_header *table;
-    u64 size = sizeof(struct acpi_table_header);
-
-    for( i = 0; i < entry_count; i++ )
-    {
-        table = acpi_os_map_memory(entry[i], size);
-        if ( ACPI_COMPARE_NAME(table->signature, signature) )
-        {
-            entry[i] = addr;
-            acpi_os_unmap_memory(table, size);
-            break;
-        }
-        acpi_os_unmap_memory(table, size);
-    }
-}
-
-static int __init acpi_create_xsdt(struct domain *d, struct membank tbl_add[])
-{
-    struct acpi_table_header *table = NULL;
-    struct acpi_table_rsdp *rsdp_tbl;
-    struct acpi_table_xsdt *xsdt = NULL;
-    u64 table_size, addr;
-    unsigned long entry_count;
-    u8 *base_ptr;
-    u8 checksum;
-
-    addr = acpi_os_get_root_pointer();
-    if ( !addr )
-    {
-        printk("Unable to get acpi root pointer\n");
-        return -EINVAL;
-    }
-    rsdp_tbl = acpi_os_map_memory(addr, sizeof(struct acpi_table_rsdp));
-    table = acpi_os_map_memory(rsdp_tbl->xsdt_physical_address,
-                               sizeof(struct acpi_table_header));
-
-    /* Add place for STAO table in XSDT table */
-    table_size = table->length + sizeof(u64);
-    entry_count = (table->length - sizeof(struct acpi_table_header))
-                  / sizeof(u64);
-    base_ptr = d->arch.efi_acpi_table
-               + acpi_get_table_offset(tbl_add, TBL_XSDT);
-    memcpy(base_ptr, table, table->length);
-    acpi_os_unmap_memory(table, sizeof(struct acpi_table_header));
-    acpi_os_unmap_memory(rsdp_tbl, sizeof(struct acpi_table_rsdp));
-
-    xsdt = (struct acpi_table_xsdt *)base_ptr;
-    acpi_xsdt_modify_entry(xsdt->table_offset_entry, entry_count,
-                           ACPI_SIG_FADT, tbl_add[TBL_FADT].start);
-    acpi_xsdt_modify_entry(xsdt->table_offset_entry, entry_count,
-                           ACPI_SIG_MADT, tbl_add[TBL_MADT].start);
-    xsdt->table_offset_entry[entry_count] = tbl_add[TBL_STAO].start;
-
-    xsdt->header.length = table_size;
-    checksum = acpi_tb_checksum(ACPI_CAST_PTR(u8, xsdt), table_size);
-    xsdt->header.checksum -= checksum;
-
-    tbl_add[TBL_XSDT].start = d->arch.efi_acpi_gpa
-                              + acpi_get_table_offset(tbl_add, TBL_XSDT);
-    tbl_add[TBL_XSDT].size = table_size;
-
-    return 0;
-}
-
-static int __init acpi_create_stao(struct domain *d, struct membank tbl_add[])
-{
-    struct acpi_table_header *table = NULL;
-    struct acpi_table_stao *stao = NULL;
-    u32 table_size = sizeof(struct acpi_table_stao);
-    u32 offset = acpi_get_table_offset(tbl_add, TBL_STAO);
-    acpi_status status;
-    u8 *base_ptr, checksum;
-
-    /* Copy OEM and ASL compiler fields from another table, use MADT */
-    status = acpi_get_table(ACPI_SIG_MADT, 0, &table);
-
-    if ( ACPI_FAILURE(status) )
-    {
-        const char *msg = acpi_format_exception(status);
-
-        printk("STAO: Failed to get MADT table, %s\n", msg);
-        return -EINVAL;
-    }
-
-    base_ptr = d->arch.efi_acpi_table + offset;
-    memcpy(base_ptr, table, sizeof(struct acpi_table_header));
-
-    stao = (struct acpi_table_stao *)base_ptr;
-    memcpy(stao->header.signature, ACPI_SIG_STAO, 4);
-    stao->header.revision = 1;
-    stao->header.length = table_size;
-    stao->ignore_uart = 1;
-    checksum = acpi_tb_checksum(ACPI_CAST_PTR(u8, stao), table_size);
-    stao->header.checksum -= checksum;
-
-    tbl_add[TBL_STAO].start = d->arch.efi_acpi_gpa + offset;
-    tbl_add[TBL_STAO].size = table_size;
-
-    return 0;
-}
-
-static int __init acpi_create_madt(struct domain *d, struct membank tbl_add[])
-{
-    struct acpi_table_header *table = NULL;
-    struct acpi_table_madt *madt = NULL;
-    struct acpi_subtable_header *header;
-    struct acpi_madt_generic_distributor *gicd;
-    u32 table_size = sizeof(struct acpi_table_madt);
-    u32 offset = acpi_get_table_offset(tbl_add, TBL_MADT);
-    int ret;
-    acpi_status status;
-    u8 *base_ptr, checksum;
-
-    status = acpi_get_table(ACPI_SIG_MADT, 0, &table);
-
-    if ( ACPI_FAILURE(status) )
-    {
-        const char *msg = acpi_format_exception(status);
-
-        printk("Failed to get MADT table, %s\n", msg);
-        return -EINVAL;
-    }
-
-    base_ptr = d->arch.efi_acpi_table + offset;
-    memcpy(base_ptr, table, table_size);
-
-    /* Add Generic Distributor. */
-    header = acpi_table_get_entry_madt(ACPI_MADT_TYPE_GENERIC_DISTRIBUTOR, 0);
-    if ( !header )
-    {
-        printk("Can't get GICD entry\n");
-        return -EINVAL;
-    }
-    gicd = container_of(header, struct acpi_madt_generic_distributor, header);
-    memcpy(base_ptr + table_size, gicd,
-                sizeof(struct acpi_madt_generic_distributor));
-    table_size += sizeof(struct acpi_madt_generic_distributor);
-
-    /* Add other subtables. */
-    ret = gic_make_hwdom_madt(d, offset + table_size);
-    if ( ret < 0 )
-    {
-        printk("Failed to get other subtables\n");
-        return -EINVAL;
-    }
-    table_size += ret;
-
-    madt = (struct acpi_table_madt *)base_ptr;
-    madt->header.length = table_size;
-    checksum = acpi_tb_checksum(ACPI_CAST_PTR(u8, madt), table_size);
-    madt->header.checksum -= checksum;
-
-    tbl_add[TBL_MADT].start = d->arch.efi_acpi_gpa + offset;
-    tbl_add[TBL_MADT].size = table_size;
-
-    return 0;
-}
-
-static int __init acpi_create_fadt(struct domain *d, struct membank tbl_add[])
-{
-    struct acpi_table_header *table = NULL;
-    struct acpi_table_fadt *fadt = NULL;
-    u64 table_size;
-    acpi_status status;
-    u8 *base_ptr;
-    u8 checksum;
-
-    status = acpi_get_table(ACPI_SIG_FADT, 0, &table);
-
-    if ( ACPI_FAILURE(status) )
-    {
-        const char *msg = acpi_format_exception(status);
-
-        printk("Failed to get FADT table, %s\n", msg);
-        return -EINVAL;
-    }
-
-    table_size = table->length;
-    base_ptr = d->arch.efi_acpi_table
-               + acpi_get_table_offset(tbl_add, TBL_FADT);
-    memcpy(base_ptr, table, table_size);
-    fadt = (struct acpi_table_fadt *)base_ptr;
-
-    /* Set PSCI_COMPLIANT and PSCI_USE_HVC */
-    fadt->arm_boot_flags |= (ACPI_FADT_PSCI_COMPLIANT | ACPI_FADT_PSCI_USE_HVC);
-    checksum = acpi_tb_checksum(ACPI_CAST_PTR(u8, fadt), table_size);
-    fadt->header.checksum -= checksum;
-
-    tbl_add[TBL_FADT].start = d->arch.efi_acpi_gpa
-                              + acpi_get_table_offset(tbl_add, TBL_FADT);
-    tbl_add[TBL_FADT].size = table_size;
-
-    return 0;
-}
-
-static int __init estimate_acpi_efi_size(struct domain *d,
-                                         struct kernel_info *kinfo)
-{
-    size_t efi_size, acpi_size, madt_size;
-    u64 addr;
-    struct acpi_table_rsdp *rsdp_tbl;
-    struct acpi_table_header *table;
-
-    efi_size = estimate_efi_size(kinfo->mem.nr_banks);
-
-    acpi_size = ROUNDUP(sizeof(struct acpi_table_fadt), 8);
-    acpi_size += ROUNDUP(sizeof(struct acpi_table_stao), 8);
-
-    madt_size = gic_get_hwdom_madt_size(d);
-    acpi_size += ROUNDUP(madt_size, 8);
-
-    addr = acpi_os_get_root_pointer();
-    if ( !addr )
-    {
-        printk("Unable to get acpi root pointer\n");
-        return -EINVAL;
-    }
-
-    rsdp_tbl = acpi_os_map_memory(addr, sizeof(struct acpi_table_rsdp));
-    if ( !rsdp_tbl )
-    {
-        printk("Unable to map RSDP table\n");
-        return -EINVAL;
-    }
-
-    table = acpi_os_map_memory(rsdp_tbl->xsdt_physical_address,
-                               sizeof(struct acpi_table_header));
-    acpi_os_unmap_memory(rsdp_tbl, sizeof(struct acpi_table_rsdp));
-    if ( !table )
-    {
-        printk("Unable to map XSDT table\n");
-        return -EINVAL;
-    }
-
-    /* Add place for STAO table in XSDT table */
-    acpi_size += ROUNDUP(table->length + sizeof(u64), 8);
-    acpi_os_unmap_memory(table, sizeof(struct acpi_table_header));
-
-    acpi_size += ROUNDUP(sizeof(struct acpi_table_rsdp), 8);
-    d->arch.efi_acpi_len = PAGE_ALIGN(ROUNDUP(efi_size, 8)
-                                      + ROUNDUP(acpi_size, 8));
-
-    return 0;
-}
-
-static int __init prepare_acpi(struct domain *d, struct kernel_info *kinfo)
-{
-    int rc = 0;
-    int order;
-    struct membank tbl_add[TBL_MMAX] = {};
-
-    rc = estimate_acpi_efi_size(d, kinfo);
-    if ( rc != 0 )
-        return rc;
-
-    order = get_order_from_bytes(d->arch.efi_acpi_len);
-    d->arch.efi_acpi_table = alloc_xenheap_pages(order, 0);
-    if ( d->arch.efi_acpi_table == NULL )
-    {
-        printk("unable to allocate memory!\n");
-        return -ENOMEM;
-    }
-    memset(d->arch.efi_acpi_table, 0, d->arch.efi_acpi_len);
-
-    /*
-     * For ACPI, Dom0 doesn't use kinfo->gnttab_start to get the grant table
-     * region. So we use it as the ACPI table mapped address. Also it needs to
-     * check if the size of grant table region is enough for those ACPI tables.
-     */
-    d->arch.efi_acpi_gpa = kinfo->gnttab_start;
-    if ( kinfo->gnttab_size < d->arch.efi_acpi_len )
-    {
-        printk("The grant table region is not enough to fit the ACPI tables!\n");
-        return -EINVAL;
-    }
-
-    rc = acpi_create_fadt(d, tbl_add);
-    if ( rc != 0 )
-        return rc;
-
-    rc = acpi_create_madt(d, tbl_add);
-    if ( rc != 0 )
-        return rc;
-
-    rc = acpi_create_stao(d, tbl_add);
-    if ( rc != 0 )
-        return rc;
-
-    rc = acpi_create_xsdt(d, tbl_add);
-    if ( rc != 0 )
-        return rc;
-
-    rc = acpi_create_rsdp(d, tbl_add);
-    if ( rc != 0 )
-        return rc;
-
-    acpi_map_other_tables(d);
-    acpi_create_efi_system_table(d, tbl_add);
-    acpi_create_efi_mmap_table(d, &kinfo->mem, tbl_add);
-
-    /* Map the EFI and ACPI tables to Dom0 */
-    rc = map_regions_p2mt(d,
-                          gaddr_to_gfn(d->arch.efi_acpi_gpa),
-                          PFN_UP(d->arch.efi_acpi_len),
-                          virt_to_mfn(d->arch.efi_acpi_table),
-                          p2m_mmio_direct_c);
-    if ( rc != 0 )
-    {
-        printk(XENLOG_ERR "Unable to map EFI/ACPI table 0x%"PRIx64
-               " - 0x%"PRIx64" in domain %d\n",
-               d->arch.efi_acpi_gpa & PAGE_MASK,
-               PAGE_ALIGN(d->arch.efi_acpi_gpa + d->arch.efi_acpi_len) - 1,
-               d->domain_id);
-        return rc;
-    }
-
-    /*
-     * Flush the cache for this region, otherwise DOM0 may read wrong data when
-     * the cache is disabled.
-     */
-    clean_and_invalidate_dcache_va_range(d->arch.efi_acpi_table,
-                                         d->arch.efi_acpi_len);
-
-    rc = create_acpi_dtb(kinfo, tbl_add);
-    if ( rc != 0 )
-        return rc;
-
-    rc = acpi_route_spis(d);
-    if ( rc != 0 )
-        return rc;
-
-    rc = acpi_iomem_deny_access(d);
-    if ( rc != 0 )
-        return rc;
-
-    /*
-     * All PPIs have been registered, allocate the event channel
-     * interrupts.
-     */
-    evtchn_allocate(d);
-
-    return 0;
-}
-#else
-static int __init prepare_acpi(struct domain *d, struct kernel_info *kinfo)
-{
-    /* Only booting with ACPI will hit here */
-    BUG();
-    return -EINVAL;
-}
-#endif
 static void __init dtb_load(struct kernel_info *kinfo)
 {
     unsigned long left;
@@ -2425,7 +1850,7 @@ static void __init initrd_load(struct kernel_info *kinfo)
  * Note that this should only be called once all PPIs used by the
  * hardware domain have been registered.
  */
-static void __init evtchn_allocate(struct domain *d)
+void __init evtchn_allocate(struct domain *d)
 {
     int res;
     u64 val;
diff --git a/xen/include/asm-arm/domain_build.h b/xen/include/asm-arm/domain_build.h
new file mode 100644
index 0000000..34ceddc
--- /dev/null
+++ b/xen/include/asm-arm/domain_build.h
@@ -0,0 +1,31 @@
+#ifndef __ASM_DOMAIN_BUILD_H__
+#define __ASM_DOMAIN_BUILD_H__
+
+#include <xen/sched.h>
+#include <asm/kernel.h>
+
+int map_irq_to_domain(struct domain *d, unsigned int irq,
+                      bool need_mapping, const char *devname);
+int make_chosen_node(const struct kernel_info *kinfo);
+void evtchn_allocate(struct domain *d);
+
+#ifndef CONFIG_ACPI
+static inline int prepare_acpi(struct domain *d, struct kernel_info *kinfo)
+{
+    /* Only booting with ACPI will hit here */
+    BUG();
+    return -EINVAL;
+}
+#else
+int prepare_acpi(struct domain *d, struct kernel_info *kinfo);
+#endif
+#endif
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 21/23] xen/vpl011: buffer out chars when the backend is xen

[Jan Beulich]

>>> On 05.10.18 at 20:47, <sstabellini@kernel.org> wrote:
> --- a/xen/drivers/char/console.c
> +++ b/xen/drivers/char/console.c
> @@ -406,6 +406,13 @@ static void dump_console_ring_key(unsigned char key)
>   */
>  static unsigned int __read_mostly console_rx = 0;
>  
> +struct domain *console_input_domain(void)
> +{
> +    if ( console_rx == 0 )
> +            return NULL;
> +    return get_domain_by_id(console_rx - 1);

This acquires a domain reference, yet I can't see that reference
getting dropped anywhere in the caller.

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 20/23] xen: support console_switching between Dom0 and DomUs on ARM

[Jan Beulich]

>>> On 05.10.18 at 20:47, <sstabellini@kernel.org> wrote:
> @@ -391,31 +394,73 @@ static void dump_console_ring_key(unsigned char key)
>      free_xenheap_pages(buf, order);
>  }
>  
> -/* CTRL-<switch_char> switches input direction between Xen and DOM0. */
> +/*
> + * CTRL-<switch_char> switches input direction between Xen, Dom0 and
> + * DomUs.
> + */

Just like the title, this comment makes it sound as if any DomU could
participate in this switching.

>  #define switch_code (opt_conswitch[0]-'a'+1)
> -static int __read_mostly xen_rx = 1; /* FALSE => input passed to domain 0. */
> +/*
> + * console_rx=0 => input to xen
> + * console_rx=1 => input to dom0
> + * console_rx=N => input dom(N-1)
> + */
> +static unsigned int __read_mostly console_rx = 0;
>  
>  static void switch_serial_input(void)
>  {
> -    static char *input_str[2] = { "DOM0", "Xen" };
> -    xen_rx = !xen_rx;
> -    printk("*** Serial input -> %s", input_str[xen_rx]);
> +    if ( console_rx++ == max_init_domid + 1 )
> +        console_rx = 0;
> +
> +    if ( !console_rx )

Please be consistent ...

> +        printk("*** Serial input to Xen");
> +    else
> +        printk("*** Serial input to DOM%d", console_rx - 1);
> +
>      if ( switch_code )
> -        printk(" (type 'CTRL-%c' three times to switch input to %s)",
> -               opt_conswitch[0], input_str[!xen_rx]);
> +        printk(" (type 'CTRL-%c' three times to switch input)",
> +               opt_conswitch[0]);
>      printk("\n");
>  }
>  
>  static void __serial_rx(char c, struct cpu_user_regs *regs)
>  {
> -    if ( xen_rx )
> +    if ( console_rx == 0 )

... in style. But perhaps ...

>          return handle_keypress(c, regs);
>  
> -    /* Deliver input to guest buffer, unless it is already full. */
> -    if ( (serial_rx_prod-serial_rx_cons) != SERIAL_RX_SIZE )
> -        serial_rx_ring[SERIAL_RX_MASK(serial_rx_prod++)] = c;
> -    /* Always notify the guest: prevents receive path from getting stuck. */
> -    send_global_virq(VIRQ_CONSOLE);
> +    if ( console_rx == 1 )

... switch() would be better to use here anyway.

> +    {
> +        /* Deliver input to hardware domain, unless it is already full. */

Looks like you've mis-edited the original comment.

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 03/23] xen/arm: document dom0less

[Julien Grall]

Hi Stefano,

On 05/10/2018 19:47, Stefano Stabellini wrote:
> Add a new document to provide information on how to use dom0less related
> features and their current limitations.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> 
> ---
> Changes in v4:
> - rename to .txt
> - improve wording
> 
> Changes in v3:
> - add patch
> ---
>   docs/misc/arm/dom0less.txt | 47 ++++++++++++++++++++++++++++++++++++++++++++++

As said on the previous version, you likely need to add an entry in 
docs/INDEX.

>   1 file changed, 47 insertions(+)
>   create mode 100644 docs/misc/arm/dom0less.txt
> 
> diff --git a/docs/misc/arm/dom0less.txt b/docs/misc/arm/dom0less.txt
> new file mode 100644
> index 0000000..df96b41
> --- /dev/null
> +++ b/docs/misc/arm/dom0less.txt
> @@ -0,0 +1,47 @@
> +Dom0less
> +========
> +
> +"Dom0less" is a set of Xen features that enable the deployment of a Xen
> +system without an hardware domain (often referred to as "dom0").
I realize I suggested the wording hardware domain. But reading this 
again, it feels that "control domain" may be the best wording here. 
Indeed what we avoid is the toolstack and domain control the domains.

You begin the document writing "it is a set of Xen featueres that enable 
deployment of a Xen system without an hardware domain". I understand 
this sentence as there would be no "hardware domain". But then you write 
"create a set of DomU alongside Dom0".

Furthermore, at some point the control domain would disappear and the 
DomID 0 may be allocated to a DomUs. Adding further confusion to a user 
seen the ID would be 0.

So I still think that using "Dom0" within the document is misleading and 
also the feature name. I don't have a good suggestion for the feature 
name. But at very least I would avoid the word "Dom0" everywhere in that 
document.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 05/23] xen/arm: introduce bootcmdlines

[Julien Grall]

On 05/10/2018 19:47, Stefano Stabellini wrote:
> Introduce a new array to store the cmdline of each boot module. It is
> separate from struct bootmodules. Remove the cmdline field from struct
> boot_module. This way, kernels and initrds with the same address in
> memory can share struct bootmodule (important because we want them to be
> free'd only once), but they can still have their separate bootcmdline
> entries.
> 
> Add a dt_name field to struct bootcmdline to make it easier to find the
> correct entry. Store the name of the "xen,domain" compatible node (for
> example "Dom1"). This is a better choice compared to the name of the
> "multiboot,kernel" compatible node, because their names are not unique.
> For instance there can be more than one "module@0x4c000000" in the
> system, but there can only be one "/chosen/Dom1".
> 
> Add a pointer to struct kernel_info to point to the cmdline for a given
> kernel.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> 
> ---
> 
> Changes in v4:
> - check that the multiboot node is under /chosen
> - use cmd and cmds as variable names for struct bootcmdline and struct
>    bootcmdline*
> - fix printk
> - use ASSERT instea of panic
> - do not add empty cmdline entries
> - add more debug printks to early_print_info
> - code style fixes
> - add comment on DT_MAX_NAME
> - increase DT_MAX_NAME to 41
> - make nr_mods unsigned int
> 
> Changes in v3:
> - introduce bootcmdlines
> - do not modify boot_fdt_cmdline
> - add comments
> 
> Changes in v2:
> - new patch
> ---
>   xen/arch/arm/bootfdt.c      | 82 +++++++++++++++++++++++++++++++++------------
>   xen/arch/arm/domain_build.c |  8 +++--
>   xen/arch/arm/kernel.h       |  1 +
>   xen/arch/arm/setup.c        | 24 +++++++++----
>   xen/include/asm-arm/setup.h | 17 ++++++++--
>   5 files changed, 99 insertions(+), 33 deletions(-)
> 
> diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
> index 8eba42c..273032b 100644
> --- a/xen/arch/arm/bootfdt.c
> +++ b/xen/arch/arm/bootfdt.c
> @@ -163,6 +163,37 @@ static void __init process_memory_node(const void *fdt, int node,
>       }
>   }
>   
> +static void __init add_boot_cmdline(const void *fdt, int node,
> +                                    const char *name, bootmodule_kind kind)
> +{
> +    struct bootcmdlines *cmds = &bootinfo.cmdlines;
> +    struct bootcmdline *cmd;
> +    const struct fdt_property *prop;
> +    int len;
> +    const char *cmdline;
> +
> +    if ( cmds->nr_mods == MAX_MODULES )
> +    {
> +        printk("Ignoring %s cmdline (too many)\n", name);
> +        return;
> +    }
> +
> +    prop = fdt_get_property(fdt, node, "bootargs", &len);
> +    if ( !prop )
> +        return;
> +
> +    cmd = &cmds->cmdline[cmds->nr_mods++];
> +    cmd->kind = kind;
> +
> +    ASSERT(strlen(name) <= DT_MAX_NAME);
> +    safe_strcpy(cmd->dt_name, name);
> +
> +    if ( len > BOOTMOD_MAX_CMDLINE )
> +        panic("module %s command line too long\n", name);
> +    cmdline = prop->data;
> +    safe_strcpy(cmd->cmdline, cmdline);
> +}
> +
>   static void __init process_multiboot_node(const void *fdt, int node,
>                                             const char *name,
>                                             u32 address_cells, u32 size_cells)
> @@ -172,8 +203,20 @@ static void __init process_multiboot_node(const void *fdt, int node,
>       const __be32 *cell;
>       bootmodule_kind kind;
>       paddr_t start, size;
> -    const char *cmdline;
> -    int len;
> +    int len = sizeof("/chosen");
> +    char path[8]; /* sizeof "/chosen" */
> +    int parent_node;
> +
> +    parent_node = fdt_parent_offset(fdt, node);
> +    ASSERT(parent_node >= 0);
> +
> +    /* Check that the node is under "chosen" */
> +    fdt_get_path(fdt, node, path, len);

It would be good if you test the code with wrong node. For instance 
fdt_get_path may not fill path (i.e if the buffer is too short). So path 
will contain garbage.

> +    if ( strncmp(path, "/chosen", len - 1) )
> +    {
> +        printk("DEBUG %s %s\n",name,path);

This looks like a left-over from your debug.

> +        return;
> +    }
As I said in the previous patch, this needs to be fixed first. By that I 
meant this should be a separate patch so it could be backported.

Also, with this patch you change correctly the behavior to deny node not 
in chosen. But you don't explain in the commit message that it affects 
the current multiboot node.

However, I still don't think this code is correct. You would allow the 
following path /chosen/foo/bar/multiboot. The parent would be "bar" and 
there are no guarantee it would be uniq (it is not under /chosen). I 
think this could be fixed by taking into account the depth of the node.
AFAICT, the multiboot node should always be at maximum depth 3.

>   
>       prop = fdt_get_property(fdt, node, "reg", &len);
>       if ( !prop )
> @@ -220,17 +263,8 @@ static void __init process_multiboot_node(const void *fdt, int node,
>               kind = BOOTMOD_XSM;
>       }
>   
> -    prop = fdt_get_property(fdt, node, "bootargs", &len);
> -    if ( prop )
> -    {
> -        if ( len > BOOTMOD_MAX_CMDLINE )
> -            panic("module %s command line too long\n", name);
> -        cmdline = prop->data;
> -    }
> -    else
> -        cmdline = NULL;
> -
> -    add_boot_module(kind, start, size, cmdline);
> +    add_boot_module(kind, start, size);
> +    add_boot_cmdline(fdt, node, fdt_get_name(fdt, parent_node, &len), kind);
>   }
>   
>   static void __init process_chosen_node(const void *fdt, int node,
> @@ -276,7 +310,7 @@ static void __init process_chosen_node(const void *fdt, int node,
>   
>       printk("Initrd %"PRIpaddr"-%"PRIpaddr"\n", start, end);
>   
> -    add_boot_module(BOOTMOD_RAMDISK, start, end-start, NULL);
> +    add_boot_module(BOOTMOD_RAMDISK, start, end-start);
>   }
>   
>   static int __init early_scan_node(const void *fdt,
> @@ -299,6 +333,7 @@ static void __init early_print_info(void)
>   {
>       struct meminfo *mi = &bootinfo.mem;
>       struct bootmodules *mods = &bootinfo.modules;
> +    struct bootcmdlines *cmds = &bootinfo.cmdlines;
>       int i, nr_rsvd;
>   
>       for ( i = 0; i < mi->nr_banks; i++ )
> @@ -307,12 +342,12 @@ static void __init early_print_info(void)
>                        mi->bank[i].start + mi->bank[i].size - 1);
>       printk("\n");
>       for ( i = 0 ; i < mods->nr_mods; i++ )
> -        printk("MODULE[%d]: %"PRIpaddr" - %"PRIpaddr" %-12s %s\n",
> +        printk("MODULE[%d]: %"PRIpaddr" - %"PRIpaddr" %-12s\n",
>                        i,
>                        mods->module[i].start,
>                        mods->module[i].start + mods->module[i].size,
> -                     boot_module_kind_as_string(mods->module[i].kind),
> -                     mods->module[i].cmdline);
> +                     boot_module_kind_as_string(mods->module[i].kind));
> +
>       nr_rsvd = fdt_num_mem_rsv(device_tree_flattened);
>       for ( i = 0; i < nr_rsvd; i++ )
>       {
> @@ -325,6 +360,11 @@ static void __init early_print_info(void)
>                        i, s, e);
>       }
>       printk("\n");
> +    for ( i = 0 ; i < cmds->nr_mods; i++ )
> +        printk("CMDLINE[%d]:%s %s\n", i,
> +               cmds->cmdline[i].dt_name,
> +               &cmds->cmdline[i].cmdline[0]);

Thank you for adding the command line. However, there are still no way 
to associate the command line with a module. For each command line, you 
should be able to know the associate module.

> +    printk("\n");
>   }
>   
>   /**
> @@ -341,7 +381,7 @@ size_t __init boot_fdt_info(const void *fdt, paddr_t paddr)
>       if ( ret < 0 )
>           panic("No valid device tree\n");
>   
> -    add_boot_module(BOOTMOD_FDT, paddr, fdt_totalsize(fdt), NULL);
> +    add_boot_module(BOOTMOD_FDT, paddr, fdt_totalsize(fdt));
>   
>       device_tree_for_each_node((void *)fdt, early_scan_node, NULL);
>       early_print_info();
> @@ -361,11 +401,11 @@ const char *boot_fdt_cmdline(const void *fdt)
>       prop = fdt_get_property(fdt, node, "xen,xen-bootargs", NULL);
>       if ( prop == NULL )
>       {
> -        struct bootmodule *dom0_mod =
> -            boot_module_find_by_kind(BOOTMOD_KERNEL);
> +        struct bootcmdline *dom0_cmdline =
> +            boot_cmdline_find_by_kind(BOOTMOD_KERNEL);
>   
>           if (fdt_get_property(fdt, node, "xen,dom0-bootargs", NULL) ||
> -            ( dom0_mod && dom0_mod->cmdline[0] ) )
> +            ( dom0_cmdline && dom0_cmdline->cmdline[0] ) )
>               prop = fdt_get_property(fdt, node, "bootargs", NULL);
>       }
>       if ( prop == NULL )
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index f552154..64f8d6b 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -375,7 +375,7 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
>       int res = 0;
>       int had_dom0_bootargs = 0;
>   
> -    const struct bootmodule *kernel = kinfo->kernel_bootmodule;
> +    const struct bootcmdline *kernel = boot_cmdline_find_by_kind(BOOTMOD_KERNEL);

Why do you need to lookup the command line here? You have just 
introduced a field in kinfo to store command line.

>   
>       if ( kernel && kernel->cmdline[0] )
>           bootargs = &kernel->cmdline[0];
> @@ -952,9 +952,9 @@ static int __init make_chosen_node(const struct kernel_info *kinfo)
>       if ( res )
>           return res;
>   
> -    if ( mod && mod->cmdline[0] )
> +    if ( kinfo->cmdline && kinfo->cmdline[0] )
>       {
> -        bootargs = &mod->cmdline[0];
> +        bootargs = &kinfo->cmdline[0];
>           res = fdt_property(fdt, "bootargs", bootargs, strlen(bootargs) + 1);
>           if ( res )
>              return res;
> @@ -2109,6 +2109,7 @@ static void __init find_gnttab_region(struct domain *d,
>   
>   int __init construct_dom0(struct domain *d)
>   {
> +    const struct bootcmdline *kernel = boot_cmdline_find_by_kind(BOOTMOD_KERNEL);
>       struct kernel_info kinfo = {};
>       struct vcpu *saved_current;
>       int rc, i, cpu;
> @@ -2154,6 +2155,7 @@ int __init construct_dom0(struct domain *d)
>   
>   #endif
>   
> +    kinfo.cmdline = kernel != NULL ? &kernel->cmdline[0] : NULL;
>       allocate_memory(d, &kinfo);
>       find_gnttab_region(d, &kinfo);
>   
> diff --git a/xen/arch/arm/kernel.h b/xen/arch/arm/kernel.h
> index 47eacb5..39b7828 100644
> --- a/xen/arch/arm/kernel.h
> +++ b/xen/arch/arm/kernel.h
> @@ -29,6 +29,7 @@ struct kernel_info {
>   
>       /* boot blob load addresses */
>       const struct bootmodule *kernel_bootmodule, *initrd_bootmodule;
> +    const char* cmdline;
>       paddr_t dtb_paddr;
>       paddr_t initrd_paddr;
>   
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index ea2495a..bc7dd97 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -201,8 +201,7 @@ void __init dt_unreserved_regions(paddr_t s, paddr_t e,
>   }
>   
>   struct bootmodule __init *add_boot_module(bootmodule_kind kind,
> -                                          paddr_t start, paddr_t size,
> -                                          const char *cmdline)
> +                                          paddr_t start, paddr_t size)
>   {
>       struct bootmodules *mods = &bootinfo.modules;
>       struct bootmodule *mod;
> @@ -218,10 +217,6 @@ struct bootmodule __init *add_boot_module(bootmodule_kind kind,
>       mod->kind = kind;
>       mod->start = start;
>       mod->size = size;
> -    if ( cmdline )
> -        safe_strcpy(mod->cmdline, cmdline);
> -    else
> -        mod->cmdline[0] = 0;
>   
>       return mod;
>   }
> @@ -240,6 +235,21 @@ struct bootmodule * __init boot_module_find_by_kind(bootmodule_kind kind)
>       return NULL;
>   }
>   
> +struct bootcmdline * __init boot_cmdline_find_by_kind(bootmodule_kind kind)

Functions dealing with a same structure should stick together. I 
actually quite dislike that add_bood_cmdline is part of bootfdt.c. There 
are no need for it to be there except the fact you moved the FDT lookup 
in the function. As I suggested in the previous patch, the lookup could 
have been left outside of the function.

> +{
> +    struct bootcmdlines *cmds = &bootinfo.cmdlines;
> +    struct bootcmdline *cmd;
> +    int i;
> +
> +    for ( i = 0 ; i < cmds->nr_mods ; i++ )
> +    {
> +        cmd = &cmds->cmdline[i];
> +        if ( cmd->kind == kind )
> +            return cmd;
> +    }
> +    return NULL;
> +}
> +
>   const char * __init boot_module_kind_as_string(bootmodule_kind kind)
>   {
>       switch ( kind )
> @@ -728,7 +738,7 @@ void __init start_xen(unsigned long boot_phys_offset,
>       /* Register Xen's load address as a boot module. */
>       xen_bootmodule = add_boot_module(BOOTMOD_XEN,
>                                (paddr_t)(uintptr_t)(_start + boot_phys_offset),
> -                             (paddr_t)(uintptr_t)(_end - _start + 1), NULL);
> +                             (paddr_t)(uintptr_t)(_end - _start + 1));
>       BUG_ON(!xen_bootmodule);
>   
>       xen_paddr = get_xen_paddr();
> diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
> index f1e4a3f..c2ed5cc 100644
> --- a/xen/include/asm-arm/setup.h
> +++ b/xen/include/asm-arm/setup.h
> @@ -35,6 +35,13 @@ struct bootmodule {
>       bootmodule_kind kind;
>       paddr_t start;
>       paddr_t size;
> +};
> +
> +/* DT_MAX_NAME is the node name max length according the DT spec */
> +#define DT_MAX_NAME 41
> +struct bootcmdline {
> +    bootmodule_kind kind;
> +    char dt_name[DT_MAX_NAME];
>       char cmdline[BOOTMOD_MAX_CMDLINE];
>   };
>   
> @@ -43,9 +50,15 @@ struct bootmodules {
>       struct bootmodule module[MAX_MODULES];
>   };
>   
> +struct bootcmdlines {
> +    unsigned int nr_mods;
> +    struct bootcmdline cmdline[MAX_MODULES];
> +};
> +
>   struct bootinfo {
>       struct meminfo mem;
>       struct bootmodules modules;
> +    struct bootcmdlines cmdlines;
>   #ifdef CONFIG_ACPI
>       struct meminfo acpi;
>   #endif
> @@ -78,9 +91,9 @@ size_t __init boot_fdt_info(const void *fdt, paddr_t paddr);
>   const char __init *boot_fdt_cmdline(const void *fdt);
>   
>   struct bootmodule *add_boot_module(bootmodule_kind kind,
> -                                   paddr_t start, paddr_t size,
> -                                   const char *cmdline);
> +                                   paddr_t start, paddr_t size);
>   struct bootmodule *boot_module_find_by_kind(bootmodule_kind kind);
> +struct bootcmdline *boot_cmdline_find_by_kind(bootmodule_kind kind);
>   const char * __init boot_module_kind_as_string(bootmodule_kind kind);
>   
>   #endif
> 

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 06/23] xen/arm: don't add duplicate boot modules, introduce domU flag

[Julien Grall]

Hi Stefano,

On 05/10/2018 19:47, Stefano Stabellini wrote:
> Don't add duplicate boot modules (same kind and same start address),
> they are freed later, we don't want to introduce double-free errors.
> 
> Introduce a domU flag in struct bootmodule and struct bootcmdline. Set
> it for kernels and ramdisks of "xen,domain" nodes to avoid getting
> confused in kernel_probe, where we try to guess which is the dom0 kernel
> and initrd to be compatible with older versions of the multiboot spec.

This comments need at least to be replicated in the structure. So we 
have an idea how to use it.

Lastly, technically it is not older versions of the specification. Your 
extension does not deal with Dom0.

> 
> boot_module_find_by_kind and boot_cmdline_find_by_kind automatically
> check for !domU entries (they are only used for non-domU modules).
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> 
> ---
> Changes in v4:
> - use unsigned int
> - better commit message
> - introduce domU flag and usage
> 
> Changes in v2:
> - new patch
> ---
>   xen/arch/arm/bootfdt.c      | 14 +++++++++-----
>   xen/arch/arm/setup.c        | 21 +++++++++++++++++----
>   xen/include/asm-arm/setup.h |  4 +++-
>   3 files changed, 29 insertions(+), 10 deletions(-)
> 
> diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
> index 273032b..349aa9d 100644
> --- a/xen/arch/arm/bootfdt.c
> +++ b/xen/arch/arm/bootfdt.c
> @@ -164,7 +164,8 @@ static void __init process_memory_node(const void *fdt, int node,
>   }
>   
>   static void __init add_boot_cmdline(const void *fdt, int node,
> -                                    const char *name, bootmodule_kind kind)
> +                                    const char *name, bootmodule_kind kind,
> +                                    bool domU)
>   {
>       struct bootcmdlines *cmds = &bootinfo.cmdlines;
>       struct bootcmdline *cmd;
> @@ -184,6 +185,7 @@ static void __init add_boot_cmdline(const void *fdt, int node,
>   
>       cmd = &cmds->cmdline[cmds->nr_mods++];
>       cmd->kind = kind;
> +    cmd->domU = domU;
>   
>       ASSERT(strlen(name) <= DT_MAX_NAME);
>       safe_strcpy(cmd->dt_name, name);
> @@ -206,6 +208,7 @@ static void __init process_multiboot_node(const void *fdt, int node,
>       int len = sizeof("/chosen");
>       char path[8]; /* sizeof "/chosen" */
>       int parent_node;
> +    bool domU;
>   
>       parent_node = fdt_parent_offset(fdt, node);
>       ASSERT(parent_node >= 0);
> @@ -263,8 +266,9 @@ static void __init process_multiboot_node(const void *fdt, int node,
>               kind = BOOTMOD_XSM;
>       }
>   
> -    add_boot_module(kind, start, size);
> -    add_boot_cmdline(fdt, node, fdt_get_name(fdt, parent_node, &len), kind);
> +    domU = fdt_node_check_compatible(fdt, parent_node, "xen,domain") == 0;
> +    add_boot_module(kind, start, size, domU);
> +    add_boot_cmdline(fdt, node, fdt_get_name(fdt, parent_node, &len), kind, domU);
>   }
>   
>   static void __init process_chosen_node(const void *fdt, int node,
> @@ -310,7 +314,7 @@ static void __init process_chosen_node(const void *fdt, int node,
>   
>       printk("Initrd %"PRIpaddr"-%"PRIpaddr"\n", start, end);
>   
> -    add_boot_module(BOOTMOD_RAMDISK, start, end-start);
> +    add_boot_module(BOOTMOD_RAMDISK, start, end-start, false);
>   }
>   
>   static int __init early_scan_node(const void *fdt,
> @@ -381,7 +385,7 @@ size_t __init boot_fdt_info(const void *fdt, paddr_t paddr)
>       if ( ret < 0 )
>           panic("No valid device tree\n");
>   
> -    add_boot_module(BOOTMOD_FDT, paddr, fdt_totalsize(fdt));
> +    add_boot_module(BOOTMOD_FDT, paddr, fdt_totalsize(fdt), false);
>   
>       device_tree_for_each_node((void *)fdt, early_scan_node, NULL);
>       early_print_info();
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index bc7dd97..dbab232 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -201,10 +201,12 @@ void __init dt_unreserved_regions(paddr_t s, paddr_t e,
>   }
>   
>   struct bootmodule __init *add_boot_module(bootmodule_kind kind,
> -                                          paddr_t start, paddr_t size)
> +                                          paddr_t start, paddr_t size,
> +                                          bool domU)
>   {
>       struct bootmodules *mods = &bootinfo.modules;
>       struct bootmodule *mod;
This at least need to be replica
> +    unsigned int i;
>   
>       if ( mods->nr_mods == MAX_MODULES )
>       {
> @@ -212,11 +214,22 @@ struct bootmodule __init *add_boot_module(bootmodule_kind kind,
>                  boot_module_kind_as_string(kind), start, start + size);
>           return NULL;
>       }
> +    for ( i = 0 ; i < mods->nr_mods ; i++ )
> +    {
> +        mod = &mods->module[i];
> +        if ( mod->kind == kind && mod->start == start )
> +        {
> +            if ( !domU )
> +                mod->domU = false;
> +            return mod;
> +        }
> +    }
>   
>       mod = &mods->module[mods->nr_mods++];
>       mod->kind = kind;
>       mod->start = start;
>       mod->size = size;
> +    mod->domU = domU;
>   
>       return mod;
>   }
> @@ -229,7 +242,7 @@ struct bootmodule * __init boot_module_find_by_kind(bootmodule_kind kind)
>       for (i = 0 ; i < mods->nr_mods ; i++ )
>       {
>           mod = &mods->module[i];
> -        if ( mod->kind == kind )
> +        if ( mod->kind == kind && !mod->domU )

 From the name of the function is it unclear why we would only return 
module with !mod->domU. So this needs some clarifications in the code.

>               return mod;
>       }
>       return NULL;
> @@ -244,7 +257,7 @@ struct bootcmdline * __init boot_cmdline_find_by_kind(bootmodule_kind kind)
>       for ( i = 0 ; i < cmds->nr_mods ; i++ )
>       {
>           cmd = &cmds->cmdline[i];
> -        if ( cmd->kind == kind )
> +        if ( cmd->kind == kind && !cmd->domU )

Ditto here.

>               return cmd;
>       }
>       return NULL;
> @@ -738,7 +751,7 @@ void __init start_xen(unsigned long boot_phys_offset,
>       /* Register Xen's load address as a boot module. */
>       xen_bootmodule = add_boot_module(BOOTMOD_XEN,
>                                (paddr_t)(uintptr_t)(_start + boot_phys_offset),
> -                             (paddr_t)(uintptr_t)(_end - _start + 1));
> +                             (paddr_t)(uintptr_t)(_end - _start + 1), false);
>       BUG_ON(!xen_bootmodule);
>   
>       xen_paddr = get_xen_paddr();
> diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
> index c2ed5cc..711b4a2 100644
> --- a/xen/include/asm-arm/setup.h
> +++ b/xen/include/asm-arm/setup.h
> @@ -33,6 +33,7 @@ struct meminfo {
>   #define BOOTMOD_MAX_CMDLINE 1024
>   struct bootmodule {
>       bootmodule_kind kind;
> +    bool domU;

This needs some documentation in the code.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 07/23] xen/arm: probe domU kernels and initrds

[Julien Grall]

Hi Stefano,

On 05/10/2018 19:47, Stefano Stabellini wrote:
> Find addresses, sizes on device tree from kernel_probe.
> Find the cmdline from the bootcmdlines array.
> 
> Introduce a new boot_module_find_by_addr_and_kind function to match not
> just on boot module kind, but also by address so that we can support
> multiple domains.
> 
> Introduce a boot_cmdline_find_by_name function to find the right struct
> cmdline based on the device tree node name of the "xen,domain"
> compatible node.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> ---
> Changes in v3:
> - retrieve cmdline from bootcmdlines array
> 
> Changes in v2:
> - fix indentation
> - unify kernel_probe with kernel_probe_domU
> - find cmdline on device_tree from kernel_probe
> ---
>   xen/arch/arm/domain_build.c |  2 +-
>   xen/arch/arm/kernel.c       | 52 +++++++++++++++++++++++++++++++++++++++------
>   xen/arch/arm/kernel.h       |  2 +-
>   xen/arch/arm/setup.c        | 29 +++++++++++++++++++++++++
>   xen/include/asm-arm/setup.h |  3 +++
>   5 files changed, 79 insertions(+), 9 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 64f8d6b..f073e6d 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -2137,7 +2137,7 @@ int __init construct_dom0(struct domain *d)
>       kinfo.unassigned_mem = dom0_mem;
>       kinfo.d = d;
>   
> -    rc = kernel_probe(&kinfo);
> +    rc = kernel_probe(&kinfo, NULL);
>       if ( rc < 0 )
>           return rc;
>   
> diff --git a/xen/arch/arm/kernel.c b/xen/arch/arm/kernel.c
> index da8410e..e5b8213 100644
> --- a/xen/arch/arm/kernel.c
> +++ b/xen/arch/arm/kernel.c
> @@ -421,22 +421,60 @@ static int __init kernel_zimage32_probe(struct kernel_info *info,
>       return 0;
>   }
>   
> -int __init kernel_probe(struct kernel_info *info)
> +int __init kernel_probe(struct kernel_info *info, struct dt_device_node *domain)
>   {
> -    struct bootmodule *mod = boot_module_find_by_kind(BOOTMOD_KERNEL);
> +    struct bootmodule *mod = NULL;
> +    struct bootcmdline *cmd = NULL;
> +    struct dt_device_node *node;
> +    u64 kernel_addr, initrd_addr, size;
> +    const char *name = NULL;

Please try to limit scope of variables. For instance, name is only used

>       int rc;
>   
> +    if ( domain == NULL )

A line explain this is for the hardware domain would be useful.

Maybe with an ASSERT(is_hardware_domain(d)) in the if.

> +    {
> +        mod = boot_module_find_by_kind(BOOTMOD_KERNEL);
> +
> +        info->kernel_bootmodule = mod;
> +        info->initrd_bootmodule = boot_module_find_by_kind(BOOTMOD_RAMDISK);
> +    } else {

Coding style:

}
else
{

> +        dt_for_each_child_node(domain, node)
> +        {
> +            if ( dt_device_is_compatible(node, "multiboot,kernel") )
> +            {
> +                u32 len;
> +                const __be32 *val;

Newline.

> +                val = dt_get_property(node, "reg", &len);
> +                dt_get_range(&val, node, &kernel_addr, &size);

Why don't you use dt_device_get_address?

> +            }
> +            else if ( dt_device_is_compatible(node, "multiboot,ramdisk") )
> +            {
> +                u32 len;
> +                const __be32 *val;

Newline.

> +                val = dt_get_property(node, "reg", &len);
> +                dt_get_range(&val, node, &initrd_addr, &size);

Ditto.

> +            }
> +            else
> +                continue;
> +        }
> +        if ( kernel_addr )

0 is a valid physical address. Please use INVALID_PADDR when checking 
whether the user specified an address.

But, why don't you assign info->kernel_bootmodule when you found the 
node associated to it?

> +            info->kernel_bootmodule = mod = boot_module_find_by_addr_and_kind(
> +                                      BOOTMOD_KERNEL, kernel_addr);

This code is not easy to read because of the indentation and the double 
assignment. As you change half the user of mod below, then you can drop 
it completely.

For the indentation, you could use a temporary variable. This would also 
help to avoid using info-> everywhere below.

> +        if ( initrd_addr )
> +            info->initrd_bootmodule = boot_module_find_by_addr_and_kind(
> +                                      BOOTMOD_RAMDISK, initrd_addr);
> +        name = dt_node_name(domain);
> +        cmd = boot_cmdline_find_by_name(name);
> +        if ( cmd )
> +            info->cmdline = &cmd->cmdline[0];

If you set command line for DomU here, then please also set the command 
line for the hwdom above. So the function has the same behavior accross 
all the domain.

> +    }
>       if ( !mod || !mod->size )
>       {
>           printk(XENLOG_ERR "Missing kernel boot module?\n");
>           return -ENOENT;
>       }
>   
> -    info->kernel_bootmodule = mod;
> -
> -    printk("Loading kernel from boot module @ %"PRIpaddr"\n", mod->start);
> -
> -    info->initrd_bootmodule = boot_module_find_by_kind(BOOTMOD_RAMDISK);
> +    printk("Loading DomU kernel from boot module @ %"PRIpaddr"\n",

This message is wrong for the hardware domain. The best solution would 
be to print dom%u.

> +           info->kernel_bootmodule->start);
>       if ( info->initrd_bootmodule )
>           printk("Loading ramdisk from boot module @ %"PRIpaddr"\n",
>                  info->initrd_bootmodule->start);
> diff --git a/xen/arch/arm/kernel.h b/xen/arch/arm/kernel.h
> index 39b7828..4a65289 100644
> --- a/xen/arch/arm/kernel.h
> +++ b/xen/arch/arm/kernel.h
> @@ -55,7 +55,7 @@ struct kernel_info {
>    *  ->type
>    *  ->load hook, and sets loader specific variables ->zimage
>    */
> -int kernel_probe(struct kernel_info *info);
> +int kernel_probe(struct kernel_info *info, struct dt_device_node *domain);
>   
>   /*
>    * Loads the kernel into guest RAM.
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index dbab232..d6d1546 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -263,6 +263,35 @@ struct bootcmdline * __init boot_cmdline_find_by_kind(bootmodule_kind kind)
>       return NULL;
>   }
>   
> +struct bootcmdline * __init boot_cmdline_find_by_name(const char *name)
> +{
> +    struct bootcmdlines *mods = &bootinfo.cmdlines;
> +    struct bootcmdline *mod;
> +    int i;

Unsigned + newline.

> +    for (i = 0 ; i < mods->nr_mods ; i++ )
> +    {
> +        mod = &mods->cmdline[i];
> +        if ( strcmp(mod->dt_name, name) == 0 )
> +            return mod;
> +    }
> +    return NULL;
> +}
> +
> +struct bootmodule * __init boot_module_find_by_addr_and_kind(bootmodule_kind kind,
> +                                                             paddr_t start)
> +{
> +    struct bootmodules *mods = &bootinfo.modules;
> +    struct bootmodule *mod; > +    int i;

Same here.

> +    for (i = 0 ; i < mods->nr_mods ; i++ )
> +    {
> +        mod = &mods->module[i];
> +        if ( mod->kind == kind && mod->start == start )
> +            return mod;
> +    }
> +    return NULL;
> +}
> +
>   const char * __init boot_module_kind_as_string(bootmodule_kind kind)
>   {
>       switch ( kind )
> diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
> index 711b4a2..177e8db 100644
> --- a/xen/include/asm-arm/setup.h
> +++ b/xen/include/asm-arm/setup.h
> @@ -95,7 +95,10 @@ const char __init *boot_fdt_cmdline(const void *fdt);
>   struct bootmodule *add_boot_module(bootmodule_kind kind,
>                                      paddr_t start, paddr_t size, bool domU);
>   struct bootmodule *boot_module_find_by_kind(bootmodule_kind kind);
> +struct bootmodule * __init boot_module_find_by_addr_and_kind(bootmodule_kind kind,
> +                                                             paddr_t start);
>   struct bootcmdline *boot_cmdline_find_by_kind(bootmodule_kind kind);
> +struct bootcmdline * __init boot_cmdline_find_by_name(const char *name);
>   const char * __init boot_module_kind_as_string(bootmodule_kind kind);
>   
>   #endif
> 

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 08/23] xen/arm: rename get_11_allocation_size to get_allocation_size

[Julien Grall]

Hi Stefano,

On 05/10/2018 19:47, Stefano Stabellini wrote:
> No functional changes.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>

Acked-by: Julien Grall <julien.grall@arm.com>

Cheers,

> 
> ---
> 
> Changes in v3:
> - no change in print messages
> - do not remove BUG_ON
> 
> Changes in v2:
> - new patch
> ---
>   xen/arch/arm/domain_build.c | 12 ++++++------
>   1 file changed, 6 insertions(+), 6 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index f073e6d..d0aff35 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -77,7 +77,7 @@ struct vcpu *__init alloc_dom0_vcpu0(struct domain *dom0)
>       return vcpu_create(dom0, 0, 0);
>   }
>   
> -static unsigned int __init get_11_allocation_size(paddr_t size)
> +static unsigned int __init get_allocation_size(paddr_t size)
>   {
>       /*
>        * get_order_from_bytes returns the order greater than or equal to
> @@ -249,7 +249,7 @@ static void __init allocate_memory(struct domain *d, struct kernel_info *kinfo)
>           get_order_from_bytes(min_t(paddr_t, dom0_mem, MB(128)));
>       const unsigned int min_order = get_order_from_bytes(MB(4));
>       struct page_info *pg;
> -    unsigned int order = get_11_allocation_size(kinfo->unassigned_mem);
> +    unsigned int order = get_allocation_size(kinfo->unassigned_mem);
>       int i;
>   
>       bool lowmem = true;
> @@ -301,7 +301,7 @@ static void __init allocate_memory(struct domain *d, struct kernel_info *kinfo)
>        * If we failed to allocate bank0 under 4GB, continue allocating
>        * memory from above 4GB and fill in banks.
>        */
> -    order = get_11_allocation_size(kinfo->unassigned_mem);
> +    order = get_allocation_size(kinfo->unassigned_mem);
>       while ( kinfo->unassigned_mem && kinfo->mem.nr_banks < NR_MEM_BANKS )
>       {
>           pg = alloc_domheap_pages(d, order, lowmem ? MEMF_bits(32) : 0);
> @@ -312,7 +312,7 @@ static void __init allocate_memory(struct domain *d, struct kernel_info *kinfo)
>               if ( lowmem && order < min_low_order)
>               {
>                   D11PRINT("Failed at min_low_order, allow high allocations\n");
> -                order = get_11_allocation_size(kinfo->unassigned_mem);
> +                order = get_allocation_size(kinfo->unassigned_mem);
>                   lowmem = false;
>                   continue;
>               }
> @@ -332,7 +332,7 @@ static void __init allocate_memory(struct domain *d, struct kernel_info *kinfo)
>               if ( lowmem )
>               {
>                   D11PRINT("Allocation below bank 0, allow high allocations\n");
> -                order = get_11_allocation_size(kinfo->unassigned_mem);
> +                order = get_allocation_size(kinfo->unassigned_mem);
>                   lowmem = false;
>                   continue;
>               }
> @@ -347,7 +347,7 @@ static void __init allocate_memory(struct domain *d, struct kernel_info *kinfo)
>            * Success, next time around try again to get the largest order
>            * allocation possible.
>            */
> -        order = get_11_allocation_size(kinfo->unassigned_mem);
> +        order = get_allocation_size(kinfo->unassigned_mem);
>       }
>   
>       if ( kinfo->unassigned_mem )
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 09/23] xen/arm: rename allocate_memory to allocate_memory_11

[Julien Grall]

Hi Stefano,

On 05/10/2018 19:47, Stefano Stabellini wrote:
> allocate_memory only deals with directly mapped memory. Rename it to
> allocate_memory_11.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>

Acked-by: Julien Grall <julien.grall@arm.com>

Cheers,

> 
> ---
> Changes in v3:
> - add patch
> ---
>   xen/arch/arm/domain_build.c | 5 +++--
>   1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index d0aff35..41f2f27 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -243,7 +243,8 @@ fail:
>    * (as described above) we allow higher allocations and continue until
>    * that runs out (or we have allocated sufficient dom0 memory).
>    */
> -static void __init allocate_memory(struct domain *d, struct kernel_info *kinfo)
> +static void __init allocate_memory_11(struct domain *d,
> +                                      struct kernel_info *kinfo)
>   {
>       const unsigned int min_low_order =
>           get_order_from_bytes(min_t(paddr_t, dom0_mem, MB(128)));
> @@ -2156,7 +2157,7 @@ int __init construct_dom0(struct domain *d)
>   #endif
>   
>       kinfo.cmdline = kernel != NULL ? &kernel->cmdline[0] : NULL;
> -    allocate_memory(d, &kinfo);
> +    allocate_memory_11(d, &kinfo);
>       find_gnttab_region(d, &kinfo);
>   
>       /* Map extra GIC MMIO, irqs and other hw stuffs to dom0. */
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 10/23] xen/arm: introduce allocate_memory

[Julien Grall]

Hi,

On 05/10/2018 19:47, Stefano Stabellini wrote:
> Introduce an allocate_memory function able to allocate memory for DomUs
> and map it at the right guest addresses, according to the guest memory
> map: GUEST_RAM0_BASE and GUEST_RAM1_BASE.

Please add something along the line: "This is under #if 0 as not used 
for now".

> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> ---
> Changes in v4:
> - move earlier, add #if 0
> - introduce allocate_bank_memory, remove insert_bank
> - allocate_bank_memory allocate memory and inserts the bank, while
>    allocate_memory specifies where to do that
> 
> Changes in v3:
> - new patch
> ---
>   xen/arch/arm/domain_build.c | 83 +++++++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 83 insertions(+)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 41f2f27..fddfd82 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -368,6 +368,89 @@ static void __init allocate_memory_11(struct domain *d,
>       }
>   }
>   
> +#if 0
> +static bool __init allocate_bank_memory(struct domain *d,
> +                                        struct kernel_info *kinfo,
> +                                        gfn_t sgfn,
> +                                        unsigned int order)
> +{
> +    int res;
> +    struct page_info *pg;
> +    struct membank *bank;
> +    paddr_t gaddr = gfn_to_gaddr(sgfn), size = pfn_to_paddr(1UL << order);

There is no need to have gaddr in a variable.

> +
> +    pg = alloc_domheap_pages(d, order, 0);

So here you impose the memory to be contiguously allocated for a given 
bank. There are quite a few case where you may not have enough memory to 
allocate contiguously.

So more likely you want to add loop in this code to allocate until order 
is reached.

> +    if ( !pg )
> +        return false;
> +
> +    res = guest_physmap_add_page(d, sgfn, page_to_mfn(pg), order);
> +    if ( res )
> +    {
> +        dprintk(XENLOG_ERR, "Failed map pages to DOMU: %d", res);
> +        goto fail;
> +    }
> +
> +    bank = &kinfo->mem.bank[kinfo->mem.nr_banks];
> +    bank->start = gaddr;
> +    bank->size = size;
> +    kinfo->mem.nr_banks++;
> +    kinfo->unassigned_mem -= size;

newline here.

> +    return true;
> +
> +fail:
> +    free_domheap_pages(pg, order);
> +    return false;
> +}
> +
> +static void __init allocate_memory(struct domain *d, struct kernel_info *kinfo)
> +{
> +    unsigned int order = get_allocation_size(kinfo->unassigned_mem);
> +    unsigned int order_req;
> +    int i;

unsigned.

> +
> +    dprintk(XENLOG_INFO, "Allocating mappings totalling %ldMB for dom%d:\n",
> +            /* Don't want format this as PRIpaddr (16 digit hex) */
> +            (unsigned long)(kinfo->unassigned_mem >> 20), d->domain_id);
> +
> +    kinfo->mem.nr_banks = 0;
> +
> +    order = get_allocation_size(kinfo->unassigned_mem);
> +    if ( order > get_order_from_bytes(GUEST_RAM0_SIZE) )
> +        order_req = get_order_from_bytes(GUEST_RAM0_SIZE);
> +    else
> +        order_req = order;
> +    if ( !allocate_bank_memory(d, kinfo, gaddr_to_gfn(GUEST_RAM0_BASE), order_req) )
> +        goto fail;
> +
> +    order -= order_req;
> +    if ( order > 0 )
> +    {
> +        if ( !allocate_bank_memory(d, kinfo, gaddr_to_gfn(GUEST_RAM1_BASE), order) )
> +            goto fail;
> +    }

How about combining the two if to avoid an extra indentation?

> +
> +    for( i = 0; i < kinfo->mem.nr_banks; i++ )
> +    {
> +        dprintk(XENLOG_INFO, "Dom%d BANK[%d] %#"PRIpaddr"-%#"PRIpaddr" (%ldMB)\n",
> +                d->domain_id,
> +                i,
> +                kinfo->mem.bank[i].start,
> +                kinfo->mem.bank[i].start + kinfo->mem.bank[i].size,
> +                /* Don't want format this as PRIpaddr (16 digit hex) */
> +                (unsigned long)(kinfo->mem.bank[i].size >> 20));
> +    }
> +
> +    return;
> +
> +fail:
> +    dprintk(XENLOG_ERR, "Failed to allocate requested domain memory."
> +            /* Don't want format this as PRIpaddr (16 digit hex) */
> +            " %ldKB unallocated. Fix the VMs configurations.\n",
> +            (unsigned long)kinfo->unassigned_mem >> 10);
> +    BUG();
> +}
> +#endif
> +
>   static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
>                                      const struct dt_device_node *node)
>   {
> 

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 11/23] xen/arm: refactor construct_dom0

[Julien Grall]

Hi,

On 05/10/2018 19:47, Stefano Stabellini wrote:
> Move generic initializations out of construct_dom0 so that they can be
> reused.
> 
> Rename prepare_dtb to prepare_dtb_hwdom to avoid confusion.
> 
> No functional changes in this patch.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> 
> ---
> Changes in v4:
> - newline and style changes
> 
> Changes in v3:
> - move setting type before allocate_memory
> - add ifdef around it and a comment
> 
> Changes in v2:
> - move discard_initial_modules() after __construct_domain()
> - remove useless blank line
> - leave safety BUG_ONs in __construct_domain
> - rename prepare_dtb to prepare_dtb_hwdom
> ---
>   xen/arch/arm/domain_build.c | 126 ++++++++++++++++++++++++--------------------
>   1 file changed, 68 insertions(+), 58 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index fddfd82..ba3dad1 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -1456,7 +1456,7 @@ static int __init handle_node(struct domain *d, struct kernel_info *kinfo,
>       return res;
>   }
>   
> -static int __init prepare_dtb(struct domain *d, struct kernel_info *kinfo)
> +static int __init prepare_dtb_hwdom(struct domain *d, struct kernel_info *kinfo)
>   {
>       const p2m_type_t default_p2mt = p2m_mmio_direct_c;
>       const void *fdt;
> @@ -2191,75 +2191,29 @@ static void __init find_gnttab_region(struct domain *d,
>              kinfo->gnttab_start, kinfo->gnttab_start + kinfo->gnttab_size);
>   }
>   
> -int __init construct_dom0(struct domain *d)
> +static int __init __construct_domain(struct domain *d, struct kernel_info *kinfo)

Why do you need to add __ in front? The more we are trying to avoid 
introducing name/variable with __ in front.

The rest of the patch looks good to me.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 12/23] xen/arm: introduce create_domUs

[Julien Grall]

Hi,

On 05/10/2018 19:47, Stefano Stabellini wrote:
> Call a new function, "create_domUs", from setup_xen to start DomU VMs.
> 
> Introduce support for the "xen,domU" compatible node on device tree.

s/xen,domU/xen,domain/

> Create new DomU VMs based on the information found on device tree under
> "xen,domU". Calls construct_domU for each domain.

Ditto

s/Calls/Call/

> 
> Introduce a simple global variable named max_init_domid to keep track of
> the initial allocated domids. It holds the max domid among the initial
> domains.
> 
> Move the discard_initial_modules after DomUs have been built.
> 
> First create domUs, then start dom0 -- no point in trying to start dom0
> when the cpu is busy.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>
> CC: andrew.cooper3@citrix.com
> CC: jbeulich@suse.com
> ---
> Changes in v4:
> - constify parameters
> - nr_spis to 0 or  GUEST_VPL011_SPI - 32 + 1 depending on vpl011
> - remove pointless initializer
> - remove change to domain_create for dom0 (useless)
> - make construct_domU return error
> 
> Changes in v3:
> - move patch earlier and introduce empty construct_domU to fix bisection
>    builds
> - fix max_init_domid to actually hold the max domid among initial
>    domains (instead of max_domid + 1)
> - move domain_unpause_by_systemcontroller(dom0) after creating domUs
> 
> Changes in v2:
> - coding style
> - set nr_spis to 32
> - introduce create_domUs
> ---
>   xen/arch/arm/domain_build.c | 51 +++++++++++++++++++++++++++++++++++++++++----
>   xen/arch/arm/setup.c        |  5 +++++
>   xen/include/asm-arm/setup.h |  3 +++
>   xen/include/asm-x86/setup.h |  2 ++
>   4 files changed, 57 insertions(+), 4 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index ba3dad1..547b624 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -7,6 +7,7 @@
>   #include <asm/irq.h>
>   #include <asm/regs.h>
>   #include <xen/errno.h>
> +#include <xen/err.h>
>   #include <xen/device_tree.h>
>   #include <xen/libfdt/libfdt.h>
>   #include <xen/guest_access.h>
> @@ -2290,6 +2291,51 @@ static int __init __construct_domain(struct domain *d, struct kernel_info *kinfo
>       return 0;
>   }
>   
> +static int __init construct_domU(struct domain *d,
> +                                 const struct dt_device_node *node)
> +{
> +    return -ENOSYS;
> +}
> +
> +void __init create_domUs(void)
> +{
> +    struct dt_device_node *node;
> +    const struct dt_device_node *chosen = dt_find_node_by_name(dt_host,
> +                                                               "chosen");

Newline

It would also be better to use dt_find_node_by_path("/chosen") because 
dt_find_node_by_name may return a different node (e.g /foo/chosen).

> +    BUG_ON(chosen == NULL);
> +    dt_for_each_child_node(chosen, node)
> +    {
> +        u32 len;
> +        struct domain *d;
> +        struct xen_domctl_createdomain d_cfg = {
> +            .arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE,
> +            .arch.nr_spis = 0,
> +            .max_vcpus = 1,
> +            .max_evtchn_port = -1,
> +            .max_grant_frames = 64,
> +            .max_maptrack_frames = 1024,
> +        };
> +
> +        if ( !dt_device_is_compatible(node, "xen,domain") )
> +            continue;
> +
> +        if ( dt_get_property(node, "vpl011", &len) != NULL )

dt_property_read_bool will do the right thing for you.

> +            d_cfg.arch.nr_spis = GUEST_VPL011_SPI - 32 + 1;
> +        dt_property_read_u32(node, "cpus", &d_cfg.max_vcpus);
> +
> +        d = domain_create(++max_init_domid, &d_cfg, false);
> +        if ( IS_ERR(d) )
> +            panic("Error creating domain %s", dt_node_name(node));
> +
> +        d->is_console = 1;

is_console is a bool, so this should be true.

> +
> +        if ( construct_domU(d, node) != 0 )
> +            panic("Could not set up domain %s", dt_node_name(node));
> +
> +        domain_unpause_by_systemcontroller(d);
> +    }
> +}
> +
>   int __init construct_dom0(struct domain *d)
>   {
>       const struct bootcmdline *kernel = boot_cmdline_find_by_kind(BOOTMOD_KERNEL);
> @@ -2342,10 +2388,7 @@ int __init construct_dom0(struct domain *d)
>       if ( rc < 0 )
>           return rc;
>   
> -    rc = __construct_domain(d, &kinfo);
> -    discard_initial_modules();
> -
> -    return rc;
> +    return __construct_domain(d, &kinfo);
>   }
>   
>   /*
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index d6d1546..8d8f180 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -64,8 +64,11 @@ static unsigned long opt_xenheap_megabytes __initdata;
>   integer_param("xenheap_megabytes", opt_xenheap_megabytes);
>   #endif
>   
> +domid_t __read_mostly max_init_domid;
> +
>   static __used void init_done(void)
>   {
> +    discard_initial_modules();
>       free_init_memory();
>       startup_cpu_idle_loop();
>   }
> @@ -926,6 +929,8 @@ void __init start_xen(unsigned long boot_phys_offset,
>       /* Must be done past setting system_state. */
>       unregister_init_virtual_region();
>   
> +    create_domUs();

I missed that bits in the previous version. I am not very sure this 
should be called after unregister_init_virtual_region(). If you call it 
after and get an hypervisor crash in init code, then the stack trace may 
miss some information (e.g bug frame).

Looking at x86, unregister_init_virtual_region() is called from 
init_done. So we probably want to move that call there.

> +
>       domain_unpause_by_systemcontroller(dom0);
>   
>       /* Switch on to the dynamically allocated stack for the idle vcpu
> diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
> index 177e8db..5620fe7 100644
> --- a/xen/include/asm-arm/setup.h
> +++ b/xen/include/asm-arm/setup.h
> @@ -68,6 +68,8 @@ struct bootinfo {
>   
>   extern struct bootinfo bootinfo;
>   
> +extern domid_t max_init_domid;
> +
>   void arch_init_memory(void);
>   
>   void copy_from_paddr(void *dst, paddr_t paddr, unsigned long len);
> @@ -84,6 +86,7 @@ void acpi_create_efi_mmap_table(struct domain *d,
>   int acpi_make_efi_nodes(void *fdt, struct membank tbl_add[]);
>   
>   int construct_dom0(struct domain *d);
> +void __init create_domUs(void);
>   
>   void discard_initial_modules(void);
>   void dt_unreserved_regions(paddr_t s, paddr_t e,
> diff --git a/xen/include/asm-x86/setup.h b/xen/include/asm-x86/setup.h
> index 42fddeb..1c80783 100644
> --- a/xen/include/asm-x86/setup.h
> +++ b/xen/include/asm-x86/setup.h
> @@ -66,4 +66,6 @@ extern bool opt_dom0_shadow;
>   #endif
>   extern bool dom0_pvh;
>   
> +#define max_init_domid (0)
> +
>   #endif
> 

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 13/23] xen/arm: implement construct_domU

[Julien Grall]

Hi,

On 05/10/2018 19:47, Stefano Stabellini wrote:
> Similar to construct_dom0, construct_domU creates a barebone DomU guest.
> 
> The device tree node passed as argument is compatible "xen,domain", see
> docs/misc/arm/device-tree/booting.txt.
> 
> Add const to kernel_probe dt_device_node parameter.

This likely belongs to patch #7 where the parameter was added.

> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> 
> ---
> Changes in v4:
> - constify kernel_probe
> - change title
> - better error messages and printed info
> - 64bit memory
> 
> Changes in v3:
> - move setting type before allocate_memory
> - add ifdef around it and a comment
> 
> Changes in v2:
> - rename mem to memory
> - make cpus and memory mandatory
> - remove wront comment from commit message
> - cpus and memory are read as integers
> - read the vpl011 option
> ---
>   xen/arch/arm/domain_build.c | 37 ++++++++++++++++++++++++++++++++++---
>   xen/arch/arm/kernel.c       |  3 ++-
>   xen/arch/arm/kernel.h       |  2 +-
>   3 files changed, 37 insertions(+), 5 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 547b624..efb530a 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -369,7 +369,6 @@ static void __init allocate_memory_11(struct domain *d,
>       }
>   }
>   
> -#if 0

Please add a word about this change in the commit message.

>   static bool __init allocate_bank_memory(struct domain *d,
>                                           struct kernel_info *kinfo,
>                                           gfn_t sgfn,
> @@ -450,7 +449,6 @@ fail:
>               (unsigned long)kinfo->unassigned_mem >> 10);
>       BUG();
>   }
> -#endif
>   
>   static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
>                                      const struct dt_device_node *node)
> @@ -2294,7 +2292,40 @@ static int __init __construct_domain(struct domain *d, struct kernel_info *kinfo
>   static int __init construct_domU(struct domain *d,
>                                    const struct dt_device_node *node)
>   {
> -    return -ENOSYS;
> +    struct kernel_info kinfo = {};
> +    int rc;
> +    u64 mem;
> +
> +    rc = dt_property_read_u64(node, "memory", &mem);
> +    if ( !rc )
> +    {
> +        printk("Error building DomU: cannot read \"memory\" property\n");
> +        return -EINVAL;
> +    }
> +    kinfo.unassigned_mem = (paddr_t)mem << 10;

I noticed I forgot to answer to:
"KB() only works for numbers, it is defined as: (_AC(_kb, ULL) << 10)"

unsigned long long is always going to be bigger than paddr_t. Also, we 
already use MB(...) in similar situation. So I am not sure to understand 
your concern here.

> +
> +    printk("*** LOADING DOMU cpus=%u memory=%luKB ***\n", d->max_vcpus, mem);
> +
> +    d->vcpu = xzalloc_array(struct vcpu *, d->max_vcpus);
> +    if ( !d->vcpu )
> +        return -ENOMEM;;

d->vcpu is already allocated by domain_create.

> +    if ( vcpu_create(d, 0, 0) == NULL )
> +        return -ENOMEM;
> +    d->max_pages = ~0U;
> +
> +    kinfo.d = d;
> +
> +    rc = kernel_probe(&kinfo, node);
> +    if ( rc < 0 )
> +        return rc;
> +
> +#ifdef CONFIG_ARM_64
> +    /* type must be set before allocate memory */
> +    d->arch.type = kinfo.type;
> +#endif
> +    allocate_memory(d, &kinfo);
> +
> +    return __construct_domain(d, &kinfo);
>   }
>   
>   void __init create_domUs(void)
> diff --git a/xen/arch/arm/kernel.c b/xen/arch/arm/kernel.c
> index e5b8213..2239a07 100644
> --- a/xen/arch/arm/kernel.c
> +++ b/xen/arch/arm/kernel.c
> @@ -421,7 +421,8 @@ static int __init kernel_zimage32_probe(struct kernel_info *info,
>       return 0;
>   }
>   
> -int __init kernel_probe(struct kernel_info *info, struct dt_device_node *domain)
> +int __init kernel_probe(struct kernel_info *info,
> +                        const struct dt_device_node *domain)
>   {
>       struct bootmodule *mod = NULL;
>       struct bootcmdline *cmd = NULL;
> diff --git a/xen/arch/arm/kernel.h b/xen/arch/arm/kernel.h
> index 4a65289..4320f72 100644
> --- a/xen/arch/arm/kernel.h
> +++ b/xen/arch/arm/kernel.h
> @@ -55,7 +55,7 @@ struct kernel_info {
>    *  ->type
>    *  ->load hook, and sets loader specific variables ->zimage
>    */
> -int kernel_probe(struct kernel_info *info, struct dt_device_node *domain);
> +int kernel_probe(struct kernel_info *info, const struct dt_device_node *domain);
>   
>   /*
>    * Loads the kernel into guest RAM.
> 

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 14/23] xen/arm: generate a simple device tree for domUs

[Julien Grall]

Hi Stefano,

On 05/10/2018 19:47, Stefano Stabellini wrote:
> +static int __init make_gic_domU_node(const struct domain *d, void *fdt)
> +{
> +    switch ( gic_hw_version() )

While I understand that today domains will use the same GIC version as 
the host, it would be best if we don't rely on this in the generation of 
the DT.

So I would use d->arch.vgic.version here.

With that change:

Acked-by: Julien Grall <julien.grall@arm.com>


> +    {
> +    case GIC_V3:
> +        return make_gicv3_domU_node(d, fdt);
> +    case GIC_V2:
> +        return make_gicv2_domU_node(d, fdt);
> +    default:
> +        panic("Unsupported GIC version");
> +    }
> +}

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 15/23] xen/arm: make set_interrupt_ppi able to handle non-PPI

[Julien Grall]

Hi Stefano,

On 05/10/2018 19:47, Stefano Stabellini wrote:
> also rename it to set_interrupt.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>

Reviewed-by: Julien Grall <julien.grall@arm.com>

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 16/23] xen/arm: generate vpl011 node on device tree for domU

[Julien Grall]

Hi,

On 05/10/2018 19:47, Stefano Stabellini wrote:
> Introduce vpl011 support to guests started from Xen: it provides a
> simple way to print output from a guest, as most guests come with a
> pl011 driver. It is also able to provide a working console with
> interrupt support.
> 
> The UART exposed to the guest is a SBSA compatible UART and not a PL011.
> SBSA UART is a subset of PL011 r1p5. A full PL011 implementation in Xen
> would just be too difficult, so guests may require some drivers changes.
> 
> Enable vpl011 conditionally if the user requested it.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> ---
> Changes in v4:
> - move rename set_interrupt_ppi and making set_interrupt_ppi generic to
>    a separate patch
> - properly name the vpl011 device node name
> - code style
> - use #define for addrcells and sizecells
> 
> Changes in v3:
> - use bool
> - retain BUG_ON(irq < 16)
> - add vpl011 bool to kinfo
> - return error of vpl011 is required but CONFIG_SBSA_VUART_CONSOLE is
>    missing
> 
> Changes in v2:
> - code style fixes
> - make set_interrupt_ppi generic
> - rename set_interrupt_ppi to set_interrupt
> - only make the vpl011 node if the option was enabled
> ---
>   xen/arch/arm/domain_build.c | 61 +++++++++++++++++++++++++++++++++++++++++++++
>   xen/arch/arm/kernel.h       |  3 +++
>   2 files changed, 64 insertions(+)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 760ebf8..049ab84 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -1605,6 +1605,54 @@ static int __init make_timer_domU_node(const struct domain *d, void *fdt)
>       return res;
>   }
>   
> +#ifdef CONFIG_SBSA_VUART_CONSOLE
> +static int __init make_vpl011_uart_node(const struct domain *d, void *fdt)
> +{
> +    int res;
> +    gic_interrupt_t intr;
> +    __be32 reg[GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS];
> +    __be32 *cells;
> +
> +    res = fdt_begin_node(fdt, "sbsa-uart@"__stringify(GUEST_PL011_BASE));
> +    if ( res )
> +        return res;
> +
> +    res = fdt_property_string(fdt, "compatible", "arm,sbsa-uart");
> +    if ( res )
> +        return res;
> +
> +    cells = &reg[0];
> +    dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS,
> +                       GUEST_ROOT_SIZE_CELLS, GUEST_PL011_BASE,
> +                       GUEST_PL011_SIZE);
> +    if ( res )
> +        return res;
> +    res = fdt_property(fdt, "reg", reg, sizeof(reg));
> +    if ( res )
> +        return res;
> +
> +    set_interrupt(intr, GUEST_VPL011_SPI, 0xf, DT_IRQ_TYPE_LEVEL_HIGH);
> +
> +    res = fdt_property(fdt, "interrupts", intr, sizeof (intr));
> +    if ( res )
> +        return res;
> +
> +    res = fdt_property_cell(fdt, "interrupt-parent",
> +                            GUEST_PHANDLE_GIC);
> +    if ( res )
> +        return res;
> +
> +    /* Use a default baud rate of 115200. */
> +    fdt_property_u32(fdt, "current-speed", 115200);
> +
> +    res = fdt_end_node(fdt);
> +    if ( res )
> +        return res;
> +
> +    return 0;
> +}
> +#endif
> +
>   /*
>    * The max size for DT is 2MB. However, the generated DT is small, 4KB
>    * are enough for now, but we might have to increase it in the future.
> @@ -1666,6 +1714,16 @@ static int __init prepare_dtb_domU(struct domain *d, struct kernel_info *kinfo)
>       if ( ret )
>           goto err;
>   
> +    if ( kinfo->vpl011 )
> +    {
> +        ret = -EINVAL;
> +#ifdef CONFIG_SBSA_VUART_CONSOLE
> +        ret = make_vpl011_uart_node(d, kinfo->fdt);
> +#endif
> +        if ( ret )
> +            goto err;
> +    }
> +
>       ret = fdt_end_node(kinfo->fdt);
>       if ( ret < 0 )
>           goto err;
> @@ -2523,6 +2581,7 @@ static int __init construct_domU(struct domain *d,
>       struct kernel_info kinfo = {};
>       int rc;
>       u64 mem;
> +    u32 len;
>   
>       rc = dt_property_read_u64(node, "memory", &mem);
>       if ( !rc )
> @@ -2534,6 +2593,8 @@ static int __init construct_domU(struct domain *d,
>   
>       printk("*** LOADING DOMU cpus=%u memory=%luKB ***\n", d->max_vcpus, mem);
>   
> +    kinfo.vpl011 = dt_get_property(node, "vpl011", &len) != NULL;

You can use dt_property_read_bool here.

> +
>       d->vcpu = xzalloc_array(struct vcpu *, d->max_vcpus);
>       if ( !d->vcpu )
>           return -ENOMEM;;
> diff --git a/xen/arch/arm/kernel.h b/xen/arch/arm/kernel.h
> index 4320f72..33f3e72 100644
> --- a/xen/arch/arm/kernel.h
> +++ b/xen/arch/arm/kernel.h
> @@ -33,6 +33,9 @@ struct kernel_info {
>       paddr_t dtb_paddr;
>       paddr_t initrd_paddr;
>   
> +    /* Enable pl011 emulation */
> +    bool vpl011;
> +
>       /* loader to use for this kernel */
>       void (*load)(struct kernel_info *info);
>       /* loader specific state */
> 

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 17/23] xen/arm: introduce a union in vpl011

[Julien Grall]

Hi Stefano,

On 05/10/2018 19:47, Stefano Stabellini wrote:
> Introduce a union in struct vpl011 to contain the console ring members.
> A later patch will add another member of the union for the case where
> the backend is in Xen.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>

Acked-by: Julien Grall <julien.grall@arm.com>

Cheers,

> ---
> Changes in v4:
> - name union "backend"
> 
> Changes in v3:
> - rename ring field to dom
> 
> Changes in v2:
> - new patch
> ---
>   xen/arch/arm/vpl011.c        | 22 ++++++++++++----------
>   xen/include/asm-arm/vpl011.h |  8 ++++++--
>   2 files changed, 18 insertions(+), 12 deletions(-)
> 
> diff --git a/xen/arch/arm/vpl011.c b/xen/arch/arm/vpl011.c
> index a281eab..ebc045e 100644
> --- a/xen/arch/arm/vpl011.c
> +++ b/xen/arch/arm/vpl011.c
> @@ -82,7 +82,7 @@ static uint8_t vpl011_read_data(struct domain *d)
>       unsigned long flags;
>       uint8_t data = 0;
>       struct vpl011 *vpl011 = &d->arch.vpl011;
> -    struct xencons_interface *intf = vpl011->ring_buf;
> +    struct xencons_interface *intf = vpl011->backend.dom.ring_buf;
>       XENCONS_RING_IDX in_cons, in_prod;
>   
>       VPL011_LOCK(d, flags);
> @@ -145,7 +145,7 @@ static uint8_t vpl011_read_data(struct domain *d)
>   static void vpl011_update_tx_fifo_status(struct vpl011 *vpl011,
>                                            unsigned int fifo_level)
>   {
> -    struct xencons_interface *intf = vpl011->ring_buf;
> +    struct xencons_interface *intf = vpl011->backend.dom.ring_buf;
>       unsigned int fifo_threshold = sizeof(intf->out) - SBSA_UART_FIFO_LEVEL;
>   
>       BUILD_BUG_ON(sizeof(intf->out) < SBSA_UART_FIFO_SIZE);
> @@ -164,7 +164,7 @@ static void vpl011_write_data(struct domain *d, uint8_t data)
>   {
>       unsigned long flags;
>       struct vpl011 *vpl011 = &d->arch.vpl011;
> -    struct xencons_interface *intf = vpl011->ring_buf;
> +    struct xencons_interface *intf = vpl011->backend.dom.ring_buf;
>       XENCONS_RING_IDX out_cons, out_prod;
>   
>       VPL011_LOCK(d, flags);
> @@ -382,7 +382,7 @@ static void vpl011_data_avail(struct domain *d)
>   {
>       unsigned long flags;
>       struct vpl011 *vpl011 = &d->arch.vpl011;
> -    struct xencons_interface *intf = vpl011->ring_buf;
> +    struct xencons_interface *intf = vpl011->backend.dom.ring_buf;
>       XENCONS_RING_IDX in_cons, in_prod, out_cons, out_prod;
>       XENCONS_RING_IDX in_fifo_level, out_fifo_level;
>   
> @@ -459,14 +459,14 @@ int domain_vpl011_init(struct domain *d, struct vpl011_init_info *info)
>       int rc;
>       struct vpl011 *vpl011 = &d->arch.vpl011;
>   
> -    if ( vpl011->ring_buf )
> +    if ( vpl011->backend.dom.ring_buf )
>           return -EINVAL;
>   
>       /* Map the guest PFN to Xen address space. */
>       rc =  prepare_ring_for_helper(d,
>                                     gfn_x(info->gfn),
> -                                  &vpl011->ring_page,
> -                                  &vpl011->ring_buf);
> +                                  &vpl011->backend.dom.ring_page,
> +                                  &vpl011->backend.dom.ring_buf);
>       if ( rc < 0 )
>           goto out;
>   
> @@ -495,7 +495,8 @@ out2:
>       vgic_free_virq(d, GUEST_VPL011_SPI);
>   
>   out1:
> -    destroy_ring_for_helper(&vpl011->ring_buf, vpl011->ring_page);
> +    destroy_ring_for_helper(&vpl011->backend.dom.ring_buf,
> +			                vpl011->backend.dom.ring_page);
>   
>   out:
>       return rc;
> @@ -505,11 +506,12 @@ void domain_vpl011_deinit(struct domain *d)
>   {
>       struct vpl011 *vpl011 = &d->arch.vpl011;
>   
> -    if ( !vpl011->ring_buf )
> +    if ( !vpl011->backend.dom.ring_buf )
>           return;
>   
>       free_xen_event_channel(d, vpl011->evtchn);
> -    destroy_ring_for_helper(&vpl011->ring_buf, vpl011->ring_page);
> +    destroy_ring_for_helper(&vpl011->backend.dom.ring_buf,
> +			                vpl011->backend.dom.ring_page);
>   }
>   
>   /*
> diff --git a/xen/include/asm-arm/vpl011.h b/xen/include/asm-arm/vpl011.h
> index db95ff8..42d7a24 100644
> --- a/xen/include/asm-arm/vpl011.h
> +++ b/xen/include/asm-arm/vpl011.h
> @@ -31,8 +31,12 @@
>   #define SBSA_UART_FIFO_SIZE 32
>   
>   struct vpl011 {
> -    void *ring_buf;
> -    struct page_info *ring_page;
> +    union {
> +        struct {
> +            void *ring_buf;
> +            struct page_info *ring_page;
> +        } dom;
> +    } backend;
>       uint32_t    uartfr;         /* Flag register */
>       uint32_t    uartcr;         /* Control register */
>       uint32_t    uartimsc;       /* Interrupt mask register*/
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 19/23] xen/arm: Allow vpl011 to be used by DomU

[Julien Grall]

On 05/10/2018 19:47, Stefano Stabellini wrote:
> Make vpl011 being able to be used without a userspace component in Dom0.
> In that case, output is printed to the Xen serial and input is received
> from the Xen serial one character at a time.
> 
> Call domain_vpl011_init during construct_domU if vpl011 is enabled.
> 
> Introduce a new ring struct with only the ring array to avoid a waste of
> memory. Introduce separate read_data and write_data functions for
> initial domains: vpl011_write_data_xen is very simple and just writes
> to the console, while vpl011_read_data_xen is a duplicate of
> vpl011_read_data. Although textually almost identical, we are forced to
> duplicate the functions because the struct layout is different.
> 
> Output characters are printed one by one, potentially leading to
> intermixed output of different domains on the console. A follow-up patch
> will solve the issue by introducing buffering.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>

Acked-by: Julien Grall <julien.grall@arm.com>

Cheers,

> ---
> Changes in v4:
> - code style
> 
> Changes in v3:
> - add in-code comments
> - improve existing comments
> - remove ifdef around domain_vpl011_init in construct_domU
> - add ASSERT
> - use SBSA_UART_FIFO_SIZE for in buffer size
> - rename ring_enable to backend_in_domain
> - rename struct xencons_in to struct vpl011_xen_backend
> - rename inring field to xen
> - rename helper functions accordingly
> - remove unnecessary stub implementation of vpl011_rx_char
> - move vpl011_rx_char_xen within the file to avoid the need of a forward
>    declaration of vpl011_data_avail
> - fix small bug in vpl011_rx_char_xen: increment in_prod before using it
>    to check xencons_queued.
> 
> Changes in v2:
> - only init if vpl011
> - rename vpl011_read_char to vpl011_rx_char
> - remove spurious change
> - fix coding style
> - use different ring struct
> - move the write_data changes to their own function
>    (vpl011_write_data_noring)
> - duplicate vpl011_read_data
> ---
>   xen/arch/arm/domain_build.c  |   9 +-
>   xen/arch/arm/vpl011.c        | 200 +++++++++++++++++++++++++++++++++++++------
>   xen/include/asm-arm/vpl011.h |   8 ++
>   3 files changed, 192 insertions(+), 25 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 049ab84..4379538 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -2618,7 +2618,14 @@ static int __init construct_domU(struct domain *d,
>       if ( rc < 0 )
>           return rc;
>   
> -    return __construct_domain(d, &kinfo);
> +    rc = __construct_domain(d, &kinfo);
> +    if ( rc < 0 )
> +        return rc;
> +
> +    if ( kinfo.vpl011 )
> +        rc = domain_vpl011_init(d, NULL);
> +
> +    return rc;
>   }
>   
>   void __init create_domUs(void)
> diff --git a/xen/arch/arm/vpl011.c b/xen/arch/arm/vpl011.c
> index 68452a8..131507e 100644
> --- a/xen/arch/arm/vpl011.c
> +++ b/xen/arch/arm/vpl011.c
> @@ -77,6 +77,91 @@ static void vpl011_update_interrupt_status(struct domain *d)
>   #endif
>   }
>   
> +/*
> + * vpl011_write_data_xen writes chars from the vpl011 out buffer to the
> + * console. Only to be used when the backend is Xen.
> + */
> +static void vpl011_write_data_xen(struct domain *d, uint8_t data)
> +{
> +    unsigned long flags;
> +    struct vpl011 *vpl011 = &d->arch.vpl011;
> +
> +    VPL011_LOCK(d, flags);
> +
> +    printk("%c", data);
> +    if (data == '\n')
> +        printk("DOM%u: ", d->domain_id);
> +
> +    vpl011->uartris |= TXI;
> +    vpl011->uartfr &= ~TXFE;
> +    vpl011_update_interrupt_status(d);
> +
> +    VPL011_UNLOCK(d, flags);
> +}
> +
> +/*
> + * vpl011_read_data_xen reads data when the backend is xen. Characters
> + * are added to the vpl011 receive buffer by vpl011_rx_char_xen.
> + */
> +static uint8_t vpl011_read_data_xen(struct domain *d)
> +{
> +    unsigned long flags;
> +    uint8_t data = 0;
> +    struct vpl011 *vpl011 = &d->arch.vpl011;
> +    struct vpl011_xen_backend *intf = vpl011->backend.xen;
> +    XENCONS_RING_IDX in_cons, in_prod;
> +
> +    VPL011_LOCK(d, flags);
> +
> +    in_cons = intf->in_cons;
> +    in_prod = intf->in_prod;
> +
> +    smp_rmb();
> +
> +    /*
> +     * It is expected that there will be data in the ring buffer when this
> +     * function is called since the guest is expected to read the data register
> +     * only if the TXFE flag is not set.
> +     * If the guest still does read when TXFE bit is set then 0 will be returned.
> +     */
> +    if ( xencons_queued(in_prod, in_cons, sizeof(intf->in)) > 0 )
> +    {
> +        unsigned int fifo_level;
> +
> +        data = intf->in[xencons_mask(in_cons, sizeof(intf->in))];
> +        in_cons += 1;
> +        smp_mb();
> +        intf->in_cons = in_cons;
> +
> +        fifo_level = xencons_queued(in_prod, in_cons, sizeof(intf->in));
> +
> +        /* If the FIFO is now empty, we clear the receive timeout interrupt. */
> +        if ( fifo_level == 0 )
> +        {
> +            vpl011->uartfr |= RXFE;
> +            vpl011->uartris &= ~RTI;
> +        }
> +
> +        /* If the FIFO is more than half empty, we clear the RX interrupt. */
> +        if ( fifo_level < sizeof(intf->in) - SBSA_UART_FIFO_LEVEL )
> +            vpl011->uartris &= ~RXI;
> +
> +        vpl011_update_interrupt_status(d);
> +    }
> +    else
> +        gprintk(XENLOG_ERR, "vpl011: Unexpected IN ring buffer empty\n");
> +
> +    /*
> +     * We have consumed a character or the FIFO was empty, so clear the
> +     * "FIFO full" bit.
> +     */
> +    vpl011->uartfr &= ~RXFF;
> +
> +    VPL011_UNLOCK(d, flags);
> +
> +    return data;
> +}
> +
>   static uint8_t vpl011_read_data(struct domain *d)
>   {
>       unsigned long flags;
> @@ -240,7 +325,10 @@ static int vpl011_mmio_read(struct vcpu *v,
>       case DR:
>           if ( !vpl011_reg32_check_access(dabt) ) goto bad_width;
>   
> -        *r = vreg_reg32_extract(vpl011_read_data(d), info);
> +        if ( vpl011->backend_in_domain )
> +            *r = vreg_reg32_extract(vpl011_read_data(d), info);
> +        else
> +            *r = vreg_reg32_extract(vpl011_read_data_xen(d), info);
>           return 1;
>   
>       case RSR:
> @@ -325,7 +413,10 @@ static int vpl011_mmio_write(struct vcpu *v,
>   
>           vreg_reg32_update(&data, r, info);
>           data &= 0xFF;
> -        vpl011_write_data(v->domain, data);
> +        if ( vpl011->backend_in_domain )
> +            vpl011_write_data(v->domain, data);
> +        else
> +            vpl011_write_data_xen(v->domain, data);
>           return 1;
>       }
>   
> @@ -430,6 +521,39 @@ static void vpl011_data_avail(struct domain *d,
>           vpl011->uartfr |= TXFE;
>   }
>   
> +/*
> + * vpl011_rx_char_xen adds a char to a domain's vpl011 receive buffer.
> + * It is only used when the vpl011 backend is in Xen.
> + */
> +void vpl011_rx_char_xen(struct domain *d, char c)
> +{
> +    unsigned long flags;
> +    struct vpl011 *vpl011 = &d->arch.vpl011;
> +    struct vpl011_xen_backend *intf = vpl011->backend.xen;
> +    XENCONS_RING_IDX in_cons, in_prod, in_fifo_level;
> +
> +    ASSERT(!vpl011->backend_in_domain);
> +    VPL011_LOCK(d, flags);
> +
> +    in_cons = intf->in_cons;
> +    in_prod = intf->in_prod;
> +    if ( xencons_queued(in_prod, in_cons, sizeof(intf->in)) == sizeof(intf->in) )
> +    {
> +        VPL011_UNLOCK(d, flags);
> +        return;
> +    }
> +
> +    intf->in[xencons_mask(in_prod, sizeof(intf->in))] = c;
> +    intf->in_prod = ++in_prod;
> +
> +    in_fifo_level = xencons_queued(in_prod,
> +                                   in_cons,
> +                                   sizeof(intf->in));
> +
> +    vpl011_data_avail(d, in_fifo_level, sizeof(intf->in), 0, SBSA_UART_FIFO_SIZE);
> +    VPL011_UNLOCK(d, flags);
> +}
> +
>   static void vpl011_notification(struct vcpu *v, unsigned int port)
>   {
>       unsigned long flags;
> @@ -470,27 +594,47 @@ int domain_vpl011_init(struct domain *d, struct vpl011_init_info *info)
>       if ( vpl011->backend.dom.ring_buf )
>           return -EINVAL;
>   
> -    /* Map the guest PFN to Xen address space. */
> -    rc =  prepare_ring_for_helper(d,
> -                                  gfn_x(info->gfn),
> -                                  &vpl011->backend.dom.ring_page,
> -                                  &vpl011->backend.dom.ring_buf);
> -    if ( rc < 0 )
> -        goto out;
> +    /*
> +     * info is NULL when the backend is in Xen.
> +     * info is != NULL when the backend is in a domain.
> +     */
> +    if ( info != NULL )
> +    {
> +        vpl011->backend_in_domain = true;
> +
> +        /* Map the guest PFN to Xen address space. */
> +        rc =  prepare_ring_for_helper(d,
> +                                      gfn_x(info->gfn),
> +                                      &vpl011->backend.dom.ring_page,
> +                                      &vpl011->backend.dom.ring_buf);
> +        if ( rc < 0 )
> +            goto out;
> +
> +        rc = alloc_unbound_xen_event_channel(d, 0, info->console_domid,
> +                                             vpl011_notification);
> +        if ( rc < 0 )
> +            goto out1;
> +
> +        vpl011->evtchn = info->evtchn = rc;
> +    }
> +    else
> +    {
> +        vpl011->backend_in_domain = false;
> +
> +        vpl011->backend.xen = xzalloc(struct vpl011_xen_backend);
> +        if ( vpl011->backend.xen == NULL )
> +        {
> +            rc = -EINVAL;
> +            goto out1;
> +        }
> +    }
>   
>       rc = vgic_reserve_virq(d, GUEST_VPL011_SPI);
>       if ( !rc )
>       {
>           rc = -EINVAL;
> -        goto out1;
> -    }
> -
> -    rc = alloc_unbound_xen_event_channel(d, 0, info->console_domid,
> -                                         vpl011_notification);
> -    if ( rc < 0 )
>           goto out2;
> -
> -    vpl011->evtchn = info->evtchn = rc;
> +    }
>   
>       spin_lock_init(&vpl011->lock);
>   
> @@ -503,8 +647,11 @@ out2:
>       vgic_free_virq(d, GUEST_VPL011_SPI);
>   
>   out1:
> -    destroy_ring_for_helper(&vpl011->backend.dom.ring_buf,
> -			                vpl011->backend.dom.ring_page);
> +    if ( vpl011->backend_in_domain )
> +        destroy_ring_for_helper(&vpl011->backend.dom.ring_buf,
> +                                vpl011->backend.dom.ring_page);
> +    else
> +        xfree(vpl011->backend.xen);
>   
>   out:
>       return rc;
> @@ -514,12 +661,17 @@ void domain_vpl011_deinit(struct domain *d)
>   {
>       struct vpl011 *vpl011 = &d->arch.vpl011;
>   
> -    if ( !vpl011->backend.dom.ring_buf )
> -        return;
> +    if ( vpl011->backend_in_domain )
> +    {
> +        if ( !vpl011->backend.dom.ring_buf )
> +            return;
>   
> -    free_xen_event_channel(d, vpl011->evtchn);
> -    destroy_ring_for_helper(&vpl011->backend.dom.ring_buf,
> -			                vpl011->backend.dom.ring_page);
> +        free_xen_event_channel(d, vpl011->evtchn);
> +        destroy_ring_for_helper(&vpl011->backend.dom.ring_buf,
> +                                vpl011->backend.dom.ring_page);
> +    }
> +    else
> +        xfree(vpl011->backend.xen);
>   }
>   
>   /*
> diff --git a/xen/include/asm-arm/vpl011.h b/xen/include/asm-arm/vpl011.h
> index 42d7a24..5eb6d25 100644
> --- a/xen/include/asm-arm/vpl011.h
> +++ b/xen/include/asm-arm/vpl011.h
> @@ -21,6 +21,7 @@
>   
>   #include <public/domctl.h>
>   #include <public/io/ring.h>
> +#include <public/io/console.h>
>   #include <asm/vreg.h>
>   #include <xen/mm.h>
>   
> @@ -29,13 +30,19 @@
>   #define VPL011_UNLOCK(d,flags) spin_unlock_irqrestore(&(d)->arch.vpl011.lock, flags)
>   
>   #define SBSA_UART_FIFO_SIZE 32
> +struct vpl011_xen_backend {
> +    char in[SBSA_UART_FIFO_SIZE];
> +    XENCONS_RING_IDX in_cons, in_prod;
> +};
>   
>   struct vpl011 {
> +    bool backend_in_domain;
>       union {
>           struct {
>               void *ring_buf;
>               struct page_info *ring_page;
>           } dom;
> +        struct vpl011_xen_backend *xen;
>       } backend;
>       uint32_t    uartfr;         /* Flag register */
>       uint32_t    uartcr;         /* Control register */
> @@ -57,6 +64,7 @@ struct vpl011_init_info {
>   int domain_vpl011_init(struct domain *d,
>                          struct vpl011_init_info *info);
>   void domain_vpl011_deinit(struct domain *d);
> +void vpl011_rx_char_xen(struct domain *d, char c);
>   #else
>   static inline int domain_vpl011_init(struct domain *d,
>                                        struct vpl011_init_info *info)
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 21/23] xen/vpl011: buffer out chars when the backend is xen

[Julien Grall]

Hi,

On 05/10/2018 19:47, Stefano Stabellini wrote:
> To avoid mixing the output of different domains on the console, buffer
> the output chars and print line by line. Unless the domain has input
> from the serial, in which case we want to print char by char for a
> smooth user experience.
> 
> The size of SBSA_UART_OUT_BUF_SIZE is arbitrary, choose the same size
> as VUART_BUT_SIZE used in vuart.c.
> 
> Export a function named console_input_domain() to allow others to know
> which domains has input at a given time.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> CC: andrew.cooper3@citrix.com
> CC: George.Dunlap@eu.citrix.com
> CC: ian.jackson@eu.citrix.com
> CC: jbeulich@suse.com
> CC: konrad.wilk@oracle.com
> CC: tim@xen.org
> CC: wei.liu2@citrix.com
> ---
> XXX: merge this patch with "xen/arm: Allow vpl011 to be used by DomU" on
>       commit

That's not going to make the series bisectable as it depends on an 
intermediate patch for console_input_domain().

The new logic looks better to me, few comments below.

> 
> Changes in v4:
> - make SBSA_UART_OUT_BUF_SIZE the same size of VUART_BUT_SIZE
> - rearrange the code to be clearer input and != input cases
> - code style
> - add \n when printing the out buffer because is full
> - don't print prefix for input domain
> ---
>   xen/arch/arm/vpl011.c        | 35 ++++++++++++++++++++++++++++++++---
>   xen/drivers/char/console.c   |  7 +++++++
>   xen/include/asm-arm/vpl011.h |  4 ++++
>   xen/include/xen/console.h    |  2 ++
>   4 files changed, 45 insertions(+), 3 deletions(-)
> 
> diff --git a/xen/arch/arm/vpl011.c b/xen/arch/arm/vpl011.c
> index 131507e..5e57ada 100644
> --- a/xen/arch/arm/vpl011.c
> +++ b/xen/arch/arm/vpl011.c
> @@ -28,6 +28,7 @@
>   #include <xen/lib.h>
>   #include <xen/mm.h>
>   #include <xen/sched.h>
> +#include <xen/console.h>
>   #include <public/domctl.h>
>   #include <public/io/console.h>
>   #include <asm/pl011-uart.h>
> @@ -85,12 +86,40 @@ static void vpl011_write_data_xen(struct domain *d, uint8_t data)
>   {
>       unsigned long flags;
>       struct vpl011 *vpl011 = &d->arch.vpl011;
> +    struct vpl011_xen_backend *intf = vpl011->backend.xen;
> +    struct domain *input = console_input_domain();
>   
>       VPL011_LOCK(d, flags);
>   
> -    printk("%c", data);
> -    if (data == '\n')
> -        printk("DOM%u: ", d->domain_id);
> +    intf->out[intf->out_prod++] = data;
> +    if ( d == input )
> +    {
> +        if ( intf->out_prod == 1 )
> +        {
> +            printk("%c", data);
> +            intf->out_prod = 0;
> +        }
> +        else
> +        {
> +            if ( data != '\n' )
> +                intf->out[intf->out_prod++] = '\n';
> +            intf->out[intf->out_prod++] = '\0';
> +            printk("%s", intf->out);
> +            intf->out_prod = 0;
> +        }
> +    }
> +    else
> +    {
> +        if ( intf->out_prod == SBSA_UART_OUT_BUF_SIZE - 2 ||
> +             data == '\n' )
> +        {
> +            if ( data != '\n' )
> +                intf->out[intf->out_prod++] = '\n';
> +            intf->out[intf->out_prod++] = '\0';
> +            printk("DOM%u: %s", d->domain_id, intf->out);
> +            intf->out_prod = 0;
> +        }
> +    }
>   
>       vpl011->uartris |= TXI;
>       vpl011->uartfr &= ~TXFE;
> diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
> index 0808bac..9a2b29a 100644
> --- a/xen/drivers/char/console.c
> +++ b/xen/drivers/char/console.c
> @@ -406,6 +406,13 @@ static void dump_console_ring_key(unsigned char key)
>    */
>   static unsigned int __read_mostly console_rx = 0;
>   
> +struct domain *console_input_domain(void)
> +{
> +    if ( console_rx == 0 )
> +            return NULL;
> +    return get_domain_by_id(console_rx - 1);
> +}
> +
>   static void switch_serial_input(void)
>   {
>       if ( console_rx++ == max_init_domid + 1 )
> diff --git a/xen/include/asm-arm/vpl011.h b/xen/include/asm-arm/vpl011.h
> index 5eb6d25..ab6fd79 100644
> --- a/xen/include/asm-arm/vpl011.h
> +++ b/xen/include/asm-arm/vpl011.h
> @@ -30,9 +30,13 @@
>   #define VPL011_UNLOCK(d,flags) spin_unlock_irqrestore(&(d)->arch.vpl011.lock, flags)
>   
>   #define SBSA_UART_FIFO_SIZE 32
> +/* Same size as VUART_BUT_SIZE, used in vuart.c */

s/BUT/BUF/

> +#define SBSA_UART_OUT_BUF_SIZE 128

You could directly use VUART_BUF_SIZE here to avoid the comment above.

>   struct vpl011_xen_backend {
>       char in[SBSA_UART_FIFO_SIZE];
> +    char out[SBSA_UART_OUT_BUF_SIZE];
>       XENCONS_RING_IDX in_cons, in_prod;
> +    XENCONS_RING_IDX out_prod;
>   };
>   
>   struct vpl011 {
> diff --git a/xen/include/xen/console.h b/xen/include/xen/console.h
> index 70c9911..c5a85c8 100644
> --- a/xen/include/xen/console.h
> +++ b/xen/include/xen/console.h
> @@ -31,6 +31,8 @@ void console_end_sync(void);
>   void console_start_log_everything(void);
>   void console_end_log_everything(void);
>   
> +struct domain *console_input_domain(void);
> +
>   /*
>    * Steal output from the console. Returns +ve identifier, else -ve error.
>    * Takes the handle of the serial line to steal, and steal callback function.
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 22/23] xen/arm: move kernel.h to asm-arm/

[Julien Grall]

Hi Stefano,

On 05/10/2018 19:47, Stefano Stabellini wrote:
> It will be #included by a file in a xen/arch/arm subdirectory.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> ---
>   xen/arch/arm/domain_build.c  |  2 +-
>   xen/arch/arm/kernel.c        |  3 +-
>   xen/arch/arm/kernel.h        | 86 --------------------------------------------
>   xen/include/asm-arm/kernel.h | 86 ++++++++++++++++++++++++++++++++++++++++++++

There are way to make git diff nicer for code movement. This seems to be 
done by default on 2.11.0. Not sure for older version. What are you using?

>   4 files changed, 88 insertions(+), 89 deletions(-)
>   delete mode 100644 xen/arch/arm/kernel.h
>   create mode 100644 xen/include/asm-arm/kernel.h
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 4379538..dc9b46e 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -16,6 +16,7 @@
>   #include <xen/warning.h>
>   #include <acpi/actables.h>
>   #include <asm/device.h>
> +#include <asm/kernel.h>
>   #include <asm/setup.h>
>   #include <asm/platform.h>
>   #include <asm/psci.h>
> @@ -24,7 +25,6 @@
>   
>   #include <xen/irq.h>
>   #include <xen/grant_table.h>
> -#include "kernel.h"
>   
>   static unsigned int __initdata opt_dom0_max_vcpus;
>   integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
> diff --git a/xen/arch/arm/kernel.c b/xen/arch/arm/kernel.c
> index 2239a07..b56aa79 100644
> --- a/xen/arch/arm/kernel.c
> +++ b/xen/arch/arm/kernel.c
> @@ -16,8 +16,7 @@
>   #include <xen/vmap.h>
>   
>   #include <asm/guest_access.h>
> -
> -#include "kernel.h"
> +#include <asm/kernel.h>
>   
>   #define UIMAGE_MAGIC          0x27051956
>   #define UIMAGE_NMLEN          32
> diff --git a/xen/arch/arm/kernel.h b/xen/arch/arm/kernel.h
> deleted file mode 100644
> index 33f3e72..0000000
> --- a/xen/arch/arm/kernel.h
> +++ /dev/null
> @@ -1,86 +0,0 @@
> -/*
> - * Kernel image loading.
> - *
> - * Copyright (C) 2011 Citrix Systems, Inc.
> - */
> -#ifndef __ARCH_ARM_KERNEL_H__
> -#define __ARCH_ARM_KERNEL_H__
> -
> -#include <xen/device_tree.h>
> -#include <asm/setup.h>
> -
> -struct kernel_info {
> -#ifdef CONFIG_ARM_64
> -    enum domain_type type;
> -#endif
> -
> -    struct domain *d;
> -
> -    void *fdt; /* flat device tree */
> -    paddr_t unassigned_mem; /* RAM not (yet) assigned to a bank */
> -    struct meminfo mem;
> -
> -    /* kernel entry point */
> -    paddr_t entry;
> -
> -    /* grant table region */
> -    paddr_t gnttab_start;
> -    paddr_t gnttab_size;
> -
> -    /* boot blob load addresses */
> -    const struct bootmodule *kernel_bootmodule, *initrd_bootmodule;
> -    const char* cmdline;
> -    paddr_t dtb_paddr;
> -    paddr_t initrd_paddr;
> -
> -    /* Enable pl011 emulation */
> -    bool vpl011;
> -
> -    /* loader to use for this kernel */
> -    void (*load)(struct kernel_info *info);
> -    /* loader specific state */
> -    union {
> -        struct {
> -            paddr_t kernel_addr;
> -            paddr_t len;
> -#ifdef CONFIG_ARM_64
> -            paddr_t text_offset; /* 64-bit Image only */
> -#endif
> -            paddr_t start; /* 32-bit zImage only */
> -        } zimage;
> -    };
> -};
> -
> -/*
> - * Probe the kernel to detemine its type and select a loader.
> - *
> - * Sets in info:
> - *  ->type
> - *  ->load hook, and sets loader specific variables ->zimage
> - */
> -int kernel_probe(struct kernel_info *info, const struct dt_device_node *domain);
> -
> -/*
> - * Loads the kernel into guest RAM.
> - *
> - * Expects to be set in info when called:
> - *  ->mem
> - *  ->fdt
> - *
> - * Sets in info:
> - *  ->entry
> - *  ->dtb_paddr
> - *  ->initrd_paddr
> - */
> -void kernel_load(struct kernel_info *info);
> -
> -#endif /* #ifdef __ARCH_ARM_KERNEL_H__ */
> -
> -/*
> - * Local variables:
> - * mode: C
> - * c-file-style: "BSD"
> - * c-basic-offset: 4
> - * indent-tabs-mode: nil
> - * End:
> - */
> diff --git a/xen/include/asm-arm/kernel.h b/xen/include/asm-arm/kernel.h
> new file mode 100644
> index 0000000..33f3e72
> --- /dev/null
> +++ b/xen/include/asm-arm/kernel.h
> @@ -0,0 +1,86 @@
> +/*
> + * Kernel image loading.
> + *
> + * Copyright (C) 2011 Citrix Systems, Inc.
> + */
> +#ifndef __ARCH_ARM_KERNEL_H__
> +#define __ARCH_ARM_KERNEL_H__
> +
> +#include <xen/device_tree.h>
> +#include <asm/setup.h>
> +
> +struct kernel_info {
> +#ifdef CONFIG_ARM_64
> +    enum domain_type type;
> +#endif
> +
> +    struct domain *d;
> +
> +    void *fdt; /* flat device tree */
> +    paddr_t unassigned_mem; /* RAM not (yet) assigned to a bank */
> +    struct meminfo mem;
> +
> +    /* kernel entry point */
> +    paddr_t entry;
> +
> +    /* grant table region */
> +    paddr_t gnttab_start;
> +    paddr_t gnttab_size;
> +
> +    /* boot blob load addresses */
> +    const struct bootmodule *kernel_bootmodule, *initrd_bootmodule;
> +    const char* cmdline;
> +    paddr_t dtb_paddr;
> +    paddr_t initrd_paddr;
> +
> +    /* Enable pl011 emulation */
> +    bool vpl011;
> +
> +    /* loader to use for this kernel */
> +    void (*load)(struct kernel_info *info);
> +    /* loader specific state */
> +    union {
> +        struct {
> +            paddr_t kernel_addr;
> +            paddr_t len;
> +#ifdef CONFIG_ARM_64
> +            paddr_t text_offset; /* 64-bit Image only */
> +#endif
> +            paddr_t start; /* 32-bit zImage only */
> +        } zimage;
> +    };
> +};
> +
> +/*
> + * Probe the kernel to detemine its type and select a loader.
> + *
> + * Sets in info:
> + *  ->type
> + *  ->load hook, and sets loader specific variables ->zimage
> + */
> +int kernel_probe(struct kernel_info *info, const struct dt_device_node *domain);
> +
> +/*
> + * Loads the kernel into guest RAM.
> + *
> + * Expects to be set in info when called:
> + *  ->mem
> + *  ->fdt
> + *
> + * Sets in info:
> + *  ->entry
> + *  ->dtb_paddr
> + *  ->initrd_paddr
> + */
> +void kernel_load(struct kernel_info *info);
> +
> +#endif /* #ifdef __ARCH_ARM_KERNEL_H__ */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 03/23] xen/arm: document dom0less

[Lars Kurth]

[-- Attachment #1.1: Type: text/plain, Size: 915 bytes --]

Hi Stefano,

> On 9 Oct 2018, at 12:52, Julien Grall <julien.grall@arm.com> wrote:
> 
> Hi Stefano,
> 
> On 05/10/2018 19:47, Stefano Stabellini wrote:
>> Add a new document to provide information on how to use dom0less related
>> features and their current limitations.
>> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
>> ---
>> Changes in v4:
>> - rename to .txt
>> - improve wording
>> Changes in v3:
>> - add patch
>> ---
>>  docs/misc/arm/dom0less.txt | 47 ++++++++++++++++++++++++++++++++++++++++++++++
> 
> As said on the previous version, you likely need to add an entry in docs/INDEX.

Agreed. Also, it may be worthwhile encoding the doc in markdown (it's not that much text) and using the .markdown extension
Also, it seems to me that docs/features is maybe a better place for this document. Not sure whether docs in this folder get picked up automatically though

Lars

[-- Attachment #1.2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 22/23] xen/arm: move kernel.h to asm-arm/

[Stefano Stabellini]

On Mon, 15 Oct 2018, Julien Grall wrote:
> Hi Stefano,
> 
> On 05/10/2018 19:47, Stefano Stabellini wrote:
> > It will be #included by a file in a xen/arch/arm subdirectory.
> > 
> > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > ---
> >   xen/arch/arm/domain_build.c  |  2 +-
> >   xen/arch/arm/kernel.c        |  3 +-
> >   xen/arch/arm/kernel.h        | 86
> > --------------------------------------------
> >   xen/include/asm-arm/kernel.h | 86
> > ++++++++++++++++++++++++++++++++++++++++++++
> 
> There are way to make git diff nicer for code movement. This seems to be done
> by default on 2.11.0. Not sure for older version. What are you using?

Git version 1.9.1 (and guilt 0.35)


> >   4 files changed, 88 insertions(+), 89 deletions(-)
> >   delete mode 100644 xen/arch/arm/kernel.h
> >   create mode 100644 xen/include/asm-arm/kernel.h
> > 
> > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > index 4379538..dc9b46e 100644
> > --- a/xen/arch/arm/domain_build.c
> > +++ b/xen/arch/arm/domain_build.c
> > @@ -16,6 +16,7 @@
> >   #include <xen/warning.h>
> >   #include <acpi/actables.h>
> >   #include <asm/device.h>
> > +#include <asm/kernel.h>
> >   #include <asm/setup.h>
> >   #include <asm/platform.h>
> >   #include <asm/psci.h>
> > @@ -24,7 +25,6 @@
> >     #include <xen/irq.h>
> >   #include <xen/grant_table.h>
> > -#include "kernel.h"
> >     static unsigned int __initdata opt_dom0_max_vcpus;
> >   integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
> > diff --git a/xen/arch/arm/kernel.c b/xen/arch/arm/kernel.c
> > index 2239a07..b56aa79 100644
> > --- a/xen/arch/arm/kernel.c
> > +++ b/xen/arch/arm/kernel.c
> > @@ -16,8 +16,7 @@
> >   #include <xen/vmap.h>
> >     #include <asm/guest_access.h>
> > -
> > -#include "kernel.h"
> > +#include <asm/kernel.h>
> >     #define UIMAGE_MAGIC          0x27051956
> >   #define UIMAGE_NMLEN          32
> > diff --git a/xen/arch/arm/kernel.h b/xen/arch/arm/kernel.h
> > deleted file mode 100644
> > index 33f3e72..0000000
> > --- a/xen/arch/arm/kernel.h
> > +++ /dev/null
> > @@ -1,86 +0,0 @@
> > -/*
> > - * Kernel image loading.
> > - *
> > - * Copyright (C) 2011 Citrix Systems, Inc.
> > - */
> > -#ifndef __ARCH_ARM_KERNEL_H__
> > -#define __ARCH_ARM_KERNEL_H__
> > -
> > -#include <xen/device_tree.h>
> > -#include <asm/setup.h>
> > -
> > -struct kernel_info {
> > -#ifdef CONFIG_ARM_64
> > -    enum domain_type type;
> > -#endif
> > -
> > -    struct domain *d;
> > -
> > -    void *fdt; /* flat device tree */
> > -    paddr_t unassigned_mem; /* RAM not (yet) assigned to a bank */
> > -    struct meminfo mem;
> > -
> > -    /* kernel entry point */
> > -    paddr_t entry;
> > -
> > -    /* grant table region */
> > -    paddr_t gnttab_start;
> > -    paddr_t gnttab_size;
> > -
> > -    /* boot blob load addresses */
> > -    const struct bootmodule *kernel_bootmodule, *initrd_bootmodule;
> > -    const char* cmdline;
> > -    paddr_t dtb_paddr;
> > -    paddr_t initrd_paddr;
> > -
> > -    /* Enable pl011 emulation */
> > -    bool vpl011;
> > -
> > -    /* loader to use for this kernel */
> > -    void (*load)(struct kernel_info *info);
> > -    /* loader specific state */
> > -    union {
> > -        struct {
> > -            paddr_t kernel_addr;
> > -            paddr_t len;
> > -#ifdef CONFIG_ARM_64
> > -            paddr_t text_offset; /* 64-bit Image only */
> > -#endif
> > -            paddr_t start; /* 32-bit zImage only */
> > -        } zimage;
> > -    };
> > -};
> > -
> > -/*
> > - * Probe the kernel to detemine its type and select a loader.
> > - *
> > - * Sets in info:
> > - *  ->type
> > - *  ->load hook, and sets loader specific variables ->zimage
> > - */
> > -int kernel_probe(struct kernel_info *info, const struct dt_device_node
> > *domain);
> > -
> > -/*
> > - * Loads the kernel into guest RAM.
> > - *
> > - * Expects to be set in info when called:
> > - *  ->mem
> > - *  ->fdt
> > - *
> > - * Sets in info:
> > - *  ->entry
> > - *  ->dtb_paddr
> > - *  ->initrd_paddr
> > - */
> > -void kernel_load(struct kernel_info *info);
> > -
> > -#endif /* #ifdef __ARCH_ARM_KERNEL_H__ */
> > -
> > -/*
> > - * Local variables:
> > - * mode: C
> > - * c-file-style: "BSD"
> > - * c-basic-offset: 4
> > - * indent-tabs-mode: nil
> > - * End:
> > - */
> > diff --git a/xen/include/asm-arm/kernel.h b/xen/include/asm-arm/kernel.h
> > new file mode 100644
> > index 0000000..33f3e72
> > --- /dev/null
> > +++ b/xen/include/asm-arm/kernel.h
> > @@ -0,0 +1,86 @@
> > +/*
> > + * Kernel image loading.
> > + *
> > + * Copyright (C) 2011 Citrix Systems, Inc.
> > + */
> > +#ifndef __ARCH_ARM_KERNEL_H__
> > +#define __ARCH_ARM_KERNEL_H__
> > +
> > +#include <xen/device_tree.h>
> > +#include <asm/setup.h>
> > +
> > +struct kernel_info {
> > +#ifdef CONFIG_ARM_64
> > +    enum domain_type type;
> > +#endif
> > +
> > +    struct domain *d;
> > +
> > +    void *fdt; /* flat device tree */
> > +    paddr_t unassigned_mem; /* RAM not (yet) assigned to a bank */
> > +    struct meminfo mem;
> > +
> > +    /* kernel entry point */
> > +    paddr_t entry;
> > +
> > +    /* grant table region */
> > +    paddr_t gnttab_start;
> > +    paddr_t gnttab_size;
> > +
> > +    /* boot blob load addresses */
> > +    const struct bootmodule *kernel_bootmodule, *initrd_bootmodule;
> > +    const char* cmdline;
> > +    paddr_t dtb_paddr;
> > +    paddr_t initrd_paddr;
> > +
> > +    /* Enable pl011 emulation */
> > +    bool vpl011;
> > +
> > +    /* loader to use for this kernel */
> > +    void (*load)(struct kernel_info *info);
> > +    /* loader specific state */
> > +    union {
> > +        struct {
> > +            paddr_t kernel_addr;
> > +            paddr_t len;
> > +#ifdef CONFIG_ARM_64
> > +            paddr_t text_offset; /* 64-bit Image only */
> > +#endif
> > +            paddr_t start; /* 32-bit zImage only */
> > +        } zimage;
> > +    };
> > +};
> > +
> > +/*
> > + * Probe the kernel to detemine its type and select a loader.
> > + *
> > + * Sets in info:
> > + *  ->type
> > + *  ->load hook, and sets loader specific variables ->zimage
> > + */
> > +int kernel_probe(struct kernel_info *info, const struct dt_device_node
> > *domain);
> > +
> > +/*
> > + * Loads the kernel into guest RAM.
> > + *
> > + * Expects to be set in info when called:
> > + *  ->mem
> > + *  ->fdt
> > + *
> > + * Sets in info:
> > + *  ->entry
> > + *  ->dtb_paddr
> > + *  ->initrd_paddr
> > + */
> > +void kernel_load(struct kernel_info *info);
> > +
> > +#endif /* #ifdef __ARCH_ARM_KERNEL_H__ */
> > +
> > +/*
> > + * Local variables:
> > + * mode: C
> > + * c-file-style: "BSD"
> > + * c-basic-offset: 4
> > + * indent-tabs-mode: nil
> > + * End:
> > + */
> > 
> 
> -- 
> Julien Grall
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 16/23] xen/arm: generate vpl011 node on device tree for domU

[Stefano Stabellini]

On Mon, 15 Oct 2018, Julien Grall wrote:
> Hi,
> 
> On 05/10/2018 19:47, Stefano Stabellini wrote:
> > Introduce vpl011 support to guests started from Xen: it provides a
> > simple way to print output from a guest, as most guests come with a
> > pl011 driver. It is also able to provide a working console with
> > interrupt support.
> > 
> > The UART exposed to the guest is a SBSA compatible UART and not a PL011.
> > SBSA UART is a subset of PL011 r1p5. A full PL011 implementation in Xen
> > would just be too difficult, so guests may require some drivers changes.
> > 
> > Enable vpl011 conditionally if the user requested it.
> > 
> > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > ---
> > Changes in v4:
> > - move rename set_interrupt_ppi and making set_interrupt_ppi generic to
> >    a separate patch
> > - properly name the vpl011 device node name
> > - code style
> > - use #define for addrcells and sizecells
> > 
> > Changes in v3:
> > - use bool
> > - retain BUG_ON(irq < 16)
> > - add vpl011 bool to kinfo
> > - return error of vpl011 is required but CONFIG_SBSA_VUART_CONSOLE is
> >    missing
> > 
> > Changes in v2:
> > - code style fixes
> > - make set_interrupt_ppi generic
> > - rename set_interrupt_ppi to set_interrupt
> > - only make the vpl011 node if the option was enabled
> > ---
> >   xen/arch/arm/domain_build.c | 61
> > +++++++++++++++++++++++++++++++++++++++++++++
> >   xen/arch/arm/kernel.h       |  3 +++
> >   2 files changed, 64 insertions(+)
> > 
> > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > index 760ebf8..049ab84 100644
> > --- a/xen/arch/arm/domain_build.c
> > +++ b/xen/arch/arm/domain_build.c
> > @@ -1605,6 +1605,54 @@ static int __init make_timer_domU_node(const struct
> > domain *d, void *fdt)
> >       return res;
> >   }
> >   +#ifdef CONFIG_SBSA_VUART_CONSOLE
> > +static int __init make_vpl011_uart_node(const struct domain *d, void *fdt)
> > +{
> > +    int res;
> > +    gic_interrupt_t intr;
> > +    __be32 reg[GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS];
> > +    __be32 *cells;
> > +
> > +    res = fdt_begin_node(fdt, "sbsa-uart@"__stringify(GUEST_PL011_BASE));
> > +    if ( res )
> > +        return res;
> > +
> > +    res = fdt_property_string(fdt, "compatible", "arm,sbsa-uart");
> > +    if ( res )
> > +        return res;
> > +
> > +    cells = &reg[0];
> > +    dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS,
> > +                       GUEST_ROOT_SIZE_CELLS, GUEST_PL011_BASE,
> > +                       GUEST_PL011_SIZE);
> > +    if ( res )
> > +        return res;
> > +    res = fdt_property(fdt, "reg", reg, sizeof(reg));
> > +    if ( res )
> > +        return res;
> > +
> > +    set_interrupt(intr, GUEST_VPL011_SPI, 0xf, DT_IRQ_TYPE_LEVEL_HIGH);
> > +
> > +    res = fdt_property(fdt, "interrupts", intr, sizeof (intr));
> > +    if ( res )
> > +        return res;
> > +
> > +    res = fdt_property_cell(fdt, "interrupt-parent",
> > +                            GUEST_PHANDLE_GIC);
> > +    if ( res )
> > +        return res;
> > +
> > +    /* Use a default baud rate of 115200. */
> > +    fdt_property_u32(fdt, "current-speed", 115200);
> > +
> > +    res = fdt_end_node(fdt);
> > +    if ( res )
> > +        return res;
> > +
> > +    return 0;
> > +}
> > +#endif
> > +
> >   /*
> >    * The max size for DT is 2MB. However, the generated DT is small, 4KB
> >    * are enough for now, but we might have to increase it in the future.
> > @@ -1666,6 +1714,16 @@ static int __init prepare_dtb_domU(struct domain *d,
> > struct kernel_info *kinfo)
> >       if ( ret )
> >           goto err;
> >   +    if ( kinfo->vpl011 )
> > +    {
> > +        ret = -EINVAL;
> > +#ifdef CONFIG_SBSA_VUART_CONSOLE
> > +        ret = make_vpl011_uart_node(d, kinfo->fdt);
> > +#endif
> > +        if ( ret )
> > +            goto err;
> > +    }
> > +
> >       ret = fdt_end_node(kinfo->fdt);
> >       if ( ret < 0 )
> >           goto err;
> > @@ -2523,6 +2581,7 @@ static int __init construct_domU(struct domain *d,
> >       struct kernel_info kinfo = {};
> >       int rc;
> >       u64 mem;
> > +    u32 len;
> >         rc = dt_property_read_u64(node, "memory", &mem);
> >       if ( !rc )
> > @@ -2534,6 +2593,8 @@ static int __init construct_domU(struct domain *d,
> >         printk("*** LOADING DOMU cpus=%u memory=%luKB ***\n", d->max_vcpus,
> > mem);
> >   +    kinfo.vpl011 = dt_get_property(node, "vpl011", &len) != NULL;
> 
> You can use dt_property_read_bool here.

I'll do


> > +
> >       d->vcpu = xzalloc_array(struct vcpu *, d->max_vcpus);
> >       if ( !d->vcpu )
> >           return -ENOMEM;;
> > diff --git a/xen/arch/arm/kernel.h b/xen/arch/arm/kernel.h
> > index 4320f72..33f3e72 100644
> > --- a/xen/arch/arm/kernel.h
> > +++ b/xen/arch/arm/kernel.h
> > @@ -33,6 +33,9 @@ struct kernel_info {
> >       paddr_t dtb_paddr;
> >       paddr_t initrd_paddr;
> >   +    /* Enable pl011 emulation */
> > +    bool vpl011;
> > +
> >       /* loader to use for this kernel */
> >       void (*load)(struct kernel_info *info);
> >       /* loader specific state */
> > 
> 
> Cheers,
> 
> -- 
> Julien Grall
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 11/23] xen/arm: refactor construct_dom0

[Stefano Stabellini]

On Mon, 15 Oct 2018, Julien Grall wrote:
> Hi,
> 
> On 05/10/2018 19:47, Stefano Stabellini wrote:
> > Move generic initializations out of construct_dom0 so that they can be
> > reused.
> > 
> > Rename prepare_dtb to prepare_dtb_hwdom to avoid confusion.
> > 
> > No functional changes in this patch.
> > 
> > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > 
> > ---
> > Changes in v4:
> > - newline and style changes
> > 
> > Changes in v3:
> > - move setting type before allocate_memory
> > - add ifdef around it and a comment
> > 
> > Changes in v2:
> > - move discard_initial_modules() after __construct_domain()
> > - remove useless blank line
> > - leave safety BUG_ONs in __construct_domain
> > - rename prepare_dtb to prepare_dtb_hwdom
> > ---
> >   xen/arch/arm/domain_build.c | 126
> > ++++++++++++++++++++++++--------------------
> >   1 file changed, 68 insertions(+), 58 deletions(-)
> > 
> > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > index fddfd82..ba3dad1 100644
> > --- a/xen/arch/arm/domain_build.c
> > +++ b/xen/arch/arm/domain_build.c
> > @@ -1456,7 +1456,7 @@ static int __init handle_node(struct domain *d, struct
> > kernel_info *kinfo,
> >       return res;
> >   }
> >   -static int __init prepare_dtb(struct domain *d, struct kernel_info
> > *kinfo)
> > +static int __init prepare_dtb_hwdom(struct domain *d, struct kernel_info
> > *kinfo)
> >   {
> >       const p2m_type_t default_p2mt = p2m_mmio_direct_c;
> >       const void *fdt;
> > @@ -2191,75 +2191,29 @@ static void __init find_gnttab_region(struct domain
> > *d,
> >              kinfo->gnttab_start, kinfo->gnttab_start + kinfo->gnttab_size);
> >   }
> >   -int __init construct_dom0(struct domain *d)
> > +static int __init __construct_domain(struct domain *d, struct kernel_info
> > *kinfo)
> 
> Why do you need to add __ in front? The more we are trying to avoid
> introducing name/variable with __ in front.

No, I can rename it to construct_domain


> The rest of the patch looks good to me.

great!

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 14/23] xen/arm: generate a simple device tree for domUs

[Stefano Stabellini]

On Mon, 15 Oct 2018, Julien Grall wrote:
> Hi Stefano,
> 
> On 05/10/2018 19:47, Stefano Stabellini wrote:
> > +static int __init make_gic_domU_node(const struct domain *d, void *fdt)
> > +{
> > +    switch ( gic_hw_version() )
> 
> While I understand that today domains will use the same GIC version as the
> host, it would be best if we don't rely on this in the generation of the DT.
> 
> So I would use d->arch.vgic.version here.
> 
> With that change:
> 
> Acked-by: Julien Grall <julien.grall@arm.com>

Done, thanks!

> 
> > +    {
> > +    case GIC_V3:
> > +        return make_gicv3_domU_node(d, fdt);
> > +    case GIC_V2:
> > +        return make_gicv2_domU_node(d, fdt);
> > +    default:
> > +        panic("Unsupported GIC version");
> > +    }
> > +}
> 
> Cheers,
> 
> -- 
> Julien Grall
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 21/23] xen/vpl011: buffer out chars when the backend is xen

[Stefano Stabellini]

On Mon, 15 Oct 2018, Julien Grall wrote:
> Hi,
> 
> On 05/10/2018 19:47, Stefano Stabellini wrote:
> > To avoid mixing the output of different domains on the console, buffer
> > the output chars and print line by line. Unless the domain has input
> > from the serial, in which case we want to print char by char for a
> > smooth user experience.
> > 
> > The size of SBSA_UART_OUT_BUF_SIZE is arbitrary, choose the same size
> > as VUART_BUT_SIZE used in vuart.c.
> > 
> > Export a function named console_input_domain() to allow others to know
> > which domains has input at a given time.
> > 
> > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > CC: andrew.cooper3@citrix.com
> > CC: George.Dunlap@eu.citrix.com
> > CC: ian.jackson@eu.citrix.com
> > CC: jbeulich@suse.com
> > CC: konrad.wilk@oracle.com
> > CC: tim@xen.org
> > CC: wei.liu2@citrix.com
> > ---
> > XXX: merge this patch with "xen/arm: Allow vpl011 to be used by DomU" on
> >       commit
> 
> That's not going to make the series bisectable as it depends on an
> intermediate patch for console_input_domain().

A tiny patch reordering will fix this. I'll do.


> The new logic looks better to me, few comments below.

Good!


> > 
> > Changes in v4:
> > - make SBSA_UART_OUT_BUF_SIZE the same size of VUART_BUT_SIZE
> > - rearrange the code to be clearer input and != input cases
> > - code style
> > - add \n when printing the out buffer because is full
> > - don't print prefix for input domain
> > ---
> >   xen/arch/arm/vpl011.c        | 35 ++++++++++++++++++++++++++++++++---
> >   xen/drivers/char/console.c   |  7 +++++++
> >   xen/include/asm-arm/vpl011.h |  4 ++++
> >   xen/include/xen/console.h    |  2 ++
> >   4 files changed, 45 insertions(+), 3 deletions(-)
> > 
> > diff --git a/xen/arch/arm/vpl011.c b/xen/arch/arm/vpl011.c
> > index 131507e..5e57ada 100644
> > --- a/xen/arch/arm/vpl011.c
> > +++ b/xen/arch/arm/vpl011.c
> > @@ -28,6 +28,7 @@
> >   #include <xen/lib.h>
> >   #include <xen/mm.h>
> >   #include <xen/sched.h>
> > +#include <xen/console.h>
> >   #include <public/domctl.h>
> >   #include <public/io/console.h>
> >   #include <asm/pl011-uart.h>
> > @@ -85,12 +86,40 @@ static void vpl011_write_data_xen(struct domain *d,
> > uint8_t data)
> >   {
> >       unsigned long flags;
> >       struct vpl011 *vpl011 = &d->arch.vpl011;
> > +    struct vpl011_xen_backend *intf = vpl011->backend.xen;
> > +    struct domain *input = console_input_domain();
> >         VPL011_LOCK(d, flags);
> >   -    printk("%c", data);
> > -    if (data == '\n')
> > -        printk("DOM%u: ", d->domain_id);
> > +    intf->out[intf->out_prod++] = data;
> > +    if ( d == input )
> > +    {
> > +        if ( intf->out_prod == 1 )
> > +        {
> > +            printk("%c", data);
> > +            intf->out_prod = 0;
> > +        }
> > +        else
> > +        {
> > +            if ( data != '\n' )
> > +                intf->out[intf->out_prod++] = '\n';
> > +            intf->out[intf->out_prod++] = '\0';
> > +            printk("%s", intf->out);
> > +            intf->out_prod = 0;
> > +        }
> > +    }
> > +    else
> > +    {
> > +        if ( intf->out_prod == SBSA_UART_OUT_BUF_SIZE - 2 ||
> > +             data == '\n' )
> > +        {
> > +            if ( data != '\n' )
> > +                intf->out[intf->out_prod++] = '\n';
> > +            intf->out[intf->out_prod++] = '\0';
> > +            printk("DOM%u: %s", d->domain_id, intf->out);
> > +            intf->out_prod = 0;
> > +        }
> > +    }
> >         vpl011->uartris |= TXI;
> >       vpl011->uartfr &= ~TXFE;
> > diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
> > index 0808bac..9a2b29a 100644
> > --- a/xen/drivers/char/console.c
> > +++ b/xen/drivers/char/console.c
> > @@ -406,6 +406,13 @@ static void dump_console_ring_key(unsigned char key)
> >    */
> >   static unsigned int __read_mostly console_rx = 0;
> >   +struct domain *console_input_domain(void)
> > +{
> > +    if ( console_rx == 0 )
> > +            return NULL;
> > +    return get_domain_by_id(console_rx - 1);
> > +}
> > +
> >   static void switch_serial_input(void)
> >   {
> >       if ( console_rx++ == max_init_domid + 1 )
> > diff --git a/xen/include/asm-arm/vpl011.h b/xen/include/asm-arm/vpl011.h
> > index 5eb6d25..ab6fd79 100644
> > --- a/xen/include/asm-arm/vpl011.h
> > +++ b/xen/include/asm-arm/vpl011.h
> > @@ -30,9 +30,13 @@
> >   #define VPL011_UNLOCK(d,flags)
> > spin_unlock_irqrestore(&(d)->arch.vpl011.lock, flags)
> >     #define SBSA_UART_FIFO_SIZE 32
> > +/* Same size as VUART_BUT_SIZE, used in vuart.c */
> 
> s/BUT/BUF/

I'll fix


> > +#define SBSA_UART_OUT_BUF_SIZE 128
> 
> You could directly use VUART_BUF_SIZE here to avoid the comment above.

That is true, but I think it is nicer to keep the two #define separate


> >   struct vpl011_xen_backend {
> >       char in[SBSA_UART_FIFO_SIZE];
> > +    char out[SBSA_UART_OUT_BUF_SIZE];
> >       XENCONS_RING_IDX in_cons, in_prod;
> > +    XENCONS_RING_IDX out_prod;
> >   };
> >     struct vpl011 {
> > diff --git a/xen/include/xen/console.h b/xen/include/xen/console.h
> > index 70c9911..c5a85c8 100644
> > --- a/xen/include/xen/console.h
> > +++ b/xen/include/xen/console.h
> > @@ -31,6 +31,8 @@ void console_end_sync(void);
> >   void console_start_log_everything(void);
> >   void console_end_log_everything(void);
> >   +struct domain *console_input_domain(void);
> > +
> >   /*
> >    * Steal output from the console. Returns +ve identifier, else -ve error.
> >    * Takes the handle of the serial line to steal, and steal callback
> > function.
> > 
> 
> -- 
> Julien Grall
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 22/23] xen/arm: move kernel.h to asm-arm/

[Julien Grall]

On 17/10/2018 15:42, Stefano Stabellini wrote:
> On Mon, 15 Oct 2018, Julien Grall wrote:
>> Hi Stefano,
>>
>> On 05/10/2018 19:47, Stefano Stabellini wrote:
>>> It will be #included by a file in a xen/arch/arm subdirectory.
>>>
>>> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
>>> ---
>>>    xen/arch/arm/domain_build.c  |  2 +-
>>>    xen/arch/arm/kernel.c        |  3 +-
>>>    xen/arch/arm/kernel.h        | 86
>>> --------------------------------------------
>>>    xen/include/asm-arm/kernel.h | 86
>>> ++++++++++++++++++++++++++++++++++++++++++++
>>
>> There are way to make git diff nicer for code movement. This seems to be done
>> by default on 2.11.0. Not sure for older version. What are you using?
> 
> Git version 1.9.1 (and guilt 0.35)

A 4 years old git version. May I ask to update to a new git (or see if 
there are an option doing the same)? This would help reviewing code 
movement.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 22/23] xen/arm: move kernel.h to asm-arm/

[Andrew Cooper]

On 17/10/18 17:11, Julien Grall wrote:
>
>
> On 17/10/2018 15:42, Stefano Stabellini wrote:
>> On Mon, 15 Oct 2018, Julien Grall wrote:
>>> Hi Stefano,
>>>
>>> On 05/10/2018 19:47, Stefano Stabellini wrote:
>>>> It will be #included by a file in a xen/arch/arm subdirectory.
>>>>
>>>> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
>>>> ---
>>>>    xen/arch/arm/domain_build.c  |  2 +-
>>>>    xen/arch/arm/kernel.c        |  3 +-
>>>>    xen/arch/arm/kernel.h        | 86
>>>> --------------------------------------------
>>>>    xen/include/asm-arm/kernel.h | 86
>>>> ++++++++++++++++++++++++++++++++++++++++++++
>>>
>>> There are way to make git diff nicer for code movement. This seems
>>> to be done
>>> by default on 2.11.0. Not sure for older version. What are you using?
>>
>> Git version 1.9.1 (and guilt 0.35)
>
> A 4 years old git version. May I ask to update to a new git (or see if
> there are an option doing the same)? This would help reviewing code
> movement.

git config --global diff.renames=copy

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 22/23] xen/arm: move kernel.h to asm-arm/

[Stefano Stabellini]

On Wed, 17 Oct 2018, Julien Grall wrote:
> On 17/10/2018 15:42, Stefano Stabellini wrote:
> > On Mon, 15 Oct 2018, Julien Grall wrote:
> > > Hi Stefano,
> > > 
> > > On 05/10/2018 19:47, Stefano Stabellini wrote:
> > > > It will be #included by a file in a xen/arch/arm subdirectory.
> > > > 
> > > > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > > > ---
> > > >    xen/arch/arm/domain_build.c  |  2 +-
> > > >    xen/arch/arm/kernel.c        |  3 +-
> > > >    xen/arch/arm/kernel.h        | 86
> > > > --------------------------------------------
> > > >    xen/include/asm-arm/kernel.h | 86
> > > > ++++++++++++++++++++++++++++++++++++++++++++
> > > 
> > > There are way to make git diff nicer for code movement. This seems to be
> > > done
> > > by default on 2.11.0. Not sure for older version. What are you using?
> > 
> > Git version 1.9.1 (and guilt 0.35)
> 
> A 4 years old git version. May I ask to update to a new git (or see if there
> are an option doing the same)? This would help reviewing code movement.

Unfortunately I get the same with 2.7.4:

 xen/arch/arm/domain_build.c  |  2 +-
 xen/arch/arm/kernel.c        |  3 +-
 xen/arch/arm/kernel.h        | 86 --------------------------------------------
 xen/include/asm-arm/kernel.h | 86 ++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 88 insertions(+), 89 deletions(-)
 delete mode 100644 xen/arch/arm/kernel.h
 create mode 100644 xen/include/asm-arm/kernel.h

FYI the diff between xen/arch/arm/kernel.h and
xen/include/asm-arm/kernel.h is 0, I don't know why git doesn't pick it
up:

stefanos@xsjstefanos:/local/repos/xen$ diff xen/arch/arm/kernel.h xen/include/asm-arm/kernel.h
stefanos@xsjstefanos:/local/repos/xen$
stefanos@xsjstefanos:/local/repos/xen$ md5sum xen/arch/arm/kernel.h xen/include/asm-arm/kernel.h
06bdad45e665f20f03ffdd2822d8f08e  xen/arch/arm/kernel.h
06bdad45e665f20f03ffdd2822d8f08e  xen/include/asm-arm/kernel.h

I hope that helps.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 10/23] xen/arm: introduce allocate_memory

[Stefano Stabellini]

On Mon, 15 Oct 2018, Julien Grall wrote:
> Hi,
> 
> On 05/10/2018 19:47, Stefano Stabellini wrote:
> > Introduce an allocate_memory function able to allocate memory for DomUs
> > and map it at the right guest addresses, according to the guest memory
> > map: GUEST_RAM0_BASE and GUEST_RAM1_BASE.
> 
> Please add something along the line: "This is under #if 0 as not used for
> now".

I'll add


> > 
> > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > ---
> > Changes in v4:
> > - move earlier, add #if 0
> > - introduce allocate_bank_memory, remove insert_bank
> > - allocate_bank_memory allocate memory and inserts the bank, while
> >    allocate_memory specifies where to do that
> > 
> > Changes in v3:
> > - new patch
> > ---
> >   xen/arch/arm/domain_build.c | 83
> > +++++++++++++++++++++++++++++++++++++++++++++
> >   1 file changed, 83 insertions(+)
> > 
> > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > index 41f2f27..fddfd82 100644
> > --- a/xen/arch/arm/domain_build.c
> > +++ b/xen/arch/arm/domain_build.c
> > @@ -368,6 +368,89 @@ static void __init allocate_memory_11(struct domain *d,
> >       }
> >   }
> >   +#if 0
> > +static bool __init allocate_bank_memory(struct domain *d,
> > +                                        struct kernel_info *kinfo,
> > +                                        gfn_t sgfn,
> > +                                        unsigned int order)
> > +{
> > +    int res;
> > +    struct page_info *pg;
> > +    struct membank *bank;
> > +    paddr_t gaddr = gfn_to_gaddr(sgfn), size = pfn_to_paddr(1UL << order);
> 
> There is no need to have gaddr in a variable.

I'll remove it.


> > +
> > +    pg = alloc_domheap_pages(d, order, 0);
> 
> So here you impose the memory to be contiguously allocated for a given bank.
> There are quite a few case where you may not have enough memory to allocate
> contiguously.
> 
> So more likely you want to add loop in this code to allocate until order is
> reached.

This case is not handled today for dom0.


> > +    if ( !pg )
> > +        return false;
> > +
> > +    res = guest_physmap_add_page(d, sgfn, page_to_mfn(pg), order);
> > +    if ( res )
> > +    {
> > +        dprintk(XENLOG_ERR, "Failed map pages to DOMU: %d", res);
> > +        goto fail;
> > +    }
> > +
> > +    bank = &kinfo->mem.bank[kinfo->mem.nr_banks];
> > +    bank->start = gaddr;
> > +    bank->size = size;
> > +    kinfo->mem.nr_banks++;
> > +    kinfo->unassigned_mem -= size;
> 
> newline here.

OK

> > +    return true;
> > +
> > +fail:
> > +    free_domheap_pages(pg, order);
> > +    return false;
> > +}
> > +
> > +static void __init allocate_memory(struct domain *d, struct kernel_info
> > *kinfo)
> > +{
> > +    unsigned int order = get_allocation_size(kinfo->unassigned_mem);
> > +    unsigned int order_req;
> > +    int i;
> 
> unsigned.

OK

> 
> > +
> > +    dprintk(XENLOG_INFO, "Allocating mappings totalling %ldMB for
> > dom%d:\n",
> > +            /* Don't want format this as PRIpaddr (16 digit hex) */
> > +            (unsigned long)(kinfo->unassigned_mem >> 20), d->domain_id);
> > +
> > +    kinfo->mem.nr_banks = 0;
> > +
> > +    order = get_allocation_size(kinfo->unassigned_mem);
> > +    if ( order > get_order_from_bytes(GUEST_RAM0_SIZE) )
> > +        order_req = get_order_from_bytes(GUEST_RAM0_SIZE);
> > +    else
> > +        order_req = order;
> > +    if ( !allocate_bank_memory(d, kinfo, gaddr_to_gfn(GUEST_RAM0_BASE),
> > order_req) )
> > +        goto fail;
> > +
> > +    order -= order_req;
> > +    if ( order > 0 )
> > +    {
> > +        if ( !allocate_bank_memory(d, kinfo, gaddr_to_gfn(GUEST_RAM1_BASE),
> > order) )
> > +            goto fail;
> > +    }
> 
> How about combining the two if to avoid an extra indentation?

OK

> > +
> > +    for( i = 0; i < kinfo->mem.nr_banks; i++ )
> > +    {
> > +        dprintk(XENLOG_INFO, "Dom%d BANK[%d] %#"PRIpaddr"-%#"PRIpaddr"
> > (%ldMB)\n",
> > +                d->domain_id,
> > +                i,
> > +                kinfo->mem.bank[i].start,
> > +                kinfo->mem.bank[i].start + kinfo->mem.bank[i].size,
> > +                /* Don't want format this as PRIpaddr (16 digit hex) */
> > +                (unsigned long)(kinfo->mem.bank[i].size >> 20));
> > +    }
> > +
> > +    return;
> > +
> > +fail:
> > +    dprintk(XENLOG_ERR, "Failed to allocate requested domain memory."
> > +            /* Don't want format this as PRIpaddr (16 digit hex) */
> > +            " %ldKB unallocated. Fix the VMs configurations.\n",
> > +            (unsigned long)kinfo->unassigned_mem >> 10);
> > +    BUG();
> > +}
> > +#endif
> > +
> >   static int __init write_properties(struct domain *d, struct kernel_info
> > *kinfo,
> >                                      const struct dt_device_node *node)
> >   {
> > 
> 
> Cheers,
> 
> -- 
> Julien Grall
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 12/23] xen/arm: introduce create_domUs

[Stefano Stabellini]

On Mon, 15 Oct 2018, Julien Grall wrote:
> Hi,
> 
> On 05/10/2018 19:47, Stefano Stabellini wrote:
> > Call a new function, "create_domUs", from setup_xen to start DomU VMs.
> > 
> > Introduce support for the "xen,domU" compatible node on device tree.
> 
> s/xen,domU/xen,domain/

OK

> > Create new DomU VMs based on the information found on device tree under
> > "xen,domU". Calls construct_domU for each domain.
> 
> Ditto
> 
> s/Calls/Call/

OK


> > 
> > Introduce a simple global variable named max_init_domid to keep track of
> > the initial allocated domids. It holds the max domid among the initial
> > domains.
> > 
> > Move the discard_initial_modules after DomUs have been built.
> > 
> > First create domUs, then start dom0 -- no point in trying to start dom0
> > when the cpu is busy.
> > 
> > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > Acked-by: Jan Beulich <jbeulich@suse.com>
> > CC: andrew.cooper3@citrix.com
> > CC: jbeulich@suse.com
> > ---
> > Changes in v4:
> > - constify parameters
> > - nr_spis to 0 or  GUEST_VPL011_SPI - 32 + 1 depending on vpl011
> > - remove pointless initializer
> > - remove change to domain_create for dom0 (useless)
> > - make construct_domU return error
> > 
> > Changes in v3:
> > - move patch earlier and introduce empty construct_domU to fix bisection
> >    builds
> > - fix max_init_domid to actually hold the max domid among initial
> >    domains (instead of max_domid + 1)
> > - move domain_unpause_by_systemcontroller(dom0) after creating domUs
> > 
> > Changes in v2:
> > - coding style
> > - set nr_spis to 32
> > - introduce create_domUs
> > ---
> >   xen/arch/arm/domain_build.c | 51
> > +++++++++++++++++++++++++++++++++++++++++----
> >   xen/arch/arm/setup.c        |  5 +++++
> >   xen/include/asm-arm/setup.h |  3 +++
> >   xen/include/asm-x86/setup.h |  2 ++
> >   4 files changed, 57 insertions(+), 4 deletions(-)
> > 
> > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > index ba3dad1..547b624 100644
> > --- a/xen/arch/arm/domain_build.c
> > +++ b/xen/arch/arm/domain_build.c
> > @@ -7,6 +7,7 @@
> >   #include <asm/irq.h>
> >   #include <asm/regs.h>
> >   #include <xen/errno.h>
> > +#include <xen/err.h>
> >   #include <xen/device_tree.h>
> >   #include <xen/libfdt/libfdt.h>
> >   #include <xen/guest_access.h>
> > @@ -2290,6 +2291,51 @@ static int __init __construct_domain(struct domain
> > *d, struct kernel_info *kinfo
> >       return 0;
> >   }
> >   +static int __init construct_domU(struct domain *d,
> > +                                 const struct dt_device_node *node)
> > +{
> > +    return -ENOSYS;
> > +}
> > +
> > +void __init create_domUs(void)
> > +{
> > +    struct dt_device_node *node;
> > +    const struct dt_device_node *chosen = dt_find_node_by_name(dt_host,
> > +                                                               "chosen");
> 
> Newline

OK


> It would also be better to use dt_find_node_by_path("/chosen") because
> dt_find_node_by_name may return a different node (e.g /foo/chosen).

Good idea


> > +    BUG_ON(chosen == NULL);
> > +    dt_for_each_child_node(chosen, node)
> > +    {
> > +        u32 len;
> > +        struct domain *d;
> > +        struct xen_domctl_createdomain d_cfg = {
> > +            .arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE,
> > +            .arch.nr_spis = 0,
> > +            .max_vcpus = 1,
> > +            .max_evtchn_port = -1,
> > +            .max_grant_frames = 64,
> > +            .max_maptrack_frames = 1024,
> > +        };
> > +
> > +        if ( !dt_device_is_compatible(node, "xen,domain") )
> > +            continue;
> > +
> > +        if ( dt_get_property(node, "vpl011", &len) != NULL )
> 
> dt_property_read_bool will do the right thing for you.

OK


> > +            d_cfg.arch.nr_spis = GUEST_VPL011_SPI - 32 + 1;
> > +        dt_property_read_u32(node, "cpus", &d_cfg.max_vcpus);
> > +
> > +        d = domain_create(++max_init_domid, &d_cfg, false);
> > +        if ( IS_ERR(d) )
> > +            panic("Error creating domain %s", dt_node_name(node));
> > +
> > +        d->is_console = 1;
> 
> is_console is a bool, so this should be true.

OK


> > +
> > +        if ( construct_domU(d, node) != 0 )
> > +            panic("Could not set up domain %s", dt_node_name(node));
> > +
> > +        domain_unpause_by_systemcontroller(d);
> > +    }
> > +}
> > +
> >   int __init construct_dom0(struct domain *d)
> >   {
> >       const struct bootcmdline *kernel =
> > boot_cmdline_find_by_kind(BOOTMOD_KERNEL);
> > @@ -2342,10 +2388,7 @@ int __init construct_dom0(struct domain *d)
> >       if ( rc < 0 )
> >           return rc;
> >   -    rc = __construct_domain(d, &kinfo);
> > -    discard_initial_modules();
> > -
> > -    return rc;
> > +    return __construct_domain(d, &kinfo);
> >   }
> >     /*
> > diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> > index d6d1546..8d8f180 100644
> > --- a/xen/arch/arm/setup.c
> > +++ b/xen/arch/arm/setup.c
> > @@ -64,8 +64,11 @@ static unsigned long opt_xenheap_megabytes __initdata;
> >   integer_param("xenheap_megabytes", opt_xenheap_megabytes);
> >   #endif
> >   +domid_t __read_mostly max_init_domid;
> > +
> >   static __used void init_done(void)
> >   {
> > +    discard_initial_modules();
> >       free_init_memory();
> >       startup_cpu_idle_loop();
> >   }
> > @@ -926,6 +929,8 @@ void __init start_xen(unsigned long boot_phys_offset,
> >       /* Must be done past setting system_state. */
> >       unregister_init_virtual_region();
> >   +    create_domUs();
> 
> I missed that bits in the previous version. I am not very sure this should be
> called after unregister_init_virtual_region(). If you call it after and get an
> hypervisor crash in init code, then the stack trace may miss some information
> (e.g bug frame).
> 
> Looking at x86, unregister_init_virtual_region() is called from init_done. So
> we probably want to move that call there.

OK



> > +
> >       domain_unpause_by_systemcontroller(dom0);
> >         /* Switch on to the dynamically allocated stack for the idle vcpu
> > diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
> > index 177e8db..5620fe7 100644
> > --- a/xen/include/asm-arm/setup.h
> > +++ b/xen/include/asm-arm/setup.h
> > @@ -68,6 +68,8 @@ struct bootinfo {
> >     extern struct bootinfo bootinfo;
> >   +extern domid_t max_init_domid;
> > +
> >   void arch_init_memory(void);
> >     void copy_from_paddr(void *dst, paddr_t paddr, unsigned long len);
> > @@ -84,6 +86,7 @@ void acpi_create_efi_mmap_table(struct domain *d,
> >   int acpi_make_efi_nodes(void *fdt, struct membank tbl_add[]);
> >     int construct_dom0(struct domain *d);
> > +void __init create_domUs(void);
> >     void discard_initial_modules(void);
> >   void dt_unreserved_regions(paddr_t s, paddr_t e,
> > diff --git a/xen/include/asm-x86/setup.h b/xen/include/asm-x86/setup.h
> > index 42fddeb..1c80783 100644
> > --- a/xen/include/asm-x86/setup.h
> > +++ b/xen/include/asm-x86/setup.h
> > @@ -66,4 +66,6 @@ extern bool opt_dom0_shadow;
> >   #endif
> >   extern bool dom0_pvh;
> >   +#define max_init_domid (0)
> > +
> >   #endif
> > 
> 
> Cheers,
> 
> -- 
> Julien Grall
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 13/23] xen/arm: implement construct_domU

[Stefano Stabellini]

On Mon, 15 Oct 2018, Julien Grall wrote:
> Hi,
> 
> On 05/10/2018 19:47, Stefano Stabellini wrote:
> > Similar to construct_dom0, construct_domU creates a barebone DomU guest.
> > 
> > The device tree node passed as argument is compatible "xen,domain", see
> > docs/misc/arm/device-tree/booting.txt.
> > 
> > Add const to kernel_probe dt_device_node parameter.
> 
> This likely belongs to patch #7 where the parameter was added.

OK


> > 
> > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > 
> > ---
> > Changes in v4:
> > - constify kernel_probe
> > - change title
> > - better error messages and printed info
> > - 64bit memory
> > 
> > Changes in v3:
> > - move setting type before allocate_memory
> > - add ifdef around it and a comment
> > 
> > Changes in v2:
> > - rename mem to memory
> > - make cpus and memory mandatory
> > - remove wront comment from commit message
> > - cpus and memory are read as integers
> > - read the vpl011 option
> > ---
> >   xen/arch/arm/domain_build.c | 37 ++++++++++++++++++++++++++++++++++---
> >   xen/arch/arm/kernel.c       |  3 ++-
> >   xen/arch/arm/kernel.h       |  2 +-
> >   3 files changed, 37 insertions(+), 5 deletions(-)
> > 
> > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > index 547b624..efb530a 100644
> > --- a/xen/arch/arm/domain_build.c
> > +++ b/xen/arch/arm/domain_build.c
> > @@ -369,7 +369,6 @@ static void __init allocate_memory_11(struct domain *d,
> >       }
> >   }
> >   -#if 0
> 
> Please add a word about this change in the commit message.

OK


> >   static bool __init allocate_bank_memory(struct domain *d,
> >                                           struct kernel_info *kinfo,
> >                                           gfn_t sgfn,
> > @@ -450,7 +449,6 @@ fail:
> >               (unsigned long)kinfo->unassigned_mem >> 10);
> >       BUG();
> >   }
> > -#endif
> >     static int __init write_properties(struct domain *d, struct kernel_info
> > *kinfo,
> >                                      const struct dt_device_node *node)
> > @@ -2294,7 +2292,40 @@ static int __init __construct_domain(struct domain
> > *d, struct kernel_info *kinfo
> >   static int __init construct_domU(struct domain *d,
> >                                    const struct dt_device_node *node)
> >   {
> > -    return -ENOSYS;
> > +    struct kernel_info kinfo = {};
> > +    int rc;
> > +    u64 mem;
> > +
> > +    rc = dt_property_read_u64(node, "memory", &mem);
> > +    if ( !rc )
> > +    {
> > +        printk("Error building DomU: cannot read \"memory\" property\n");
> > +        return -EINVAL;
> > +    }
> > +    kinfo.unassigned_mem = (paddr_t)mem << 10;
> 
> I noticed I forgot to answer to:
> "KB() only works for numbers, it is defined as: (_AC(_kb, ULL) << 10)"
> 
> unsigned long long is always going to be bigger than paddr_t. Also, we already
> use MB(...) in similar situation. So I am not sure to understand your concern
> here.

I admit that my explanation was so bad that even I had to go back to
figure out what I meant :-)

What I wanted to say is that KB() only works for constants, not
variables. So KB(10) works, but KB(mem) does not.


> > +
> > +    printk("*** LOADING DOMU cpus=%u memory=%luKB ***\n", d->max_vcpus,
> > mem);
> > +
> > +    d->vcpu = xzalloc_array(struct vcpu *, d->max_vcpus);
> > +    if ( !d->vcpu )
> > +        return -ENOMEM;;
> 
> d->vcpu is already allocated by domain_create.

OK


> > +    if ( vcpu_create(d, 0, 0) == NULL )
> > +        return -ENOMEM;
> > +    d->max_pages = ~0U;
> > +
> > +    kinfo.d = d;
> > +
> > +    rc = kernel_probe(&kinfo, node);
> > +    if ( rc < 0 )
> > +        return rc;
> > +
> > +#ifdef CONFIG_ARM_64
> > +    /* type must be set before allocate memory */
> > +    d->arch.type = kinfo.type;
> > +#endif
> > +    allocate_memory(d, &kinfo);
> > +
> > +    return __construct_domain(d, &kinfo);
> >   }
> >     void __init create_domUs(void)
> > diff --git a/xen/arch/arm/kernel.c b/xen/arch/arm/kernel.c
> > index e5b8213..2239a07 100644
> > --- a/xen/arch/arm/kernel.c
> > +++ b/xen/arch/arm/kernel.c
> > @@ -421,7 +421,8 @@ static int __init kernel_zimage32_probe(struct
> > kernel_info *info,
> >       return 0;
> >   }
> >   -int __init kernel_probe(struct kernel_info *info, struct dt_device_node
> > *domain)
> > +int __init kernel_probe(struct kernel_info *info,
> > +                        const struct dt_device_node *domain)
> >   {
> >       struct bootmodule *mod = NULL;
> >       struct bootcmdline *cmd = NULL;
> > diff --git a/xen/arch/arm/kernel.h b/xen/arch/arm/kernel.h
> > index 4a65289..4320f72 100644
> > --- a/xen/arch/arm/kernel.h
> > +++ b/xen/arch/arm/kernel.h
> > @@ -55,7 +55,7 @@ struct kernel_info {
> >    *  ->type
> >    *  ->load hook, and sets loader specific variables ->zimage
> >    */
> > -int kernel_probe(struct kernel_info *info, struct dt_device_node *domain);
> > +int kernel_probe(struct kernel_info *info, const struct dt_device_node
> > *domain);
> >     /*
> >    * Loads the kernel into guest RAM.
> > 
> 
> Cheers,
> 
> -- 
> Julien Grall
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 20/23] xen: support console_switching between Dom0 and DomUs on ARM

[Stefano Stabellini]

On Mon, 8 Oct 2018, Jan Beulich wrote:
> >>> On 05.10.18 at 20:47, <sstabellini@kernel.org> wrote:
> > @@ -391,31 +394,73 @@ static void dump_console_ring_key(unsigned char key)
> >      free_xenheap_pages(buf, order);
> >  }
> >  
> > -/* CTRL-<switch_char> switches input direction between Xen and DOM0. */
> > +/*
> > + * CTRL-<switch_char> switches input direction between Xen, Dom0 and
> > + * DomUs.
> > + */
> 
> Just like the title, this comment makes it sound as if any DomU could
> participate in this switching.

Is the following any better?

  CTRL-<switch_char> changes input direction, rotating among Xen, Dom0, and DomUs.

Otherwise please suggest your favorite wording and I'll use it.


> >  #define switch_code (opt_conswitch[0]-'a'+1)
> > -static int __read_mostly xen_rx = 1; /* FALSE => input passed to domain 0. */
> > +/*
> > + * console_rx=0 => input to xen
> > + * console_rx=1 => input to dom0
> > + * console_rx=N => input dom(N-1)
> > + */
> > +static unsigned int __read_mostly console_rx = 0;
> >  
> >  static void switch_serial_input(void)
> >  {
> > -    static char *input_str[2] = { "DOM0", "Xen" };
> > -    xen_rx = !xen_rx;
> > -    printk("*** Serial input -> %s", input_str[xen_rx]);
> > +    if ( console_rx++ == max_init_domid + 1 )
> > +        console_rx = 0;
> > +
> > +    if ( !console_rx )
> 
> Please be consistent ...

I'll change it to console_rx == 0


> > +        printk("*** Serial input to Xen");
> > +    else
> > +        printk("*** Serial input to DOM%d", console_rx - 1);
> > +
> >      if ( switch_code )
> > -        printk(" (type 'CTRL-%c' three times to switch input to %s)",
> > -               opt_conswitch[0], input_str[!xen_rx]);
> > +        printk(" (type 'CTRL-%c' three times to switch input)",
> > +               opt_conswitch[0]);
> >      printk("\n");
> >  }
> >  
> >  static void __serial_rx(char c, struct cpu_user_regs *regs)
> >  {
> > -    if ( xen_rx )
> > +    if ( console_rx == 0 )
> 
> ... in style. But perhaps ...
> 
> >          return handle_keypress(c, regs);
> >  
> > -    /* Deliver input to guest buffer, unless it is already full. */
> > -    if ( (serial_rx_prod-serial_rx_cons) != SERIAL_RX_SIZE )
> > -        serial_rx_ring[SERIAL_RX_MASK(serial_rx_prod++)] = c;
> > -    /* Always notify the guest: prevents receive path from getting stuck. */
> > -    send_global_virq(VIRQ_CONSOLE);
> > +    if ( console_rx == 1 )
> 
> ... switch() would be better to use here anyway.

I'll replace these with a switch


> > +    {
> > +        /* Deliver input to hardware domain, unless it is already full. */
> 
> Looks like you've mis-edited the original comment.

I take that you mean that the word "buffer" is missing, right? It should
be:

  Deliver input to the hardware domain buffer, unless it is already full.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 21/23] xen/vpl011: buffer out chars when the backend is xen

[Stefano Stabellini]

On Mon, 8 Oct 2018, Jan Beulich wrote:
> >>> On 05.10.18 at 20:47, <sstabellini@kernel.org> wrote:
> > --- a/xen/drivers/char/console.c
> > +++ b/xen/drivers/char/console.c
> > @@ -406,6 +406,13 @@ static void dump_console_ring_key(unsigned char key)
> >   */
> >  static unsigned int __read_mostly console_rx = 0;
> >  
> > +struct domain *console_input_domain(void)
> > +{
> > +    if ( console_rx == 0 )
> > +            return NULL;
> > +    return get_domain_by_id(console_rx - 1);
> 
> This acquires a domain reference, yet I can't see that reference
> getting dropped anywhere in the caller.

Well spotted!

I'll add a comment here as a reminder, and I'll drop the domain
reference in the caller (xen/arch/arm/vpl011.c:vpl011_write_data_xen).

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 06/23] xen/arm: don't add duplicate boot modules, introduce domU flag

[Stefano Stabellini]

On Tue, 9 Oct 2018, Julien Grall wrote:
> Hi Stefano,
> 
> On 05/10/2018 19:47, Stefano Stabellini wrote:
> > Don't add duplicate boot modules (same kind and same start address),
> > they are freed later, we don't want to introduce double-free errors.
> > 
> > Introduce a domU flag in struct bootmodule and struct bootcmdline. Set
> > it for kernels and ramdisks of "xen,domain" nodes to avoid getting
> > confused in kernel_probe, where we try to guess which is the dom0 kernel
> > and initrd to be compatible with older versions of the multiboot spec.
> 
> This comments need at least to be replicated in the structure. So we have an
> idea how to use it.
> 
> Lastly, technically it is not older versions of the specification. Your
> extension does not deal with Dom0.

OK


> > 
> > boot_module_find_by_kind and boot_cmdline_find_by_kind automatically
> > check for !domU entries (they are only used for non-domU modules).
> > 
> > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > 
> > ---
> > Changes in v4:
> > - use unsigned int
> > - better commit message
> > - introduce domU flag and usage
> > 
> > Changes in v2:
> > - new patch
> > ---
> >   xen/arch/arm/bootfdt.c      | 14 +++++++++-----
> >   xen/arch/arm/setup.c        | 21 +++++++++++++++++----
> >   xen/include/asm-arm/setup.h |  4 +++-
> >   3 files changed, 29 insertions(+), 10 deletions(-)
> > 
> > diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
> > index 273032b..349aa9d 100644
> > --- a/xen/arch/arm/bootfdt.c
> > +++ b/xen/arch/arm/bootfdt.c
> > @@ -164,7 +164,8 @@ static void __init process_memory_node(const void *fdt,
> > int node,
> >   }
> >     static void __init add_boot_cmdline(const void *fdt, int node,
> > -                                    const char *name, bootmodule_kind kind)
> > +                                    const char *name, bootmodule_kind kind,
> > +                                    bool domU)
> >   {
> >       struct bootcmdlines *cmds = &bootinfo.cmdlines;
> >       struct bootcmdline *cmd;
> > @@ -184,6 +185,7 @@ static void __init add_boot_cmdline(const void *fdt, int
> > node,
> >         cmd = &cmds->cmdline[cmds->nr_mods++];
> >       cmd->kind = kind;
> > +    cmd->domU = domU;
> >         ASSERT(strlen(name) <= DT_MAX_NAME);
> >       safe_strcpy(cmd->dt_name, name);
> > @@ -206,6 +208,7 @@ static void __init process_multiboot_node(const void
> > *fdt, int node,
> >       int len = sizeof("/chosen");
> >       char path[8]; /* sizeof "/chosen" */
> >       int parent_node;
> > +    bool domU;
> >         parent_node = fdt_parent_offset(fdt, node);
> >       ASSERT(parent_node >= 0);
> > @@ -263,8 +266,9 @@ static void __init process_multiboot_node(const void
> > *fdt, int node,
> >               kind = BOOTMOD_XSM;
> >       }
> >   -    add_boot_module(kind, start, size);
> > -    add_boot_cmdline(fdt, node, fdt_get_name(fdt, parent_node, &len),
> > kind);
> > +    domU = fdt_node_check_compatible(fdt, parent_node, "xen,domain") == 0;
> > +    add_boot_module(kind, start, size, domU);
> > +    add_boot_cmdline(fdt, node, fdt_get_name(fdt, parent_node, &len), kind,
> > domU);
> >   }
> >     static void __init process_chosen_node(const void *fdt, int node,
> > @@ -310,7 +314,7 @@ static void __init process_chosen_node(const void *fdt,
> > int node,
> >         printk("Initrd %"PRIpaddr"-%"PRIpaddr"\n", start, end);
> >   -    add_boot_module(BOOTMOD_RAMDISK, start, end-start);
> > +    add_boot_module(BOOTMOD_RAMDISK, start, end-start, false);
> >   }
> >     static int __init early_scan_node(const void *fdt,
> > @@ -381,7 +385,7 @@ size_t __init boot_fdt_info(const void *fdt, paddr_t
> > paddr)
> >       if ( ret < 0 )
> >           panic("No valid device tree\n");
> >   -    add_boot_module(BOOTMOD_FDT, paddr, fdt_totalsize(fdt));
> > +    add_boot_module(BOOTMOD_FDT, paddr, fdt_totalsize(fdt), false);
> >         device_tree_for_each_node((void *)fdt, early_scan_node, NULL);
> >       early_print_info();
> > diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> > index bc7dd97..dbab232 100644
> > --- a/xen/arch/arm/setup.c
> > +++ b/xen/arch/arm/setup.c
> > @@ -201,10 +201,12 @@ void __init dt_unreserved_regions(paddr_t s, paddr_t
> > e,
> >   }
> >     struct bootmodule __init *add_boot_module(bootmodule_kind kind,
> > -                                          paddr_t start, paddr_t size)
> > +                                          paddr_t start, paddr_t size,
> > +                                          bool domU)
> >   {
> >       struct bootmodules *mods = &bootinfo.modules;
> >       struct bootmodule *mod;
> This at least need to be replica

I take this was a typo?


> > +    unsigned int i;
> >         if ( mods->nr_mods == MAX_MODULES )
> >       {
> > @@ -212,11 +214,22 @@ struct bootmodule __init
> > *add_boot_module(bootmodule_kind kind,
> >                  boot_module_kind_as_string(kind), start, start + size);
> >           return NULL;
> >       }
> > +    for ( i = 0 ; i < mods->nr_mods ; i++ )
> > +    {
> > +        mod = &mods->module[i];
> > +        if ( mod->kind == kind && mod->start == start )
> > +        {
> > +            if ( !domU )
> > +                mod->domU = false;
> > +            return mod;
> > +        }
> > +    }
> >         mod = &mods->module[mods->nr_mods++];
> >       mod->kind = kind;
> >       mod->start = start;
> >       mod->size = size;
> > +    mod->domU = domU;
> >         return mod;
> >   }
> > @@ -229,7 +242,7 @@ struct bootmodule * __init
> > boot_module_find_by_kind(bootmodule_kind kind)
> >       for (i = 0 ; i < mods->nr_mods ; i++ )
> >       {
> >           mod = &mods->module[i];
> > -        if ( mod->kind == kind )
> > +        if ( mod->kind == kind && !mod->domU )
> 
> From the name of the function is it unclear why we would only return module
> with !mod->domU. So this needs some clarifications in the code.

I'll add a comment

> 
> >               return mod;
> >       }
> >       return NULL;
> > @@ -244,7 +257,7 @@ struct bootcmdline * __init
> > boot_cmdline_find_by_kind(bootmodule_kind kind)
> >       for ( i = 0 ; i < cmds->nr_mods ; i++ )
> >       {
> >           cmd = &cmds->cmdline[i];
> > -        if ( cmd->kind == kind )
> > +        if ( cmd->kind == kind && !cmd->domU )
> 
> Ditto here.

I'll add a comment


> >               return cmd;
> >       }
> >       return NULL;
> > @@ -738,7 +751,7 @@ void __init start_xen(unsigned long boot_phys_offset,
> >       /* Register Xen's load address as a boot module. */
> >       xen_bootmodule = add_boot_module(BOOTMOD_XEN,
> >                                (paddr_t)(uintptr_t)(_start +
> > boot_phys_offset),
> > -                             (paddr_t)(uintptr_t)(_end - _start + 1));
> > +                             (paddr_t)(uintptr_t)(_end - _start + 1),
> > false);
> >       BUG_ON(!xen_bootmodule);
> >         xen_paddr = get_xen_paddr();
> > diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
> > index c2ed5cc..711b4a2 100644
> > --- a/xen/include/asm-arm/setup.h
> > +++ b/xen/include/asm-arm/setup.h
> > @@ -33,6 +33,7 @@ struct meminfo {
> >   #define BOOTMOD_MAX_CMDLINE 1024
> >   struct bootmodule {
> >       bootmodule_kind kind;
> > +    bool domU;
> 
> This needs some documentation in the code.

OK

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 07/23] xen/arm: probe domU kernels and initrds

[Stefano Stabellini]

On Mon, 15 Oct 2018, Julien Grall wrote:
> Hi Stefano,
> 
> On 05/10/2018 19:47, Stefano Stabellini wrote:
> > Find addresses, sizes on device tree from kernel_probe.
> > Find the cmdline from the bootcmdlines array.
> > 
> > Introduce a new boot_module_find_by_addr_and_kind function to match not
> > just on boot module kind, but also by address so that we can support
> > multiple domains.
> > 
> > Introduce a boot_cmdline_find_by_name function to find the right struct
> > cmdline based on the device tree node name of the "xen,domain"
> > compatible node.
> > 
> > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > ---
> > Changes in v3:
> > - retrieve cmdline from bootcmdlines array
> > 
> > Changes in v2:
> > - fix indentation
> > - unify kernel_probe with kernel_probe_domU
> > - find cmdline on device_tree from kernel_probe
> > ---
> >   xen/arch/arm/domain_build.c |  2 +-
> >   xen/arch/arm/kernel.c       | 52
> > +++++++++++++++++++++++++++++++++++++++------
> >   xen/arch/arm/kernel.h       |  2 +-
> >   xen/arch/arm/setup.c        | 29 +++++++++++++++++++++++++
> >   xen/include/asm-arm/setup.h |  3 +++
> >   5 files changed, 79 insertions(+), 9 deletions(-)
> > 
> > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > index 64f8d6b..f073e6d 100644
> > --- a/xen/arch/arm/domain_build.c
> > +++ b/xen/arch/arm/domain_build.c
> > @@ -2137,7 +2137,7 @@ int __init construct_dom0(struct domain *d)
> >       kinfo.unassigned_mem = dom0_mem;
> >       kinfo.d = d;
> >   -    rc = kernel_probe(&kinfo);
> > +    rc = kernel_probe(&kinfo, NULL);
> >       if ( rc < 0 )
> >           return rc;
> >   diff --git a/xen/arch/arm/kernel.c b/xen/arch/arm/kernel.c
> > index da8410e..e5b8213 100644
> > --- a/xen/arch/arm/kernel.c
> > +++ b/xen/arch/arm/kernel.c
> > @@ -421,22 +421,60 @@ static int __init kernel_zimage32_probe(struct
> > kernel_info *info,
> >       return 0;
> >   }
> >   -int __init kernel_probe(struct kernel_info *info)
> > +int __init kernel_probe(struct kernel_info *info, struct dt_device_node
> > *domain)
> >   {
> > -    struct bootmodule *mod = boot_module_find_by_kind(BOOTMOD_KERNEL);
> > +    struct bootmodule *mod = NULL;
> > +    struct bootcmdline *cmd = NULL;
> > +    struct dt_device_node *node;
> > +    u64 kernel_addr, initrd_addr, size;
> > +    const char *name = NULL;
> 
> Please try to limit scope of variables. For instance, name is only used

I'll move it below


> >       int rc;
> >   +    if ( domain == NULL )
> 
> A line explain this is for the hardware domain would be useful.
> 
> Maybe with an ASSERT(is_hardware_domain(d)) in the if.

I'll add a comment and the ASSERT


> > +    {
> > +        mod = boot_module_find_by_kind(BOOTMOD_KERNEL);
> > +
> > +        info->kernel_bootmodule = mod;
> > +        info->initrd_bootmodule =
> > boot_module_find_by_kind(BOOTMOD_RAMDISK);
> > +    } else {
> 
> Coding style:
> 
> }
> else
> {

I'll fix


> > +        dt_for_each_child_node(domain, node)
> > +        {
> > +            if ( dt_device_is_compatible(node, "multiboot,kernel") )
> > +            {
> > +                u32 len;
> > +                const __be32 *val;
> 
> Newline.

OK

> 
> > +                val = dt_get_property(node, "reg", &len);
> > +                dt_get_range(&val, node, &kernel_addr, &size);
> 
> Why don't you use dt_device_get_address?

I tried but it doesn't work, it fails to retrieve address and size. The
reason is that dt_match_bus fails
(dt_device_get_address->dt_get_address->dt_match_bus). I guess the
reason is that /chosen is not a bus.


> > +            }
> > +            else if ( dt_device_is_compatible(node, "multiboot,ramdisk") )
> > +            {
> > +                u32 len;
> > +                const __be32 *val;
> 
> Newline.

OK

> > +                val = dt_get_property(node, "reg", &len);
> > +                dt_get_range(&val, node, &initrd_addr, &size);
> 
> Ditto.
>
> > +            }
> > +            else
> > +                continue;
> > +        }
> > +        if ( kernel_addr )
> 
> 0 is a valid physical address. Please use INVALID_PADDR when checking whether
> the user specified an address.
> 
> But, why don't you assign info->kernel_bootmodule when you found the node
> associated to it?

OK


> > +            info->kernel_bootmodule = mod =
> > boot_module_find_by_addr_and_kind(
> > +                                      BOOTMOD_KERNEL, kernel_addr);
> 
> This code is not easy to read because of the indentation and the double
> assignment. As you change half the user of mod below, then you can drop it
> completely.
> 
> For the indentation, you could use a temporary variable. This would also help
> to avoid using info-> everywhere below.

I'll improve the code and avoid the double assignment, but I would like
to keep mod around. It is better than the alternative.


> > +        if ( initrd_addr )
> > +            info->initrd_bootmodule = boot_module_find_by_addr_and_kind(
> > +                                      BOOTMOD_RAMDISK, initrd_addr);
> > +        name = dt_node_name(domain);
> > +        cmd = boot_cmdline_find_by_name(name);
> > +        if ( cmd )
> > +            info->cmdline = &cmd->cmdline[0];
> 
> If you set command line for DomU here, then please also set the command line
> for the hwdom above. So the function has the same behavior accross all the
> domain.

OK


> > +    }
> >       if ( !mod || !mod->size )
> >       {
> >           printk(XENLOG_ERR "Missing kernel boot module?\n");
> >           return -ENOENT;
> >       }
> >   -    info->kernel_bootmodule = mod;
> > -
> > -    printk("Loading kernel from boot module @ %"PRIpaddr"\n", mod->start);
> > -
> > -    info->initrd_bootmodule = boot_module_find_by_kind(BOOTMOD_RAMDISK);
> > +    printk("Loading DomU kernel from boot module @ %"PRIpaddr"\n",
> 
> This message is wrong for the hardware domain. The best solution would be to
> print dom%u.

OK


> > +           info->kernel_bootmodule->start);
> >       if ( info->initrd_bootmodule )
> >           printk("Loading ramdisk from boot module @ %"PRIpaddr"\n",
> >                  info->initrd_bootmodule->start);
> > diff --git a/xen/arch/arm/kernel.h b/xen/arch/arm/kernel.h
> > index 39b7828..4a65289 100644
> > --- a/xen/arch/arm/kernel.h
> > +++ b/xen/arch/arm/kernel.h
> > @@ -55,7 +55,7 @@ struct kernel_info {
> >    *  ->type
> >    *  ->load hook, and sets loader specific variables ->zimage
> >    */
> > -int kernel_probe(struct kernel_info *info);
> > +int kernel_probe(struct kernel_info *info, struct dt_device_node *domain);
> >     /*
> >    * Loads the kernel into guest RAM.
> > diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> > index dbab232..d6d1546 100644
> > --- a/xen/arch/arm/setup.c
> > +++ b/xen/arch/arm/setup.c
> > @@ -263,6 +263,35 @@ struct bootcmdline * __init
> > boot_cmdline_find_by_kind(bootmodule_kind kind)
> >       return NULL;
> >   }
> >   +struct bootcmdline * __init boot_cmdline_find_by_name(const char *name)
> > +{
> > +    struct bootcmdlines *mods = &bootinfo.cmdlines;
> > +    struct bootcmdline *mod;
> > +    int i;
> 
> Unsigned + newline.

OK


> > +    for (i = 0 ; i < mods->nr_mods ; i++ )
> > +    {
> > +        mod = &mods->cmdline[i];
> > +        if ( strcmp(mod->dt_name, name) == 0 )
> > +            return mod;
> > +    }
> > +    return NULL;
> > +}
> > +
> > +struct bootmodule * __init
> > boot_module_find_by_addr_and_kind(bootmodule_kind kind,
> > +                                                             paddr_t start)
> > +{
> > +    struct bootmodules *mods = &bootinfo.modules;
> > +    struct bootmodule *mod; > +    int i;
> 
> Same here.

OK

> > +    for (i = 0 ; i < mods->nr_mods ; i++ )
> > +    {
> > +        mod = &mods->module[i];
> > +        if ( mod->kind == kind && mod->start == start )
> > +            return mod;
> > +    }
> > +    return NULL;
> > +}
> > +
> >   const char * __init boot_module_kind_as_string(bootmodule_kind kind)
> >   {
> >       switch ( kind )
> > diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
> > index 711b4a2..177e8db 100644
> > --- a/xen/include/asm-arm/setup.h
> > +++ b/xen/include/asm-arm/setup.h
> > @@ -95,7 +95,10 @@ const char __init *boot_fdt_cmdline(const void *fdt);
> >   struct bootmodule *add_boot_module(bootmodule_kind kind,
> >                                      paddr_t start, paddr_t size, bool
> > domU);
> >   struct bootmodule *boot_module_find_by_kind(bootmodule_kind kind);
> > +struct bootmodule * __init
> > boot_module_find_by_addr_and_kind(bootmodule_kind kind,
> > +                                                             paddr_t
> > start);
> >   struct bootcmdline *boot_cmdline_find_by_kind(bootmodule_kind kind);
> > +struct bootcmdline * __init boot_cmdline_find_by_name(const char *name);
> >   const char * __init boot_module_kind_as_string(bootmodule_kind kind);
> >     #endif
> > 
> 
> Cheers,
> 
> -- 
> Julien Grall
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 22/23] xen/arm: move kernel.h to asm-arm/

[Julien Grall]

Hi Stefano,

On 10/19/18 10:54 PM, Stefano Stabellini wrote:
> On Wed, 17 Oct 2018, Julien Grall wrote:
>> On 17/10/2018 15:42, Stefano Stabellini wrote:
>>> On Mon, 15 Oct 2018, Julien Grall wrote:
>>>> Hi Stefano,
>>>>
>>>> On 05/10/2018 19:47, Stefano Stabellini wrote:
>>>>> It will be #included by a file in a xen/arch/arm subdirectory.
>>>>>
>>>>> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
>>>>> ---
>>>>>     xen/arch/arm/domain_build.c  |  2 +-
>>>>>     xen/arch/arm/kernel.c        |  3 +-
>>>>>     xen/arch/arm/kernel.h        | 86
>>>>> --------------------------------------------
>>>>>     xen/include/asm-arm/kernel.h | 86
>>>>> ++++++++++++++++++++++++++++++++++++++++++++
>>>>
>>>> There are way to make git diff nicer for code movement. This seems to be
>>>> done
>>>> by default on 2.11.0. Not sure for older version. What are you using?
>>>
>>> Git version 1.9.1 (and guilt 0.35)
>>
>> A 4 years old git version. May I ask to update to a new git (or see if there
>> are an option doing the same)? This would help reviewing code movement.
> 
> Unfortunately I get the same with 2.7.4:
> 
>   xen/arch/arm/domain_build.c  |  2 +-
>   xen/arch/arm/kernel.c        |  3 +-
>   xen/arch/arm/kernel.h        | 86 --------------------------------------------
>   xen/include/asm-arm/kernel.h | 86 ++++++++++++++++++++++++++++++++++++++++++++
>   4 files changed, 88 insertions(+), 89 deletions(-)
>   delete mode 100644 xen/arch/arm/kernel.h
>   create mode 100644 xen/include/asm-arm/kernel.h
> 
> FYI the diff between xen/arch/arm/kernel.h and
> xen/include/asm-arm/kernel.h is 0, I don't know why git doesn't pick it
> up:

D'oh :/. Probably some configurations in your .config. Thank you for 
confirming they are both the same.

Cheers,

> 
> stefanos@xsjstefanos:/local/repos/xen$ diff xen/arch/arm/kernel.h xen/include/asm-arm/kernel.h
> stefanos@xsjstefanos:/local/repos/xen$
> stefanos@xsjstefanos:/local/repos/xen$ md5sum xen/arch/arm/kernel.h xen/include/asm-arm/kernel.h
> 06bdad45e665f20f03ffdd2822d8f08e  xen/arch/arm/kernel.h
> 06bdad45e665f20f03ffdd2822d8f08e  xen/include/asm-arm/kernel.h
> 
> I hope that helps.
> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 10/23] xen/arm: introduce allocate_memory

[Julien Grall]

Hi Stefano,

On 10/19/18 11:28 PM, Stefano Stabellini wrote:
> On Mon, 15 Oct 2018, Julien Grall wrote:
>> Hi,
>>
>> On 05/10/2018 19:47, Stefano Stabellini wrote:
>>> Introduce an allocate_memory function able to allocate memory for DomUs
>>> and map it at the right guest addresses, according to the guest memory
>>> map: GUEST_RAM0_BASE and GUEST_RAM1_BASE.
>>
>> Please add something along the line: "This is under #if 0 as not used for
>> now".
> 
> I'll add
> 
> 
>>>
>>> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
>>> ---
>>> Changes in v4:
>>> - move earlier, add #if 0
>>> - introduce allocate_bank_memory, remove insert_bank
>>> - allocate_bank_memory allocate memory and inserts the bank, while
>>>     allocate_memory specifies where to do that
>>>
>>> Changes in v3:
>>> - new patch
>>> ---
>>>    xen/arch/arm/domain_build.c | 83
>>> +++++++++++++++++++++++++++++++++++++++++++++
>>>    1 file changed, 83 insertions(+)
>>>
>>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>>> index 41f2f27..fddfd82 100644
>>> --- a/xen/arch/arm/domain_build.c
>>> +++ b/xen/arch/arm/domain_build.c
>>> @@ -368,6 +368,89 @@ static void __init allocate_memory_11(struct domain *d,
>>>        }
>>>    }
>>>    +#if 0
>>> +static bool __init allocate_bank_memory(struct domain *d,
>>> +                                        struct kernel_info *kinfo,
>>> +                                        gfn_t sgfn,
>>> +                                        unsigned int order)
>>> +{
>>> +    int res;
>>> +    struct page_info *pg;
>>> +    struct membank *bank;
>>> +    paddr_t gaddr = gfn_to_gaddr(sgfn), size = pfn_to_paddr(1UL << order);
>>
>> There is no need to have gaddr in a variable.
> 
> I'll remove it.
> 
> 
>>> +
>>> +    pg = alloc_domheap_pages(d, order, 0);
>>
>> So here you impose the memory to be contiguously allocated for a given bank.
>> There are quite a few case where you may not have enough memory to allocate
>> contiguously.
>>
>> So more likely you want to add loop in this code to allocate until order is
>> reached.
> 
> This case is not handled today for dom0.

I am afraid this is slightly untrue. Dom0 memory is direct mapped, Xen 
will first try to allocate the using the biggest order. If it fails, the 
order will be decreased until the size is suitable. We will always try 
to allocate the maximum of memory.

There are three major problems with your code:
	1) There may not be enough contiguous space in the host memory for a 
4GB region.
	2) The memory specify by the user may not be in power of 2 pages. For 
instance, a user can specify 40KB. With your algo, we will end to 
allocate 32KB in the first bank and 8KB in the second bank. However what 
we want is allocate 40KB in a single bank.
	3) You don't check whether the leftover memory is bigger than the size 
of the second bank.

In the case where the memory is not direct mapped, we should be able to 
allocate the whole region with multiple non-contiguous memory. I think 
the algorithm in used in libxc (see populate_guest_memory) is working 
quite well for this purpose.

It is also possible to have a simpler solution within the hypervisor. I 
quickly wrote a patch (not compiled, nor tested) that should address the 
3 problems above. See: https://pastebin.com/FQ9k9CbL

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 13/23] xen/arm: implement construct_domU

[Julien Grall]

Hi Stefano,

On 10/19/18 11:53 PM, Stefano Stabellini wrote:
> On Mon, 15 Oct 2018, Julien Grall wrote:
>> Hi,
>>
>> On 05/10/2018 19:47, Stefano Stabellini wrote:
>>> Similar to construct_dom0, construct_domU creates a barebone DomU guest.
>>>
>>> The device tree node passed as argument is compatible "xen,domain", see
>>> docs/misc/arm/device-tree/booting.txt.
>>>
>>> Add const to kernel_probe dt_device_node parameter.
>>
>> This likely belongs to patch #7 where the parameter was added.
> 
> OK
> 
> 
>>>
>>> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
>>>
>>> ---
>>> Changes in v4:
>>> - constify kernel_probe
>>> - change title
>>> - better error messages and printed info
>>> - 64bit memory
>>>
>>> Changes in v3:
>>> - move setting type before allocate_memory
>>> - add ifdef around it and a comment
>>>
>>> Changes in v2:
>>> - rename mem to memory
>>> - make cpus and memory mandatory
>>> - remove wront comment from commit message
>>> - cpus and memory are read as integers
>>> - read the vpl011 option
>>> ---
>>>    xen/arch/arm/domain_build.c | 37 ++++++++++++++++++++++++++++++++++---
>>>    xen/arch/arm/kernel.c       |  3 ++-
>>>    xen/arch/arm/kernel.h       |  2 +-
>>>    3 files changed, 37 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>>> index 547b624..efb530a 100644
>>> --- a/xen/arch/arm/domain_build.c
>>> +++ b/xen/arch/arm/domain_build.c
>>> @@ -369,7 +369,6 @@ static void __init allocate_memory_11(struct domain *d,
>>>        }
>>>    }
>>>    -#if 0
>>
>> Please add a word about this change in the commit message.
> 
> OK
> 
> 
>>>    static bool __init allocate_bank_memory(struct domain *d,
>>>                                            struct kernel_info *kinfo,
>>>                                            gfn_t sgfn,
>>> @@ -450,7 +449,6 @@ fail:
>>>                (unsigned long)kinfo->unassigned_mem >> 10);
>>>        BUG();
>>>    }
>>> -#endif
>>>      static int __init write_properties(struct domain *d, struct kernel_info
>>> *kinfo,
>>>                                       const struct dt_device_node *node)
>>> @@ -2294,7 +2292,40 @@ static int __init __construct_domain(struct domain
>>> *d, struct kernel_info *kinfo
>>>    static int __init construct_domU(struct domain *d,
>>>                                     const struct dt_device_node *node)
>>>    {
>>> -    return -ENOSYS;
>>> +    struct kernel_info kinfo = {};
>>> +    int rc;
>>> +    u64 mem;
>>> +
>>> +    rc = dt_property_read_u64(node, "memory", &mem);
>>> +    if ( !rc )
>>> +    {
>>> +        printk("Error building DomU: cannot read \"memory\" property\n");
>>> +        return -EINVAL;
>>> +    }
>>> +    kinfo.unassigned_mem = (paddr_t)mem << 10;
>>
>> I noticed I forgot to answer to:
>> "KB() only works for numbers, it is defined as: (_AC(_kb, ULL) << 10)"
>>
>> unsigned long long is always going to be bigger than paddr_t. Also, we already
>> use MB(...) in similar situation. So I am not sure to understand your concern
>> here.
> 
> I admit that my explanation was so bad that even I had to go back to
> figure out what I meant :-)
> 
> What I wanted to say is that KB() only works for constants, not
> variables. So KB(10) works, but KB(mem) does not.

Oh, that's annoying. Can you use mem * SZ_1K then?

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 21/23] xen/vpl011: buffer out chars when the backend is xen

[Julien Grall]

Hi Stefano,

On 10/20/18 12:20 AM, Stefano Stabellini wrote:
> On Mon, 8 Oct 2018, Jan Beulich wrote:
>>>>> On 05.10.18 at 20:47, <sstabellini@kernel.org> wrote:
>>> --- a/xen/drivers/char/console.c
>>> +++ b/xen/drivers/char/console.c
>>> @@ -406,6 +406,13 @@ static void dump_console_ring_key(unsigned char key)
>>>    */
>>>   static unsigned int __read_mostly console_rx = 0;
>>>   
>>> +struct domain *console_input_domain(void)
>>> +{
>>> +    if ( console_rx == 0 )
>>> +            return NULL;
>>> +    return get_domain_by_id(console_rx - 1);
>>
>> This acquires a domain reference, yet I can't see that reference
>> getting dropped anywhere in the caller.
> 
> Well spotted!
> 
> I'll add a comment here as a reminder, and I'll drop the domain
> reference in the caller (xen/arch/arm/vpl011.c:vpl011_write_data_xen).

Why do you use get_domain_by_id() and not rcu_lock_domain_by_id()? The 
latter is recommend for short live reference.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 06/23] xen/arm: don't add duplicate boot modules, introduce domU flag

[Julien Grall]

Hi Stefano,

On 10/20/18 12:36 AM, Stefano Stabellini wrote:
> On Tue, 9 Oct 2018, Julien Grall wrote:
>>> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
>>> index bc7dd97..dbab232 100644
>>> --- a/xen/arch/arm/setup.c
>>> +++ b/xen/arch/arm/setup.c
>>> @@ -201,10 +201,12 @@ void __init dt_unreserved_regions(paddr_t s, paddr_t
>>> e,
>>>    }
>>>      struct bootmodule __init *add_boot_module(bootmodule_kind kind,
>>> -                                          paddr_t start, paddr_t size)
>>> +                                          paddr_t start, paddr_t size,
>>> +                                          bool domU)
>>>    {
>>>        struct bootmodules *mods = &bootinfo.modules;
>>>        struct bootmodule *mod;
>> This at least need to be replica
> 
> I take this was a typo?

I can't remember. Let's consider a typo until the next version :).

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 13/23] xen/arm: implement construct_domU

[Stefano Stabellini]

On Sat, 20 Oct 2018, Julien Grall wrote:
> Hi Stefano,
> 
> On 10/19/18 11:53 PM, Stefano Stabellini wrote:
> > On Mon, 15 Oct 2018, Julien Grall wrote:
> > > Hi,
> > > 
> > > On 05/10/2018 19:47, Stefano Stabellini wrote:
> > > > Similar to construct_dom0, construct_domU creates a barebone DomU guest.
> > > > 
> > > > The device tree node passed as argument is compatible "xen,domain", see
> > > > docs/misc/arm/device-tree/booting.txt.
> > > > 
> > > > Add const to kernel_probe dt_device_node parameter.
> > > 
> > > This likely belongs to patch #7 where the parameter was added.
> > 
> > OK
> > 
> > 
> > > > 
> > > > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > > > 
> > > > ---
> > > > Changes in v4:
> > > > - constify kernel_probe
> > > > - change title
> > > > - better error messages and printed info
> > > > - 64bit memory
> > > > 
> > > > Changes in v3:
> > > > - move setting type before allocate_memory
> > > > - add ifdef around it and a comment
> > > > 
> > > > Changes in v2:
> > > > - rename mem to memory
> > > > - make cpus and memory mandatory
> > > > - remove wront comment from commit message
> > > > - cpus and memory are read as integers
> > > > - read the vpl011 option
> > > > ---
> > > >    xen/arch/arm/domain_build.c | 37
> > > > ++++++++++++++++++++++++++++++++++---
> > > >    xen/arch/arm/kernel.c       |  3 ++-
> > > >    xen/arch/arm/kernel.h       |  2 +-
> > > >    3 files changed, 37 insertions(+), 5 deletions(-)
> > > > 
> > > > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > > > index 547b624..efb530a 100644
> > > > --- a/xen/arch/arm/domain_build.c
> > > > +++ b/xen/arch/arm/domain_build.c
> > > > @@ -369,7 +369,6 @@ static void __init allocate_memory_11(struct domain
> > > > *d,
> > > >        }
> > > >    }
> > > >    -#if 0
> > > 
> > > Please add a word about this change in the commit message.
> > 
> > OK
> > 
> > 
> > > >    static bool __init allocate_bank_memory(struct domain *d,
> > > >                                            struct kernel_info *kinfo,
> > > >                                            gfn_t sgfn,
> > > > @@ -450,7 +449,6 @@ fail:
> > > >                (unsigned long)kinfo->unassigned_mem >> 10);
> > > >        BUG();
> > > >    }
> > > > -#endif
> > > >      static int __init write_properties(struct domain *d, struct
> > > > kernel_info
> > > > *kinfo,
> > > >                                       const struct dt_device_node *node)
> > > > @@ -2294,7 +2292,40 @@ static int __init __construct_domain(struct
> > > > domain
> > > > *d, struct kernel_info *kinfo
> > > >    static int __init construct_domU(struct domain *d,
> > > >                                     const struct dt_device_node *node)
> > > >    {
> > > > -    return -ENOSYS;
> > > > +    struct kernel_info kinfo = {};
> > > > +    int rc;
> > > > +    u64 mem;
> > > > +
> > > > +    rc = dt_property_read_u64(node, "memory", &mem);
> > > > +    if ( !rc )
> > > > +    {
> > > > +        printk("Error building DomU: cannot read \"memory\"
> > > > property\n");
> > > > +        return -EINVAL;
> > > > +    }
> > > > +    kinfo.unassigned_mem = (paddr_t)mem << 10;
> > > 
> > > I noticed I forgot to answer to:
> > > "KB() only works for numbers, it is defined as: (_AC(_kb, ULL) << 10)"
> > > 
> > > unsigned long long is always going to be bigger than paddr_t. Also, we
> > > already
> > > use MB(...) in similar situation. So I am not sure to understand your
> > > concern
> > > here.
> > 
> > I admit that my explanation was so bad that even I had to go back to
> > figure out what I meant :-)
> > 
> > What I wanted to say is that KB() only works for constants, not
> > variables. So KB(10) works, but KB(mem) does not.
> 
> Oh, that's annoying. Can you use mem * SZ_1K then?

OK

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 21/23] xen/vpl011: buffer out chars when the backend is xen

[Stefano Stabellini]

On Sat, 20 Oct 2018, Julien Grall wrote:
> Hi Stefano,
> 
> On 10/20/18 12:20 AM, Stefano Stabellini wrote:
> > On Mon, 8 Oct 2018, Jan Beulich wrote:
> > > > > > On 05.10.18 at 20:47, <sstabellini@kernel.org> wrote:
> > > > --- a/xen/drivers/char/console.c
> > > > +++ b/xen/drivers/char/console.c
> > > > @@ -406,6 +406,13 @@ static void dump_console_ring_key(unsigned char
> > > > key)
> > > >    */
> > > >   static unsigned int __read_mostly console_rx = 0;
> > > >   +struct domain *console_input_domain(void)
> > > > +{
> > > > +    if ( console_rx == 0 )
> > > > +            return NULL;
> > > > +    return get_domain_by_id(console_rx - 1);
> > > 
> > > This acquires a domain reference, yet I can't see that reference
> > > getting dropped anywhere in the caller.
> > 
> > Well spotted!
> > 
> > I'll add a comment here as a reminder, and I'll drop the domain
> > reference in the caller (xen/arch/arm/vpl011.c:vpl011_write_data_xen).
> 
> Why do you use get_domain_by_id() and not rcu_lock_domain_by_id()? The latter
> is recommend for short live reference.

Yes, you are right, that is a better solution. I'll change it to use
rcu_lock_domain_by_id. We still need the corresponding rcu_unlock_domain
call, so I'll still add a new comment and the corresponding unlock call.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 10/23] xen/arm: introduce allocate_memory

[Stefano Stabellini]

On Sat, 20 Oct 2018, Julien Grall wrote:
> > > > +
> > > > +    pg = alloc_domheap_pages(d, order, 0);
> > > 
> > > So here you impose the memory to be contiguously allocated for a given
> > > bank.
> > > There are quite a few case where you may not have enough memory to
> > > allocate
> > > contiguously.
> > > 
> > > So more likely you want to add loop in this code to allocate until order
> > > is
> > > reached.
> > 
> > This case is not handled today for dom0.
> 
> I am afraid this is slightly untrue. Dom0 memory is direct mapped, Xen will
> first try to allocate the using the biggest order. If it fails, the order will
> be decreased until the size is suitable. We will always try to allocate the
> maximum of memory.
> 
> There are three major problems with your code:
> 	1) There may not be enough contiguous space in the host memory for a
> 4GB region.
> 	2) The memory specify by the user may not be in power of 2 pages. For
> instance, a user can specify 40KB. With your algo, we will end to allocate
> 32KB in the first bank and 8KB in the second bank. However what we want is
> allocate 40KB in a single bank.
> 	3) You don't check whether the leftover memory is bigger than the size
> of the second bank.
> 
> In the case where the memory is not direct mapped, we should be able to
> allocate the whole region with multiple non-contiguous memory. I think the
> algorithm in used in libxc (see populate_guest_memory) is working quite well
> for this purpose.
> 
> It is also possible to have a simpler solution within the hypervisor. I
> quickly wrote a patch (not compiled, nor tested) that should address the 3
> problems above. See: https://pastebin.com/FQ9k9CbL

Thank you for your help. Actually I had already written something
along the same lines, I don't know why I didn't say it in my reply... I
double checked the implementation with yours here and they look
equivalent. Good.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 03/23] xen/arm: document dom0less

[Stefano Stabellini]

On Tue, 16 Oct 2018, Lars Kurth wrote:
> Hi Stefano,
> 
> > On 9 Oct 2018, at 12:52, Julien Grall <julien.grall@arm.com> wrote:
> > 
> > Hi Stefano,
> > 
> > On 05/10/2018 19:47, Stefano Stabellini wrote:
> >> Add a new document to provide information on how to use dom0less related
> >> features and their current limitations.
> >> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> >> ---
> >> Changes in v4:
> >> - rename to .txt
> >> - improve wording
> >> Changes in v3:
> >> - add patch
> >> ---
> >>  docs/misc/arm/dom0less.txt | 47 ++++++++++++++++++++++++++++++++++++++++++++++
> > 
> > As said on the previous version, you likely need to add an entry in docs/INDEX.
> 
> Agreed. Also, it may be worthwhile encoding the doc in markdown (it's not that much text) and using the .markdown extension
> Also, it seems to me that docs/features is maybe a better place for this document. Not sure whether docs in this folder get picked up automatically though

I can move it to docs/features and I am OK with changing it to markdown.
But please don't ask me to convert it to pandoc (all other docs under
docs/features are pandoc).

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 03/23] xen/arm: document dom0less

[Stefano Stabellini]

On Tue, 9 Oct 2018, Julien Grall wrote:
> Hi Stefano,
> 
> On 05/10/2018 19:47, Stefano Stabellini wrote:
> > Add a new document to provide information on how to use dom0less related
> > features and their current limitations.
> > 
> > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > 
> > ---
> > Changes in v4:
> > - rename to .txt
> > - improve wording
> > 
> > Changes in v3:
> > - add patch
> > ---
> >   docs/misc/arm/dom0less.txt | 47
> > ++++++++++++++++++++++++++++++++++++++++++++++
> 
> As said on the previous version, you likely need to add an entry in
> docs/INDEX.

OK

> >   1 file changed, 47 insertions(+)
> >   create mode 100644 docs/misc/arm/dom0less.txt
> > 
> > diff --git a/docs/misc/arm/dom0less.txt b/docs/misc/arm/dom0less.txt
> > new file mode 100644
> > index 0000000..df96b41
> > --- /dev/null
> > +++ b/docs/misc/arm/dom0less.txt
> > @@ -0,0 +1,47 @@
> > +Dom0less
> > +========
> > +
> > +"Dom0less" is a set of Xen features that enable the deployment of a Xen
> > +system without an hardware domain (often referred to as "dom0").
> I realize I suggested the wording hardware domain. But reading this again, it
> feels that "control domain" may be the best wording here. Indeed what we avoid
> is the toolstack and domain control the domains.

OK


> You begin the document writing "it is a set of Xen featueres that enable
> deployment of a Xen system without an hardware domain". I understand this
> sentence as there would be no "hardware domain". But then you write "create a
> set of DomU alongside Dom0".
> 
> Furthermore, at some point the control domain would disappear and the DomID 0
> may be allocated to a DomUs. Adding further confusion to a user seen the ID
> would be 0.
> 
> So I still think that using "Dom0" within the document is misleading and also
> the feature name. I don't have a good suggestion for the feature name. But at
> very least I would avoid the word "Dom0" everywhere in that document.
 
I'll reword the document to avoid "Dom0"

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 05/23] xen/arm: introduce bootcmdlines

[Stefano Stabellini]

On Tue, 9 Oct 2018, Julien Grall wrote:
> On 05/10/2018 19:47, Stefano Stabellini wrote:
> > Introduce a new array to store the cmdline of each boot module. It is
> > separate from struct bootmodules. Remove the cmdline field from struct
> > boot_module. This way, kernels and initrds with the same address in
> > memory can share struct bootmodule (important because we want them to be
> > free'd only once), but they can still have their separate bootcmdline
> > entries.
> > 
> > Add a dt_name field to struct bootcmdline to make it easier to find the
> > correct entry. Store the name of the "xen,domain" compatible node (for
> > example "Dom1"). This is a better choice compared to the name of the
> > "multiboot,kernel" compatible node, because their names are not unique.
> > For instance there can be more than one "module@0x4c000000" in the
> > system, but there can only be one "/chosen/Dom1".
> > 
> > Add a pointer to struct kernel_info to point to the cmdline for a given
> > kernel.
> > 
> > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > 
> > ---
> > 
> > Changes in v4:
> > - check that the multiboot node is under /chosen
> > - use cmd and cmds as variable names for struct bootcmdline and struct
> >    bootcmdline*
> > - fix printk
> > - use ASSERT instea of panic
> > - do not add empty cmdline entries
> > - add more debug printks to early_print_info
> > - code style fixes
> > - add comment on DT_MAX_NAME
> > - increase DT_MAX_NAME to 41
> > - make nr_mods unsigned int
> > 
> > Changes in v3:
> > - introduce bootcmdlines
> > - do not modify boot_fdt_cmdline
> > - add comments
> > 
> > Changes in v2:
> > - new patch
> > ---
> >   xen/arch/arm/bootfdt.c      | 82
> > +++++++++++++++++++++++++++++++++------------
> >   xen/arch/arm/domain_build.c |  8 +++--
> >   xen/arch/arm/kernel.h       |  1 +
> >   xen/arch/arm/setup.c        | 24 +++++++++----
> >   xen/include/asm-arm/setup.h | 17 ++++++++--
> >   5 files changed, 99 insertions(+), 33 deletions(-)
> > 
> > diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
> > index 8eba42c..273032b 100644
> > --- a/xen/arch/arm/bootfdt.c
> > +++ b/xen/arch/arm/bootfdt.c
> > @@ -163,6 +163,37 @@ static void __init process_memory_node(const void *fdt,
> > int node,
> >       }
> >   }
> >   +static void __init add_boot_cmdline(const void *fdt, int node,
> > +                                    const char *name, bootmodule_kind kind)
> > +{
> > +    struct bootcmdlines *cmds = &bootinfo.cmdlines;
> > +    struct bootcmdline *cmd;
> > +    const struct fdt_property *prop;
> > +    int len;
> > +    const char *cmdline;
> > +
> > +    if ( cmds->nr_mods == MAX_MODULES )
> > +    {
> > +        printk("Ignoring %s cmdline (too many)\n", name);
> > +        return;
> > +    }
> > +
> > +    prop = fdt_get_property(fdt, node, "bootargs", &len);
> > +    if ( !prop )
> > +        return;
> > +
> > +    cmd = &cmds->cmdline[cmds->nr_mods++];
> > +    cmd->kind = kind;
> > +
> > +    ASSERT(strlen(name) <= DT_MAX_NAME);
> > +    safe_strcpy(cmd->dt_name, name);
> > +
> > +    if ( len > BOOTMOD_MAX_CMDLINE )
> > +        panic("module %s command line too long\n", name);
> > +    cmdline = prop->data;
> > +    safe_strcpy(cmd->cmdline, cmdline);
> > +}
> > +
> >   static void __init process_multiboot_node(const void *fdt, int node,
> >                                             const char *name,
> >                                             u32 address_cells, u32
> > size_cells)
> > @@ -172,8 +203,20 @@ static void __init process_multiboot_node(const void
> > *fdt, int node,
> >       const __be32 *cell;
> >       bootmodule_kind kind;
> >       paddr_t start, size;
> > -    const char *cmdline;
> > -    int len;
> > +    int len = sizeof("/chosen");
> > +    char path[8]; /* sizeof "/chosen" */
> > +    int parent_node;
> > +
> > +    parent_node = fdt_parent_offset(fdt, node);
> > +    ASSERT(parent_node >= 0);
> > +
> > +    /* Check that the node is under "chosen" */
> > +    fdt_get_path(fdt, node, path, len);
> 
> It would be good if you test the code with wrong node. For instance
> fdt_get_path may not fill path (i.e if the buffer is too short). So path will
> contain garbage.

I'll do


> > +    if ( strncmp(path, "/chosen", len - 1) )
> > +    {
> > +        printk("DEBUG %s %s\n",name,path);
> 
> This looks like a left-over from your debug.

I'll remove


> > +        return;
> > +    }
> As I said in the previous patch, this needs to be fixed first. By that I meant
> this should be a separate patch so it could be backported.

I'll separate it out


> Also, with this patch you change correctly the behavior to deny node not in
> chosen. But you don't explain in the commit message that it affects the
> current multiboot node.

I'll mention it.


> However, I still don't think this code is correct. You would allow the
> following path /chosen/foo/bar/multiboot. The parent would be "bar" and there
> are no guarantee it would be uniq (it is not under /chosen). I think this
> could be fixed by taking into account the depth of the node.
> AFAICT, the multiboot node should always be at maximum depth 3.

I'll add a check.


> >         prop = fdt_get_property(fdt, node, "reg", &len);
> >       if ( !prop )
> > @@ -220,17 +263,8 @@ static void __init process_multiboot_node(const void
> > *fdt, int node,
> >               kind = BOOTMOD_XSM;
> >       }
> >   -    prop = fdt_get_property(fdt, node, "bootargs", &len);
> > -    if ( prop )
> > -    {
> > -        if ( len > BOOTMOD_MAX_CMDLINE )
> > -            panic("module %s command line too long\n", name);
> > -        cmdline = prop->data;
> > -    }
> > -    else
> > -        cmdline = NULL;
> > -
> > -    add_boot_module(kind, start, size, cmdline);
> > +    add_boot_module(kind, start, size);
> > +    add_boot_cmdline(fdt, node, fdt_get_name(fdt, parent_node, &len),
> > kind);
> >   }
> >     static void __init process_chosen_node(const void *fdt, int node,
> > @@ -276,7 +310,7 @@ static void __init process_chosen_node(const void *fdt,
> > int node,
> >         printk("Initrd %"PRIpaddr"-%"PRIpaddr"\n", start, end);
> >   -    add_boot_module(BOOTMOD_RAMDISK, start, end-start, NULL);
> > +    add_boot_module(BOOTMOD_RAMDISK, start, end-start);
> >   }
> >     static int __init early_scan_node(const void *fdt,
> > @@ -299,6 +333,7 @@ static void __init early_print_info(void)
> >   {
> >       struct meminfo *mi = &bootinfo.mem;
> >       struct bootmodules *mods = &bootinfo.modules;
> > +    struct bootcmdlines *cmds = &bootinfo.cmdlines;
> >       int i, nr_rsvd;
> >         for ( i = 0; i < mi->nr_banks; i++ )
> > @@ -307,12 +342,12 @@ static void __init early_print_info(void)
> >                        mi->bank[i].start + mi->bank[i].size - 1);
> >       printk("\n");
> >       for ( i = 0 ; i < mods->nr_mods; i++ )
> > -        printk("MODULE[%d]: %"PRIpaddr" - %"PRIpaddr" %-12s %s\n",
> > +        printk("MODULE[%d]: %"PRIpaddr" - %"PRIpaddr" %-12s\n",
> >                        i,
> >                        mods->module[i].start,
> >                        mods->module[i].start + mods->module[i].size,
> > -                     boot_module_kind_as_string(mods->module[i].kind),
> > -                     mods->module[i].cmdline);
> > +                     boot_module_kind_as_string(mods->module[i].kind));
> > +
> >       nr_rsvd = fdt_num_mem_rsv(device_tree_flattened);
> >       for ( i = 0; i < nr_rsvd; i++ )
> >       {
> > @@ -325,6 +360,11 @@ static void __init early_print_info(void)
> >                        i, s, e);
> >       }
> >       printk("\n");
> > +    for ( i = 0 ; i < cmds->nr_mods; i++ )
> > +        printk("CMDLINE[%d]:%s %s\n", i,
> > +               cmds->cmdline[i].dt_name,
> > +               &cmds->cmdline[i].cmdline[0]);
> 
> Thank you for adding the command line. However, there are still no way to
> associate the command line with a module. For each command line, you should be
> able to know the associate module.

It is possible once the whole series is committed, by using
boot_cmdline_find_by_name and the domU field introduced later in this
series. I could move boot_cmdline_find_by_name earlier, but since there
are no callers, I would have to #if 0 it.  My choice would be to leave
things as they are.


> > +    printk("\n");
> >   }
> >     /**
> > @@ -341,7 +381,7 @@ size_t __init boot_fdt_info(const void *fdt, paddr_t
> > paddr)
> >       if ( ret < 0 )
> >           panic("No valid device tree\n");
> >   -    add_boot_module(BOOTMOD_FDT, paddr, fdt_totalsize(fdt), NULL);
> > +    add_boot_module(BOOTMOD_FDT, paddr, fdt_totalsize(fdt));
> >         device_tree_for_each_node((void *)fdt, early_scan_node, NULL);
> >       early_print_info();
> > @@ -361,11 +401,11 @@ const char *boot_fdt_cmdline(const void *fdt)
> >       prop = fdt_get_property(fdt, node, "xen,xen-bootargs", NULL);
> >       if ( prop == NULL )
> >       {
> > -        struct bootmodule *dom0_mod =
> > -            boot_module_find_by_kind(BOOTMOD_KERNEL);
> > +        struct bootcmdline *dom0_cmdline =
> > +            boot_cmdline_find_by_kind(BOOTMOD_KERNEL);
> >             if (fdt_get_property(fdt, node, "xen,dom0-bootargs", NULL) ||
> > -            ( dom0_mod && dom0_mod->cmdline[0] ) )
> > +            ( dom0_cmdline && dom0_cmdline->cmdline[0] ) )
> >               prop = fdt_get_property(fdt, node, "bootargs", NULL);
> >       }
> >       if ( prop == NULL )
> > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > index f552154..64f8d6b 100644
> > --- a/xen/arch/arm/domain_build.c
> > +++ b/xen/arch/arm/domain_build.c
> > @@ -375,7 +375,7 @@ static int __init write_properties(struct domain *d,
> > struct kernel_info *kinfo,
> >       int res = 0;
> >       int had_dom0_bootargs = 0;
> >   -    const struct bootmodule *kernel = kinfo->kernel_bootmodule;
> > +    const struct bootcmdline *kernel =
> > boot_cmdline_find_by_kind(BOOTMOD_KERNEL);
> 
> Why do you need to lookup the command line here? You have just introduced a
> field in kinfo to store command line.

good point, I'll use the new field here


> >         if ( kernel && kernel->cmdline[0] )
> >           bootargs = &kernel->cmdline[0];
> > @@ -952,9 +952,9 @@ static int __init make_chosen_node(const struct
> > kernel_info *kinfo)
> >       if ( res )
> >           return res;
> >   -    if ( mod && mod->cmdline[0] )
> > +    if ( kinfo->cmdline && kinfo->cmdline[0] )
> >       {
> > -        bootargs = &mod->cmdline[0];
> > +        bootargs = &kinfo->cmdline[0];
> >           res = fdt_property(fdt, "bootargs", bootargs, strlen(bootargs) +
> > 1);
> >           if ( res )
> >              return res;
> > @@ -2109,6 +2109,7 @@ static void __init find_gnttab_region(struct domain
> > *d,
> >     int __init construct_dom0(struct domain *d)
> >   {
> > +    const struct bootcmdline *kernel =
> > boot_cmdline_find_by_kind(BOOTMOD_KERNEL);
> >       struct kernel_info kinfo = {};
> >       struct vcpu *saved_current;
> >       int rc, i, cpu;
> > @@ -2154,6 +2155,7 @@ int __init construct_dom0(struct domain *d)
> >     #endif
> >   +    kinfo.cmdline = kernel != NULL ? &kernel->cmdline[0] : NULL;
> >       allocate_memory(d, &kinfo);
> >       find_gnttab_region(d, &kinfo);
> >   diff --git a/xen/arch/arm/kernel.h b/xen/arch/arm/kernel.h
> > index 47eacb5..39b7828 100644
> > --- a/xen/arch/arm/kernel.h
> > +++ b/xen/arch/arm/kernel.h
> > @@ -29,6 +29,7 @@ struct kernel_info {
> >         /* boot blob load addresses */
> >       const struct bootmodule *kernel_bootmodule, *initrd_bootmodule;
> > +    const char* cmdline;
> >       paddr_t dtb_paddr;
> >       paddr_t initrd_paddr;
> >   diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> > index ea2495a..bc7dd97 100644
> > --- a/xen/arch/arm/setup.c
> > +++ b/xen/arch/arm/setup.c
> > @@ -201,8 +201,7 @@ void __init dt_unreserved_regions(paddr_t s, paddr_t e,
> >   }
> >     struct bootmodule __init *add_boot_module(bootmodule_kind kind,
> > -                                          paddr_t start, paddr_t size,
> > -                                          const char *cmdline)
> > +                                          paddr_t start, paddr_t size)
> >   {
> >       struct bootmodules *mods = &bootinfo.modules;
> >       struct bootmodule *mod;
> > @@ -218,10 +217,6 @@ struct bootmodule __init
> > *add_boot_module(bootmodule_kind kind,
> >       mod->kind = kind;
> >       mod->start = start;
> >       mod->size = size;
> > -    if ( cmdline )
> > -        safe_strcpy(mod->cmdline, cmdline);
> > -    else
> > -        mod->cmdline[0] = 0;
> >         return mod;
> >   }
> > @@ -240,6 +235,21 @@ struct bootmodule * __init
> > boot_module_find_by_kind(bootmodule_kind kind)
> >       return NULL;
> >   }
> >   +struct bootcmdline * __init boot_cmdline_find_by_kind(bootmodule_kind
> > kind)
> 
> Functions dealing with a same structure should stick together. I actually
> quite dislike that add_bood_cmdline is part of bootfdt.c. There are no need
> for it to be there except the fact you moved the FDT lookup in the function.
> As I suggested in the previous patch, the lookup could have been left outside
> of the function.

I'll move add_bood_cmdline to setup.c.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v4 20/23] xen: support console_switching between Dom0 and DomUs on ARM

[Jan Beulich]

>>> On 20.10.18 at 01:10, <sstabellini@kernel.org> wrote:
> On Mon, 8 Oct 2018, Jan Beulich wrote:
>> >>> On 05.10.18 at 20:47, <sstabellini@kernel.org> wrote:
>> > @@ -391,31 +394,73 @@ static void dump_console_ring_key(unsigned char key)
>> >      free_xenheap_pages(buf, order);
>> >  }
>> >  
>> > -/* CTRL-<switch_char> switches input direction between Xen and DOM0. */
>> > +/*
>> > + * CTRL-<switch_char> switches input direction between Xen, Dom0 and
>> > + * DomUs.
>> > + */
>> 
>> Just like the title, this comment makes it sound as if any DomU could
>> participate in this switching.
> 
> Is the following any better?
> 
>   CTRL-<switch_char> changes input direction, rotating among Xen, Dom0, and 
> DomUs.
> 
> Otherwise please suggest your favorite wording and I'll use it.

"..., and the <whatever-term-you-mean-to-us-for-boot-time-
created-ones> DomUs"

>> > +    {
>> > +        /* Deliver input to hardware domain, unless it is already full. */
>> 
>> Looks like you've mis-edited the original comment.
> 
> I take that you mean that the word "buffer" is missing, right? It should
> be:
> 
>   Deliver input to the hardware domain buffer, unless it is already full.

Yes, thanks.

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel