[PATCH RFC 01/15] xen: allow console_io hypercalls from DomUs on ARM

[PATCH RFC 01/15] xen: allow console_io hypercalls from DomUs on ARM

[Stefano Stabellini]

This is very useful when starting multiple domains from Xen without
xenstore access. It will allow them to print out to the Xen console.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
CC: andrew.cooper3@citrix.com
CC: George.Dunlap@eu.citrix.com
CC: ian.jackson@eu.citrix.com
CC: jbeulich@suse.com
CC: konrad.wilk@oracle.com
CC: tim@xen.org
CC: wei.liu2@citrix.com
CC: dgdegra@tycho.nsa.gov
---
If there is a better way to do this with XSM, please advise.

---
 xen/drivers/char/console.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
index 0f05369..dc9e0bb 100644
--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -555,9 +555,11 @@ long do_console_io(int cmd, int count, XEN_GUEST_HANDLE_PARAM(char) buffer)
     long rc;
     unsigned int idx, len;
 
+#ifndef CONFIG_ARM
     rc = xsm_console_io(XSM_OTHER, current->domain, cmd);
     if ( rc )
         return rc;
+#endif
 
     switch ( cmd )
     {
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH RFC 02/15] xen/arm: move a few guest related #defines to public/arch-arm.h

[Stefano Stabellini]

Move a few constants defined by libxl_arm.c to
xen/include/public/arch-arm.h, so that they are together with the other
guest related #defines such as GUEST_GICD_BASE and GUEST_VPL011_SPI.
Also, this way they can be reused by hypervisor code.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
CC: wei.liu2@citrix.com
CC: ian.jackson@eu.citrix.com
---
 tools/libxl/libxl_arm.c       | 26 --------------------------
 xen/include/public/arch-arm.h | 26 ++++++++++++++++++++++++++
 2 files changed, 26 insertions(+), 26 deletions(-)

diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
index 8af9f6f..89a417f 100644
--- a/tools/libxl/libxl_arm.c
+++ b/tools/libxl/libxl_arm.c
@@ -8,23 +8,6 @@
 #include <libfdt.h>
 #include <assert.h>
 
-/**
- * IRQ line type.
- * DT_IRQ_TYPE_NONE            - default, unspecified type
- * DT_IRQ_TYPE_EDGE_RISING     - rising edge triggered
- * DT_IRQ_TYPE_EDGE_FALLING    - falling edge triggered
- * DT_IRQ_TYPE_EDGE_BOTH       - rising and falling edge triggered
- * DT_IRQ_TYPE_LEVEL_HIGH      - high level triggered
- * DT_IRQ_TYPE_LEVEL_LOW       - low level triggered
- */
-#define DT_IRQ_TYPE_NONE           0x00000000
-#define DT_IRQ_TYPE_EDGE_RISING    0x00000001
-#define DT_IRQ_TYPE_EDGE_FALLING   0x00000002
-#define DT_IRQ_TYPE_EDGE_BOTH                           \
-    (DT_IRQ_TYPE_EDGE_FALLING | DT_IRQ_TYPE_EDGE_RISING)
-#define DT_IRQ_TYPE_LEVEL_HIGH     0x00000004
-#define DT_IRQ_TYPE_LEVEL_LOW      0x00000008
-
 static const char *gicv_to_string(libxl_gic_version gic_version)
 {
     switch (gic_version) {
@@ -165,18 +148,9 @@ static struct arch_info {
     {"xen-3.0-aarch64", "arm,armv8-timer", "arm,armv8" },
 };
 
-/*
- * The device tree compiler (DTC) is allocating the phandle from 1 to
- * onwards. Reserve a high value for the GIC phandle.
- */
-#define PHANDLE_GIC (65000)
-
 typedef uint32_t be32;
 typedef be32 gic_interrupt[3];
 
-#define ROOT_ADDRESS_CELLS 2
-#define ROOT_SIZE_CELLS 2
-
 #define PROP_INITRD_START "linux,initrd-start"
 #define PROP_INITRD_END "linux,initrd-end"
 
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index eb424e8..cb88168 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -456,6 +456,32 @@ typedef uint64_t xen_callback_t;
 #define PSCI_cpu_on      2
 #define PSCI_migrate     3
 
+/*
+ * The device tree compiler (DTC) is allocating the phandle from 1 to
+ * onwards. Reserve a high value for the GIC phandle.
+ */
+#define PHANDLE_GIC (65000)
+
+#define ROOT_ADDRESS_CELLS 2
+#define ROOT_SIZE_CELLS 2
+
+/**
+ * IRQ line type.
+ * DT_IRQ_TYPE_NONE            - default, unspecified type
+ * DT_IRQ_TYPE_EDGE_RISING     - rising edge triggered
+ * DT_IRQ_TYPE_EDGE_FALLING    - falling edge triggered
+ * DT_IRQ_TYPE_EDGE_BOTH       - rising and falling edge triggered
+ * DT_IRQ_TYPE_LEVEL_HIGH      - high level triggered
+ * DT_IRQ_TYPE_LEVEL_LOW       - low level triggered
+ */
+#define DT_IRQ_TYPE_NONE           0x00000000
+#define DT_IRQ_TYPE_EDGE_RISING    0x00000001
+#define DT_IRQ_TYPE_EDGE_FALLING   0x00000002
+#define DT_IRQ_TYPE_EDGE_BOTH                           \
+    (DT_IRQ_TYPE_EDGE_FALLING | DT_IRQ_TYPE_EDGE_RISING)
+#define DT_IRQ_TYPE_LEVEL_HIGH     0x00000004
+#define DT_IRQ_TYPE_LEVEL_LOW      0x00000008
+
 #endif
 
 #ifndef __ASSEMBLY__
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH RFC 03/15] xen/arm: extend device tree based multiboot protocol

[Stefano Stabellini]

Extend the existing device tree based multiboot protocol to include
information regarding other domUs to boot.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
---
 docs/misc/arm/device-tree/booting.txt | 102 ++++++++++++++++++++++++++++++++++
 1 file changed, 102 insertions(+)

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index ce2d0dc..95255e5 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -119,3 +119,105 @@ For those you would hardcode the Xen commandline in the DTB under
 line by writing bootargs (as for native Linux).
 A Xen-aware bootloader would set xen,xen-bootargs for Xen, xen,dom0-bootargs
 for Dom0 and bootargs for native Linux.
+
+
+Creating DomUs directly from Xen
+================================
+
+It is possible to have Xen create other domains, in addition to dom0,
+out of the information provided via device tree. A Kernel and initrd
+(optional) need to be specified for each guest.
+
+For each DomU to be created there needs to be one node under /chosen
+with the following properties:
+
+- compatible
+
+    "xen,domU"
+
+- mem (optional)
+
+    A string specifying the amount of RAM to allocate to the guest. If
+    not specified it defaults to "64M". The format of the string is the same
+    as the one for the mem= parameter in xl config files.
+
+- cpus (optional)
+
+    A string specifying the number of vcpus to allocate to the guest. If
+    not specified it defaults to "1".
+
+- #address-cells and #size-cells
+
+    Both #address-cells and #size-cells need to be specified because
+    both sub-nodes (described shortly) have reg properties.
+
+Under the "xen,domU" compatible node, one or more sub-nodes are present
+for the DomU kernel and ramdisk.
+
+The kernel sub-node has the following properties:
+
+- compatible
+
+    "multiboot,domU-kernel"
+
+- reg
+
+    Specifies the physical address of the kernel in RAM and its
+    length.
+
+- bootargs (optional)
+
+    Command line parameters for the guest kernel.
+
+The ramdisk sub-node has the following properties:
+
+- compatible
+
+    "multiboot,domU-ramdisk"
+
+- reg
+
+    Specifies the physical address of the ramdisk in RAM and its
+    length.
+
+
+Example
+=======
+
+domU1 {
+    compatible = "xen,domU";
+    #address-cells = <0x2>;
+    #size-cells = <0x1>;
+    mem = "128M";
+    cpus = "2";
+
+    module@0x4a000000 {
+        compatible = "multiboot,domU-kernel";
+        reg = <0x0 0x4a000000 0xffffff>;
+        bootargs = "console=ttyAMA0 init=/bin/sh";
+    };
+
+    module@0x4b000000 {
+        compatible = "multiboot,domU-ramdisk";
+        reg = <0x0 0x4b000000 0xffffff>;
+    };
+};
+
+domU2 {
+    compatible = "xen,domU";
+    #address-cells = <0x2>;
+    #size-cells = <0x1>;
+    mem = "64M";
+    cpus = "1";
+
+    module@0x4c000000 {
+        compatible = "multiboot,domU-kernel";
+        reg = <0x0 0x4c000000 0xffffff>;
+        bootargs = "console=ttyAMA0 init=/bin/sh";
+    };
+
+    module@0x4d000000 {
+        compatible = "multiboot,domU-ramdisk";
+        reg = <0x0 0x4d000000 0xffffff>;
+    };
+};
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH RFC 04/15] xen/arm: do not pass dt_host to make_memory_node and make_hypervisor_node

[Stefano Stabellini]

In order to make make_memory_node and make_hypervisor_node more
reusable, do not pass them dt_host. As they only use it to calculate
addrcells and sizecells, pass addrcells and sizecells directly.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
---
 xen/arch/arm/domain_build.c   | 20 ++++++++++----------
 xen/common/device_tree.c      |  6 +++---
 xen/include/xen/device_tree.h |  2 +-
 3 files changed, 14 insertions(+), 14 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 11cdf05..bb88e09 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -533,11 +533,11 @@ static int fdt_property_interrupts(void *fdt, gic_interrupt_t *intr,
 
 static int make_memory_node(const struct domain *d,
                             void *fdt,
-                            const struct dt_device_node *parent,
+                            int addrcells, int sizecells,
                             const struct kernel_info *kinfo)
 {
     int res, i;
-    int reg_size = dt_child_n_addr_cells(parent) + dt_child_n_size_cells(parent);
+    int reg_size = addrcells + sizecells;
     int nr_cells = reg_size*kinfo->mem.nr_banks;
     __be32 reg[nr_cells];
     __be32 *cells;
@@ -563,7 +563,7 @@ static int make_memory_node(const struct domain *d,
         dt_dprintk("  Bank %d: %#"PRIx64"->%#"PRIx64"\n",
                    i, start, start + size);
 
-        dt_child_set_range(&cells, parent, start, size);
+        dt_child_set_range(&cells, addrcells, sizecells, start, size);
     }
 
     res = fdt_property(fdt, "reg", reg, sizeof(reg));
@@ -579,7 +579,7 @@ static void evtchn_allocate(struct domain *d);
 
 static int make_hypervisor_node(struct domain *d,
                                 const struct kernel_info *kinfo,
-                                const struct dt_device_node *parent)
+                                int addrcells, int sizecells)
 {
     const char compat[] =
         "xen,xen-"__stringify(XEN_VERSION)"."__stringify(XEN_SUBVERSION)"\0"
@@ -588,9 +588,6 @@ static int make_hypervisor_node(struct domain *d,
     gic_interrupt_t intr;
     __be32 *cells;
     int res;
-    /* Convenience alias */
-    int addrcells = dt_child_n_addr_cells(parent);
-    int sizecells = dt_child_n_size_cells(parent);
     void *fdt = kinfo->fdt;
 
     dt_dprintk("Create hypervisor node\n");
@@ -615,7 +612,8 @@ static int make_hypervisor_node(struct domain *d,
 
     /* reg 0 is grant table space */
     cells = &reg[0];
-    dt_child_set_range(&cells, parent, kinfo->gnttab_start, kinfo->gnttab_size);
+    dt_child_set_range(&cells, addrcells, sizecells,
+                       kinfo->gnttab_start, kinfo->gnttab_size);
     res = fdt_property(fdt, "reg", reg,
                        dt_cells_to_size(addrcells + sizecells));
     if ( res )
@@ -1292,11 +1290,13 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
 
     if ( node == dt_host )
     {
+        int addrcells = dt_child_n_addr_cells(node);
+        int sizecells = dt_child_n_size_cells(node);
         /*
          * The hypervisor node should always be created after all nodes
          * from the host DT have been parsed.
          */
-        res = make_hypervisor_node(d, kinfo, node);
+        res = make_hypervisor_node(d, kinfo, addrcells, sizecells);
         if ( res )
             return res;
 
@@ -1308,7 +1308,7 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
         if ( res )
             return res;
 
-        res = make_memory_node(d, kinfo->fdt, node, kinfo);
+        res = make_memory_node(d, kinfo->fdt, addrcells, sizecells, kinfo);
         if ( res )
             return res;
 
diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index 7b009ea..8fc401d 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -112,11 +112,11 @@ void dt_set_range(__be32 **cellp, const struct dt_device_node *np,
     dt_set_cell(cellp, dt_n_size_cells(np), size);
 }
 
-void dt_child_set_range(__be32 **cellp, const struct dt_device_node *parent,
+void dt_child_set_range(__be32 **cellp, int addrcells, int sizecells,
                         u64 address, u64 size)
 {
-    dt_set_cell(cellp, dt_child_n_addr_cells(parent), address);
-    dt_set_cell(cellp, dt_child_n_size_cells(parent), size);
+    dt_set_cell(cellp, addrcells, address);
+    dt_set_cell(cellp, sizecells, size);
 }
 
 static void __init *unflatten_dt_alloc(unsigned long *mem, unsigned long size,
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index 0aecbe0..01040a6 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -699,7 +699,7 @@ void dt_set_range(__be32 **cellp, const struct dt_device_node *np,
  * Write a range into a series of cells and update cellp to point to the
  * cell just after.
  */
-void dt_child_set_range(__be32 **cellp, const struct dt_device_node *parent,
+void dt_child_set_range(__be32 **cellp, int addrcells, int sizecells,
                         u64 address, u64 size);
 
 /**
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH RFC 05/15] xen/arm: rename acpi_make_chosen_node to make_chosen_node

[Stefano Stabellini]

acpi_make_chosen_node is actually generic and can be reused. Rename it
to make_chosen_node and make it available to non-ACPI builds.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
---
 xen/arch/arm/domain_build.c | 84 ++++++++++++++++++++++-----------------------
 1 file changed, 42 insertions(+), 42 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index bb88e09..4e4cd19 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -935,6 +935,47 @@ static int make_timer_node(const struct domain *d, void *fdt,
     return res;
 }
 
+static int make_chosen_node(const struct kernel_info *kinfo)
+{
+    int res;
+    const char *bootargs = NULL;
+    const struct bootmodule *mod = kinfo->kernel_bootmodule;
+    void *fdt = kinfo->fdt;
+
+    dt_dprintk("Create chosen node\n");
+    res = fdt_begin_node(fdt, "chosen");
+    if ( res )
+        return res;
+
+    if ( mod && mod->cmdline[0] )
+    {
+        bootargs = &mod->cmdline[0];
+        res = fdt_property(fdt, "bootargs", bootargs, strlen(bootargs) + 1);
+        if ( res )
+           return res;
+    }
+
+    /*
+     * If the bootloader provides an initrd, we must create a placeholder
+     * for the initrd properties. The values will be replaced later.
+     */
+    if ( mod && mod->size )
+    {
+        u64 a = 0;
+        res = fdt_property(kinfo->fdt, "linux,initrd-start", &a, sizeof(a));
+        if ( res )
+            return res;
+
+        res = fdt_property(kinfo->fdt, "linux,initrd-end", &a, sizeof(a));
+        if ( res )
+            return res;
+    }
+
+    res = fdt_end_node(fdt);
+
+    return res;
+}
+
 static int map_irq_to_domain(struct domain *d, unsigned int irq,
                              bool need_mapping, const char *devname)
 
@@ -1420,47 +1461,6 @@ static int acpi_route_spis(struct domain *d)
     return 0;
 }
 
-static int acpi_make_chosen_node(const struct kernel_info *kinfo)
-{
-    int res;
-    const char *bootargs = NULL;
-    const struct bootmodule *mod = kinfo->kernel_bootmodule;
-    void *fdt = kinfo->fdt;
-
-    dt_dprintk("Create chosen node\n");
-    res = fdt_begin_node(fdt, "chosen");
-    if ( res )
-        return res;
-
-    if ( mod && mod->cmdline[0] )
-    {
-        bootargs = &mod->cmdline[0];
-        res = fdt_property(fdt, "bootargs", bootargs, strlen(bootargs) + 1);
-        if ( res )
-           return res;
-    }
-
-    /*
-     * If the bootloader provides an initrd, we must create a placeholder
-     * for the initrd properties. The values will be replaced later.
-     */
-    if ( mod && mod->size )
-    {
-        u64 a = 0;
-        res = fdt_property(kinfo->fdt, "linux,initrd-start", &a, sizeof(a));
-        if ( res )
-            return res;
-
-        res = fdt_property(kinfo->fdt, "linux,initrd-end", &a, sizeof(a));
-        if ( res )
-            return res;
-    }
-
-    res = fdt_end_node(fdt);
-
-    return res;
-}
-
 static int acpi_make_hypervisor_node(const struct kernel_info *kinfo,
                                      struct membank tbl_add[])
 {
@@ -1532,7 +1532,7 @@ static int create_acpi_dtb(struct kernel_info *kinfo, struct membank tbl_add[])
         return ret;
 
     /* Create a chosen node for DOM0 */
-    ret = acpi_make_chosen_node(kinfo);
+    ret = make_chosen_node(kinfo);
     if ( ret )
         goto err;
 
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH RFC 00/15] dom0less step1: boot multiple domains from device tree

[Stefano Stabellini]

Hi all,

This is first step toward "dom0less" as discussed in the various
certifications related threads and discussions.

The goal of this series is to enable Xen to boot multiple domains in
parallel, in addition to dom0, out of information found on device tree.

The device tree based boot protocol is extended to carry information
about DomUs. Based on that information, Xen creates and starts one or
more DomUs. DomUs created this way don't have access to xenstore, as
xenstore is not started yet. This is actually OK, because this is meant
for mission critical applications that typically only access directly
assigned devices. They cannot tolerate interference or increased IRQ
latency due to PV protocols. Device assignment is not actually covered
by this series, it will be added later.

DomUs can print to the Xen serial using the CONSOLEIO hypercalls. A
virtual PL011 is also emulated for them so that they can use their
regular PL011 driver. This allows unmodified guests to run as Xen on ARM
guests -- no Xen support needed at all. Console input also comes from
the Xen serial: the Ctrl-AAA switching mechanism is extended to switch
among domUs, dom0, and Xen.

Cheers,

Stefano


Stefano Stabellini (15):
      xen: allow console_io hypercalls from DomUs on ARM
      xen/arm: move a few guest related #defines to public/arch-arm.h
      xen/arm: extend device tree based multiboot protocol
      xen/arm: do not pass dt_host to make_memory_node and make_hypervisor_node
      xen/arm: rename acpi_make_chosen_node to make_chosen_node
      xen/arm: add BOOTMOD_DOMU_KERNEL/RAMDISK
      xen/arm: increase MAX_MODULES
      xen/arm: probe domU kernels and initrds
      xen/arm: refactor construct_dom0
      xen/arm: introduce construct_domU
      xen/arm: generate a simple device tree for domUs
      xen/arm: generate vpl011 node on device tree for domU
      xen/arm: Allow vpl011 to be used by DomU
      xen/arm: call construct_domU from start_xen and start DomU VMs
      xen: support console_switching between Dom0 and DomUs on ARM

 docs/misc/arm/device-tree/booting.txt | 102 +++++++
 tools/libxl/libxl_arm.c               |  26 --
 xen/arch/arm/bootfdt.c                |   4 +
 xen/arch/arm/domain_build.c           | 533 +++++++++++++++++++++++++++-------
 xen/arch/arm/kernel.c                 |  54 ++++
 xen/arch/arm/kernel.h                 |   2 +
 xen/arch/arm/setup.c                  |  52 +++-
 xen/arch/arm/vpl011.c                 |  98 +++++--
 xen/common/device_tree.c              |   6 +-
 xen/drivers/char/console.c            |  51 +++-
 xen/include/asm-arm/setup.h           |  10 +-
 xen/include/asm-arm/vpl011.h          |   2 +
 xen/include/asm-x86/setup.h           |   2 +
 xen/include/public/arch-arm.h         |  26 ++
 xen/include/xen/device_tree.h         |   2 +-
 15 files changed, 789 insertions(+), 181 deletions(-)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH RFC 06/15] xen/arm: add BOOTMOD_DOMU_KERNEL/RAMDISK

[Stefano Stabellini]

Introduce bootmod types for domU kernels and initrds.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
---
 xen/arch/arm/bootfdt.c      | 4 ++++
 xen/arch/arm/setup.c        | 2 ++
 xen/include/asm-arm/setup.h | 2 ++
 3 files changed, 8 insertions(+)

diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
index 8eba42c..6c88ec4 100644
--- a/xen/arch/arm/bootfdt.c
+++ b/xen/arch/arm/bootfdt.c
@@ -194,6 +194,10 @@ static void __init process_multiboot_node(const void *fdt, int node,
         kind = BOOTMOD_RAMDISK;
     else if ( fdt_node_check_compatible(fdt, node, "xen,xsm-policy") == 0 )
         kind = BOOTMOD_XSM;
+    else if ( fdt_node_check_compatible(fdt, node, "multiboot,domU-kernel") == 0 )
+        kind = BOOTMOD_DOMU_KERNEL;
+    else if ( fdt_node_check_compatible(fdt, node, "multiboot,domU-ramdisk") == 0 )
+        kind = BOOTMOD_DOMU_RAMDISK;
     else
         kind = BOOTMOD_UNKNOWN;
 
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 1d6f6bf..82593c8 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -249,6 +249,8 @@ const char * __init boot_module_kind_as_string(bootmodule_kind kind)
     case BOOTMOD_FDT:     return "Device Tree";
     case BOOTMOD_KERNEL:  return "Kernel";
     case BOOTMOD_RAMDISK: return "Ramdisk";
+    case BOOTMOD_DOMU_KERNEL:  return "DomU Kernel";
+    case BOOTMOD_DOMU_RAMDISK: return "DomU Ramdisk";
     case BOOTMOD_XSM:     return "XSM";
     case BOOTMOD_UNKNOWN: return "Unknown";
     default: BUG();
diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
index 0cc3330..e190451 100644
--- a/xen/include/asm-arm/setup.h
+++ b/xen/include/asm-arm/setup.h
@@ -16,6 +16,8 @@ typedef enum {
     BOOTMOD_KERNEL,
     BOOTMOD_RAMDISK,
     BOOTMOD_XSM,
+    BOOTMOD_DOMU_KERNEL,
+    BOOTMOD_DOMU_RAMDISK,
     BOOTMOD_UNKNOWN
 }  bootmodule_kind;
 
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH RFC 07/15] xen/arm: increase MAX_MODULES

[Stefano Stabellini]

Xen boot modules need to account not just for Dom0 but also for a few
potential DomUs, each of them coming with their own kernel and initrd.
Increase MAX_MODULES to 32 to allow for more DomUs.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
---
 xen/include/asm-arm/setup.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
index e190451..86aac0e 100644
--- a/xen/include/asm-arm/setup.h
+++ b/xen/include/asm-arm/setup.h
@@ -8,7 +8,7 @@
 
 #define NR_MEM_BANKS 128
 
-#define MAX_MODULES 5 /* Current maximum useful modules */
+#define MAX_MODULES 32 /* Current maximum useful modules */
 
 typedef enum {
     BOOTMOD_XEN,
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH RFC 08/15] xen/arm: probe domU kernels and initrds

[Stefano Stabellini]

Find addresses and sizes on device tree.
Introduce a new boot_module_find_by_addr_and_kind function to match not
just on boot module kind, but also by address so that we can support
multiple domUs.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
---
 xen/arch/arm/kernel.c       | 54 +++++++++++++++++++++++++++++++++++++++++++++
 xen/arch/arm/kernel.h       |  2 ++
 xen/arch/arm/setup.c        | 15 +++++++++++++
 xen/include/asm-arm/setup.h |  2 ++
 4 files changed, 73 insertions(+)

diff --git a/xen/arch/arm/kernel.c b/xen/arch/arm/kernel.c
index 8fdfd91..c41092e 100644
--- a/xen/arch/arm/kernel.c
+++ b/xen/arch/arm/kernel.c
@@ -496,6 +496,60 @@ err:
     return rc;
 }
 
+int kernel_probe_domU(struct kernel_info *info, struct dt_device_node *domU) 
+{
+    struct bootmodule *mod;
+    struct dt_device_node *node;
+    u64 kernel_addr, initrd_addr, size;
+    int rc;
+
+    dt_for_each_child_node(domU, node)
+    {
+        if ( dt_device_is_compatible(node, "multiboot,domU-kernel") )
+        {
+            u32 len;
+            const __be32 *val;
+            val = dt_get_property(node, "reg", &len);
+            dt_get_range(&val, node, &kernel_addr, &size);
+        }
+        else if ( dt_device_is_compatible(node, "multiboot,domU-ramdisk") )
+        {
+            u32 len;
+            const __be32 *val;
+            val = dt_get_property(node, "reg", &len);
+            dt_get_range(&val, node, &initrd_addr, &size);
+        }
+        else
+            continue;
+    }
+    info->kernel_bootmodule = mod = boot_module_find_by_addr_and_kind(
+									    BOOTMOD_DOMU_KERNEL, kernel_addr);
+    info->initrd_bootmodule = boot_module_find_by_addr_and_kind(
+							      BOOTMOD_DOMU_RAMDISK, initrd_addr);
+    printk("Loading DomU kernel from boot module @ %"PRIpaddr"\n",
+		   info->kernel_bootmodule->start);
+    if ( info->initrd_bootmodule )
+        printk("Loading ramdisk from boot module @ %"PRIpaddr"\n",
+               info->initrd_bootmodule->start);
+
+    /* if it is a gzip'ed image, 32bit or 64bit, uncompress it */
+    rc = kernel_decompress(mod);
+    if (rc < 0 && rc != -EINVAL)
+        return rc;
+
+#ifdef CONFIG_ARM_64
+    rc = kernel_zimage64_probe(info, mod->start, mod->size);
+    if (rc < 0)
+#endif
+        rc = kernel_uimage_probe(info, mod->start, mod->size);
+    if (rc < 0)
+        rc = kernel_zimage32_probe(info, mod->start, mod->size);
+    if (rc < 0)
+        rc = kernel_elf_probe(info, mod->start, mod->size);
+
+    return rc;
+}
+
 int kernel_probe(struct kernel_info *info)
 {
     struct bootmodule *mod = boot_module_find_by_kind(BOOTMOD_KERNEL);
diff --git a/xen/arch/arm/kernel.h b/xen/arch/arm/kernel.h
index 6d69509..8e1614b 100644
--- a/xen/arch/arm/kernel.h
+++ b/xen/arch/arm/kernel.h
@@ -64,6 +64,8 @@ struct kernel_info {
  */
 int kernel_probe(struct kernel_info *info);
 
+int kernel_probe_domU(struct kernel_info *info, struct dt_device_node *node);
+
 /*
  * Loads the kernel into guest RAM.
  *
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 82593c8..98bdb24 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -241,6 +241,21 @@ struct bootmodule * __init boot_module_find_by_kind(bootmodule_kind kind)
     return NULL;
 }
 
+struct bootmodule * __init boot_module_find_by_addr_and_kind(bootmodule_kind kind,
+                                                             paddr_t start)
+{
+    struct bootmodules *mods = &bootinfo.modules;
+    struct bootmodule *mod;
+    int i;
+    for (i = 0 ; i < mods->nr_mods ; i++ )
+    {
+        mod = &mods->module[i];
+        if ( mod->kind == kind && mod->start == start )
+            return mod;
+    }
+    return NULL;
+}
+
 const char * __init boot_module_kind_as_string(bootmodule_kind kind)
 {
     switch ( kind )
diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
index 86aac0e..903782f 100644
--- a/xen/include/asm-arm/setup.h
+++ b/xen/include/asm-arm/setup.h
@@ -83,6 +83,8 @@ struct bootmodule *add_boot_module(bootmodule_kind kind,
                                    paddr_t start, paddr_t size,
                                    const char *cmdline);
 struct bootmodule *boot_module_find_by_kind(bootmodule_kind kind);
+struct bootmodule * __init boot_module_find_by_addr_and_kind(bootmodule_kind kind,
+                                                             paddr_t start);
 const char * __init boot_module_kind_as_string(bootmodule_kind kind);
 
 #endif
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH RFC 09/15] xen/arm: refactor construct_dom0

[Stefano Stabellini]

Move generic initializations out of construct_dom0 so that they can be
reused.

No functional changes in this patch.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
---
 xen/arch/arm/domain_build.c | 124 ++++++++++++++++++++++++--------------------
 1 file changed, 67 insertions(+), 57 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 4e4cd19..b31c563 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2092,73 +2092,27 @@ static void __init find_gnttab_region(struct domain *d,
            kinfo->gnttab_start, kinfo->gnttab_start + kinfo->gnttab_size);
 }
 
-int __init construct_dom0(struct domain *d)
+int __init __construct_domain(struct domain *d, struct kernel_info *kinfo)
 {
-    struct kernel_info kinfo = {};
     struct vcpu *saved_current;
-    int rc, i, cpu;
+    int i, cpu;
 
     struct vcpu *v = d->vcpu[0];
     struct cpu_user_regs *regs = &v->arch.cpu_info->guest_cpu_user_regs;
 
-    /* Sanity! */
-    BUG_ON(d->domain_id != 0);
-    BUG_ON(d->vcpu[0] == NULL);
-    BUG_ON(v->is_initialised);
-
-    printk("*** LOADING DOMAIN 0 ***\n");
-    if ( dom0_mem <= 0 )
-    {
-        warning_add("PLEASE SPECIFY dom0_mem PARAMETER - USING 512M FOR NOW\n");
-        dom0_mem = MB(512);
-    }
-
-
-    iommu_hwdom_init(d);
-
-    d->max_pages = ~0U;
-
-    kinfo.unassigned_mem = dom0_mem;
-    kinfo.d = d;
-
-    rc = kernel_probe(&kinfo);
-    if ( rc < 0 )
-        return rc;
-
 #ifdef CONFIG_ARM_64
     /* if aarch32 mode is not supported at EL1 do not allow 32-bit domain */
-    if ( !(cpu_has_el1_32) && kinfo.type == DOMAIN_32BIT )
+    if ( !(cpu_has_el1_32) && kinfo->type == DOMAIN_32BIT )
     {
         printk("Platform does not support 32-bit domain\n");
         return -EINVAL;
     }
-    d->arch.type = kinfo.type;
 
     if ( is_64bit_domain(d) )
         vcpu_switch_to_aarch64_mode(v);
 
 #endif
 
-    allocate_memory(d, &kinfo);
-    find_gnttab_region(d, &kinfo);
-
-    /* Map extra GIC MMIO, irqs and other hw stuffs to dom0. */
-    rc = gic_map_hwdom_extra_mappings(d);
-    if ( rc < 0 )
-        return rc;
-
-    rc = platform_specific_mapping(d);
-    if ( rc < 0 )
-        return rc;
-
-    if ( acpi_disabled )
-        rc = prepare_dtb(d, &kinfo);
-    else
-        rc = prepare_acpi(d, &kinfo);
-
-    if ( rc < 0 )
-        return rc;
-
     /*
      * The following loads use the domain's p2m and require current to
      * be a vcpu of the domain, temporarily switch
@@ -2171,20 +2125,18 @@ int __init construct_dom0(struct domain *d)
      * kernel_load will determine the placement of the kernel as well
      * as the initrd & fdt in RAM, so call it first.
      */
-    kernel_load(&kinfo);
+    kernel_load(kinfo);
     /* initrd_load will fix up the fdt, so call it before dtb_load */
-    initrd_load(&kinfo);
-    dtb_load(&kinfo);
+    initrd_load(kinfo);
+    dtb_load(kinfo);
 
     /* Now that we are done restore the original p2m and current. */
     set_current(saved_current);
     p2m_restore_state(saved_current);
 
-    discard_initial_modules();
-
     memset(regs, 0, sizeof(*regs));
 
-    regs->pc = (register_t)kinfo.entry;
+    regs->pc = (register_t)kinfo->entry;
 
     if ( is_32bit_domain(d) )
     {
@@ -2202,14 +2154,14 @@ int __init construct_dom0(struct domain *d)
          */
         regs->r0 = 0; /* SBZ */
         regs->r1 = 0xffffffff; /* We use DTB therefore no machine id */
-        regs->r2 = kinfo.dtb_paddr;
+        regs->r2 = kinfo->dtb_paddr;
     }
 #ifdef CONFIG_ARM_64
     else
     {
         regs->cpsr = PSR_GUEST64_INIT;
         /* From linux/Documentation/arm64/booting.txt */
-        regs->x0 = kinfo.dtb_paddr;
+        regs->x0 = kinfo->dtb_paddr;
         regs->x1 = 0; /* Reserved for future use */
         regs->x2 = 0; /* Reserved for future use */
         regs->x3 = 0; /* Reserved for future use */
@@ -2235,6 +2187,64 @@ int __init construct_dom0(struct domain *d)
     return 0;
 }
 
+int __init construct_dom0(struct domain *d)
+{
+    struct kernel_info kinfo = {};
+    int rc;
+
+    struct vcpu *v = d->vcpu[0];
+
+    /* Sanity! */
+    BUG_ON(d->domain_id != 0);
+    BUG_ON(d->vcpu[0] == NULL);
+    BUG_ON(v->is_initialised);
+
+    printk("*** LOADING DOMAIN 0 ***\n");
+    if ( dom0_mem <= 0 )
+    {
+        warning_add("PLEASE SPECIFY dom0_mem PARAMETER - USING 512M FOR NOW\n");
+        dom0_mem = MB(512);
+    }
+
+
+    iommu_hwdom_init(d);
+
+    d->max_pages = ~0U;
+
+    kinfo.unassigned_mem = dom0_mem;
+    kinfo.d = d;
+
+    rc = kernel_probe(&kinfo);
+    if ( rc < 0 )
+        return rc;
+
+    allocate_memory(d, &kinfo);
+    find_gnttab_region(d, &kinfo);
+
+    /* Map extra GIC MMIO, irqs and other hw stuffs to dom0. */
+    rc = gic_map_hwdom_extra_mappings(d);
+    if ( rc < 0 )
+        return rc;
+
+    rc = platform_specific_mapping(d);
+    if ( rc < 0 )
+        return rc;
+
+    d->arch.type = kinfo.type;
+
+    if ( acpi_disabled )
+        rc = prepare_dtb(d, &kinfo);
+    else
+        rc = prepare_acpi(d, &kinfo);
+
+    if ( rc < 0 )
+        return rc;
+
+    discard_initial_modules();
+
+    return __construct_domain(d, &kinfo);
+}
+
 /*
  * Local variables:
  * mode: C
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH RFC 10/15] xen/arm: introduce construct_domU

[Stefano Stabellini]

Similar to construct_dom0, construct_domU creates a barebone DomU guest.
Default to 1 max vcpu and 64MB of memory if not specified otherwise.

The device tree node passed as argument is compatible "xen,domU", see
docs/misc/arm/device-tree/booting.txt.

Allocate all vcpus on cpu0 initially.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
---
 xen/arch/arm/domain_build.c | 37 +++++++++++++++++++++++++++++++++++++
 xen/include/asm-arm/setup.h |  2 ++
 2 files changed, 39 insertions(+)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index b31c563..02a7f94 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2187,6 +2187,43 @@ int __init __construct_domain(struct domain *d, struct kernel_info *kinfo)
     return 0;
 }
 
+int __init construct_domU(struct domain *d, struct dt_device_node *node)
+{
+    struct kernel_info kinfo = {};
+    int rc;
+    const char *cpus = NULL, *mem = NULL;
+
+    printk("*** LOADING DOMU ***\n");
+
+    d->max_vcpus = 1;
+    rc = dt_property_read_string(node, "cpus", &cpus);
+    if ( !rc )
+        d->max_vcpus = simple_strtoul(cpus, &cpus, 0);
+
+    kinfo.unassigned_mem = MB(64);
+    rc = dt_property_read_string(node, "mem", &mem);
+    if ( !rc )
+        kinfo.unassigned_mem = parse_size_and_unit(mem, &mem);
+
+    d->vcpu = xzalloc_array(struct vcpu *, d->max_vcpus);
+    if ( !d->vcpu )
+        return -ENOMEM;;
+    if ( alloc_vcpu(d, 0, 0) == NULL )
+        return -ENOMEM;
+    d->max_pages = ~0U;
+
+    kinfo.d = d;
+
+    rc = kernel_probe_domU(&kinfo, node);
+    if ( rc < 0 )
+        return rc;
+
+    d->arch.type = kinfo.type;
+    allocate_memory(d, &kinfo);
+
+    return __construct_domain(d, &kinfo);
+}
+
 int __init construct_dom0(struct domain *d)
 {
     struct kernel_info kinfo = {};
diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
index 903782f..e9f9905 100644
--- a/xen/include/asm-arm/setup.h
+++ b/xen/include/asm-arm/setup.h
@@ -2,6 +2,7 @@
 #define __ARM_SETUP_H_
 
 #include <public/version.h>
+#include <xen/device_tree.h>
 
 #define MIN_FDT_ALIGN 8
 #define MAX_FDT_SIZE SZ_2M
@@ -71,6 +72,7 @@ void acpi_create_efi_mmap_table(struct domain *d,
 int acpi_make_efi_nodes(void *fdt, struct membank tbl_add[]);
 
 int construct_dom0(struct domain *d);
+int construct_domU(struct domain *d, struct dt_device_node *node);
 
 void discard_initial_modules(void);
 void dt_unreserved_regions(paddr_t s, paddr_t e,
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH RFC 11/15] xen/arm: generate a simple device tree for domUs

[Stefano Stabellini]

Introduce functions to generate a basic domU device tree, similar to the
existing functions in tools/libxl/libxl_arm.c.

Rename existing prepare_dtb to prepare_dtb_dom0 to avoid confusion.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
---
 xen/arch/arm/domain_build.c | 195 +++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 193 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 02a7f94..b4f560f 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1360,7 +1360,194 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
     return res;
 }
 
-static int prepare_dtb(struct domain *d, struct kernel_info *kinfo)
+static int make_gic_domU_node(const struct domain *d, void *fdt, int addrcells, int sizecells)
+{
+    int res = 0;
+    int reg_size = addrcells + sizecells;
+    int nr_cells = reg_size * 2;
+    __be32 reg[nr_cells];
+    __be32 *cells;
+
+    res = fdt_begin_node(fdt, "interrupt-controller");
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "#address-cells", 0);
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "#interrupt-cells", 3);
+    if ( res )
+        return res;
+
+    res = fdt_property(fdt, "interrupt-controller", NULL, 0);
+    if ( res )
+        return res;
+
+    if (gic_hw_version() == GIC_V3)
+    {
+        const uint64_t gicd_base = GUEST_GICV3_GICD_BASE;
+        const uint64_t gicd_size = GUEST_GICV3_GICD_SIZE;
+        const uint64_t gicr0_base = GUEST_GICV3_GICR0_BASE;
+        const uint64_t gicr0_size = GUEST_GICV3_GICR0_SIZE;
+
+        res = fdt_property_string(fdt, "compatible", "arm,gic-v3");
+        if ( res )
+            return res;
+
+        cells = &reg[0];
+        dt_child_set_range(&cells, addrcells, sizecells, gicd_base, gicd_size);
+        dt_child_set_range(&cells, addrcells, sizecells, gicr0_base, gicr0_size);
+        res = fdt_property(fdt, "reg", reg, sizeof(reg));
+    }
+    else if (gic_hw_version() == GIC_V3)
+    {
+        const uint64_t gicd_base = GUEST_GICD_BASE;
+        const uint64_t gicd_size = GUEST_GICD_SIZE;
+        const uint64_t gicc_base = GUEST_GICC_BASE;
+        const uint64_t gicc_size = GUEST_GICC_SIZE;
+
+        res = fdt_property_string(fdt, "compatible", "arm,cortex-a15-gic");
+        if ( res )
+            return res;
+
+        cells = &reg[0];
+        dt_child_set_range(&cells, addrcells, sizecells, gicd_base, gicd_size);
+        dt_child_set_range(&cells, addrcells, sizecells, gicc_base, gicc_size);
+    }
+
+    res = fdt_property(fdt, "reg", reg, sizeof(reg));        
+    if (res)
+        return res;
+
+    res = fdt_property_cell(fdt, "linux,phandle", PHANDLE_GIC);
+    if (res)
+        return res;
+
+    res = fdt_property_cell(fdt, "phandle", PHANDLE_GIC);
+    if (res)
+        return res;
+
+    res = fdt_end_node(fdt);
+
+    return res;
+}
+
+static int make_timer_domU_node(const struct domain *d, void *fdt)
+{
+    int res;
+    gic_interrupt_t intrs[3];
+
+    res = fdt_begin_node(fdt, "timer");
+    if ( res )
+        return res;
+
+    if (!is_64bit_domain(d))
+    {
+        res = fdt_property_string(fdt, "compatible", "arm,armv7-timer");
+        if ( res )
+            return res;
+    } else {
+        res = fdt_property_string(fdt, "compatible", "arm,armv8-timer");
+        if ( res )
+            return res;
+    }
+
+    set_interrupt_ppi(intrs[0], GUEST_TIMER_PHYS_S_PPI, 0xf, DT_IRQ_TYPE_LEVEL_LOW);
+    set_interrupt_ppi(intrs[1], GUEST_TIMER_PHYS_NS_PPI, 0xf, DT_IRQ_TYPE_LEVEL_LOW);
+    set_interrupt_ppi(intrs[2], GUEST_TIMER_VIRT_PPI, 0xf, DT_IRQ_TYPE_LEVEL_LOW);
+
+    res = fdt_property(fdt, "interrupts", intrs, sizeof (intrs[0]) * 3);
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "interrupt-parent",
+                            PHANDLE_GIC);
+    if (res)
+        return res;
+
+    res = fdt_end_node(fdt);
+    return res;
+}
+
+#define DOMU_DTB_SIZE 4096
+static int prepare_dtb_domU(struct domain *d, struct kernel_info *kinfo)
+{
+    int addrcells, sizecells;
+    int ret;
+
+    addrcells = dt_child_n_addr_cells(dt_host);
+    sizecells = dt_child_n_size_cells(dt_host);
+
+    kinfo->fdt = xmalloc_bytes(DOMU_DTB_SIZE);
+    if ( kinfo->fdt == NULL )
+        return -ENOMEM;
+
+    ret = fdt_create(kinfo->fdt, DOMU_DTB_SIZE);
+    if ( ret < 0 )
+        goto err;
+
+    ret = fdt_finish_reservemap(kinfo->fdt);
+    if ( ret < 0 )
+        goto err;
+
+    ret = fdt_begin_node(kinfo->fdt, "/");
+    if ( ret < 0 )
+        goto err;
+
+    ret = fdt_property_cell(kinfo->fdt, "#address-cells", addrcells);
+    if ( ret )
+        goto err;
+
+    ret = fdt_property_cell(kinfo->fdt, "#size-cells", sizecells);
+    if ( ret )
+        goto err;
+
+    ret = make_chosen_node(kinfo);
+    if ( ret )
+        goto err;
+
+    ret = make_hypervisor_node(d, kinfo, addrcells, sizecells);
+    if ( ret )
+        goto err;
+
+    ret = make_psci_node(kinfo->fdt, NULL);
+    if ( ret )
+        goto err;
+
+    ret = make_cpus_node(d, kinfo->fdt, NULL);
+    if ( ret )
+        goto err;
+
+    ret = make_memory_node(d, kinfo->fdt, addrcells, sizecells, kinfo);
+    if ( ret )
+        goto err;
+
+    ret = make_gic_domU_node(d, kinfo->fdt, addrcells, sizecells);
+    if ( ret )
+        goto err;
+
+    ret = make_timer_domU_node(d, kinfo->fdt);
+    if ( ret )
+        goto err;
+
+    ret = fdt_end_node(kinfo->fdt);
+    if ( ret < 0 )
+        goto err;
+
+    ret = fdt_finish(kinfo->fdt);
+    if ( ret < 0 )
+        goto err;
+
+    return 0;
+
+  err:
+    printk("Device tree generation failed (%d).\n", ret);
+    xfree(kinfo->fdt);
+    return -EINVAL;
+}
+
+static int prepare_dtb_dom0(struct domain *d, struct kernel_info *kinfo)
 {
     const p2m_type_t default_p2mt = p2m_mmio_direct_c;
     const void *fdt;
@@ -2221,6 +2408,10 @@ int __init construct_domU(struct domain *d, struct dt_device_node *node)
     d->arch.type = kinfo.type;
     allocate_memory(d, &kinfo);
 
+    rc = prepare_dtb_domU(d, &kinfo);
+    if ( rc < 0 )
+        return rc;
+
     return __construct_domain(d, &kinfo);
 }
 
@@ -2270,7 +2461,7 @@ int __init construct_dom0(struct domain *d)
     d->arch.type = kinfo.type;
 
     if ( acpi_disabled )
-        rc = prepare_dtb(d, &kinfo);
+        rc = prepare_dtb_dom0(d, &kinfo);
     else
         rc = prepare_acpi(d, &kinfo);
 
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH RFC 12/15] xen/arm: generate vpl011 node on device tree for domU

[Stefano Stabellini]

Introduce vpl011 support to guests started from Xen: it provides a
simple way to print output from a guest, as most guests come with a
pl011 driver. It is also able to provide a working console with
interrupt support.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
---
 xen/arch/arm/domain_build.c | 70 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 70 insertions(+)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index b4f560f..ff65057 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1470,6 +1470,70 @@ static int make_timer_domU_node(const struct domain *d, void *fdt)
     return res;
 }
 
+static void set_interrupt(gic_interrupt_t *interrupt, unsigned int irq,
+                          unsigned int cpumask, unsigned int level)
+{
+    __be32 *cells = (__be32 *) interrupt;
+    int is_ppi = (irq < 32);
+
+    irq -= (is_ppi) ? 16: 32; /* PPIs start at 16, SPIs at 32 */
+
+    /* See linux Documentation/devictree/bindings/arm/gic.txt */
+    dt_set_cell(&cells, 1, is_ppi); /* is a PPI? */
+    dt_set_cell(&cells, 1, irq);
+    dt_set_cell(&cells, 1, (cpumask << 8) | level);
+}
+
+#ifdef CONFIG_SBSA_VUART_CONSOLE
+static int make_vpl011_uart_node(const struct domain *d, void *fdt,
+                                 int addrcells, int sizecells)
+{
+    int res;
+    gic_interrupt_t intr;
+    int reg_size = addrcells + sizecells;
+    int nr_cells = reg_size;
+    __be32 reg[nr_cells];
+    __be32 *cells;
+
+    res = fdt_begin_node(fdt, "sbsa-pl011");
+    if (res)
+        return res;
+
+    res = fdt_property_string(fdt, "compatible", "arm,sbsa-uart");
+    if (res)
+        return res;
+
+    cells = &reg[0];
+    dt_child_set_range(&cells, addrcells, sizecells, GUEST_PL011_BASE,
+            GUEST_PL011_SIZE);
+    if (res)
+        return res;
+    res = fdt_property(fdt, "reg", reg, sizeof(reg));
+    if (res)
+        return res;
+
+    set_interrupt(&intr, GUEST_VPL011_SPI, 0xf, DT_IRQ_TYPE_LEVEL_HIGH);
+
+    res = fdt_property(fdt, "interrupts", intr, sizeof (intr));
+    if (res)
+        return res;
+
+    res = fdt_property_cell(fdt, "interrupt-parent",
+                            PHANDLE_GIC);
+    if (res)
+        return res;
+
+    /* Use a default baud rate of 115200. */
+    fdt_property_u32(fdt, "current-speed", 115200);
+
+    res = fdt_end_node(fdt);
+    if (res)
+        return res;
+
+    return 0;
+}
+#endif
+
 #define DOMU_DTB_SIZE 4096
 static int prepare_dtb_domU(struct domain *d, struct kernel_info *kinfo)
 {
@@ -1531,6 +1595,12 @@ static int prepare_dtb_domU(struct domain *d, struct kernel_info *kinfo)
     if ( ret )
         goto err;
 
+#ifdef CONFIG_SBSA_VUART_CONSOLE
+    ret = make_vpl011_uart_node(d, kinfo->fdt, addrcells, sizecells);
+    if ( ret )
+        goto err;
+#endif
+
     ret = fdt_end_node(kinfo->fdt);
     if ( ret < 0 )
         goto err;
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH RFC 13/15] xen/arm: Allow vpl011 to be used by DomU

[Stefano Stabellini]

Make vpl011 being able to be used without a userspace component in Dom0.
In that case, output is printed to the Xen serial and input is received
from the Xen serial one character at a time.

Call domain_vpl011_init during construct_domU.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
---
 xen/arch/arm/domain_build.c  |  9 +++-
 xen/arch/arm/vpl011.c        | 98 +++++++++++++++++++++++++++++++++-----------
 xen/include/asm-arm/vpl011.h |  2 +
 3 files changed, 84 insertions(+), 25 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index ff65057..97f14ca 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2482,7 +2482,14 @@ int __init construct_domU(struct domain *d, struct dt_device_node *node)
     if ( rc < 0 )
         return rc;
 
-    return __construct_domain(d, &kinfo);
+    rc = __construct_domain(d, &kinfo);
+    if ( rc < 0 )
+        return rc;
+
+#ifdef CONFIG_SBSA_VUART_CONSOLE
+    rc = domain_vpl011_init(d, NULL);
+#endif
+    return rc;
 }
 
 int __init construct_dom0(struct domain *d)
diff --git a/xen/arch/arm/vpl011.c b/xen/arch/arm/vpl011.c
index a281eab..5f1dc7a 100644
--- a/xen/arch/arm/vpl011.c
+++ b/xen/arch/arm/vpl011.c
@@ -34,6 +34,8 @@
 #include <asm/vgic-emul.h>
 #include <asm/vpl011.h>
 
+static void vpl011_data_avail(struct domain *d);
+
 /*
  * Since pl011 registers are 32-bit registers, all registers
  * are handled similarly allowing 8-bit, 16-bit and 32-bit
@@ -77,6 +79,29 @@ static void vpl011_update_interrupt_status(struct domain *d)
 #endif
 }
 
+void vpl011_read_char(struct domain *d, char c)
+{
+    unsigned long flags;
+    XENCONS_RING_IDX in_cons, in_prod;
+    struct xencons_interface *intf = d->arch.vpl011.ring_buf;
+
+    VPL011_LOCK(d, flags);
+
+    in_cons = intf->in_cons;
+    in_prod = intf->in_prod;
+    if (xencons_queued(in_prod, in_cons, sizeof(intf->in)) == sizeof(intf->in))
+    {
+        VPL011_UNLOCK(d, flags);
+        return;
+    }
+
+    intf->in[xencons_mask(in_prod, sizeof(intf->in))] = c;
+    intf->in_prod = in_prod + 1;
+
+    VPL011_UNLOCK(d, flags);
+    vpl011_data_avail(d);
+}
+
 static uint8_t vpl011_read_data(struct domain *d)
 {
     unsigned long flags;
@@ -166,9 +191,18 @@ static void vpl011_write_data(struct domain *d, uint8_t data)
     struct vpl011 *vpl011 = &d->arch.vpl011;
     struct xencons_interface *intf = vpl011->ring_buf;
     XENCONS_RING_IDX out_cons, out_prod;
+    unsigned int fifo_level = 0;
 
     VPL011_LOCK(d, flags);
 
+    if ( vpl011->ring_page == NULL )
+    {
+        printk("%c", data);
+        if (data == '\n')
+            printk("DOM%u: ", d->domain_id);
+        goto done;
+    }
+
     out_cons = intf->out_cons;
     out_prod = intf->out_prod;
 
@@ -184,13 +218,10 @@ static void vpl011_write_data(struct domain *d, uint8_t data)
     if ( xencons_queued(out_prod, out_cons, sizeof(intf->out)) !=
          sizeof (intf->out) )
     {
-        unsigned int fifo_level;
-
         intf->out[xencons_mask(out_prod, sizeof(intf->out))] = data;
         out_prod += 1;
         smp_wmb();
         intf->out_prod = out_prod;
-
         fifo_level = xencons_queued(out_prod, out_cons, sizeof(intf->out));
 
         if ( fifo_level == sizeof(intf->out) )
@@ -205,14 +236,15 @@ static void vpl011_write_data(struct domain *d, uint8_t data)
              */
             vpl011->uartfr |= BUSY;
         }
-
-        vpl011_update_tx_fifo_status(vpl011, fifo_level);
-
-        vpl011_update_interrupt_status(d);
     }
     else
         gprintk(XENLOG_ERR, "vpl011: Unexpected OUT ring buffer full\n");
 
+done:
+    vpl011_update_tx_fifo_status(vpl011, fifo_level);
+
+    vpl011_update_interrupt_status(d);
+
     vpl011->uartfr &= ~TXFE;
 
     VPL011_UNLOCK(d, flags);
@@ -462,13 +494,30 @@ int domain_vpl011_init(struct domain *d, struct vpl011_init_info *info)
     if ( vpl011->ring_buf )
         return -EINVAL;
 
-    /* Map the guest PFN to Xen address space. */
-    rc =  prepare_ring_for_helper(d,
-                                  gfn_x(info->gfn),
-                                  &vpl011->ring_page,
-                                  &vpl011->ring_buf);
-    if ( rc < 0 )
-        goto out;
+    if ( info != NULL )
+    {
+        /* Map the guest PFN to Xen address space. */
+        rc =  prepare_ring_for_helper(d,
+                gfn_x(info->gfn),
+                &vpl011->ring_page,
+                &vpl011->ring_buf);
+        if ( rc < 0 )
+            goto out;
+
+        rc = alloc_unbound_xen_event_channel(d, 0, info->console_domid,
+                vpl011_notification);
+        if ( rc < 0 )
+            goto out2;
+
+        vpl011->evtchn = info->evtchn = rc;
+    } else {
+        vpl011->ring_buf = xzalloc(struct xencons_interface);
+        if ( vpl011->ring_buf == NULL )
+        {
+            rc = -EINVAL;
+            goto out;
+        }
+    }
 
     rc = vgic_reserve_virq(d, GUEST_VPL011_SPI);
     if ( !rc )
@@ -477,13 +526,6 @@ int domain_vpl011_init(struct domain *d, struct vpl011_init_info *info)
         goto out1;
     }
 
-    rc = alloc_unbound_xen_event_channel(d, 0, info->console_domid,
-                                         vpl011_notification);
-    if ( rc < 0 )
-        goto out2;
-
-    vpl011->evtchn = info->evtchn = rc;
-
     spin_lock_init(&vpl011->lock);
 
     register_mmio_handler(d, &vpl011_mmio_handler,
@@ -495,7 +537,10 @@ out2:
     vgic_free_virq(d, GUEST_VPL011_SPI);
 
 out1:
-    destroy_ring_for_helper(&vpl011->ring_buf, vpl011->ring_page);
+    if ( vpl011->ring_page == NULL && vpl011->ring_buf != NULL )
+        xfree(vpl011->ring_buf);
+    else
+        destroy_ring_for_helper(&vpl011->ring_buf, vpl011->ring_page);
 
 out:
     return rc;
@@ -508,8 +553,13 @@ void domain_vpl011_deinit(struct domain *d)
     if ( !vpl011->ring_buf )
         return;
 
-    free_xen_event_channel(d, vpl011->evtchn);
-    destroy_ring_for_helper(&vpl011->ring_buf, vpl011->ring_page);
+    if ( vpl011->ring_page == NULL && vpl011->ring_buf != NULL )
+    {
+        xfree(vpl011->ring_buf);
+    } else {
+        free_xen_event_channel(d, vpl011->evtchn);
+        destroy_ring_for_helper(&vpl011->ring_buf, vpl011->ring_page);
+    }
 }
 
 /*
diff --git a/xen/include/asm-arm/vpl011.h b/xen/include/asm-arm/vpl011.h
index db95ff8..8d9b0da 100644
--- a/xen/include/asm-arm/vpl011.h
+++ b/xen/include/asm-arm/vpl011.h
@@ -53,6 +53,7 @@ struct vpl011_init_info {
 int domain_vpl011_init(struct domain *d,
                        struct vpl011_init_info *info);
 void domain_vpl011_deinit(struct domain *d);
+void vpl011_read_char(struct domain *d, char c);
 #else
 static inline int domain_vpl011_init(struct domain *d,
                                      struct vpl011_init_info *info)
@@ -61,6 +62,7 @@ static inline int domain_vpl011_init(struct domain *d,
 }
 
 static inline void domain_vpl011_deinit(struct domain *d) { }
+static inline void vpl011_read_char(struct domain *d, char c) { }
 #endif
 #endif  /* _VPL011_H_ */
 
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH RFC 14/15] xen/arm: call construct_domU from start_xen and start DomU VMs

[Stefano Stabellini]

Introduce support for the "xen,domU" compatible node on device tree.
Create new DomU VMs based on the information found on device tree under
"xen,domU".

Introduce a simple global variable named max_init_domid to keep track of
the initial allocated domids.

Move the discard_initial_modules after DomUs have been built

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
---
 xen/arch/arm/domain_build.c |  2 --
 xen/arch/arm/setup.c        | 35 ++++++++++++++++++++++++++++++++++-
 xen/include/asm-arm/setup.h |  2 ++
 xen/include/asm-x86/setup.h |  2 ++
 4 files changed, 38 insertions(+), 3 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 97f14ca..e2d370f 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2545,8 +2545,6 @@ int __init construct_dom0(struct domain *d)
     if ( rc < 0 )
         return rc;
 
-    discard_initial_modules();
-
     return __construct_domain(d, &kinfo);
 }
 
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 98bdb24..3723704 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -63,6 +63,8 @@ static unsigned long opt_xenheap_megabytes __initdata;
 integer_param("xenheap_megabytes", opt_xenheap_megabytes);
 #endif
 
+domid_t __read_mostly max_init_domid = 0;
+
 static __used void init_done(void)
 {
     free_init_memory();
@@ -711,6 +713,8 @@ void __init start_xen(unsigned long boot_phys_offset,
     struct bootmodule *xen_bootmodule;
     struct domain *dom0;
     struct xen_domctl_createdomain dom0_cfg = {};
+    struct dt_device_node *chosen;
+    struct dt_device_node *node;
 
     dcache_line_bytes = read_dcache_line_bytes();
 
@@ -860,7 +864,7 @@ void __init start_xen(unsigned long boot_phys_offset,
     dom0_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
     dom0_cfg.arch.nr_spis = gic_number_lines() - 32;
 
-    dom0 = domain_create(0, &dom0_cfg);
+    dom0 = domain_create(max_init_domid++, &dom0_cfg);
     if ( IS_ERR(dom0) || (alloc_dom0_vcpu0(dom0) == NULL) )
             panic("Error creating domain 0");
 
@@ -886,6 +890,35 @@ void __init start_xen(unsigned long boot_phys_offset,
 
     domain_unpause_by_systemcontroller(dom0);
 
+    chosen = dt_find_node_by_name(dt_host, "chosen");
+    if ( chosen != NULL )
+    {
+        dt_for_each_child_node(chosen, node)
+        {
+            struct domain *d;
+            struct xen_domctl_createdomain d_cfg = {};
+
+            if ( !dt_device_is_compatible(node, "xen,domU") )
+                continue;
+
+            d_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
+            d_cfg.arch.nr_spis = gic_number_lines() - 32;
+
+            d = domain_create(max_init_domid++, &d_cfg);
+            if ( IS_ERR(d))
+                panic("Error creating domU");
+
+            d->is_privileged = 0;
+            d->target = NULL;
+
+            if ( construct_domU(d, node) != 0)
+                printk("Could not set up DOMU guest OS");
+
+            domain_unpause_by_systemcontroller(d);
+        }
+    }
+    discard_initial_modules();
+
     /* Switch on to the dynamically allocated stack for the idle vcpu
      * since the static one we're running on is about to be freed. */
     memcpy(idle_vcpu[0]->arch.cpu_info, get_cpu_info(),
diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
index e9f9905..578f3b9 100644
--- a/xen/include/asm-arm/setup.h
+++ b/xen/include/asm-arm/setup.h
@@ -56,6 +56,8 @@ struct bootinfo {
 
 extern struct bootinfo bootinfo;
 
+extern domid_t max_init_domid;
+
 void arch_init_memory(void);
 
 void copy_from_paddr(void *dst, paddr_t paddr, unsigned long len);
diff --git a/xen/include/asm-x86/setup.h b/xen/include/asm-x86/setup.h
index 19232af..2fb9529 100644
--- a/xen/include/asm-x86/setup.h
+++ b/xen/include/asm-x86/setup.h
@@ -73,4 +73,6 @@ extern bool opt_dom0_shadow;
 #endif
 extern bool dom0_pvh;
 
+#define max_init_domid (1)
+
 #endif
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH RFC 15/15] xen: support console_switching between Dom0 and DomUs on ARM

[Stefano Stabellini]

Today Ctrl-AAA is used to switch between Xen and Dom0. Extend the
mechanism to allow for switching between Xen, Dom0, and any of the
initial DomU created from Xen alongside Dom0 out of information provided
via device tree.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
CC: andrew.cooper3@citrix.com
CC: George.Dunlap@eu.citrix.com
CC: ian.jackson@eu.citrix.com
CC: jbeulich@suse.com
CC: konrad.wilk@oracle.com
CC: tim@xen.org
CC: wei.liu2@citrix.com
---
 xen/drivers/char/console.c | 49 ++++++++++++++++++++++++++++++++--------------
 1 file changed, 34 insertions(+), 15 deletions(-)

diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
index dc9e0bb..07a89c6 100644
--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -31,10 +31,13 @@
 #include <xen/early_printk.h>
 #include <xen/warning.h>
 #include <xen/pv_console.h>
+#include <asm/setup.h>
 
 #ifdef CONFIG_X86
 #include <xen/consoled.h>
 #include <asm/guest.h>
+#else
+#include <asm/vpl011.h>
 #endif
 
 /* console: comma-separated list of console outputs. */
@@ -389,29 +392,48 @@ static void dump_console_ring_key(unsigned char key)
     free_xenheap_pages(buf, order);
 }
 
-/* CTRL-<switch_char> switches input direction between Xen and DOM0. */
+/*
+ * CTRL-<switch_char> switches input direction between Xen, Dom0 and
+ * DomUs.
+ */
 #define switch_code (opt_conswitch[0]-'a'+1)
-static int __read_mostly xen_rx = 1; /* FALSE => input passed to domain 0. */
+static int __read_mostly xen_rx = 1; /* 1 => input passed to domain 0. */
 
 static void switch_serial_input(void)
 {
-    static char *input_str[2] = { "DOM0", "Xen" };
-    xen_rx = !xen_rx;
-    printk("*** Serial input -> %s", input_str[xen_rx]);
+    xen_rx++;
+    if ( xen_rx == max_init_domid + 1 )
+        xen_rx = 0;
+
+    if ( !xen_rx )
+        printk("*** Serial input xen_rx=%d -> %s", xen_rx, "Xen");
+    else
+        printk("*** Serial input xen_rx=%d -> DOM%d", xen_rx, xen_rx - 1);
+
     if ( switch_code )
-        printk(" (type 'CTRL-%c' three times to switch input to %s)",
-               opt_conswitch[0], input_str[!xen_rx]);
+        printk(" (type 'CTRL-%c' three times to switch input)",
+               opt_conswitch[0]);
     printk("\n");
 }
 
 static void __serial_rx(char c, struct cpu_user_regs *regs)
 {
-    if ( xen_rx )
+    if ( xen_rx == 0 )
         return handle_keypress(c, regs);
 
-    /* Deliver input to guest buffer, unless it is already full. */
-    if ( (serial_rx_prod-serial_rx_cons) != SERIAL_RX_SIZE )
-        serial_rx_ring[SERIAL_RX_MASK(serial_rx_prod++)] = c;
+    if ( xen_rx == 1 )
+    {
+        /* Deliver input to guest buffer, unless it is already full. */
+        if ( (serial_rx_prod-serial_rx_cons) != SERIAL_RX_SIZE )
+            serial_rx_ring[SERIAL_RX_MASK(serial_rx_prod++)] = c;
+    }
+#ifdef CONFIG_ARM
+    else
+    {
+        struct domain *d = get_domain_by_id(xen_rx - 1);
+        vpl011_read_char(d, c);
+    }
+#endif
     /* Always notify the guest: prevents receive path from getting stuck. */
     send_global_virq(VIRQ_CONSOLE);
 
@@ -925,7 +947,7 @@ void __init console_endboot(void)
      * a useful 'how to switch' message.
      */
     if ( opt_conswitch[1] == 'x' )
-        xen_rx = !xen_rx;
+        xen_rx = 0;
 
     register_keyhandler('w', dump_console_ring_key,
                         "synchronously dump console ring buffer (dmesg)", 0);
@@ -935,9 +957,6 @@ void __init console_endboot(void)
                             "decrease log level threshold", 0);
     register_irq_keyhandler('G', &do_toggle_guest,
                             "toggle host/guest log level adjustment", 0);
-
-    /* Serial input is directed to DOM0 by default. */
-    switch_serial_input();
 }
 
 int __init console_has(const char *device)
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 01/15] xen: allow console_io hypercalls from DomUs on ARM

[Julien Grall]

Hi Stefano,

On 13/06/18 23:15, Stefano Stabellini wrote:
> This is very useful when starting multiple domains from Xen without
> xenstore access. It will allow them to print out to the Xen console.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> CC: andrew.cooper3@citrix.com
> CC: George.Dunlap@eu.citrix.com
> CC: ian.jackson@eu.citrix.com
> CC: jbeulich@suse.com
> CC: konrad.wilk@oracle.com
> CC: tim@xen.org
> CC: wei.liu2@citrix.com
> CC: dgdegra@tycho.nsa.gov
> ---
> If there is a better way to do this with XSM, please advise.

We definitely need to keep the XSM around to avoid opening a hole. We 
also don't want all the domain to access the console.

Looking at the implementation, any domain with is_privileged will be 
able to access the console. IHMO, I don't think we should set that for 
DomU created by Xen.

So I would suggest to introduce a new variable is_console and to tell 
whether a domain can access the console. xsm_console_io(...) would then 
need to be updated accordingly.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 02/15] xen/arm: move a few guest related #defines to public/arch-arm.h

[Julien Grall]

Hi Stefano,

On 13/06/18 23:15, Stefano Stabellini wrote:
> Move a few constants defined by libxl_arm.c to
> xen/include/public/arch-arm.h, so that they are together with the other
> guest related #defines such as GUEST_GICD_BASE and GUEST_VPL011_SPI.
> Also, this way they can be reused by hypervisor code.

All variables moved to arch-arm.h should be prefixed with GUEST_* to 
avoid clash with the rest of Xen.

> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> CC: wei.liu2@citrix.com
> CC: ian.jackson@eu.citrix.com
> ---
>   tools/libxl/libxl_arm.c       | 26 --------------------------
>   xen/include/public/arch-arm.h | 26 ++++++++++++++++++++++++++
>   2 files changed, 26 insertions(+), 26 deletions(-)
> 
> diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
> index 8af9f6f..89a417f 100644
> --- a/tools/libxl/libxl_arm.c
> +++ b/tools/libxl/libxl_arm.c
> @@ -8,23 +8,6 @@
>   #include <libfdt.h>
>   #include <assert.h>
>   
> -/**
> - * IRQ line type.
> - * DT_IRQ_TYPE_NONE            - default, unspecified type
> - * DT_IRQ_TYPE_EDGE_RISING     - rising edge triggered
> - * DT_IRQ_TYPE_EDGE_FALLING    - falling edge triggered
> - * DT_IRQ_TYPE_EDGE_BOTH       - rising and falling edge triggered
> - * DT_IRQ_TYPE_LEVEL_HIGH      - high level triggered
> - * DT_IRQ_TYPE_LEVEL_LOW       - low level triggered
> - */
> -#define DT_IRQ_TYPE_NONE           0x00000000
> -#define DT_IRQ_TYPE_EDGE_RISING    0x00000001
> -#define DT_IRQ_TYPE_EDGE_FALLING   0x00000002
> -#define DT_IRQ_TYPE_EDGE_BOTH                           \
> -    (DT_IRQ_TYPE_EDGE_FALLING | DT_IRQ_TYPE_EDGE_RISING)
> -#define DT_IRQ_TYPE_LEVEL_HIGH     0x00000004
> -#define DT_IRQ_TYPE_LEVEL_LOW      0x00000008
> -

Those defines have nothing to do with the guest itself. They are 
currently define in Xen without the DT_ prefix.

>   static const char *gicv_to_string(libxl_gic_version gic_version)
>   {
>       switch (gic_version) {
> @@ -165,18 +148,9 @@ static struct arch_info {
>       {"xen-3.0-aarch64", "arm,armv8-timer", "arm,armv8" },
>   };
>   
> -/*
> - * The device tree compiler (DTC) is allocating the phandle from 1 to
> - * onwards. Reserve a high value for the GIC phandle.
> - */
> -#define PHANDLE_GIC (65000)
> -
>   typedef uint32_t be32;
>   typedef be32 gic_interrupt[3];
>   
> -#define ROOT_ADDRESS_CELLS 2
> -#define ROOT_SIZE_CELLS 2
> -
>   #define PROP_INITRD_START "linux,initrd-start"
>   #define PROP_INITRD_END "linux,initrd-end"
>   
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> index eb424e8..cb88168 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -456,6 +456,32 @@ typedef uint64_t xen_callback_t;
>   #define PSCI_cpu_on      2
>   #define PSCI_migrate     3
>   
> +/*
> + * The device tree compiler (DTC) is allocating the phandle from 1 to
> + * onwards. Reserve a high value for the GIC phandle.
> + */
> +#define PHANDLE_GIC (65000)
> +
> +#define ROOT_ADDRESS_CELLS 2
> +#define ROOT_SIZE_CELLS 2
> +
> +/**
> + * IRQ line type.
> + * DT_IRQ_TYPE_NONE            - default, unspecified type
> + * DT_IRQ_TYPE_EDGE_RISING     - rising edge triggered
> + * DT_IRQ_TYPE_EDGE_FALLING    - falling edge triggered
> + * DT_IRQ_TYPE_EDGE_BOTH       - rising and falling edge triggered
> + * DT_IRQ_TYPE_LEVEL_HIGH      - high level triggered
> + * DT_IRQ_TYPE_LEVEL_LOW       - low level triggered
> + */
> +#define DT_IRQ_TYPE_NONE           0x00000000
> +#define DT_IRQ_TYPE_EDGE_RISING    0x00000001
> +#define DT_IRQ_TYPE_EDGE_FALLING   0x00000002
> +#define DT_IRQ_TYPE_EDGE_BOTH                           \
> +    (DT_IRQ_TYPE_EDGE_FALLING | DT_IRQ_TYPE_EDGE_RISING)
> +#define DT_IRQ_TYPE_LEVEL_HIGH     0x00000004
> +#define DT_IRQ_TYPE_LEVEL_LOW      0x00000008
> +
>   #endif
>   
>   #ifndef __ASSEMBLY__
> 

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 03/15] xen/arm: extend device tree based multiboot protocol

[Julien Grall]

Hi Stefano,

On 13/06/18 23:15, Stefano Stabellini wrote:
> Extend the existing device tree based multiboot protocol to include
> information regarding other domUs to boot.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> ---
>   docs/misc/arm/device-tree/booting.txt | 102 ++++++++++++++++++++++++++++++++++
>   1 file changed, 102 insertions(+)
> 
> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
> index ce2d0dc..95255e5 100644
> --- a/docs/misc/arm/device-tree/booting.txt
> +++ b/docs/misc/arm/device-tree/booting.txt
> @@ -119,3 +119,105 @@ For those you would hardcode the Xen commandline in the DTB under
>   line by writing bootargs (as for native Linux).
>   A Xen-aware bootloader would set xen,xen-bootargs for Xen, xen,dom0-bootargs
>   for Dom0 and bootargs for native Linux.
> +
> +
> +Creating DomUs directly from Xen
> +================================
> +
> +It is possible to have Xen create other domains, in addition to dom0,
> +out of the information provided via device tree. A Kernel and initrd

NIT: s/Kernel/kernel/

> +(optional) need to be specified for each guest.
> +
> +For each DomU to be created there needs to be one node under /chosen
> +with the following properties:

I think it would be better to make this domain agnostic. I.e allow Dom0 
to be created the same way but still supporting the current bindings.

We could differentiate Dom0 from the other with a property 
"xen,initial-domain". Note that I am not asking to add this property in 
this series. This is more a forward looking of the use of this binding.

> +
> +- compatible
> +
> +    "xen,domU"

If we follow my suggestion, this would be renamed "xen,domain".

> +
> +- mem (optional)

I would prefer the full name "memory".

> +
> +    A string specifying the amount of RAM to allocate to the guest. If
> +    not specified it defaults to "64M". The format of the string is the same
> +    as the one for the mem= parameter in xl config files.

I don't much like default values because they are pretty arbitrary. This 
also does not match the default value for Dom0. Why don't you mandate 
the property?

I would also prefer if the size is specified in the same way number are 
described in Device-Tree. I.e using cells. You could impose 2 cells for 
the size here.

> +
> +- cpus (optional)
> +
> +    A string specifying the number of vcpus to allocate to the guest. If
> +    not specified it defaults to "1".

Same remarks as for "mem".

> +
> +- #address-cells and #size-cells
> +
> +    Both #address-cells and #size-cells need to be specified because
> +    both sub-nodes (described shortly) have reg properties.
> +
> +Under the "xen,domU" compatible node, one or more sub-nodes are present
> +for the DomU kernel and ramdisk.
> +
> +The kernel sub-node has the following properties:
> +
> +- compatible
> +
> +    "multiboot,domU-kernel"

I don't think we need to specify a new compatible here. We could re-use 
"multiboot,kernel" here because the parent node will contain "xen,domU".

> +
> +- reg
> +
> +    Specifies the physical address of the kernel in RAM and its
> +    length.
> +
> +- bootargs (optional)
> +
> +    Command line parameters for the guest kernel.
> +
> +The ramdisk sub-node has the following properties:
> +
> +- compatible
> +
> +    "multiboot,domU-ramdisk"

Same here, we could re-use "multiboot,ramdisk".

> +
> +- reg
> +
> +    Specifies the physical address of the ramdisk in RAM and its
> +    length.

We should mention somewhere that this should be described in the /chosen 
node of the device-tree.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 04/15] xen/arm: do not pass dt_host to make_memory_node and make_hypervisor_node

[Julien Grall]

Hi Stefano,

On 13/06/18 23:15, Stefano Stabellini wrote:
> In order to make make_memory_node and make_hypervisor_node more
> reusable, do not pass them dt_host. As they only use it to calculate
> addrcells and sizecells, pass addrcells and sizecells directly.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> ---
>   xen/arch/arm/domain_build.c   | 20 ++++++++++----------
>   xen/common/device_tree.c      |  6 +++---
>   xen/include/xen/device_tree.h |  2 +-
>   3 files changed, 14 insertions(+), 14 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 11cdf05..bb88e09 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -533,11 +533,11 @@ static int fdt_property_interrupts(void *fdt, gic_interrupt_t *intr,
>   
>   static int make_memory_node(const struct domain *d,
>                               void *fdt,
> -                            const struct dt_device_node *parent,
> +                            int addrcells, int sizecells,
>                               const struct kernel_info *kinfo)
>   {
>       int res, i;
> -    int reg_size = dt_child_n_addr_cells(parent) + dt_child_n_size_cells(parent);
> +    int reg_size = addrcells + sizecells;
>       int nr_cells = reg_size*kinfo->mem.nr_banks;
>       __be32 reg[nr_cells];
>       __be32 *cells;
> @@ -563,7 +563,7 @@ static int make_memory_node(const struct domain *d,
>           dt_dprintk("  Bank %d: %#"PRIx64"->%#"PRIx64"\n",
>                      i, start, start + size);
>   
> -        dt_child_set_range(&cells, parent, start, size);
> +        dt_child_set_range(&cells, addrcells, sizecells, start, size);
>       }
>   
>       res = fdt_property(fdt, "reg", reg, sizeof(reg));
> @@ -579,7 +579,7 @@ static void evtchn_allocate(struct domain *d);
>   
>   static int make_hypervisor_node(struct domain *d,
>                                   const struct kernel_info *kinfo,
> -                                const struct dt_device_node *parent)
> +                                int addrcells, int sizecells)
>   {
>       const char compat[] =
>           "xen,xen-"__stringify(XEN_VERSION)"."__stringify(XEN_SUBVERSION)"\0"
> @@ -588,9 +588,6 @@ static int make_hypervisor_node(struct domain *d,
>       gic_interrupt_t intr;
>       __be32 *cells;
>       int res;
> -    /* Convenience alias */
> -    int addrcells = dt_child_n_addr_cells(parent);
> -    int sizecells = dt_child_n_size_cells(parent);
>       void *fdt = kinfo->fdt;
>   
>       dt_dprintk("Create hypervisor node\n");
> @@ -615,7 +612,8 @@ static int make_hypervisor_node(struct domain *d,
>   
>       /* reg 0 is grant table space */
>       cells = &reg[0];
> -    dt_child_set_range(&cells, parent, kinfo->gnttab_start, kinfo->gnttab_size);
> +    dt_child_set_range(&cells, addrcells, sizecells,
> +                       kinfo->gnttab_start, kinfo->gnttab_size);
>       res = fdt_property(fdt, "reg", reg,
>                          dt_cells_to_size(addrcells + sizecells));
>       if ( res )
> @@ -1292,11 +1290,13 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
>   
>       if ( node == dt_host )
>       {
> +        int addrcells = dt_child_n_addr_cells(node);
> +        int sizecells = dt_child_n_size_cells(node);

Newline here please.

>           /*
>            * The hypervisor node should always be created after all nodes
>            * from the host DT have been parsed.
>            */
> -        res = make_hypervisor_node(d, kinfo, node);
> +        res = make_hypervisor_node(d, kinfo, addrcells, sizecells);
>           if ( res )
>               return res;
>   
> @@ -1308,7 +1308,7 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
>           if ( res )
>               return res;
>   
> -        res = make_memory_node(d, kinfo->fdt, node, kinfo);
> +        res = make_memory_node(d, kinfo->fdt, addrcells, sizecells, kinfo);
>           if ( res )
>               return res;
>   
> diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
> index 7b009ea..8fc401d 100644
> --- a/xen/common/device_tree.c
> +++ b/xen/common/device_tree.c
> @@ -112,11 +112,11 @@ void dt_set_range(__be32 **cellp, const struct dt_device_node *np,
>       dt_set_cell(cellp, dt_n_size_cells(np), size);
>   }
>   
> -void dt_child_set_range(__be32 **cellp, const struct dt_device_node *parent,
> +void dt_child_set_range(__be32 **cellp, int addrcells, int sizecells,
>                           u64 address, u64 size)
>   {
> -    dt_set_cell(cellp, dt_child_n_addr_cells(parent), address);
> -    dt_set_cell(cellp, dt_child_n_size_cells(parent), size);
> +    dt_set_cell(cellp, addrcells, address);
> +    dt_set_cell(cellp, sizecells, size);
>   }
>   
>   static void __init *unflatten_dt_alloc(unsigned long *mem, unsigned long size,
> diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
> index 0aecbe0..01040a6 100644
> --- a/xen/include/xen/device_tree.h
> +++ b/xen/include/xen/device_tree.h
> @@ -699,7 +699,7 @@ void dt_set_range(__be32 **cellp, const struct dt_device_node *np,
>    * Write a range into a series of cells and update cellp to point to the
>    * cell just after.
>    */
> -void dt_child_set_range(__be32 **cellp, const struct dt_device_node *parent,
> +void dt_child_set_range(__be32 **cellp, int addrcells, int sizecells,
>                           u64 address, u64 size);
>   
>   /**
> 

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 05/15] xen/arm: rename acpi_make_chosen_node to make_chosen_node

[Julien Grall]

Hi Stefano,

On 13/06/18 23:15, Stefano Stabellini wrote:
> acpi_make_chosen_node is actually generic and can be reused. Rename it
> to make_chosen_node and make it available to non-ACPI builds.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> ---
>   xen/arch/arm/domain_build.c | 84 ++++++++++++++++++++++-----------------------

Probably not the best patch to make the comment but this is the first 
place where you start to have big changes in the file.

This series is basically adding another 400 lines in a file that is 
already 2245 lines. After this series we are going to handle:
	- DT generation when booting using ACPI
	- DT generation when booting using DT
	- DT generation for guest

This is probably a bit too much for a single file.

>   1 file changed, 42 insertions(+), 42 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index bb88e09..4e4cd19 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -935,6 +935,47 @@ static int make_timer_node(const struct domain *d, void *fdt,
>       return res;
>   }
>   
> +static int make_chosen_node(const struct kernel_info *kinfo)

Can you document it who is going to use it? This name gives the 
impression that will be used for creating any DT. But in fact, it will 
only be the case for ACPI based Dom0 and DomU.

> +{
> +    int res;
> +    const char *bootargs = NULL;
> +    const struct bootmodule *mod = kinfo->kernel_bootmodule;
> +    void *fdt = kinfo->fdt;
> +
> +    dt_dprintk("Create chosen node\n");
> +    res = fdt_begin_node(fdt, "chosen");
> +    if ( res )
> +        return res;
> +
> +    if ( mod && mod->cmdline[0] )
> +    {
> +        bootargs = &mod->cmdline[0];
> +        res = fdt_property(fdt, "bootargs", bootargs, strlen(bootargs) + 1);
> +        if ( res )
> +           return res;
> +    }
> +
> +    /*
> +     * If the bootloader provides an initrd, we must create a placeholder
> +     * for the initrd properties. The values will be replaced later.
> +     */
> +    if ( mod && mod->size )
> +    {
> +        u64 a = 0;
> +        res = fdt_property(kinfo->fdt, "linux,initrd-start", &a, sizeof(a));
> +        if ( res )
> +            return res;
> +
> +        res = fdt_property(kinfo->fdt, "linux,initrd-end", &a, sizeof(a));
> +        if ( res )
> +            return res;
> +    }
> +
> +    res = fdt_end_node(fdt);
> +
> +    return res;
> +}
> +
>   static int map_irq_to_domain(struct domain *d, unsigned int irq,
>                                bool need_mapping, const char *devname)
>   
> @@ -1420,47 +1461,6 @@ static int acpi_route_spis(struct domain *d)
>       return 0;
>   }
>   
> -static int acpi_make_chosen_node(const struct kernel_info *kinfo)
> -{
> -    int res;
> -    const char *bootargs = NULL;
> -    const struct bootmodule *mod = kinfo->kernel_bootmodule;
> -    void *fdt = kinfo->fdt;
> -
> -    dt_dprintk("Create chosen node\n");
> -    res = fdt_begin_node(fdt, "chosen");
> -    if ( res )
> -        return res;
> -
> -    if ( mod && mod->cmdline[0] )
> -    {
> -        bootargs = &mod->cmdline[0];
> -        res = fdt_property(fdt, "bootargs", bootargs, strlen(bootargs) + 1);
> -        if ( res )
> -           return res;
> -    }
> -
> -    /*
> -     * If the bootloader provides an initrd, we must create a placeholder
> -     * for the initrd properties. The values will be replaced later.
> -     */
> -    if ( mod && mod->size )
> -    {
> -        u64 a = 0;
> -        res = fdt_property(kinfo->fdt, "linux,initrd-start", &a, sizeof(a));
> -        if ( res )
> -            return res;
> -
> -        res = fdt_property(kinfo->fdt, "linux,initrd-end", &a, sizeof(a));
> -        if ( res )
> -            return res;
> -    }
> -
> -    res = fdt_end_node(fdt);
> -
> -    return res;
> -}
> -
>   static int acpi_make_hypervisor_node(const struct kernel_info *kinfo,
>                                        struct membank tbl_add[])
>   {
> @@ -1532,7 +1532,7 @@ static int create_acpi_dtb(struct kernel_info *kinfo, struct membank tbl_add[])
>           return ret;
>   
>       /* Create a chosen node for DOM0 */
> -    ret = acpi_make_chosen_node(kinfo);
> +    ret = make_chosen_node(kinfo);
>       if ( ret )
>           goto err;
>   
> 

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 08/15] xen/arm: probe domU kernels and initrds

[Julien Grall]

Hi Stefano,

On 13/06/18 23:15, Stefano Stabellini wrote:
> Find addresses and sizes on device tree.
> Introduce a new boot_module_find_by_addr_and_kind function to match not
> just on boot module kind, but also by address so that we can support
> multiple domUs.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> ---
>   xen/arch/arm/kernel.c       | 54 +++++++++++++++++++++++++++++++++++++++++++++
>   xen/arch/arm/kernel.h       |  2 ++
>   xen/arch/arm/setup.c        | 15 +++++++++++++
>   xen/include/asm-arm/setup.h |  2 ++
>   4 files changed, 73 insertions(+)
> 
> diff --git a/xen/arch/arm/kernel.c b/xen/arch/arm/kernel.c
> index 8fdfd91..c41092e 100644
> --- a/xen/arch/arm/kernel.c
> +++ b/xen/arch/arm/kernel.c
> @@ -496,6 +496,60 @@ err:
>       return rc;
>   }
>   
> +int kernel_probe_domU(struct kernel_info *info, struct dt_device_node *domU)
> +{
> +    struct bootmodule *mod;
> +    struct dt_device_node *node;
> +    u64 kernel_addr, initrd_addr, size;
> +    int rc;
> +
> +    dt_for_each_child_node(domU, node)
> +    {
> +        if ( dt_device_is_compatible(node, "multiboot,domU-kernel") )
> +        {
> +            u32 len;
> +            const __be32 *val;
> +            val = dt_get_property(node, "reg", &len);
> +            dt_get_range(&val, node, &kernel_addr, &size);
> +        }
> +        else if ( dt_device_is_compatible(node, "multiboot,domU-ramdisk") )
> +        {
> +            u32 len;
> +            const __be32 *val;
> +            val = dt_get_property(node, "reg", &len);
> +            dt_get_range(&val, node, &initrd_addr, &size);
> +        }
> +        else
> +            continue;
> +    }
> +    info->kernel_bootmodule = mod = boot_module_find_by_addr_and_kind(
> +									    BOOTMOD_DOMU_KERNEL, kernel_addr);

This line contains hard tab.

But I don't think this will work as you expect. Imagine the kernel is 
the same for each guest. It would be fine to have the Image loaded once 
in memory and therefore specify the same physical address for all 
domU-kernel compatible node. However, the command line may be different.

So you would end up to use the wrong module here.

> +    info->initrd_bootmodule = boot_module_find_by_addr_and_kind(
> +							      BOOTMOD_DOMU_RAMDISK, initrd_addr);

Same here.

> +    printk("Loading DomU kernel from boot module @ %"PRIpaddr"\n",
> +		   info->kernel_bootmodule->start);

The indentation is wrong here.

> +    if ( info->initrd_bootmodule )
> +        printk("Loading ramdisk from boot module @ %"PRIpaddr"\n",
> +               info->initrd_bootmodule->start);
> +
> +    /* if it is a gzip'ed image, 32bit or 64bit, uncompress it */
> +    rc = kernel_decompress(mod);
> +    if (rc < 0 && rc != -EINVAL)
> +        return rc;
> +
> +#ifdef CONFIG_ARM_64
> +    rc = kernel_zimage64_probe(info, mod->start, mod->size);
> +    if (rc < 0)
> +#endif
> +        rc = kernel_uimage_probe(info, mod->start, mod->size);
> +    if (rc < 0)
> +        rc = kernel_zimage32_probe(info, mod->start, mod->size);
> +    if (rc < 0)
> +        rc = kernel_elf_probe(info, mod->start, mod->size);

Most of this code is the same as kernel_probe. How about reworking 
kernel_probe to handle any domain?

> +
> +    return rc;
> +}
> +
>   int kernel_probe(struct kernel_info *info)
>   {
>       struct bootmodule *mod = boot_module_find_by_kind(BOOTMOD_KERNEL);
> diff --git a/xen/arch/arm/kernel.h b/xen/arch/arm/kernel.h
> index 6d69509..8e1614b 100644
> --- a/xen/arch/arm/kernel.h
> +++ b/xen/arch/arm/kernel.h
> @@ -64,6 +64,8 @@ struct kernel_info {
>    */
>   int kernel_probe(struct kernel_info *info);
>   
> +int kernel_probe_domU(struct kernel_info *info, struct dt_device_node *node);
> +
>   /*
>    * Loads the kernel into guest RAM.
>    *
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index 82593c8..98bdb24 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -241,6 +241,21 @@ struct bootmodule * __init boot_module_find_by_kind(bootmodule_kind kind)
>       return NULL;
>   }
>   
> +struct bootmodule * __init boot_module_find_by_addr_and_kind(bootmodule_kind kind,
> +                                                             paddr_t start)
> +{
> +    struct bootmodules *mods = &bootinfo.modules;
> +    struct bootmodule *mod;
> +    int i;
> +    for (i = 0 ; i < mods->nr_mods ; i++ )
> +    {
> +        mod = &mods->module[i];
> +        if ( mod->kind == kind && mod->start == start )
> +            return mod;
> +    }
> +    return NULL;
> +}
> +
>   const char * __init boot_module_kind_as_string(bootmodule_kind kind)
>   {
>       switch ( kind )
> diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
> index 86aac0e..903782f 100644
> --- a/xen/include/asm-arm/setup.h
> +++ b/xen/include/asm-arm/setup.h
> @@ -83,6 +83,8 @@ struct bootmodule *add_boot_module(bootmodule_kind kind,
>                                      paddr_t start, paddr_t size,
>                                      const char *cmdline);
>   struct bootmodule *boot_module_find_by_kind(bootmodule_kind kind);
> +struct bootmodule * __init boot_module_find_by_addr_and_kind(bootmodule_kind kind,
> +                                                             paddr_t start);
>   const char * __init boot_module_kind_as_string(bootmodule_kind kind);
>   
>   #endif
> 

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 09/15] xen/arm: refactor construct_dom0

[Julien Grall]

Hi Stefano,

On 13/06/18 23:15, Stefano Stabellini wrote:
> Move generic initializations out of construct_dom0 so that they can be
> reused.
> 
> No functional changes in this patch.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> ---
>   xen/arch/arm/domain_build.c | 124 ++++++++++++++++++++++++--------------------
>   1 file changed, 67 insertions(+), 57 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 4e4cd19..b31c563 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -2092,73 +2092,27 @@ static void __init find_gnttab_region(struct domain *d,
>              kinfo->gnttab_start, kinfo->gnttab_start + kinfo->gnttab_size);
>   }
>   
> -int __init construct_dom0(struct domain *d)
> +int __init __construct_domain(struct domain *d, struct kernel_info *kinfo)
>   {
> -    struct kernel_info kinfo = {};
>       struct vcpu *saved_current;
> -    int rc, i, cpu;
> +    int i, cpu;
>   
>       struct vcpu *v = d->vcpu[0];
>       struct cpu_user_regs *regs = &v->arch.cpu_info->guest_cpu_user_regs;
>   
> -    /* Sanity! */
> -    BUG_ON(d->domain_id != 0);
> -    BUG_ON(d->vcpu[0] == NULL);
> -    BUG_ON(v->is_initialised);

I think we want to keep the last two BUG_ON here. "d->vcpu[0] == NULL" 
is making sure that the value used when setting current is sane.

"v->is_initalised" is making sure this function is not called twice by 
mistake. Indeed the variable will be set at the end of __construct_domain.

> -
> -    printk("*** LOADING DOMAIN 0 ***\n");
> -    if ( dom0_mem <= 0 )
> -    {
> -        warning_add("PLEASE SPECIFY dom0_mem PARAMETER - USING 512M FOR NOW\n");
> -        dom0_mem = MB(512);
> -    }
> -
> -
> -    iommu_hwdom_init(d);
> -
> -    d->max_pages = ~0U;
> -
> -    kinfo.unassigned_mem = dom0_mem;
> -    kinfo.d = d;
> -
> -    rc = kernel_probe(&kinfo);
> -    if ( rc < 0 )
> -        return rc;
> -
>   #ifdef CONFIG_ARM_64
>       /* if aarch32 mode is not supported at EL1 do not allow 32-bit domain */
> -    if ( !(cpu_has_el1_32) && kinfo.type == DOMAIN_32BIT )
> +    if ( !(cpu_has_el1_32) && kinfo->type == DOMAIN_32BIT )
>       {
>           printk("Platform does not support 32-bit domain\n");
>           return -EINVAL;
>       }
> -    d->arch.type = kinfo.type;

Any reason to move this out?

>   
>       if ( is_64bit_domain(d) )
>           vcpu_switch_to_aarch64_mode(v);
>   
>   #endif
>   
> -    allocate_memory(d, &kinfo);

I think this could stay here.

> -    find_gnttab_region(d, &kinfo);
> -
> -    /* Map extra GIC MMIO, irqs and other hw stuffs to dom0. */
> -    rc = gic_map_hwdom_extra_mappings(d);
> -    if ( rc < 0 )
> -        return rc;
> -
> -    rc = platform_specific_mapping(d);
> -    if ( rc < 0 )
> -        return rc;
> -
> -    if ( acpi_disabled )
> -        rc = prepare_dtb(d, &kinfo);
> -    else
> -        rc = prepare_acpi(d, &kinfo);

I think this is probably a call to add "hwdom" in the name of the 2 
functions.

> -
> -    if ( rc < 0 )
> -        return rc;
> -
>       /*
>        * The following loads use the domain's p2m and require current to
>        * be a vcpu of the domain, temporarily switch
> @@ -2171,20 +2125,18 @@ int __init construct_dom0(struct domain *d)
>        * kernel_load will determine the placement of the kernel as well
>        * as the initrd & fdt in RAM, so call it first.
>        */
> -    kernel_load(&kinfo);
> +    kernel_load(kinfo);
>       /* initrd_load will fix up the fdt, so call it before dtb_load */
> -    initrd_load(&kinfo);
> -    dtb_load(&kinfo);
> +    initrd_load(kinfo);
> +    dtb_load(kinfo);
>   
>       /* Now that we are done restore the original p2m and current. */
>       set_current(saved_current);
>       p2m_restore_state(saved_current);
>   
> -    discard_initial_modules();
> -
>       memset(regs, 0, sizeof(*regs));
>   
> -    regs->pc = (register_t)kinfo.entry;
> +    regs->pc = (register_t)kinfo->entry;
>   
>       if ( is_32bit_domain(d) )
>       {
> @@ -2202,14 +2154,14 @@ int __init construct_dom0(struct domain *d)
>            */
>           regs->r0 = 0; /* SBZ */
>           regs->r1 = 0xffffffff; /* We use DTB therefore no machine id */
> -        regs->r2 = kinfo.dtb_paddr;
> +        regs->r2 = kinfo->dtb_paddr;
>       }
>   #ifdef CONFIG_ARM_64
>       else
>       {
>           regs->cpsr = PSR_GUEST64_INIT;
>           /* From linux/Documentation/arm64/booting.txt */
> -        regs->x0 = kinfo.dtb_paddr;
> +        regs->x0 = kinfo->dtb_paddr;
>           regs->x1 = 0; /* Reserved for future use */
>           regs->x2 = 0; /* Reserved for future use */
>           regs->x3 = 0; /* Reserved for future use */
> @@ -2235,6 +2187,64 @@ int __init construct_dom0(struct domain *d)
>       return 0;
>   }
>   
> +int __init construct_dom0(struct domain *d)
> +{
> +    struct kernel_info kinfo = {};
> +    int rc;
> +
> +    struct vcpu *v = d->vcpu[0];
> +
> +    /* Sanity! */
> +    BUG_ON(d->domain_id != 0);
> +    BUG_ON(d->vcpu[0] == NULL);
> +    BUG_ON(v->is_initialised);
> +
> +    printk("*** LOADING DOMAIN 0 ***\n");
> +    if ( dom0_mem <= 0 )
> +    {
> +        warning_add("PLEASE SPECIFY dom0_mem PARAMETER - USING 512M FOR NOW\n");
> +        dom0_mem = MB(512);
> +    }
> +
> +

No need to copy the second newline :).

> +    iommu_hwdom_init(d);
> +
> +    d->max_pages = ~0U;
> +
> +    kinfo.unassigned_mem = dom0_mem;
> +    kinfo.d = d;
> +
> +    rc = kernel_probe(&kinfo);
> +    if ( rc < 0 )
> +        return rc;
> +
> +    allocate_memory(d, &kinfo);
> +    find_gnttab_region(d, &kinfo);
> +
> +    /* Map extra GIC MMIO, irqs and other hw stuffs to dom0. */
> +    rc = gic_map_hwdom_extra_mappings(d);
> +    if ( rc < 0 )
> +        return rc;
> +
> +    rc = platform_specific_mapping(d);
> +    if ( rc < 0 )
> +        return rc;
> +
> +    d->arch.type = kinfo.type;
> +
> +    if ( acpi_disabled )
> +        rc = prepare_dtb(d, &kinfo);
> +    else
> +        rc = prepare_acpi(d, &kinfo);
> +
> +    if ( rc < 0 )
> +        return rc;
> +
> +    discard_initial_modules();

You say "no functional change" in this patch. But this is one. The 
module are now discard much earlier. This imply that memory baking the 
Image/Initrd will be free to be re-used at any time.

I don't think this is what we want. Unless you can promise no memory is 
allocated in __construct_domain().

> +
> +    return __construct_domain(d, &kinfo);
> +}
> +
>   /*
>    * Local variables:
>    * mode: C
> 

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 10/15] xen/arm: introduce construct_domU

[Julien Grall]

Hi,

On 13/06/18 23:15, Stefano Stabellini wrote:
> Similar to construct_dom0, construct_domU creates a barebone DomU guest.
> Default to 1 max vcpu and 64MB of memory if not specified otherwise.
> 
> The device tree node passed as argument is compatible "xen,domU", see
> docs/misc/arm/device-tree/booting.txt.
> 
> Allocate all vcpus on cpu0 initially.

I don't think this comment is true. __construct_domain will allocate 
vCPUs in cycle.

> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> ---
>   xen/arch/arm/domain_build.c | 37 +++++++++++++++++++++++++++++++++++++
>   xen/include/asm-arm/setup.h |  2 ++
>   2 files changed, 39 insertions(+)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index b31c563..02a7f94 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -2187,6 +2187,43 @@ int __init __construct_domain(struct domain *d, struct kernel_info *kinfo)
>       return 0;
>   }
>   
> +int __init construct_domU(struct domain *d, struct dt_device_node *node)
> +{
> +    struct kernel_info kinfo = {};
> +    int rc;
> +    const char *cpus = NULL, *mem = NULL;
> +
> +    printk("*** LOADING DOMU ***\n");
> +
> +    d->max_vcpus = 1;
> +    rc = dt_property_read_string(node, "cpus", &cpus);
> +    if ( !rc )
> +        d->max_vcpus = simple_strtoul(cpus, &cpus, 0);
> +
> +    kinfo.unassigned_mem = MB(64);
> +    rc = dt_property_read_string(node, "mem", &mem);
> +    if ( !rc )
> +        kinfo.unassigned_mem = parse_size_and_unit(mem, &mem);
> +
> +    d->vcpu = xzalloc_array(struct vcpu *, d->max_vcpus);
> +    if ( !d->vcpu )
> +        return -ENOMEM;;
> +    if ( alloc_vcpu(d, 0, 0) == NULL )
> +        return -ENOMEM;
> +    d->max_pages = ~0U;
> +
> +    kinfo.d = d;
> +
> +    rc = kernel_probe_domU(&kinfo, node);
> +    if ( rc < 0 )
> +        return rc;
> +
> +    d->arch.type = kinfo.type;
> +    allocate_memory(d, &kinfo);

allocate_memory() will allocate direct mapped memory but you impose a 
static memory layout for the guest. Both are not going to work very well 
together.

So you probably want to extend allocate_memory to support allocating 
memory for a given region.

> +
> +    return __construct_domain(d, &kinfo);
> +}
> +
>   int __init construct_dom0(struct domain *d)
>   {
>       struct kernel_info kinfo = {};
> diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
> index 903782f..e9f9905 100644
> --- a/xen/include/asm-arm/setup.h
> +++ b/xen/include/asm-arm/setup.h
> @@ -2,6 +2,7 @@
>   #define __ARM_SETUP_H_
>   
>   #include <public/version.h>
> +#include <xen/device_tree.h>
>   
>   #define MIN_FDT_ALIGN 8
>   #define MAX_FDT_SIZE SZ_2M
> @@ -71,6 +72,7 @@ void acpi_create_efi_mmap_table(struct domain *d,
>   int acpi_make_efi_nodes(void *fdt, struct membank tbl_add[]);
>   
>   int construct_dom0(struct domain *d);
> +int construct_domU(struct domain *d, struct dt_device_node *node);
>   
>   void discard_initial_modules(void);
>   void dt_unreserved_regions(paddr_t s, paddr_t e,
> 

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 11/15] xen/arm: generate a simple device tree for domUs

[Julien Grall]

Hi Stefano,

On 13/06/18 23:15, Stefano Stabellini wrote:
> Introduce functions to generate a basic domU device tree, similar to the
> existing functions in tools/libxl/libxl_arm.c.
> 
> Rename existing prepare_dtb to prepare_dtb_dom0 to avoid confusion.

Ah this is were the rename is. It might have make sense to do that in 
patch #9. Also, you want to name it "hwdom" as this is the preferred 
name nowadays.

> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> ---
>   xen/arch/arm/domain_build.c | 195 +++++++++++++++++++++++++++++++++++++++++++-
>   1 file changed, 193 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 02a7f94..b4f560f 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -1360,7 +1360,194 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
>       return res;
>   }
>   
> -static int prepare_dtb(struct domain *d, struct kernel_info *kinfo)
> +static int make_gic_domU_node(const struct domain *d, void *fdt, int addrcells, int sizecells)
> +{
> +    int res = 0;
> +    int reg_size = addrcells + sizecells;
> +    int nr_cells = reg_size * 2;
> +    __be32 reg[nr_cells]; > +    __be32 *cells;
> +
> +    res = fdt_begin_node(fdt, "interrupt-controller");

Per the DT spec, node name should contain @unit-address when a "reg" 
property present. "unit-address" been the first address of the "reg" 
property.

Per the DT spec, node should have @base when a "regs" property is present.

> +    if ( res )
> +        return res;
> +
> +    res = fdt_property_cell(fdt, "#address-cells", 0);
> +    if ( res )
> +        return res;
> +
> +    res = fdt_property_cell(fdt, "#interrupt-cells", 3);
> +    if ( res )
> +        return res;
> +
> +    res = fdt_property(fdt, "interrupt-controller", NULL, 0);
> +    if ( res )
> +        return res;
> +
> +    if (gic_hw_version() == GIC_V3)

Could we use a switch please? This would allow to catch easily new 
hardware version.

> +    {
> +        const uint64_t gicd_base = GUEST_GICV3_GICD_BASE;
> +        const uint64_t gicd_size = GUEST_GICV3_GICD_SIZE;
> +        const uint64_t gicr0_base = GUEST_GICV3_GICR0_BASE;
> +        const uint64_t gicr0_size = GUEST_GICV3_GICR0_SIZE;
> +
> +        res = fdt_property_string(fdt, "compatible", "arm,gic-v3");
> +        if ( res )
> +            return res;
> +
> +        cells = &reg[0];
> +        dt_child_set_range(&cells, addrcells, sizecells, gicd_base, gicd_size);
> +        dt_child_set_range(&cells, addrcells, sizecells, gicr0_base, gicr0_size);
> +        res = fdt_property(fdt, "reg", reg, sizeof(reg));
> +    }
> +    else if (gic_hw_version() == GIC_V3)

I think this should be GIC_V2 here.

> +    {
> +        const uint64_t gicd_base = GUEST_GICD_BASE;
> +        const uint64_t gicd_size = GUEST_GICD_SIZE;
> +        const uint64_t gicc_base = GUEST_GICC_BASE;
> +        const uint64_t gicc_size = GUEST_GICC_SIZE;
> +
> +        res = fdt_property_string(fdt, "compatible", "arm,cortex-a15-gic");

I know that we use that property in libxl. But this is a bit weird to 
use it for Arm64 guest :). It would be better to use arm,gic-400 here.

> +        if ( res )
> +            return res;
> +
> +        cells = &reg[0];
> +        dt_child_set_range(&cells, addrcells, sizecells, gicd_base, gicd_size);
> +        dt_child_set_range(&cells, addrcells, sizecells, gicc_base, gicc_size);
> +    }
> +
> +    res = fdt_property(fdt, "reg", reg, sizeof(reg));
> +    if (res)
> +        return res;
> +
> +    res = fdt_property_cell(fdt, "linux,phandle", PHANDLE_GIC);
> +    if (res)
> +        return res;
> +
> +    res = fdt_property_cell(fdt, "phandle", PHANDLE_GIC);
> +    if (res)
> +        return res;
> +
> +    res = fdt_end_node(fdt);
> +
> +    return res;
> +}
> +
> +static int make_timer_domU_node(const struct domain *d, void *fdt)
> +{
> +    int res;
> +    gic_interrupt_t intrs[3];
> +
> +    res = fdt_begin_node(fdt, "timer");
> +    if ( res )
> +        return res;
> +
> +    if (!is_64bit_domain(d))
> +    {
> +        res = fdt_property_string(fdt, "compatible", "arm,armv7-timer");
> +        if ( res )
> +            return res;
> +    } else {
> +        res = fdt_property_string(fdt, "compatible", "arm,armv8-timer");
> +        if ( res )
> +            return res;
> +    }
> +
> +    set_interrupt_ppi(intrs[0], GUEST_TIMER_PHYS_S_PPI, 0xf, DT_IRQ_TYPE_LEVEL_LOW);
> +    set_interrupt_ppi(intrs[1], GUEST_TIMER_PHYS_NS_PPI, 0xf, DT_IRQ_TYPE_LEVEL_LOW);
> +    set_interrupt_ppi(intrs[2], GUEST_TIMER_VIRT_PPI, 0xf, DT_IRQ_TYPE_LEVEL_LOW);
> +
> +    res = fdt_property(fdt, "interrupts", intrs, sizeof (intrs[0]) * 3);
> +    if ( res )
> +        return res;
> +
> +    res = fdt_property_cell(fdt, "interrupt-parent",
> +                            PHANDLE_GIC);
> +    if (res)
> +        return res;
> +
> +    res = fdt_end_node(fdt);
> +    return res;
> +}
> +
> +#define DOMU_DTB_SIZE 4096

Is this going to be enough? Per the documentation, the maximum size of a 
DT is 2MB.

> +static int prepare_dtb_domU(struct domain *d, struct kernel_info *kinfo)
> +{
> +    int addrcells, sizecells;
> +    int ret;
> +
> +    addrcells = dt_child_n_addr_cells(dt_host);
> +    sizecells = dt_child_n_size_cells(dt_host);
> +
> +    kinfo->fdt = xmalloc_bytes(DOMU_DTB_SIZE);
> +    if ( kinfo->fdt == NULL )
> +        return -ENOMEM;
> +
> +    ret = fdt_create(kinfo->fdt, DOMU_DTB_SIZE);
> +    if ( ret < 0 )
> +        goto err;
> +
> +    ret = fdt_finish_reservemap(kinfo->fdt);
> +    if ( ret < 0 )
> +        goto err;
> +
> +    ret = fdt_begin_node(kinfo->fdt, "/");
> +    if ( ret < 0 )
> +        goto err;
> +
> +    ret = fdt_property_cell(kinfo->fdt, "#address-cells", addrcells);
> +    if ( ret )
> +        goto err;
> +
> +    ret = fdt_property_cell(kinfo->fdt, "#size-cells", sizecells);
> +    if ( ret )
> +        goto err;
> +
> +    ret = make_chosen_node(kinfo);
> +    if ( ret )
> +        goto err;
> +
> +    ret = make_hypervisor_node(d, kinfo, addrcells, sizecells);

make_hypervisor_node() will allocate an PPIfor the event channel based 
on what's available. However, the PPI for the guests is static and the 
timer is allocated afterwards.

So you probably want to rework make_hypervisor_node to pass the 
interrupt in parameter.

> +    if ( ret )
> +        goto err;
> +
> +    ret = make_psci_node(kinfo->fdt, NULL);
> +    if ( ret )
> +        goto err;
> +
> +    ret = make_cpus_node(d, kinfo->fdt, NULL);
> +    if ( ret )
> +        goto err;
> +
> +    ret = make_memory_node(d, kinfo->fdt, addrcells, sizecells, kinfo);
> +    if ( ret )
> +        goto err;
> +
> +    ret = make_gic_domU_node(d, kinfo->fdt, addrcells, sizecells);
> +    if ( ret )
> +        goto err;
> +
> +    ret = make_timer_domU_node(d, kinfo->fdt);
> +    if ( ret )
> +        goto err;
> +
> +    ret = fdt_end_node(kinfo->fdt);
> +    if ( ret < 0 )
> +        goto err;
> +
> +    ret = fdt_finish(kinfo->fdt);
> +    if ( ret < 0 )
> +        goto err;
> +
> +    return 0;
> +
> +  err:
> +    printk("Device tree generation failed (%d).\n", ret);
> +    xfree(kinfo->fdt);
> +    return -EINVAL;
> +}
> +
> +static int prepare_dtb_dom0(struct domain *d, struct kernel_info *kinfo)
>   {
>       const p2m_type_t default_p2mt = p2m_mmio_direct_c;
>       const void *fdt;
> @@ -2221,6 +2408,10 @@ int __init construct_domU(struct domain *d, struct dt_device_node *node)
>       d->arch.type = kinfo.type;
>       allocate_memory(d, &kinfo);
>   
> +    rc = prepare_dtb_domU(d, &kinfo);
> +    if ( rc < 0 )
> +        return rc;
> +
>       return __construct_domain(d, &kinfo);
>   }
>   
> @@ -2270,7 +2461,7 @@ int __init construct_dom0(struct domain *d)
>       d->arch.type = kinfo.type;
>   
>       if ( acpi_disabled )
> -        rc = prepare_dtb(d, &kinfo);
> +        rc = prepare_dtb_dom0(d, &kinfo);
>       else
>           rc = prepare_acpi(d, &kinfo);
>   
> 

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 00/15] dom0less step1: boot multiple domains from device tree

[Edgar E. Iglesias]

On Wed, Jun 13, 2018 at 03:15:09PM -0700, Stefano Stabellini wrote:
> Hi all,
> 
> This is first step toward "dom0less" as discussed in the various
> certifications related threads and discussions.
> 
> The goal of this series is to enable Xen to boot multiple domains in
> parallel, in addition to dom0, out of information found on device tree.
> 
> The device tree based boot protocol is extended to carry information
> about DomUs. Based on that information, Xen creates and starts one or
> more DomUs. DomUs created this way don't have access to xenstore, as
> xenstore is not started yet. This is actually OK, because this is meant
> for mission critical applications that typically only access directly
> assigned devices. They cannot tolerate interference or increased IRQ
> latency due to PV protocols. Device assignment is not actually covered
> by this series, it will be added later.
> 
> DomUs can print to the Xen serial using the CONSOLEIO hypercalls. A
> virtual PL011 is also emulated for them so that they can use their
> regular PL011 driver. This allows unmodified guests to run as Xen on ARM
> guests -- no Xen support needed at all. Console input also comes from
> the Xen serial: the Ctrl-AAA switching mechanism is extended to switch
> among domUs, dom0, and Xen.

Hi Stefano,

I think this looks very promising!

A question, do you have any thoughts on how we in the long run could
try to keep xl configs somewhat on par with the format here?

Or would the device-tree naming of options eventually have to diverge.

Eventually we'll need support for describing arbitrary memory maps and
device passthrough via device-tree for these domUs.

Thanks & Best regards,
Edgar


> 
> Cheers,
> 
> Stefano
> 
> 
> Stefano Stabellini (15):
>       xen: allow console_io hypercalls from DomUs on ARM
>       xen/arm: move a few guest related #defines to public/arch-arm.h
>       xen/arm: extend device tree based multiboot protocol
>       xen/arm: do not pass dt_host to make_memory_node and make_hypervisor_node
>       xen/arm: rename acpi_make_chosen_node to make_chosen_node
>       xen/arm: add BOOTMOD_DOMU_KERNEL/RAMDISK
>       xen/arm: increase MAX_MODULES
>       xen/arm: probe domU kernels and initrds
>       xen/arm: refactor construct_dom0
>       xen/arm: introduce construct_domU
>       xen/arm: generate a simple device tree for domUs
>       xen/arm: generate vpl011 node on device tree for domU
>       xen/arm: Allow vpl011 to be used by DomU
>       xen/arm: call construct_domU from start_xen and start DomU VMs
>       xen: support console_switching between Dom0 and DomUs on ARM
> 
>  docs/misc/arm/device-tree/booting.txt | 102 +++++++
>  tools/libxl/libxl_arm.c               |  26 --
>  xen/arch/arm/bootfdt.c                |   4 +
>  xen/arch/arm/domain_build.c           | 533 +++++++++++++++++++++++++++-------
>  xen/arch/arm/kernel.c                 |  54 ++++
>  xen/arch/arm/kernel.h                 |   2 +
>  xen/arch/arm/setup.c                  |  52 +++-
>  xen/arch/arm/vpl011.c                 |  98 +++++--
>  xen/common/device_tree.c              |   6 +-
>  xen/drivers/char/console.c            |  51 +++-
>  xen/include/asm-arm/setup.h           |  10 +-
>  xen/include/asm-arm/vpl011.h          |   2 +
>  xen/include/asm-x86/setup.h           |   2 +
>  xen/include/public/arch-arm.h         |  26 ++
>  xen/include/xen/device_tree.h         |   2 +-
>  15 files changed, 789 insertions(+), 181 deletions(-)
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xenproject.org
> https://lists.xenproject.org/mailman/listinfo/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 00/15] dom0less step1: boot multiple domains from device tree

[Stefano Stabellini]

On Thu, 14 Jun 2018, Edgar E. Iglesias wrote:
> On Wed, Jun 13, 2018 at 03:15:09PM -0700, Stefano Stabellini wrote:
> > Hi all,
> > 
> > This is first step toward "dom0less" as discussed in the various
> > certifications related threads and discussions.
> > 
> > The goal of this series is to enable Xen to boot multiple domains in
> > parallel, in addition to dom0, out of information found on device tree.
> > 
> > The device tree based boot protocol is extended to carry information
> > about DomUs. Based on that information, Xen creates and starts one or
> > more DomUs. DomUs created this way don't have access to xenstore, as
> > xenstore is not started yet. This is actually OK, because this is meant
> > for mission critical applications that typically only access directly
> > assigned devices. They cannot tolerate interference or increased IRQ
> > latency due to PV protocols. Device assignment is not actually covered
> > by this series, it will be added later.
> > 
> > DomUs can print to the Xen serial using the CONSOLEIO hypercalls. A
> > virtual PL011 is also emulated for them so that they can use their
> > regular PL011 driver. This allows unmodified guests to run as Xen on ARM
> > guests -- no Xen support needed at all. Console input also comes from
> > the Xen serial: the Ctrl-AAA switching mechanism is extended to switch
> > among domUs, dom0, and Xen.
> 
> Hi Stefano,
> 
> I think this looks very promising!
> 
> A question, do you have any thoughts on how we in the long run could
> try to keep xl configs somewhat on par with the format here?
> 
> Or would the device-tree naming of options eventually have to diverge.
> 
> Eventually we'll need support for describing arbitrary memory maps and
> device passthrough via device-tree for these domUs.

I think it makes sense to try to keep the two set of options somewhat
similar. I am keen on minimizing meaningless differences between the
two. Especially arbitrary memory maps and device passthrough
configurations, that are complex options -- it is worth trying to keep
them the same.

However, in general, I don't think we can promise to keep them exactly
identical. There are a lot of xl config options that won't be supported
as part of device tree booting. And when they are supported, there might
be additional parameters that are ignored.
docs/misc/arm/device-tree/booting.txt will be the official doc for
device tree booting options, while docs/man/xl.cfg.pod.5 will be the
official doc for xl config file options.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 02/15] xen/arm: move a few guest related #defines to public/arch-arm.h

[Stefano Stabellini]

On Thu, 14 Jun 2018, Julien Grall wrote:
> Hi Stefano,
> 
> On 13/06/18 23:15, Stefano Stabellini wrote:
> > Move a few constants defined by libxl_arm.c to
> > xen/include/public/arch-arm.h, so that they are together with the other
> > guest related #defines such as GUEST_GICD_BASE and GUEST_VPL011_SPI.
> > Also, this way they can be reused by hypervisor code.
> 
> All variables moved to arch-arm.h should be prefixed with GUEST_* to avoid
> clash with the rest of Xen.

I'll do.

> > 
> > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > CC: wei.liu2@citrix.com
> > CC: ian.jackson@eu.citrix.com
> > ---
> >   tools/libxl/libxl_arm.c       | 26 --------------------------
> >   xen/include/public/arch-arm.h | 26 ++++++++++++++++++++++++++
> >   2 files changed, 26 insertions(+), 26 deletions(-)
> > 
> > diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
> > index 8af9f6f..89a417f 100644
> > --- a/tools/libxl/libxl_arm.c
> > +++ b/tools/libxl/libxl_arm.c
> > @@ -8,23 +8,6 @@
> >   #include <libfdt.h>
> >   #include <assert.h>
> >   -/**
> > - * IRQ line type.
> > - * DT_IRQ_TYPE_NONE            - default, unspecified type
> > - * DT_IRQ_TYPE_EDGE_RISING     - rising edge triggered
> > - * DT_IRQ_TYPE_EDGE_FALLING    - falling edge triggered
> > - * DT_IRQ_TYPE_EDGE_BOTH       - rising and falling edge triggered
> > - * DT_IRQ_TYPE_LEVEL_HIGH      - high level triggered
> > - * DT_IRQ_TYPE_LEVEL_LOW       - low level triggered
> > - */
> > -#define DT_IRQ_TYPE_NONE           0x00000000
> > -#define DT_IRQ_TYPE_EDGE_RISING    0x00000001
> > -#define DT_IRQ_TYPE_EDGE_FALLING   0x00000002
> > -#define DT_IRQ_TYPE_EDGE_BOTH                           \
> > -    (DT_IRQ_TYPE_EDGE_FALLING | DT_IRQ_TYPE_EDGE_RISING)
> > -#define DT_IRQ_TYPE_LEVEL_HIGH     0x00000004
> > -#define DT_IRQ_TYPE_LEVEL_LOW      0x00000008
> > -
> 
> Those defines have nothing to do with the guest itself. They are currently
> define in Xen without the DT_ prefix.

Sounds like we want to get rid of the DT_IRQ_TYPE_* definitions
completely, move the IRQ_TYPE_* definitions from device_tree.h to here,
and start using them in tools/libxl/libxl_arm.c (which involves a
renaming s/DT_IRQ_TYPE/IRQ_TYPE/g).

Is that what you had in mind?


> >   static const char *gicv_to_string(libxl_gic_version gic_version)
> >   {
> >       switch (gic_version) {
> > @@ -165,18 +148,9 @@ static struct arch_info {
> >       {"xen-3.0-aarch64", "arm,armv8-timer", "arm,armv8" },
> >   };
> >   -/*
> > - * The device tree compiler (DTC) is allocating the phandle from 1 to
> > - * onwards. Reserve a high value for the GIC phandle.
> > - */
> > -#define PHANDLE_GIC (65000)
> > -
> >   typedef uint32_t be32;
> >   typedef be32 gic_interrupt[3];
> >   -#define ROOT_ADDRESS_CELLS 2
> > -#define ROOT_SIZE_CELLS 2
> > -
> >   #define PROP_INITRD_START "linux,initrd-start"
> >   #define PROP_INITRD_END "linux,initrd-end"
> >   diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> > index eb424e8..cb88168 100644
> > --- a/xen/include/public/arch-arm.h
> > +++ b/xen/include/public/arch-arm.h
> > @@ -456,6 +456,32 @@ typedef uint64_t xen_callback_t;
> >   #define PSCI_cpu_on      2
> >   #define PSCI_migrate     3
> >   +/*
> > + * The device tree compiler (DTC) is allocating the phandle from 1 to
> > + * onwards. Reserve a high value for the GIC phandle.
> > + */
> > +#define PHANDLE_GIC (65000)
> > +
> > +#define ROOT_ADDRESS_CELLS 2
> > +#define ROOT_SIZE_CELLS 2
> > +
> > +/**
> > + * IRQ line type.
> > + * DT_IRQ_TYPE_NONE            - default, unspecified type
> > + * DT_IRQ_TYPE_EDGE_RISING     - rising edge triggered
> > + * DT_IRQ_TYPE_EDGE_FALLING    - falling edge triggered
> > + * DT_IRQ_TYPE_EDGE_BOTH       - rising and falling edge triggered
> > + * DT_IRQ_TYPE_LEVEL_HIGH      - high level triggered
> > + * DT_IRQ_TYPE_LEVEL_LOW       - low level triggered
> > + */
> > +#define DT_IRQ_TYPE_NONE           0x00000000
> > +#define DT_IRQ_TYPE_EDGE_RISING    0x00000001
> > +#define DT_IRQ_TYPE_EDGE_FALLING   0x00000002
> > +#define DT_IRQ_TYPE_EDGE_BOTH                           \
> > +    (DT_IRQ_TYPE_EDGE_FALLING | DT_IRQ_TYPE_EDGE_RISING)
> > +#define DT_IRQ_TYPE_LEVEL_HIGH     0x00000004
> > +#define DT_IRQ_TYPE_LEVEL_LOW      0x00000008
> > +
> >   #endif
> >     #ifndef __ASSEMBLY__
> > 
> 
> Cheers,
> 
> -- 
> Julien Grall
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 04/15] xen/arm: do not pass dt_host to make_memory_node and make_hypervisor_node

[Stefano Stabellini]

On Thu, 14 Jun 2018, Julien Grall wrote:
> Hi Stefano,
> 
> On 13/06/18 23:15, Stefano Stabellini wrote:
> > In order to make make_memory_node and make_hypervisor_node more
> > reusable, do not pass them dt_host. As they only use it to calculate
> > addrcells and sizecells, pass addrcells and sizecells directly.
> > 
> > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > ---
> >   xen/arch/arm/domain_build.c   | 20 ++++++++++----------
> >   xen/common/device_tree.c      |  6 +++---
> >   xen/include/xen/device_tree.h |  2 +-
> >   3 files changed, 14 insertions(+), 14 deletions(-)
> > 
> > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > index 11cdf05..bb88e09 100644
> > --- a/xen/arch/arm/domain_build.c
> > +++ b/xen/arch/arm/domain_build.c
> > @@ -533,11 +533,11 @@ static int fdt_property_interrupts(void *fdt,
> > gic_interrupt_t *intr,
> >     static int make_memory_node(const struct domain *d,
> >                               void *fdt,
> > -                            const struct dt_device_node *parent,
> > +                            int addrcells, int sizecells,
> >                               const struct kernel_info *kinfo)
> >   {
> >       int res, i;
> > -    int reg_size = dt_child_n_addr_cells(parent) +
> > dt_child_n_size_cells(parent);
> > +    int reg_size = addrcells + sizecells;
> >       int nr_cells = reg_size*kinfo->mem.nr_banks;
> >       __be32 reg[nr_cells];
> >       __be32 *cells;
> > @@ -563,7 +563,7 @@ static int make_memory_node(const struct domain *d,
> >           dt_dprintk("  Bank %d: %#"PRIx64"->%#"PRIx64"\n",
> >                      i, start, start + size);
> >   -        dt_child_set_range(&cells, parent, start, size);
> > +        dt_child_set_range(&cells, addrcells, sizecells, start, size);
> >       }
> >         res = fdt_property(fdt, "reg", reg, sizeof(reg));
> > @@ -579,7 +579,7 @@ static void evtchn_allocate(struct domain *d);
> >     static int make_hypervisor_node(struct domain *d,
> >                                   const struct kernel_info *kinfo,
> > -                                const struct dt_device_node *parent)
> > +                                int addrcells, int sizecells)
> >   {
> >       const char compat[] =
> >           "xen,xen-"__stringify(XEN_VERSION)"."__stringify(XEN_SUBVERSION)"\0"
> > @@ -588,9 +588,6 @@ static int make_hypervisor_node(struct domain *d,
> >       gic_interrupt_t intr;
> >       __be32 *cells;
> >       int res;
> > -    /* Convenience alias */
> > -    int addrcells = dt_child_n_addr_cells(parent);
> > -    int sizecells = dt_child_n_size_cells(parent);
> >       void *fdt = kinfo->fdt;
> >         dt_dprintk("Create hypervisor node\n");
> > @@ -615,7 +612,8 @@ static int make_hypervisor_node(struct domain *d,
> >         /* reg 0 is grant table space */
> >       cells = &reg[0];
> > -    dt_child_set_range(&cells, parent, kinfo->gnttab_start,
> > kinfo->gnttab_size);
> > +    dt_child_set_range(&cells, addrcells, sizecells,
> > +                       kinfo->gnttab_start, kinfo->gnttab_size);
> >       res = fdt_property(fdt, "reg", reg,
> >                          dt_cells_to_size(addrcells + sizecells));
> >       if ( res )
> > @@ -1292,11 +1290,13 @@ static int handle_node(struct domain *d, struct
> > kernel_info *kinfo,
> >         if ( node == dt_host )
> >       {
> > +        int addrcells = dt_child_n_addr_cells(node);
> > +        int sizecells = dt_child_n_size_cells(node);
> 
> Newline here please.

I'll do


> >           /*
> >            * The hypervisor node should always be created after all nodes
> >            * from the host DT have been parsed.
> >            */
> > -        res = make_hypervisor_node(d, kinfo, node);
> > +        res = make_hypervisor_node(d, kinfo, addrcells, sizecells);
> >           if ( res )
> >               return res;
> >   @@ -1308,7 +1308,7 @@ static int handle_node(struct domain *d, struct
> > kernel_info *kinfo,
> >           if ( res )
> >               return res;
> >   -        res = make_memory_node(d, kinfo->fdt, node, kinfo);
> > +        res = make_memory_node(d, kinfo->fdt, addrcells, sizecells, kinfo);
> >           if ( res )
> >               return res;
> >   diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
> > index 7b009ea..8fc401d 100644
> > --- a/xen/common/device_tree.c
> > +++ b/xen/common/device_tree.c
> > @@ -112,11 +112,11 @@ void dt_set_range(__be32 **cellp, const struct
> > dt_device_node *np,
> >       dt_set_cell(cellp, dt_n_size_cells(np), size);
> >   }
> >   -void dt_child_set_range(__be32 **cellp, const struct dt_device_node
> > *parent,
> > +void dt_child_set_range(__be32 **cellp, int addrcells, int sizecells,
> >                           u64 address, u64 size)
> >   {
> > -    dt_set_cell(cellp, dt_child_n_addr_cells(parent), address);
> > -    dt_set_cell(cellp, dt_child_n_size_cells(parent), size);
> > +    dt_set_cell(cellp, addrcells, address);
> > +    dt_set_cell(cellp, sizecells, size);
> >   }
> >     static void __init *unflatten_dt_alloc(unsigned long *mem, unsigned long
> > size,
> > diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
> > index 0aecbe0..01040a6 100644
> > --- a/xen/include/xen/device_tree.h
> > +++ b/xen/include/xen/device_tree.h
> > @@ -699,7 +699,7 @@ void dt_set_range(__be32 **cellp, const struct
> > dt_device_node *np,
> >    * Write a range into a series of cells and update cellp to point to the
> >    * cell just after.
> >    */
> > -void dt_child_set_range(__be32 **cellp, const struct dt_device_node
> > *parent,
> > +void dt_child_set_range(__be32 **cellp, int addrcells, int sizecells,
> >                           u64 address, u64 size);
> >     /**
> > 
> 
> Cheers,
> 
> -- 
> Julien Grall
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 05/15] xen/arm: rename acpi_make_chosen_node to make_chosen_node

[Stefano Stabellini]

On Thu, 14 Jun 2018, Julien Grall wrote:
> Hi Stefano,
> 
> On 13/06/18 23:15, Stefano Stabellini wrote:
> > acpi_make_chosen_node is actually generic and can be reused. Rename it
> > to make_chosen_node and make it available to non-ACPI builds.
> > 
> > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > ---
> >   xen/arch/arm/domain_build.c | 84
> > ++++++++++++++++++++++-----------------------
> 
> Probably not the best patch to make the comment but this is the first place
> where you start to have big changes in the file.
> 
> This series is basically adding another 400 lines in a file that is already
> 2245 lines. After this series we are going to handle:
> 	- DT generation when booting using ACPI
> 	- DT generation when booting using DT
> 	- DT generation for guest
> 
> This is probably a bit too much for a single file.

To make it easier to review, I am going to add a separate patch that
will only do code movement at the end of the series.


> >   1 file changed, 42 insertions(+), 42 deletions(-)
> > 
> > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > index bb88e09..4e4cd19 100644
> > --- a/xen/arch/arm/domain_build.c
> > +++ b/xen/arch/arm/domain_build.c
> > @@ -935,6 +935,47 @@ static int make_timer_node(const struct domain *d, void
> > *fdt,
> >       return res;
> >   }
> >   +static int make_chosen_node(const struct kernel_info *kinfo)
> 
> Can you document it who is going to use it? This name gives the impression
> that will be used for creating any DT. But in fact, it will only be the case
> for ACPI based Dom0 and DomU.

I'll add:

 * This function is used as part of the device tree generation for Dom0
 * on ACPI systems, and DomUs started directly from Xen based on device
 * tree information.


> > +{
> > +    int res;
> > +    const char *bootargs = NULL;
> > +    const struct bootmodule *mod = kinfo->kernel_bootmodule;
> > +    void *fdt = kinfo->fdt;
> > +
> > +    dt_dprintk("Create chosen node\n");
> > +    res = fdt_begin_node(fdt, "chosen");
> > +    if ( res )
> > +        return res;
> > +
> > +    if ( mod && mod->cmdline[0] )
> > +    {
> > +        bootargs = &mod->cmdline[0];
> > +        res = fdt_property(fdt, "bootargs", bootargs, strlen(bootargs) +
> > 1);
> > +        if ( res )
> > +           return res;
> > +    }
> > +
> > +    /*
> > +     * If the bootloader provides an initrd, we must create a placeholder
> > +     * for the initrd properties. The values will be replaced later.
> > +     */
> > +    if ( mod && mod->size )
> > +    {
> > +        u64 a = 0;
> > +        res = fdt_property(kinfo->fdt, "linux,initrd-start", &a,
> > sizeof(a));
> > +        if ( res )
> > +            return res;
> > +
> > +        res = fdt_property(kinfo->fdt, "linux,initrd-end", &a, sizeof(a));
> > +        if ( res )
> > +            return res;
> > +    }
> > +
> > +    res = fdt_end_node(fdt);
> > +
> > +    return res;
> > +}
> > +
> >   static int map_irq_to_domain(struct domain *d, unsigned int irq,
> >                                bool need_mapping, const char *devname)
> >   @@ -1420,47 +1461,6 @@ static int acpi_route_spis(struct domain *d)
> >       return 0;
> >   }
> >   -static int acpi_make_chosen_node(const struct kernel_info *kinfo)
> > -{
> > -    int res;
> > -    const char *bootargs = NULL;
> > -    const struct bootmodule *mod = kinfo->kernel_bootmodule;
> > -    void *fdt = kinfo->fdt;
> > -
> > -    dt_dprintk("Create chosen node\n");
> > -    res = fdt_begin_node(fdt, "chosen");
> > -    if ( res )
> > -        return res;
> > -
> > -    if ( mod && mod->cmdline[0] )
> > -    {
> > -        bootargs = &mod->cmdline[0];
> > -        res = fdt_property(fdt, "bootargs", bootargs, strlen(bootargs) +
> > 1);
> > -        if ( res )
> > -           return res;
> > -    }
> > -
> > -    /*
> > -     * If the bootloader provides an initrd, we must create a placeholder
> > -     * for the initrd properties. The values will be replaced later.
> > -     */
> > -    if ( mod && mod->size )
> > -    {
> > -        u64 a = 0;
> > -        res = fdt_property(kinfo->fdt, "linux,initrd-start", &a,
> > sizeof(a));
> > -        if ( res )
> > -            return res;
> > -
> > -        res = fdt_property(kinfo->fdt, "linux,initrd-end", &a, sizeof(a));
> > -        if ( res )
> > -            return res;
> > -    }
> > -
> > -    res = fdt_end_node(fdt);
> > -
> > -    return res;
> > -}
> > -
> >   static int acpi_make_hypervisor_node(const struct kernel_info *kinfo,
> >                                        struct membank tbl_add[])
> >   {
> > @@ -1532,7 +1532,7 @@ static int create_acpi_dtb(struct kernel_info *kinfo,
> > struct membank tbl_add[])
> >           return ret;
> >         /* Create a chosen node for DOM0 */
> > -    ret = acpi_make_chosen_node(kinfo);
> > +    ret = make_chosen_node(kinfo);
> >       if ( ret )
> >           goto err;
> >   
> 
> Cheers,
> 
> -- 
> Julien Grall
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 09/15] xen/arm: refactor construct_dom0

[Stefano Stabellini]

On Thu, 14 Jun 2018, Julien Grall wrote:
> Hi Stefano,
> 
> On 13/06/18 23:15, Stefano Stabellini wrote:
> > Move generic initializations out of construct_dom0 so that they can be
> > reused.
> > 
> > No functional changes in this patch.
> > 
> > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > ---
> >   xen/arch/arm/domain_build.c | 124
> > ++++++++++++++++++++++++--------------------
> >   1 file changed, 67 insertions(+), 57 deletions(-)
> > 
> > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > index 4e4cd19..b31c563 100644
> > --- a/xen/arch/arm/domain_build.c
> > +++ b/xen/arch/arm/domain_build.c
> > @@ -2092,73 +2092,27 @@ static void __init find_gnttab_region(struct domain
> > *d,
> >              kinfo->gnttab_start, kinfo->gnttab_start + kinfo->gnttab_size);
> >   }
> >   -int __init construct_dom0(struct domain *d)
> > +int __init __construct_domain(struct domain *d, struct kernel_info *kinfo)
> >   {
> > -    struct kernel_info kinfo = {};
> >       struct vcpu *saved_current;
> > -    int rc, i, cpu;
> > +    int i, cpu;
> >         struct vcpu *v = d->vcpu[0];
> >       struct cpu_user_regs *regs = &v->arch.cpu_info->guest_cpu_user_regs;
> >   -    /* Sanity! */
> > -    BUG_ON(d->domain_id != 0);
> > -    BUG_ON(d->vcpu[0] == NULL);
> > -    BUG_ON(v->is_initialised);
> 
> I think we want to keep the last two BUG_ON here. "d->vcpu[0] == NULL" is
> making sure that the value used when setting current is sane.
> 
> "v->is_initalised" is making sure this function is not called twice by
> mistake. Indeed the variable will be set at the end of __construct_domain.

OK, I'll make the change.


> > -
> > -    printk("*** LOADING DOMAIN 0 ***\n");
> > -    if ( dom0_mem <= 0 )
> > -    {
> > -        warning_add("PLEASE SPECIFY dom0_mem PARAMETER - USING 512M FOR
> > NOW\n");
> > -        dom0_mem = MB(512);
> > -    }
> > -
> > -
> > -    iommu_hwdom_init(d);
> > -
> > -    d->max_pages = ~0U;
> > -
> > -    kinfo.unassigned_mem = dom0_mem;
> > -    kinfo.d = d;
> > -
> > -    rc = kernel_probe(&kinfo);
> > -    if ( rc < 0 )
> > -        return rc;
> > -
> >   #ifdef CONFIG_ARM_64
> >       /* if aarch32 mode is not supported at EL1 do not allow 32-bit domain
> > */
> > -    if ( !(cpu_has_el1_32) && kinfo.type == DOMAIN_32BIT )
> > +    if ( !(cpu_has_el1_32) && kinfo->type == DOMAIN_32BIT )
> >       {
> >           printk("Platform does not support 32-bit domain\n");
> >           return -EINVAL;
> >       }
> > -    d->arch.type = kinfo.type;
> 
> Any reason to move this out?

Yeah, initially I left it there but it didn't work. It needs to be set
before calling allocate_memory() for domUs otherwise memory allocations
fail.


> >         if ( is_64bit_domain(d) )
> >           vcpu_switch_to_aarch64_mode(v);
> >     #endif
> >   -    allocate_memory(d, &kinfo);
> 
> I think this could stay here.

Same as before, if I leave it, it doesn't work. allocate_memory() needs
be called before prepare_dtb_domU.


> > -    find_gnttab_region(d, &kinfo);
> > -
> > -    /* Map extra GIC MMIO, irqs and other hw stuffs to dom0. */
> > -    rc = gic_map_hwdom_extra_mappings(d);
> > -    if ( rc < 0 )
> > -        return rc;
> > -
> > -    rc = platform_specific_mapping(d);
> > -    if ( rc < 0 )
> > -        return rc;
> > -
> > -    if ( acpi_disabled )
> > -        rc = prepare_dtb(d, &kinfo);
> > -    else
> > -        rc = prepare_acpi(d, &kinfo);
> 
> I think this is probably a call to add "hwdom" in the name of the 2 functions.

I'll do the rename in a later patch (and I'll use hwdom instead of dom0).


> > -
> > -    if ( rc < 0 )
> > -        return rc;
> > -
> >       /*
> >        * The following loads use the domain's p2m and require current to
> >        * be a vcpu of the domain, temporarily switch
> > @@ -2171,20 +2125,18 @@ int __init construct_dom0(struct domain *d)
> >        * kernel_load will determine the placement of the kernel as well
> >        * as the initrd & fdt in RAM, so call it first.
> >        */
> > -    kernel_load(&kinfo);
> > +    kernel_load(kinfo);
> >       /* initrd_load will fix up the fdt, so call it before dtb_load */
> > -    initrd_load(&kinfo);
> > -    dtb_load(&kinfo);
> > +    initrd_load(kinfo);
> > +    dtb_load(kinfo);
> >         /* Now that we are done restore the original p2m and current. */
> >       set_current(saved_current);
> >       p2m_restore_state(saved_current);
> >   -    discard_initial_modules();
> > -
> >       memset(regs, 0, sizeof(*regs));
> >   -    regs->pc = (register_t)kinfo.entry;
> > +    regs->pc = (register_t)kinfo->entry;
> >         if ( is_32bit_domain(d) )
> >       {
> > @@ -2202,14 +2154,14 @@ int __init construct_dom0(struct domain *d)
> >            */
> >           regs->r0 = 0; /* SBZ */
> >           regs->r1 = 0xffffffff; /* We use DTB therefore no machine id */
> > -        regs->r2 = kinfo.dtb_paddr;
> > +        regs->r2 = kinfo->dtb_paddr;
> >       }
> >   #ifdef CONFIG_ARM_64
> >       else
> >       {
> >           regs->cpsr = PSR_GUEST64_INIT;
> >           /* From linux/Documentation/arm64/booting.txt */
> > -        regs->x0 = kinfo.dtb_paddr;
> > +        regs->x0 = kinfo->dtb_paddr;
> >           regs->x1 = 0; /* Reserved for future use */
> >           regs->x2 = 0; /* Reserved for future use */
> >           regs->x3 = 0; /* Reserved for future use */
> > @@ -2235,6 +2187,64 @@ int __init construct_dom0(struct domain *d)
> >       return 0;
> >   }
> >   +int __init construct_dom0(struct domain *d)
> > +{
> > +    struct kernel_info kinfo = {};
> > +    int rc;
> > +
> > +    struct vcpu *v = d->vcpu[0];
> > +
> > +    /* Sanity! */
> > +    BUG_ON(d->domain_id != 0);
> > +    BUG_ON(d->vcpu[0] == NULL);
> > +    BUG_ON(v->is_initialised);
> > +
> > +    printk("*** LOADING DOMAIN 0 ***\n");
> > +    if ( dom0_mem <= 0 )
> > +    {
> > +        warning_add("PLEASE SPECIFY dom0_mem PARAMETER - USING 512M FOR
> > NOW\n");
> > +        dom0_mem = MB(512);
> > +    }
> > +
> > +
> 
> No need to copy the second newline :).

:-)


> > +    iommu_hwdom_init(d);
> > +
> > +    d->max_pages = ~0U;
> > +
> > +    kinfo.unassigned_mem = dom0_mem;
> > +    kinfo.d = d;
> > +
> > +    rc = kernel_probe(&kinfo);
> > +    if ( rc < 0 )
> > +        return rc;
> > +
> > +    allocate_memory(d, &kinfo);
> > +    find_gnttab_region(d, &kinfo);
> > +
> > +    /* Map extra GIC MMIO, irqs and other hw stuffs to dom0. */
> > +    rc = gic_map_hwdom_extra_mappings(d);
> > +    if ( rc < 0 )
> > +        return rc;
> > +
> > +    rc = platform_specific_mapping(d);
> > +    if ( rc < 0 )
> > +        return rc;
> > +
> > +    d->arch.type = kinfo.type;
> > +
> > +    if ( acpi_disabled )
> > +        rc = prepare_dtb(d, &kinfo);
> > +    else
> > +        rc = prepare_acpi(d, &kinfo);
> > +
> > +    if ( rc < 0 )
> > +        return rc;
> > +
> > +    discard_initial_modules();
> 
> You say "no functional change" in this patch. But this is one. The module are
> now discard much earlier. This imply that memory baking the Image/Initrd will
> be free to be re-used at any time.
>
> I don't think this is what we want. Unless you can promise no memory is
> allocated in __construct_domain().

discard_initial_modules() will be moved later by patch #14, but I think
it makes sense to call discard_initial_modules() after
__construct_domain() here.


> > +
> > +    return __construct_domain(d, &kinfo);
> > +}
> > +
> >   /*
> >    * Local variables:
> >    * mode: C
> > 
> 
> Cheers,
> 
> -- 
> Julien Grall
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 02/15] xen/arm: move a few guest related #defines to public/arch-arm.h

[Julien Grall]

Hi Stefano,

On 06/14/2018 10:15 PM, Stefano Stabellini wrote:
> On Thu, 14 Jun 2018, Julien Grall wrote:
>> On 13/06/18 23:15, Stefano Stabellini wrote:
>>> Move a few constants defined by libxl_arm.c to
>>> xen/include/public/arch-arm.h, so that they are together with the other
>>> guest related #defines such as GUEST_GICD_BASE and GUEST_VPL011_SPI.
>>> Also, this way they can be reused by hypervisor code.
>>
>> All variables moved to arch-arm.h should be prefixed with GUEST_* to avoid
>> clash with the rest of Xen.
> 
> I'll do.
> 
>>>
>>> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
>>> CC: wei.liu2@citrix.com
>>> CC: ian.jackson@eu.citrix.com
>>> ---
>>>    tools/libxl/libxl_arm.c       | 26 --------------------------
>>>    xen/include/public/arch-arm.h | 26 ++++++++++++++++++++++++++
>>>    2 files changed, 26 insertions(+), 26 deletions(-)
>>>
>>> diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
>>> index 8af9f6f..89a417f 100644
>>> --- a/tools/libxl/libxl_arm.c
>>> +++ b/tools/libxl/libxl_arm.c
>>> @@ -8,23 +8,6 @@
>>>    #include <libfdt.h>
>>>    #include <assert.h>
>>>    -/**
>>> - * IRQ line type.
>>> - * DT_IRQ_TYPE_NONE            - default, unspecified type
>>> - * DT_IRQ_TYPE_EDGE_RISING     - rising edge triggered
>>> - * DT_IRQ_TYPE_EDGE_FALLING    - falling edge triggered
>>> - * DT_IRQ_TYPE_EDGE_BOTH       - rising and falling edge triggered
>>> - * DT_IRQ_TYPE_LEVEL_HIGH      - high level triggered
>>> - * DT_IRQ_TYPE_LEVEL_LOW       - low level triggered
>>> - */
>>> -#define DT_IRQ_TYPE_NONE           0x00000000
>>> -#define DT_IRQ_TYPE_EDGE_RISING    0x00000001
>>> -#define DT_IRQ_TYPE_EDGE_FALLING   0x00000002
>>> -#define DT_IRQ_TYPE_EDGE_BOTH                           \
>>> -    (DT_IRQ_TYPE_EDGE_FALLING | DT_IRQ_TYPE_EDGE_RISING)
>>> -#define DT_IRQ_TYPE_LEVEL_HIGH     0x00000004
>>> -#define DT_IRQ_TYPE_LEVEL_LOW      0x00000008
>>> -
>>
>> Those defines have nothing to do with the guest itself. They are currently
>> define in Xen without the DT_ prefix.
> 
> Sounds like we want to get rid of the DT_IRQ_TYPE_* definitions
> completely, move the IRQ_TYPE_* definitions from device_tree.h to here,
> and start using them in tools/libxl/libxl_arm.c (which involves a
> renaming s/DT_IRQ_TYPE/IRQ_TYPE/g).
> 
> Is that what you had in mind?

Even if DT is Arm only today, the DT code is in common code and 
therefore header device_tree.h should contain every thing necessary to 
use a DT.

If we still want to share constant with libxl then I would prefer to 
introduce a new header (similar to acpi/acconfig.h) that provide all the 
common values.

Note that the hypervisor one don't have the DT_ prefix because they are 
use to describe IRQ for both DT and ACPI in Xen. It is not that nice, we 
might want to introduce aliases in that case. So we keep DT_* in libxl.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 09/15] xen/arm: refactor construct_dom0

[Julien Grall]

Hi Stefano,

On 06/15/2018 12:35 AM, Stefano Stabellini wrote:
> On Thu, 14 Jun 2018, Julien Grall wrote:
>> On 13/06/18 23:15, Stefano Stabellini wrote:
>>> -
>>> -    printk("*** LOADING DOMAIN 0 ***\n");
>>> -    if ( dom0_mem <= 0 )
>>> -    {
>>> -        warning_add("PLEASE SPECIFY dom0_mem PARAMETER - USING 512M FOR
>>> NOW\n");
>>> -        dom0_mem = MB(512);
>>> -    }
>>> -
>>> -
>>> -    iommu_hwdom_init(d);
>>> -
>>> -    d->max_pages = ~0U;
>>> -
>>> -    kinfo.unassigned_mem = dom0_mem;
>>> -    kinfo.d = d;
>>> -
>>> -    rc = kernel_probe(&kinfo);
>>> -    if ( rc < 0 )
>>> -        return rc;
>>> -
>>>    #ifdef CONFIG_ARM_64
>>>        /* if aarch32 mode is not supported at EL1 do not allow 32-bit domain
>>> */
>>> -    if ( !(cpu_has_el1_32) && kinfo.type == DOMAIN_32BIT )
>>> +    if ( !(cpu_has_el1_32) && kinfo->type == DOMAIN_32BIT )
>>>        {
>>>            printk("Platform does not support 32-bit domain\n");
>>>            return -EINVAL;
>>>        }
>>> -    d->arch.type = kinfo.type;
>>
>> Any reason to move this out?
> 
> Yeah, initially I left it there but it didn't work. It needs to be set
> before calling allocate_memory() for domUs otherwise memory allocations
> fail.

Oh because allocate_domain(d) rely on is_domain_32bit, right? I don't 
much like the duplication here just because of prepare_dtb_domU. I am 
wondering if we could do:

if ( !is_hardware_domain(d) )
   prepare_dtb_domU(...);
else if ( acpi_disabled )
   prepare_acpi_hwdom(...);
else
   prepare_dt_hwdom(....);

>>> +    if ( acpi_disabled )
>>> +        rc = prepare_dtb(d, &kinfo);
>>> +    else
>>> +        rc = prepare_acpi(d, &kinfo);
>>> +
>>> +    if ( rc < 0 )
>>> +        return rc;
>>> +
>>> +    discard_initial_modules();
>>
>> You say "no functional change" in this patch. But this is one. The module are
>> now discard much earlier. This imply that memory baking the Image/Initrd will
>> be free to be re-used at any time.
>>
>> I don't think this is what we want. Unless you can promise no memory is
>> allocated in __construct_domain().
> 
> discard_initial_modules() will be moved later by patch #14, but I think
> it makes sense to call discard_initial_modules() after
> __construct_domain() here.

Yeah, I noticed you moved the discard_initial_modules() later on. But I 
would like to have the series bisectable if possible :).

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 12/15] xen/arm: generate vpl011 node on device tree for domU

[Julien Grall]

Hi Stefano,

On 06/13/2018 11:15 PM, Stefano Stabellini wrote:
> Introduce vpl011 support to guests started from Xen: it provides a
> simple way to print output from a guest, as most guests come with a
> pl011 driver. It is also able to provide a working console with
> interrupt support.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> ---
>   xen/arch/arm/domain_build.c | 70 +++++++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 70 insertions(+)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index b4f560f..ff65057 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -1470,6 +1470,70 @@ static int make_timer_domU_node(const struct domain *d, void *fdt)
>       return res;
>   }
>   
> +static void set_interrupt(gic_interrupt_t *interrupt, unsigned int irq,

The definition of interrupt looks suspicious. gic_interrupt_t is defined 
as be32[3]. Here you pass a pointer, so interrupt type would be __be32 
**, that you crudely cast to __be32* below.

Most likely you don't want to pass a pointer here and just use the type 
gic_interrupt_t. Because it is an array, then there will be no issue.

> +                          unsigned int cpumask, unsigned int level)
> +{
> +    __be32 *cells = (__be32 *) interrupt;

Explicit cast are always a bad idea. If you need one, then mostly likely 
you did something wrong :). In that case interrupt type is __be32** and 
you cast to __be32*. If you change the type as suggested above, then the 
cast will not be necessary here.

> +    int is_ppi = (irq < 32);
> +
> +    irq -= (is_ppi) ? 16: 32; /* PPIs start at 16, SPIs at 32 */
> +
> +    /* See linux Documentation/devictree/bindings/arm/gic.txt */
> +    dt_set_cell(&cells, 1, is_ppi); /* is a PPI? */
> +    dt_set_cell(&cells, 1, irq);
> +    dt_set_cell(&cells, 1, (cpumask << 8) | level);
> +}

We already have a function to generate PPI interrupt 
(set_interrupt_ppi). Would it be possible to extend it to support interrupt?

Most likely, you will want to use set_interrupt(...) everywhere and just 
drop set_interrupt_ppi.

> +
> +#ifdef CONFIG_SBSA_VUART_CONSOLE
> +static int make_vpl011_uart_node(const struct domain *d, void *fdt,
> +                                 int addrcells, int sizecells)
> +{
> +    int res;
> +    gic_interrupt_t intr;
> +    int reg_size = addrcells + sizecells;
> +    int nr_cells = reg_size;
> +    __be32 reg[nr_cells];
> +    __be32 *cells;
> +
> +    res = fdt_begin_node(fdt, "sbsa-pl011");
> +    if (res)

Coding style:

if ( ... )

> +        return res;
> +
> +    res = fdt_property_string(fdt, "compatible", "arm,sbsa-uart");

To make clear, you are exposing a SBSA compatible UART and not a PL011. 
SBSA UART is a subset of PL011 r1p5. A full PL011 implementation in Xen 
would just be too difficult, so your guest may require some changes in 
their driver.

I think this is a small price to pay, but I wanted to make sure you 
don't expect the guest to drive the UART the same way a PL011.

> +    if (res)

Coding style

> +        return res;
> +
> +    cells = &reg[0];
> +    dt_child_set_range(&cells, addrcells, sizecells, GUEST_PL011_BASE,
> +            GUEST_PL011_SIZE);

The indentation looks wrong here.

> +    if (res)

Coding style

> +        return res;
> +    res = fdt_property(fdt, "reg", reg, sizeof(reg));
> +    if (res)

Coding style

> +        return res;
> +
> +    set_interrupt(&intr, GUEST_VPL011_SPI, 0xf, DT_IRQ_TYPE_LEVEL_HIGH);
> +
> +    res = fdt_property(fdt, "interrupts", intr, sizeof (intr));
> +    if (res)

Coding style

> +        return res;
> +
> +    res = fdt_property_cell(fdt, "interrupt-parent",
> +                            PHANDLE_GIC);
> +    if (res)

Coding style

> +        return res;
> +
> +    /* Use a default baud rate of 115200. */
> +    fdt_property_u32(fdt, "current-speed", 115200);
> +
> +    res = fdt_end_node(fdt);
> +    if (res)

Coding style

> +        return res;
> +
> +    return 0;
> +}
> +#endif
> +
>   #define DOMU_DTB_SIZE 4096
>   static int prepare_dtb_domU(struct domain *d, struct kernel_info *kinfo)
>   {
> @@ -1531,6 +1595,12 @@ static int prepare_dtb_domU(struct domain *d, struct kernel_info *kinfo)
>       if ( ret )
>           goto err;
>   
> +#ifdef CONFIG_SBSA_VUART_CONSOLE.
> +    ret = make_vpl011_uart_node(d, kinfo->fdt, addrcells, sizecells);

I would prefer if don't expose the pl011 by default to a guest and 
provide a way to enable it for a given guest

> +    if ( ret )
> +        goto err;
> +#endif
> +
>       ret = fdt_end_node(kinfo->fdt);
>       if ( ret < 0 )
>           goto err;
> 

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 13/15] xen/arm: Allow vpl011 to be used by DomU

[Julien Grall]

Hi Stefano,

On 06/13/2018 11:15 PM, Stefano Stabellini wrote:
> Make vpl011 being able to be used without a userspace component in Dom0.
> In that case, output is printed to the Xen serial and input is received
> from the Xen serial one character at a time.
> 
> Call domain_vpl011_init during construct_domU.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> ---
>   xen/arch/arm/domain_build.c  |  9 +++-
>   xen/arch/arm/vpl011.c        | 98 +++++++++++++++++++++++++++++++++-----------
>   xen/include/asm-arm/vpl011.h |  2 +
>   3 files changed, 84 insertions(+), 25 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index ff65057..97f14ca 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -2482,7 +2482,14 @@ int __init construct_domU(struct domain *d, struct dt_device_node *node)
>       if ( rc < 0 )
>           return rc;
>   
> -    return __construct_domain(d, &kinfo);
> +    rc = __construct_domain(d, &kinfo);
> +    if ( rc < 0 )
> +        return rc;
> +
> +#ifdef CONFIG_SBSA_VUART_CONSOLE
> +    rc = domain_vpl011_init(d, NULL);

See my remark on the previous patch about exposing vpl011 by default.

> +#endif
> +    return rc;
>   }
>   
>   int __init construct_dom0(struct domain *d)
> diff --git a/xen/arch/arm/vpl011.c b/xen/arch/arm/vpl011.c
> index a281eab..5f1dc7a 100644
> --- a/xen/arch/arm/vpl011.c
> +++ b/xen/arch/arm/vpl011.c
> @@ -34,6 +34,8 @@
>   #include <asm/vgic-emul.h>
>   #include <asm/vpl011.h>
>   
> +static void vpl011_data_avail(struct domain *d);
> +
>   /*
>    * Since pl011 registers are 32-bit registers, all registers
>    * are handled similarly allowing 8-bit, 16-bit and 32-bit
> @@ -77,6 +79,29 @@ static void vpl011_update_interrupt_status(struct domain *d)
>   #endif
>   }
>   
> +void vpl011_read_char(struct domain *d, char c)

The name is slightly odd. From the name, I would expect that a character 
is returned. But in fact, you write a character you received in the 
ring. So a better name would be vpl011_rx_char.

> +{
> +    unsigned long flags;
> +    XENCONS_RING_IDX in_cons, in_prod;
> +    struct xencons_interface *intf = d->arch.vpl011.ring_buf;
> +
> +    VPL011_LOCK(d, flags);
> +
> +    in_cons = intf->in_cons;
> +    in_prod = intf->in_prod;
> +    if (xencons_queued(in_prod, in_cons, sizeof(intf->in)) == sizeof(intf->in))
> +    {
> +        VPL011_UNLOCK(d, flags);
> +        return;
> +    }
> +
> +    intf->in[xencons_mask(in_prod, sizeof(intf->in))] = c;
> +    intf->in_prod = in_prod + 1;
> +
> +    VPL011_UNLOCK(d, flags);
> +    vpl011_data_avail(d);
> +}
> +
>   static uint8_t vpl011_read_data(struct domain *d)
>   {
>       unsigned long flags;
> @@ -166,9 +191,18 @@ static void vpl011_write_data(struct domain *d, uint8_t data)
>       struct vpl011 *vpl011 = &d->arch.vpl011;
>       struct xencons_interface *intf = vpl011->ring_buf;
>       XENCONS_RING_IDX out_cons, out_prod;
> +    unsigned int fifo_level = 0;
>   
>       VPL011_LOCK(d, flags);
>   
> +    if ( vpl011->ring_page == NULL )
> +    {
> +        printk("%c", data);
> +        if (data == '\n')
> +            printk("DOM%u: ", d->domain_id);
> +        goto done;
> +    }
> +

I would rather introduce separate function to read/write data for the 
case without PV console. And use it where appropriate. This would make 
the code slightly easier to understand because "ring_page == NULL" is 
slightly untuitive.

An idea would be introduce callback and set them during the 
initialization of the vpl011 for the domain.

>       out_cons = intf->out_cons;
>       out_prod = intf->out_prod;
>   
> @@ -184,13 +218,10 @@ static void vpl011_write_data(struct domain *d, uint8_t data)
>       if ( xencons_queued(out_prod, out_cons, sizeof(intf->out)) !=
>            sizeof (intf->out) )
>       {
> -        unsigned int fifo_level;
> -
>           intf->out[xencons_mask(out_prod, sizeof(intf->out))] = data;
>           out_prod += 1;
>           smp_wmb();
>           intf->out_prod = out_prod;
> -

Spurious change.

>           fifo_level = xencons_queued(out_prod, out_cons, sizeof(intf->out));
>   
>           if ( fifo_level == sizeof(intf->out) )
> @@ -205,14 +236,15 @@ static void vpl011_write_data(struct domain *d, uint8_t data)
>                */
>               vpl011->uartfr |= BUSY;
>           }
> -
> -        vpl011_update_tx_fifo_status(vpl011, fifo_level);
> -
> -        vpl011_update_interrupt_status(d);
>       }
>       else
>           gprintk(XENLOG_ERR, "vpl011: Unexpected OUT ring buffer full\n");
>   
> +done:
> +    vpl011_update_tx_fifo_status(vpl011, fifo_level);
> +
> +    vpl011_update_interrupt_status(d);

Hmmm, now you will call vpl011_update_* in the error case when writing 
to the case. If you want to keep that, this should at least be explained 
in the commit message or probably be a separate patch.

> +
>       vpl011->uartfr &= ~TXFE;
>   
>       VPL011_UNLOCK(d, flags);
> @@ -462,13 +494,30 @@ int domain_vpl011_init(struct domain *d, struct vpl011_init_info *info)
>       if ( vpl011->ring_buf )
>           return -EINVAL;
>   
> -    /* Map the guest PFN to Xen address space. */
> -    rc =  prepare_ring_for_helper(d,
> -                                  gfn_x(info->gfn),
> -                                  &vpl011->ring_page,
> -                                  &vpl011->ring_buf);
> -    if ( rc < 0 )
> -        goto out;
> +    if ( info != NULL )

Please document how info could be NULL here.

> +    {
> +        /* Map the guest PFN to Xen address space. */
> +        rc =  prepare_ring_for_helper(d,
> +                gfn_x(info->gfn),
> +                &vpl011->ring_page,
> +                &vpl011->ring_buf);
> +        if ( rc < 0 )
> +            goto out;
> +
> +        rc = alloc_unbound_xen_event_channel(d, 0, info->console_domid,
> +                vpl011_notification);
> +        if ( rc < 0 )
> +            goto out2;

When you move code around, you should also look at the error path. In 
that case, you are going to free a virq that does not exist (because not 
yet allocated), and if vgic_reserve_virq below fails, you will not free 
the event channel allocated.

> +
> +        vpl011->evtchn = info->evtchn = rc;
> +    } else {
> +        vpl011->ring_buf = xzalloc(struct xencons_interface);

Using ring_buf is such waste of memory. You basically only allow 1024 
character but still using 4K.

Furthermore, the way you use ring_page is really confusing. A bit more 
documentation might help. Although, this new code does not feel 
integrated with the rest of the vpl011.

It looks like you want to rework the vpl011 code to have move anything 
related to ring in separate function. Once it is done, we could then 
introduce new callback for the guest started in Xen.

> +        if ( vpl011->ring_buf == NULL )
> +        {
> +            rc = -EINVAL;
> +            goto out;
> +        }
> +    }
>   
>       rc = vgic_reserve_virq(d, GUEST_VPL011_SPI);
>       if ( !rc )
> @@ -477,13 +526,6 @@ int domain_vpl011_init(struct domain *d, struct vpl011_init_info *info)
>           goto out1;
>       }
>   
> -    rc = alloc_unbound_xen_event_channel(d, 0, info->console_domid,
> -                                         vpl011_notification);
> -    if ( rc < 0 )
> -        goto out2;
> -
> -    vpl011->evtchn = info->evtchn = rc;
> -
>       spin_lock_init(&vpl011->lock);
>   
>       register_mmio_handler(d, &vpl011_mmio_handler,
> @@ -495,7 +537,10 @@ out2:
>       vgic_free_virq(d, GUEST_VPL011_SPI);
>   
>   out1:
> -    destroy_ring_for_helper(&vpl011->ring_buf, vpl011->ring_page);
> +    if ( vpl011->ring_page == NULL && vpl011->ring_buf != NULL )
> +        xfree(vpl011->ring_buf);
> +    else
> +        destroy_ring_for_helper(&vpl011->ring_buf, vpl011->ring_page);
>   
>   out:
>       return rc;
> @@ -508,8 +553,13 @@ void domain_vpl011_deinit(struct domain *d)
>       if ( !vpl011->ring_buf )
>           return;
>   
> -    free_xen_event_channel(d, vpl011->evtchn);
> -    destroy_ring_for_helper(&vpl011->ring_buf, vpl011->ring_page);
> +    if ( vpl011->ring_page == NULL && vpl011->ring_buf != NULL )
> +    {
> +        xfree(vpl011->ring_buf);
> +    } else {

Coding style

}
else
{

> +        free_xen_event_channel(d, vpl011->evtchn);
> +        destroy_ring_for_helper(&vpl011->ring_buf, vpl011->ring_page);
> +    }
>   }
>   
>   /*
> diff --git a/xen/include/asm-arm/vpl011.h b/xen/include/asm-arm/vpl011.h
> index db95ff8..8d9b0da 100644
> --- a/xen/include/asm-arm/vpl011.h
> +++ b/xen/include/asm-arm/vpl011.h
> @@ -53,6 +53,7 @@ struct vpl011_init_info {
>   int domain_vpl011_init(struct domain *d,
>                          struct vpl011_init_info *info);
>   void domain_vpl011_deinit(struct domain *d);
> +void vpl011_read_char(struct domain *d, char c);
>   #else
>   static inline int domain_vpl011_init(struct domain *d,
>                                        struct vpl011_init_info *info)
> @@ -61,6 +62,7 @@ static inline int domain_vpl011_init(struct domain *d,
>   }
>   
>   static inline void domain_vpl011_deinit(struct domain *d) { }
> +static inline void vpl011_read_char(struct domain *d, char c) { }
>   #endif
>   #endif  /* _VPL011_H_ */
>   
> 

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 14/15] xen/arm: call construct_domU from start_xen and start DomU VMs

[Julien Grall]

On 06/13/2018 11:15 PM, Stefano Stabellini wrote:
> Introduce support for the "xen,domU" compatible node on device tree.
> Create new DomU VMs based on the information found on device tree under
> "xen,domU".

While I like the idea of having multiple domain created by Xen, I think 
there are still few open questions here:
	1) The domains will be listed via "xl list". So are they still 
manageable via DOMCTL?
	2) Is it possible to restart those domains?
	3) If a domain crash, what will happen? Are they just going to 	sit 
there using resources until the platform rebooted?
	4) How do you handle scheduling? Is it still possible to do it via 
Dom0? How about the dom0less situation?

> 
> Introduce a simple global variable named max_init_domid to keep track of
> the initial allocated domids.

What is the exact goal of this new variable?

> 
> Move the discard_initial_modules after DomUs have been built
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> ---
>   xen/arch/arm/domain_build.c |  2 --
>   xen/arch/arm/setup.c        | 35 ++++++++++++++++++++++++++++++++++-
>   xen/include/asm-arm/setup.h |  2 ++
>   xen/include/asm-x86/setup.h |  2 ++

You need to CC x86 maintainers for this change.

>   4 files changed, 38 insertions(+), 3 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 97f14ca..e2d370f 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -2545,8 +2545,6 @@ int __init construct_dom0(struct domain *d)
>       if ( rc < 0 )
>           return rc;
>   
> -    discard_initial_modules();
> -

Please mention this move in the commit message.

>       return __construct_domain(d, &kinfo);
>   }
>   
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index 98bdb24..3723704 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -63,6 +63,8 @@ static unsigned long opt_xenheap_megabytes __initdata;
>   integer_param("xenheap_megabytes", opt_xenheap_megabytes);
>   #endif
>   
> +domid_t __read_mostly max_init_domid = 0;
> +
>   static __used void init_done(void)
>   {
>       free_init_memory();
> @@ -711,6 +713,8 @@ void __init start_xen(unsigned long boot_phys_offset,
>       struct bootmodule *xen_bootmodule;
>       struct domain *dom0;
>       struct xen_domctl_createdomain dom0_cfg = {};
> +    struct dt_device_node *chosen;
> +    struct dt_device_node *node;
>   
>       dcache_line_bytes = read_dcache_line_bytes();
>   
> @@ -860,7 +864,7 @@ void __init start_xen(unsigned long boot_phys_offset,
>       dom0_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
>       dom0_cfg.arch.nr_spis = gic_number_lines() - 32;
>   
> -    dom0 = domain_create(0, &dom0_cfg);
> +    dom0 = domain_create(max_init_domid++, &dom0_cfg);
>       if ( IS_ERR(dom0) || (alloc_dom0_vcpu0(dom0) == NULL) )
>               panic("Error creating domain 0");
>   
> @@ -886,6 +890,35 @@ void __init start_xen(unsigned long boot_phys_offset,
>   
>       domain_unpause_by_systemcontroller(dom0);
>   
> +    chosen = dt_find_node_by_name(dt_host, "chosen");
> +    if ( chosen != NULL )
> +    {
> +        dt_for_each_child_node(chosen, node)
> +        {
> +            struct domain *d;
> +            struct xen_domctl_createdomain d_cfg = {};

There are quite a few field in xen_domctl_createdomain that we may want 
to allow the user setting them. I am thinking of ssidref for XSM. How is 
this going to be done?

> +
> +            if ( !dt_device_is_compatible(node, "xen,domU") )
> +                continue;
> +
> +            d_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;

Any reason to impose using the native GIC here?

> +            d_cfg.arch.nr_spis = gic_number_lines() - 32;

That's a bit unfortunate. So you are imposing to use 1020 IRQs and the 
waste of memory associated when only 32 SPIs is enough at the moment.

> +
> +            d = domain_create(max_init_domid++, &d_cfg);
> +            if ( IS_ERR(d))

Coding style ( ... )

> +                panic("Error creating domU");
> +
> +            d->is_privileged = 0;
> +            d->target = NULL;

Why do you set them? They are zeroed by default.

> +
> +            if ( construct_domU(d, node) != 0)

Coding style ( ... )

> +                printk("Could not set up DOMU guest OS");
> +
> +            domain_unpause_by_systemcontroller(d);
> +        }
> +    }

Please introduce a new function, this would avoid to increate 
start_xen() too much.


> +    discard_initial_modules();
> +
>       /* Switch on to the dynamically allocated stack for the idle vcpu
>        * since the static one we're running on is about to be freed. */
>       memcpy(idle_vcpu[0]->arch.cpu_info, get_cpu_info(),
> diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
> index e9f9905..578f3b9 100644
> --- a/xen/include/asm-arm/setup.h
> +++ b/xen/include/asm-arm/setup.h
> @@ -56,6 +56,8 @@ struct bootinfo {
>   
>   extern struct bootinfo bootinfo;
>   
> +extern domid_t max_init_domid;
> +
>   void arch_init_memory(void);
>   
>   void copy_from_paddr(void *dst, paddr_t paddr, unsigned long len);
> diff --git a/xen/include/asm-x86/setup.h b/xen/include/asm-x86/setup.h
> index 19232af..2fb9529 100644
> --- a/xen/include/asm-x86/setup.h
> +++ b/xen/include/asm-x86/setup.h
> @@ -73,4 +73,6 @@ extern bool opt_dom0_shadow;
>   #endif
>   extern bool dom0_pvh;
>   
> +#define max_init_domid (1)
> +
>   #endif
> 

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 02/15] xen/arm: move a few guest related #defines to public/arch-arm.h

[Wei Liu]

On Wed, Jun 13, 2018 at 03:15:05PM -0700, Stefano Stabellini wrote:
> Move a few constants defined by libxl_arm.c to
> xen/include/public/arch-arm.h, so that they are together with the other
> guest related #defines such as GUEST_GICD_BASE and GUEST_VPL011_SPI.
> Also, this way they can be reused by hypervisor code.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> CC: wei.liu2@citrix.com
> CC: ian.jackson@eu.citrix.com

FAOD I will defer this to Julien.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 02/15] xen/arm: move a few guest related #defines to public/arch-arm.h

[Stefano Stabellini]

On Fri, 15 Jun 2018, Julien Grall wrote:
> Hi Stefano,
> 
> On 06/14/2018 10:15 PM, Stefano Stabellini wrote:
> > On Thu, 14 Jun 2018, Julien Grall wrote:
> > > On 13/06/18 23:15, Stefano Stabellini wrote:
> > > > Move a few constants defined by libxl_arm.c to
> > > > xen/include/public/arch-arm.h, so that they are together with the other
> > > > guest related #defines such as GUEST_GICD_BASE and GUEST_VPL011_SPI.
> > > > Also, this way they can be reused by hypervisor code.
> > > 
> > > All variables moved to arch-arm.h should be prefixed with GUEST_* to avoid
> > > clash with the rest of Xen.
> > 
> > I'll do.
> > 
> > > > 
> > > > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > > > CC: wei.liu2@citrix.com
> > > > CC: ian.jackson@eu.citrix.com
> > > > ---
> > > >    tools/libxl/libxl_arm.c       | 26 --------------------------
> > > >    xen/include/public/arch-arm.h | 26 ++++++++++++++++++++++++++
> > > >    2 files changed, 26 insertions(+), 26 deletions(-)
> > > > 
> > > > diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
> > > > index 8af9f6f..89a417f 100644
> > > > --- a/tools/libxl/libxl_arm.c
> > > > +++ b/tools/libxl/libxl_arm.c
> > > > @@ -8,23 +8,6 @@
> > > >    #include <libfdt.h>
> > > >    #include <assert.h>
> > > >    -/**
> > > > - * IRQ line type.
> > > > - * DT_IRQ_TYPE_NONE            - default, unspecified type
> > > > - * DT_IRQ_TYPE_EDGE_RISING     - rising edge triggered
> > > > - * DT_IRQ_TYPE_EDGE_FALLING    - falling edge triggered
> > > > - * DT_IRQ_TYPE_EDGE_BOTH       - rising and falling edge triggered
> > > > - * DT_IRQ_TYPE_LEVEL_HIGH      - high level triggered
> > > > - * DT_IRQ_TYPE_LEVEL_LOW       - low level triggered
> > > > - */
> > > > -#define DT_IRQ_TYPE_NONE           0x00000000
> > > > -#define DT_IRQ_TYPE_EDGE_RISING    0x00000001
> > > > -#define DT_IRQ_TYPE_EDGE_FALLING   0x00000002
> > > > -#define DT_IRQ_TYPE_EDGE_BOTH                           \
> > > > -    (DT_IRQ_TYPE_EDGE_FALLING | DT_IRQ_TYPE_EDGE_RISING)
> > > > -#define DT_IRQ_TYPE_LEVEL_HIGH     0x00000004
> > > > -#define DT_IRQ_TYPE_LEVEL_LOW      0x00000008
> > > > -
> > > 
> > > Those defines have nothing to do with the guest itself. They are currently
> > > define in Xen without the DT_ prefix.
> > 
> > Sounds like we want to get rid of the DT_IRQ_TYPE_* definitions
> > completely, move the IRQ_TYPE_* definitions from device_tree.h to here,
> > and start using them in tools/libxl/libxl_arm.c (which involves a
> > renaming s/DT_IRQ_TYPE/IRQ_TYPE/g).
> > 
> > Is that what you had in mind?
> 
> Even if DT is Arm only today, the DT code is in common code and therefore
> header device_tree.h should contain every thing necessary to use a DT.
> 
> If we still want to share constant with libxl then I would prefer to introduce
> a new header (similar to acpi/acconfig.h) that provide all the common values.

OK, I can do that. I'll introduce a new header file.


> Note that the hypervisor one don't have the DT_ prefix because they are use to
> describe IRQ for both DT and ACPI in Xen. It is not that nice, we might want
> to introduce aliases in that case. So we keep DT_* in libxl.

With the new header file we'll be able to reuse the same #defines in
libxl and xen. I think it would be nicer to avoid the aliases and just
use the regular definitions in libxl too? Changing libxl to use
IRQ_TYPE_ directly only requires a small patch.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 02/15] xen/arm: move a few guest related #defines to public/arch-arm.h

[Julien Grall]

Hi,

On 02/07/2018 21:37, Stefano Stabellini wrote:
> On Fri, 15 Jun 2018, Julien Grall wrote:
>> Hi Stefano,
>>
>> On 06/14/2018 10:15 PM, Stefano Stabellini wrote:
>>> On Thu, 14 Jun 2018, Julien Grall wrote:
>>>> On 13/06/18 23:15, Stefano Stabellini wrote:
>>>>> Move a few constants defined by libxl_arm.c to
>>>>> xen/include/public/arch-arm.h, so that they are together with the other
>>>>> guest related #defines such as GUEST_GICD_BASE and GUEST_VPL011_SPI.
>>>>> Also, this way they can be reused by hypervisor code.
>>>>
>>>> All variables moved to arch-arm.h should be prefixed with GUEST_* to avoid
>>>> clash with the rest of Xen.
>>>
>>> I'll do.
>>>
>>>>>
>>>>> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
>>>>> CC: wei.liu2@citrix.com
>>>>> CC: ian.jackson@eu.citrix.com
>>>>> ---
>>>>>     tools/libxl/libxl_arm.c       | 26 --------------------------
>>>>>     xen/include/public/arch-arm.h | 26 ++++++++++++++++++++++++++
>>>>>     2 files changed, 26 insertions(+), 26 deletions(-)
>>>>>
>>>>> diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
>>>>> index 8af9f6f..89a417f 100644
>>>>> --- a/tools/libxl/libxl_arm.c
>>>>> +++ b/tools/libxl/libxl_arm.c
>>>>> @@ -8,23 +8,6 @@
>>>>>     #include <libfdt.h>
>>>>>     #include <assert.h>
>>>>>     -/**
>>>>> - * IRQ line type.
>>>>> - * DT_IRQ_TYPE_NONE            - default, unspecified type
>>>>> - * DT_IRQ_TYPE_EDGE_RISING     - rising edge triggered
>>>>> - * DT_IRQ_TYPE_EDGE_FALLING    - falling edge triggered
>>>>> - * DT_IRQ_TYPE_EDGE_BOTH       - rising and falling edge triggered
>>>>> - * DT_IRQ_TYPE_LEVEL_HIGH      - high level triggered
>>>>> - * DT_IRQ_TYPE_LEVEL_LOW       - low level triggered
>>>>> - */
>>>>> -#define DT_IRQ_TYPE_NONE           0x00000000
>>>>> -#define DT_IRQ_TYPE_EDGE_RISING    0x00000001
>>>>> -#define DT_IRQ_TYPE_EDGE_FALLING   0x00000002
>>>>> -#define DT_IRQ_TYPE_EDGE_BOTH                           \
>>>>> -    (DT_IRQ_TYPE_EDGE_FALLING | DT_IRQ_TYPE_EDGE_RISING)
>>>>> -#define DT_IRQ_TYPE_LEVEL_HIGH     0x00000004
>>>>> -#define DT_IRQ_TYPE_LEVEL_LOW      0x00000008
>>>>> -
>>>>
>>>> Those defines have nothing to do with the guest itself. They are currently
>>>> define in Xen without the DT_ prefix.
>>>
>>> Sounds like we want to get rid of the DT_IRQ_TYPE_* definitions
>>> completely, move the IRQ_TYPE_* definitions from device_tree.h to here,
>>> and start using them in tools/libxl/libxl_arm.c (which involves a
>>> renaming s/DT_IRQ_TYPE/IRQ_TYPE/g).
>>>
>>> Is that what you had in mind?
>>
>> Even if DT is Arm only today, the DT code is in common code and therefore
>> header device_tree.h should contain every thing necessary to use a DT.
>>
>> If we still want to share constant with libxl then I would prefer to introduce
>> a new header (similar to acpi/acconfig.h) that provide all the common values.
> 
> OK, I can do that. I'll introduce a new header file.
> 
> 
>> Note that the hypervisor one don't have the DT_ prefix because they are use to
>> describe IRQ for both DT and ACPI in Xen. It is not that nice, we might want
>> to introduce aliases in that case. So we keep DT_* in libxl.
> 
> With the new header file we'll be able to reuse the same #defines in
> libxl and xen. I think it would be nicer to avoid the aliases and just
> use the regular definitions in libxl too? Changing libxl to use
> IRQ_TYPE_ directly only requires a small patch.

I really don't want that. As I explained, IRQ_TYPE have been 
conveniently chosen to avoid converting DT value to IRQ_TYPE. They are 
not meant to be use like that libxl (or any header publicly shared) and 
will provide more confusion that anything for other bits than the 
hypervisor.

So in the hypervisor case we want to:
	1) Define DT_* in a separate header
	2) Alias IRQ_TYPE_* to corresponding one.

In the libxl case we want to use the new header containing DT_*.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 03/15] xen/arm: extend device tree based multiboot protocol

[Stefano Stabellini]

On Thu, 14 Jun 2018, Julien Grall wrote:
> Hi Stefano,
> 
> On 13/06/18 23:15, Stefano Stabellini wrote:
> > Extend the existing device tree based multiboot protocol to include
> > information regarding other domUs to boot.
> > 
> > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > ---
> >   docs/misc/arm/device-tree/booting.txt | 102
> > ++++++++++++++++++++++++++++++++++
> >   1 file changed, 102 insertions(+)
> > 
> > diff --git a/docs/misc/arm/device-tree/booting.txt
> > b/docs/misc/arm/device-tree/booting.txt
> > index ce2d0dc..95255e5 100644
> > --- a/docs/misc/arm/device-tree/booting.txt
> > +++ b/docs/misc/arm/device-tree/booting.txt
> > @@ -119,3 +119,105 @@ For those you would hardcode the Xen commandline in
> > the DTB under
> >   line by writing bootargs (as for native Linux).
> >   A Xen-aware bootloader would set xen,xen-bootargs for Xen,
> > xen,dom0-bootargs
> >   for Dom0 and bootargs for native Linux.
> > +
> > +
> > +Creating DomUs directly from Xen
> > +================================
> > +
> > +It is possible to have Xen create other domains, in addition to dom0,
> > +out of the information provided via device tree. A Kernel and initrd
> 
> NIT: s/Kernel/kernel/

OK


> > +(optional) need to be specified for each guest.
> > +
> > +For each DomU to be created there needs to be one node under /chosen
> > +with the following properties:
> 
> I think it would be better to make this domain agnostic. I.e allow Dom0 to be
> created the same way but still supporting the current bindings.
> 
> We could differentiate Dom0 from the other with a property
> "xen,initial-domain". Note that I am not asking to add this property in this
> series. This is more a forward looking of the use of this binding.

It sounds like a good idea, I'll add it.


> > +
> > +- compatible
> > +
> > +    "xen,domU"
> 
> If we follow my suggestion, this would be renamed "xen,domain".

OK


> > +
> > +- mem (optional)
> 
> I would prefer the full name "memory".
 
Yes, especially given that the corresponding xl config file option is
called "memory".


> > +
> > +    A string specifying the amount of RAM to allocate to the guest. If
> > +    not specified it defaults to "64M". The format of the string is the
> > same
> > +    as the one for the mem= parameter in xl config files.
> 
> I don't much like default values because they are pretty arbitrary. This also
> does not match the default value for Dom0. Why don't you mandate the property?

Yes, I can do that.


> I would also prefer if the size is specified in the same way number are
> described in Device-Tree. I.e using cells. You could impose 2 cells for the
> size here.

see below


> > +
> > +- cpus (optional)
> > +
> > +    A string specifying the number of vcpus to allocate to the guest. If
> > +    not specified it defaults to "1".
> 
> Same remarks as for "mem".

I think it would be nicer if we kept the same format used for xl config
files for these two options. Especially given that we already have the
functions in the hypervisor to parse them (Xen knows how to parse
dom0_max_vcpus and dom0_mem for instance). It is going to be easier to
use and it doesn't come with a cost for Xen.

Edgar, what do you think?


> > +
> > +- #address-cells and #size-cells
> > +
> > +    Both #address-cells and #size-cells need to be specified because
> > +    both sub-nodes (described shortly) have reg properties.
> > +
> > +Under the "xen,domU" compatible node, one or more sub-nodes are present
> > +for the DomU kernel and ramdisk.
> > +
> > +The kernel sub-node has the following properties:
> > +
> > +- compatible
> > +
> > +    "multiboot,domU-kernel"
> 
> I don't think we need to specify a new compatible here. We could re-use
> "multiboot,kernel" here because the parent node will contain "xen,domU".

Yes you are right


> > +
> > +- reg
> > +
> > +    Specifies the physical address of the kernel in RAM and its
> > +    length.
> > +
> > +- bootargs (optional)
> > +
> > +    Command line parameters for the guest kernel.
> > +
> > +The ramdisk sub-node has the following properties:
> > +
> > +- compatible
> > +
> > +    "multiboot,domU-ramdisk"
> 
> Same here, we could re-use "multiboot,ramdisk".

OK


> > +
> > +- reg
> > +
> > +    Specifies the physical address of the ramdisk in RAM and its
> > +    length.
> 
> We should mention somewhere that this should be described in the /chosen node
> of the device-tree.

OK

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 02/15] xen/arm: move a few guest related #defines to public/arch-arm.h

[Stefano Stabellini]

On Mon, 2 Jul 2018, Julien Grall wrote:
> Hi,
> 
> On 02/07/2018 21:37, Stefano Stabellini wrote:
> > On Fri, 15 Jun 2018, Julien Grall wrote:
> > > Hi Stefano,
> > > 
> > > On 06/14/2018 10:15 PM, Stefano Stabellini wrote:
> > > > On Thu, 14 Jun 2018, Julien Grall wrote:
> > > > > On 13/06/18 23:15, Stefano Stabellini wrote:
> > > > > > Move a few constants defined by libxl_arm.c to
> > > > > > xen/include/public/arch-arm.h, so that they are together with the
> > > > > > other
> > > > > > guest related #defines such as GUEST_GICD_BASE and GUEST_VPL011_SPI.
> > > > > > Also, this way they can be reused by hypervisor code.
> > > > > 
> > > > > All variables moved to arch-arm.h should be prefixed with GUEST_* to
> > > > > avoid
> > > > > clash with the rest of Xen.
> > > > 
> > > > I'll do.
> > > > 
> > > > > > 
> > > > > > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > > > > > CC: wei.liu2@citrix.com
> > > > > > CC: ian.jackson@eu.citrix.com
> > > > > > ---
> > > > > >     tools/libxl/libxl_arm.c       | 26 --------------------------
> > > > > >     xen/include/public/arch-arm.h | 26 ++++++++++++++++++++++++++
> > > > > >     2 files changed, 26 insertions(+), 26 deletions(-)
> > > > > > 
> > > > > > diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
> > > > > > index 8af9f6f..89a417f 100644
> > > > > > --- a/tools/libxl/libxl_arm.c
> > > > > > +++ b/tools/libxl/libxl_arm.c
> > > > > > @@ -8,23 +8,6 @@
> > > > > >     #include <libfdt.h>
> > > > > >     #include <assert.h>
> > > > > >     -/**
> > > > > > - * IRQ line type.
> > > > > > - * DT_IRQ_TYPE_NONE            - default, unspecified type
> > > > > > - * DT_IRQ_TYPE_EDGE_RISING     - rising edge triggered
> > > > > > - * DT_IRQ_TYPE_EDGE_FALLING    - falling edge triggered
> > > > > > - * DT_IRQ_TYPE_EDGE_BOTH       - rising and falling edge triggered
> > > > > > - * DT_IRQ_TYPE_LEVEL_HIGH      - high level triggered
> > > > > > - * DT_IRQ_TYPE_LEVEL_LOW       - low level triggered
> > > > > > - */
> > > > > > -#define DT_IRQ_TYPE_NONE           0x00000000
> > > > > > -#define DT_IRQ_TYPE_EDGE_RISING    0x00000001
> > > > > > -#define DT_IRQ_TYPE_EDGE_FALLING   0x00000002
> > > > > > -#define DT_IRQ_TYPE_EDGE_BOTH                           \
> > > > > > -    (DT_IRQ_TYPE_EDGE_FALLING | DT_IRQ_TYPE_EDGE_RISING)
> > > > > > -#define DT_IRQ_TYPE_LEVEL_HIGH     0x00000004
> > > > > > -#define DT_IRQ_TYPE_LEVEL_LOW      0x00000008
> > > > > > -
> > > > > 
> > > > > Those defines have nothing to do with the guest itself. They are
> > > > > currently
> > > > > define in Xen without the DT_ prefix.
> > > > 
> > > > Sounds like we want to get rid of the DT_IRQ_TYPE_* definitions
> > > > completely, move the IRQ_TYPE_* definitions from device_tree.h to here,
> > > > and start using them in tools/libxl/libxl_arm.c (which involves a
> > > > renaming s/DT_IRQ_TYPE/IRQ_TYPE/g).
> > > > 
> > > > Is that what you had in mind?
> > > 
> > > Even if DT is Arm only today, the DT code is in common code and therefore
> > > header device_tree.h should contain every thing necessary to use a DT.
> > > 
> > > If we still want to share constant with libxl then I would prefer to
> > > introduce
> > > a new header (similar to acpi/acconfig.h) that provide all the common
> > > values.
> > 
> > OK, I can do that. I'll introduce a new header file.
> > 
> > 
> > > Note that the hypervisor one don't have the DT_ prefix because they are
> > > use to
> > > describe IRQ for both DT and ACPI in Xen. It is not that nice, we might
> > > want
> > > to introduce aliases in that case. So we keep DT_* in libxl.
> > 
> > With the new header file we'll be able to reuse the same #defines in
> > libxl and xen. I think it would be nicer to avoid the aliases and just
> > use the regular definitions in libxl too? Changing libxl to use
> > IRQ_TYPE_ directly only requires a small patch.
> 
> I really don't want that. As I explained, IRQ_TYPE have been conveniently
> chosen to avoid converting DT value to IRQ_TYPE. They are not meant to be use
> like that libxl (or any header publicly shared) and will provide more
> confusion that anything for other bits than the hypervisor.

I don't follow the explanation. Why would they be confusing in public
headers? What is the important difference between DT_IRQ_TYPE_* and
IRQ_TYPE_* that we are trying to keep? Why IRQ_TYPE_* shouldn't be used
by libxl?


> So in the hypervisor case we want to:
> 	1) Define DT_* in a separate header
> 	2) Alias IRQ_TYPE_* to corresponding one.
> 
> In the libxl case we want to use the new header containing DT_*.

I understand what you are asking now, that is clear. I don't understand
why though.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 03/15] xen/arm: extend device tree based multiboot protocol

[Edgar E. Iglesias]

[-- Attachment #1.1: Type: text/plain, Size: 5938 bytes --]

Hi, Sorry for the top post (I'm not managing todo in line reply with my
phone).

Yes, in the long run it there are some benefits if the format could be kept
similar when possible. We could reuse some of the documentation and perhaps
some of the code to parse. Allthough I'm guessing that most of the xl
parsing is not readily reusable from the hypervisor itself.

On the other hand, perhaps the xl format parser is too complex and not
something we would like be reused from within the hypervisor itself?
Considering that one of the use cases for dom0 less is certification.

So an alternative if we don't try to reuse the xl format as much as
possible could be to create a new syntax that is as easy as possible to
parse.

I'm not very convinced on what path is best at the moment.

Regarding strings vs dtb numbers with cells, don't we already have a
generic dtb function that parses numbers and considers the #cells that
could be reused?

Best regards,
Edgar

On Tue, Jul 3, 2018, 06:32 Stefano Stabellini <sstabellini@kernel.org>
wrote:

> On Thu, 14 Jun 2018, Julien Grall wrote:
> > Hi Stefano,
> >
> > On 13/06/18 23:15, Stefano Stabellini wrote:
> > > Extend the existing device tree based multiboot protocol to include
> > > information regarding other domUs to boot.
> > >
> > > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > > ---
> > >   docs/misc/arm/device-tree/booting.txt | 102
> > > ++++++++++++++++++++++++++++++++++
> > >   1 file changed, 102 insertions(+)
> > >
> > > diff --git a/docs/misc/arm/device-tree/booting.txt
> > > b/docs/misc/arm/device-tree/booting.txt
> > > index ce2d0dc..95255e5 100644
> > > --- a/docs/misc/arm/device-tree/booting.txt
> > > +++ b/docs/misc/arm/device-tree/booting.txt
> > > @@ -119,3 +119,105 @@ For those you would hardcode the Xen commandline
> in
> > > the DTB under
> > >   line by writing bootargs (as for native Linux).
> > >   A Xen-aware bootloader would set xen,xen-bootargs for Xen,
> > > xen,dom0-bootargs
> > >   for Dom0 and bootargs for native Linux.
> > > +
> > > +
> > > +Creating DomUs directly from Xen
> > > +================================
> > > +
> > > +It is possible to have Xen create other domains, in addition to dom0,
> > > +out of the information provided via device tree. A Kernel and initrd
> >
> > NIT: s/Kernel/kernel/
>
> OK
>
>
> > > +(optional) need to be specified for each guest.
> > > +
> > > +For each DomU to be created there needs to be one node under /chosen
> > > +with the following properties:
> >
> > I think it would be better to make this domain agnostic. I.e allow Dom0
> to be
> > created the same way but still supporting the current bindings.
> >
> > We could differentiate Dom0 from the other with a property
> > "xen,initial-domain". Note that I am not asking to add this property in
> this
> > series. This is more a forward looking of the use of this binding.
>
> It sounds like a good idea, I'll add it.
>
>
> > > +
> > > +- compatible
> > > +
> > > +    "xen,domU"
> >
> > If we follow my suggestion, this would be renamed "xen,domain".
>
> OK
>
>
> > > +
> > > +- mem (optional)
> >
> > I would prefer the full name "memory".
>
> Yes, especially given that the corresponding xl config file option is
> called "memory".
>
>
> > > +
> > > +    A string specifying the amount of RAM to allocate to the guest. If
> > > +    not specified it defaults to "64M". The format of the string is
> the
> > > same
> > > +    as the one for the mem= parameter in xl config files.
> >
> > I don't much like default values because they are pretty arbitrary. This
> also
> > does not match the default value for Dom0. Why don't you mandate the
> property?
>
> Yes, I can do that.
>
>
> > I would also prefer if the size is specified in the same way number are
> > described in Device-Tree. I.e using cells. You could impose 2 cells for
> the
> > size here.
>
> see below
>
>
> > > +
> > > +- cpus (optional)
> > > +
> > > +    A string specifying the number of vcpus to allocate to the guest.
> If
> > > +    not specified it defaults to "1".
> >
> > Same remarks as for "mem".
>
> I think it would be nicer if we kept the same format used for xl config
> files for these two options. Especially given that we already have the
> functions in the hypervisor to parse them (Xen knows how to parse
> dom0_max_vcpus and dom0_mem for instance). It is going to be easier to
> use and it doesn't come with a cost for Xen.
>
> Edgar, what do you think?
>
>
> > > +
> > > +- #address-cells and #size-cells
> > > +
> > > +    Both #address-cells and #size-cells need to be specified because
> > > +    both sub-nodes (described shortly) have reg properties.
> > > +
> > > +Under the "xen,domU" compatible node, one or more sub-nodes are
> present
> > > +for the DomU kernel and ramdisk.
> > > +
> > > +The kernel sub-node has the following properties:
> > > +
> > > +- compatible
> > > +
> > > +    "multiboot,domU-kernel"
> >
> > I don't think we need to specify a new compatible here. We could re-use
> > "multiboot,kernel" here because the parent node will contain "xen,domU".
>
> Yes you are right
>
>
> > > +
> > > +- reg
> > > +
> > > +    Specifies the physical address of the kernel in RAM and its
> > > +    length.
> > > +
> > > +- bootargs (optional)
> > > +
> > > +    Command line parameters for the guest kernel.
> > > +
> > > +The ramdisk sub-node has the following properties:
> > > +
> > > +- compatible
> > > +
> > > +    "multiboot,domU-ramdisk"
> >
> > Same here, we could re-use "multiboot,ramdisk".
>
> OK
>
>
> > > +
> > > +- reg
> > > +
> > > +    Specifies the physical address of the ramdisk in RAM and its
> > > +    length.
> >
> > We should mention somewhere that this should be described in the /chosen
> node
> > of the device-tree.
>
> OK
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xenproject.org
> https://lists.xenproject.org/mailman/listinfo/xen-devel

[-- Attachment #1.2: Type: text/html, Size: 8100 bytes --]

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 02/15] xen/arm: move a few guest related #defines to public/arch-arm.h

[Julien Grall]

Hi,

On 02/07/18 22:38, Stefano Stabellini wrote:
> On Mon, 2 Jul 2018, Julien Grall wrote:
>> Hi,
>>
>> On 02/07/2018 21:37, Stefano Stabellini wrote:
>>> On Fri, 15 Jun 2018, Julien Grall wrote:
>>>> Hi Stefano,
>>>>
>>>> On 06/14/2018 10:15 PM, Stefano Stabellini wrote:
>>>>> On Thu, 14 Jun 2018, Julien Grall wrote:
>>>>>> On 13/06/18 23:15, Stefano Stabellini wrote:
>>>>>>> Move a few constants defined by libxl_arm.c to
>>>>>>> xen/include/public/arch-arm.h, so that they are together with the
>>>>>>> other
>>>>>>> guest related #defines such as GUEST_GICD_BASE and GUEST_VPL011_SPI.
>>>>>>> Also, this way they can be reused by hypervisor code.
>>>>>>
>>>>>> All variables moved to arch-arm.h should be prefixed with GUEST_* to
>>>>>> avoid
>>>>>> clash with the rest of Xen.
>>>>>
>>>>> I'll do.
>>>>>
>>>>>>>
>>>>>>> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
>>>>>>> CC: wei.liu2@citrix.com
>>>>>>> CC: ian.jackson@eu.citrix.com
>>>>>>> ---
>>>>>>>      tools/libxl/libxl_arm.c       | 26 --------------------------
>>>>>>>      xen/include/public/arch-arm.h | 26 ++++++++++++++++++++++++++
>>>>>>>      2 files changed, 26 insertions(+), 26 deletions(-)
>>>>>>>
>>>>>>> diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
>>>>>>> index 8af9f6f..89a417f 100644
>>>>>>> --- a/tools/libxl/libxl_arm.c
>>>>>>> +++ b/tools/libxl/libxl_arm.c
>>>>>>> @@ -8,23 +8,6 @@
>>>>>>>      #include <libfdt.h>
>>>>>>>      #include <assert.h>
>>>>>>>      -/**
>>>>>>> - * IRQ line type.
>>>>>>> - * DT_IRQ_TYPE_NONE            - default, unspecified type
>>>>>>> - * DT_IRQ_TYPE_EDGE_RISING     - rising edge triggered
>>>>>>> - * DT_IRQ_TYPE_EDGE_FALLING    - falling edge triggered
>>>>>>> - * DT_IRQ_TYPE_EDGE_BOTH       - rising and falling edge triggered
>>>>>>> - * DT_IRQ_TYPE_LEVEL_HIGH      - high level triggered
>>>>>>> - * DT_IRQ_TYPE_LEVEL_LOW       - low level triggered
>>>>>>> - */
>>>>>>> -#define DT_IRQ_TYPE_NONE           0x00000000
>>>>>>> -#define DT_IRQ_TYPE_EDGE_RISING    0x00000001
>>>>>>> -#define DT_IRQ_TYPE_EDGE_FALLING   0x00000002
>>>>>>> -#define DT_IRQ_TYPE_EDGE_BOTH                           \
>>>>>>> -    (DT_IRQ_TYPE_EDGE_FALLING | DT_IRQ_TYPE_EDGE_RISING)
>>>>>>> -#define DT_IRQ_TYPE_LEVEL_HIGH     0x00000004
>>>>>>> -#define DT_IRQ_TYPE_LEVEL_LOW      0x00000008
>>>>>>> -
>>>>>>
>>>>>> Those defines have nothing to do with the guest itself. They are
>>>>>> currently
>>>>>> define in Xen without the DT_ prefix.
>>>>>
>>>>> Sounds like we want to get rid of the DT_IRQ_TYPE_* definitions
>>>>> completely, move the IRQ_TYPE_* definitions from device_tree.h to here,
>>>>> and start using them in tools/libxl/libxl_arm.c (which involves a
>>>>> renaming s/DT_IRQ_TYPE/IRQ_TYPE/g).
>>>>>
>>>>> Is that what you had in mind?
>>>>
>>>> Even if DT is Arm only today, the DT code is in common code and therefore
>>>> header device_tree.h should contain every thing necessary to use a DT.
>>>>
>>>> If we still want to share constant with libxl then I would prefer to
>>>> introduce
>>>> a new header (similar to acpi/acconfig.h) that provide all the common
>>>> values.
>>>
>>> OK, I can do that. I'll introduce a new header file.
>>>
>>>
>>>> Note that the hypervisor one don't have the DT_ prefix because they are
>>>> use to
>>>> describe IRQ for both DT and ACPI in Xen. It is not that nice, we might
>>>> want
>>>> to introduce aliases in that case. So we keep DT_* in libxl.
>>>
>>> With the new header file we'll be able to reuse the same #defines in
>>> libxl and xen. I think it would be nicer to avoid the aliases and just
>>> use the regular definitions in libxl too? Changing libxl to use
>>> IRQ_TYPE_ directly only requires a small patch.
>>
>> I really don't want that. As I explained, IRQ_TYPE have been conveniently
>> chosen to avoid converting DT value to IRQ_TYPE. They are not meant to be use
>> like that libxl (or any header publicly shared) and will provide more
>> confusion that anything for other bits than the hypervisor.
> 
> I don't follow the explanation. Why would they be confusing in public
> headers? What is the important difference between DT_IRQ_TYPE_* and
> IRQ_TYPE_* that we are trying to keep? Why IRQ_TYPE_* shouldn't be used
> by libxl?

Because IRQ_TYPE_* does not have the prefix DT_ in them. There are no 
way from the name to say this is for DT only and might be misused 
outside of the hypervisor.

IRQ_TYPE_* used to have the prefix DT_ but it was dropped as in the 
hypervisor they are used to represent irq type for both ACPI and DT. If 
we end up to expose those values to libxl then we should define 
DT_IRQ_TYPE_* and provide aliases for IRQ_TYPE_.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 03/15] xen/arm: extend device tree based multiboot protocol

[Julien Grall]

Hi Stefano,

On 02/07/18 22:31, Stefano Stabellini wrote:
> On Thu, 14 Jun 2018, Julien Grall wrote:
>> On 13/06/18 23:15, Stefano Stabellini wrote:
>>> +
>>> +- cpus (optional)
>>> +
>>> +    A string specifying the number of vcpus to allocate to the guest. If
>>> +    not specified it defaults to "1".
>>
>> Same remarks as for "mem".
> 
> I think it would be nicer if we kept the same format used for xl config
> files for these two options.Especially given that we already have the
> functions in the hypervisor to parse them (Xen knows how to parse
> dom0_max_vcpus and dom0_mem for instance). It is going to be easier to > use and it doesn't come with a cost for Xen.

I can't see any reason to keep those 2 options similar to libxl when all 
the rest is embracing DT format. The user would already have to spend 
some times to look at the new bindings anyway, not using string is not 
going to be a big difficulty.

While I agree that Xen knows how to parse dom0_mem, dom0_max_vcpus, Xen 
also knows how to read numbers from cells which is widely used within 
the hypervisor. So both solution are going to be easy to use in the 
hypervisor. However, the former is always going to be faster, less error 
prone than the latter.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 02/15] xen/arm: move a few guest related #defines to public/arch-arm.h

[Stefano Stabellini]

On Tue, 3 Jul 2018, Julien Grall wrote:
> Hi,
> 
> On 02/07/18 22:38, Stefano Stabellini wrote:
> > On Mon, 2 Jul 2018, Julien Grall wrote:
> > > Hi,
> > > 
> > > On 02/07/2018 21:37, Stefano Stabellini wrote:
> > > > On Fri, 15 Jun 2018, Julien Grall wrote:
> > > > > Hi Stefano,
> > > > > 
> > > > > On 06/14/2018 10:15 PM, Stefano Stabellini wrote:
> > > > > > On Thu, 14 Jun 2018, Julien Grall wrote:
> > > > > > > On 13/06/18 23:15, Stefano Stabellini wrote:
> > > > > > > > Move a few constants defined by libxl_arm.c to
> > > > > > > > xen/include/public/arch-arm.h, so that they are together with
> > > > > > > > the
> > > > > > > > other
> > > > > > > > guest related #defines such as GUEST_GICD_BASE and
> > > > > > > > GUEST_VPL011_SPI.
> > > > > > > > Also, this way they can be reused by hypervisor code.
> > > > > > > 
> > > > > > > All variables moved to arch-arm.h should be prefixed with GUEST_*
> > > > > > > to
> > > > > > > avoid
> > > > > > > clash with the rest of Xen.
> > > > > > 
> > > > > > I'll do.
> > > > > > 
> > > > > > > > 
> > > > > > > > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > > > > > > > CC: wei.liu2@citrix.com
> > > > > > > > CC: ian.jackson@eu.citrix.com
> > > > > > > > ---
> > > > > > > >      tools/libxl/libxl_arm.c       | 26
> > > > > > > > --------------------------
> > > > > > > >      xen/include/public/arch-arm.h | 26
> > > > > > > > ++++++++++++++++++++++++++
> > > > > > > >      2 files changed, 26 insertions(+), 26 deletions(-)
> > > > > > > > 
> > > > > > > > diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
> > > > > > > > index 8af9f6f..89a417f 100644
> > > > > > > > --- a/tools/libxl/libxl_arm.c
> > > > > > > > +++ b/tools/libxl/libxl_arm.c
> > > > > > > > @@ -8,23 +8,6 @@
> > > > > > > >      #include <libfdt.h>
> > > > > > > >      #include <assert.h>
> > > > > > > >      -/**
> > > > > > > > - * IRQ line type.
> > > > > > > > - * DT_IRQ_TYPE_NONE            - default, unspecified type
> > > > > > > > - * DT_IRQ_TYPE_EDGE_RISING     - rising edge triggered
> > > > > > > > - * DT_IRQ_TYPE_EDGE_FALLING    - falling edge triggered
> > > > > > > > - * DT_IRQ_TYPE_EDGE_BOTH       - rising and falling edge
> > > > > > > > triggered
> > > > > > > > - * DT_IRQ_TYPE_LEVEL_HIGH      - high level triggered
> > > > > > > > - * DT_IRQ_TYPE_LEVEL_LOW       - low level triggered
> > > > > > > > - */
> > > > > > > > -#define DT_IRQ_TYPE_NONE           0x00000000
> > > > > > > > -#define DT_IRQ_TYPE_EDGE_RISING    0x00000001
> > > > > > > > -#define DT_IRQ_TYPE_EDGE_FALLING   0x00000002
> > > > > > > > -#define DT_IRQ_TYPE_EDGE_BOTH                           \
> > > > > > > > -    (DT_IRQ_TYPE_EDGE_FALLING | DT_IRQ_TYPE_EDGE_RISING)
> > > > > > > > -#define DT_IRQ_TYPE_LEVEL_HIGH     0x00000004
> > > > > > > > -#define DT_IRQ_TYPE_LEVEL_LOW      0x00000008
> > > > > > > > -
> > > > > > > 
> > > > > > > Those defines have nothing to do with the guest itself. They are
> > > > > > > currently
> > > > > > > define in Xen without the DT_ prefix.
> > > > > > 
> > > > > > Sounds like we want to get rid of the DT_IRQ_TYPE_* definitions
> > > > > > completely, move the IRQ_TYPE_* definitions from device_tree.h to
> > > > > > here,
> > > > > > and start using them in tools/libxl/libxl_arm.c (which involves a
> > > > > > renaming s/DT_IRQ_TYPE/IRQ_TYPE/g).
> > > > > > 
> > > > > > Is that what you had in mind?
> > > > > 
> > > > > Even if DT is Arm only today, the DT code is in common code and
> > > > > therefore
> > > > > header device_tree.h should contain every thing necessary to use a DT.
> > > > > 
> > > > > If we still want to share constant with libxl then I would prefer to
> > > > > introduce
> > > > > a new header (similar to acpi/acconfig.h) that provide all the common
> > > > > values.
> > > > 
> > > > OK, I can do that. I'll introduce a new header file.
> > > > 
> > > > 
> > > > > Note that the hypervisor one don't have the DT_ prefix because they
> > > > > are
> > > > > use to
> > > > > describe IRQ for both DT and ACPI in Xen. It is not that nice, we
> > > > > might
> > > > > want
> > > > > to introduce aliases in that case. So we keep DT_* in libxl.
> > > > 
> > > > With the new header file we'll be able to reuse the same #defines in
> > > > libxl and xen. I think it would be nicer to avoid the aliases and just
> > > > use the regular definitions in libxl too? Changing libxl to use
> > > > IRQ_TYPE_ directly only requires a small patch.
> > > 
> > > I really don't want that. As I explained, IRQ_TYPE have been conveniently
> > > chosen to avoid converting DT value to IRQ_TYPE. They are not meant to be
> > > use
> > > like that libxl (or any header publicly shared) and will provide more
> > > confusion that anything for other bits than the hypervisor.
> > 
> > I don't follow the explanation. Why would they be confusing in public
> > headers? What is the important difference between DT_IRQ_TYPE_* and
> > IRQ_TYPE_* that we are trying to keep? Why IRQ_TYPE_* shouldn't be used
> > by libxl?
> 
> Because IRQ_TYPE_* does not have the prefix DT_ in them. There are no way from
> the name to say this is for DT only and might be misused outside of the
> hypervisor.
> 
> IRQ_TYPE_* used to have the prefix DT_ but it was dropped as in the hypervisor
> they are used to represent irq type for both ACPI and DT. If we end up to
> expose those values to libxl then we should define DT_IRQ_TYPE_* and provide
> aliases for IRQ_TYPE_.

The irq types defined as IRQ_TYPE_* are not just used in device tree
code but also in ACPI. This is the reason why they were originally
renamed from DT_IRQ_TYPE_* to IRQ_TYPE_.

Libxl is also carrying a copy of the same definition as DT_IRQ_TYPE_*.

I am suggesting to remove the code duplication and use the same
IRQ_TYPE_* #defines everywhere: DT in Xen, ACPI in Xen and DT in libxl.

Your suggestion is to use DT_IRQ_TYPE_* for DT in libxl and introduce
IRQ_TYPE_* aliases for DT and ACPI in Xen. The reason is that they are
meant for device tree usage so it should be clear that they are for
device tree. (I hope I got it right.)

The two goals are not mutually exclusive. I don't care about what is
called IRQ_TYPE_* and what is called DT_IRQ_TYPE_*, I am just trying to
use the same defines in libxl DT and Xen DT code. I think those two
should be the same, whether they are named DT_IRQ_TYPE_* or simply
IRQ_TYPE_*. For that, I could introduce a new header with DT_IRQ_TYPE_*
and use it both in Xen and Libxl. I think that would satisfy your
requirements?

It remains to be seen what to do for the ACPI defines. I would probably
just use the DT_* defines in the ACPI code, given that now the
DT_IRQ_TYPE_* defines become available outside of device_tree.h.
Another option is to introduce ACPI specific aliases, such as
ACPI_IRQ_TYPE_*. I don't particularly care.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 03/15] xen/arm: extend device tree based multiboot protocol

[Stefano Stabellini]

On Tue, 3 Jul 2018, Julien Grall wrote:
> Hi Stefano,
> 
> On 02/07/18 22:31, Stefano Stabellini wrote:
> > On Thu, 14 Jun 2018, Julien Grall wrote:
> > > On 13/06/18 23:15, Stefano Stabellini wrote:
> > > > +
> > > > +- cpus (optional)
> > > > +
> > > > +    A string specifying the number of vcpus to allocate to the guest.
> > > > If
> > > > +    not specified it defaults to "1".
> > > 
> > > Same remarks as for "mem".
> > 
> > I think it would be nicer if we kept the same format used for xl config
> > files for these two options.Especially given that we already have the
> > functions in the hypervisor to parse them (Xen knows how to parse
> > dom0_max_vcpus and dom0_mem for instance). It is going to be easier to > use
> > and it doesn't come with a cost for Xen.
> 
> I can't see any reason to keep those 2 options similar to libxl when all the
> rest is embracing DT format. The user would already have to spend some times
> to look at the new bindings anyway, not using string is not going to be a big
> difficulty.
>
> While I agree that Xen knows how to parse dom0_mem, dom0_max_vcpus, Xen also
> knows how to read numbers from cells which is widely used within the
> hypervisor. So both solution are going to be easy to use in the hypervisor.
> However, the former is always going to be faster, less error prone than the
> latter.

Memory and cpus are the only two VM options we have so far. Kernels and
ramdisks are part of the multiboot protocol, they are about passing
binaries from the bootloader to Xen in memory, rather than specifying VM
properties. They describe something in memory, a bit like a device on
device tree, while memory and cpus are options a bit like the existing
bootargs. There are "mem" and "dom0_mem" options as part of the common
bootargs but they are passed as strings, not as device tree fields.

So I don't think we have anything else "embracing DT format" yet.

For instance, if we were to implement this on x86 (which I am not), I
would image we would be using the x86 version of multiboot to pass the
additional kernels and ramdisks, and we would also add an extra strings
field with the VM options, similar to the string field used as command
line arguments.

This is to say that I don't think we necessarily have to abide to device
tree best recommendations to pass the VM options (but we could still
choose to). There is an argument for using the same format as per the VM
config file to make it easier for users, but as Edgar also pointed out,
we don't want to do that if it means adding a complex parser. There is
also an argument for using a format that could be reused on non-device
tree platforms.

This is why I think that for cpus and memory is relatively simple, I
would just use the same format as dom0_mem and dom0_max_vcpus for
simplicity of everybody involved (xen and the user). The other options
we'll need are iomem, which takes an address and size, irqs that takes a
list of integers, and dtdev which takes a device tree path. We could
make them all a single string such as:

    domU2 {
			compatible = "xen,domain";
			#address-cells = <0x2>;
			#size-cells = <0x1>;

            vmargs = "memory=1G cpus=2 iomem=0x400000,0x1000 irqs=112,113 dtdev=/soc/ethernet@fff51000"

			module@0x4c000000 {
				compatible = "multiboot,kernel", "multiboot,module";
				reg = <0x0 0x4c000000 0xffffff>;
				bootargs = "console=ttyAMA0 init=/bin/sh";
			};

			module@0x4d000000 {
				compatible = "multiboot,ramdisk", "multiboot,module";
				reg = <0x0 0x4d000000 0xffffff>;
			};
		};

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 03/15] xen/arm: extend device tree based multiboot protocol

[Stefano Stabellini]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 7348 bytes --]

Hi Edgar,

Yes, we certainly don't want the xl parser in the hypervisor. We only
need a minimal subset of options. We do already have a device tree
parser that understands cells. We also have a parser for a set of
command line options, some of them similar to the VM options we need to
pass. I think we'll need introduce our own extra-simple format.

I forgot to CC you to my other reply on the subject: 

https://marc.info/?l=xen-devel&m=153065625412272

On Tue, 3 Jul 2018, Edgar E. Iglesias wrote:
> Hi, Sorry for the top post (I'm not managing todo in line reply with my phone). 
> 
> Yes, in the long run it there are some benefits if the format could be kept similar when possible. We could reuse some of the
> documentation and perhaps some of the code to parse. Allthough I'm guessing that most of the xl parsing is not readily reusable
> from the hypervisor itself.
> 
> On the other hand, perhaps the xl format parser is too complex and not something we would like be reused from within the
> hypervisor itself? Considering that one of the use cases for dom0 less is certification.
> 
> So an alternative if we don't try to reuse the xl format as much as possible could be to create a new syntax that is as easy as
> possible to parse.
> 
> I'm not very convinced on what path is best at the moment.
> 
> Regarding strings vs dtb numbers with cells, don't we already have a generic dtb function that parses numbers and considers the
> #cells that could be reused?
> 
> Best regards, 
> Edgar 
> 
> On Tue, Jul 3, 2018, 06:32 Stefano Stabellini <sstabellini@kernel.org> wrote:
>       On Thu, 14 Jun 2018, Julien Grall wrote:
>       > Hi Stefano,
>       >
>       > On 13/06/18 23:15, Stefano Stabellini wrote:
>       > > Extend the existing device tree based multiboot protocol to include
>       > > information regarding other domUs to boot.
>       > >
>       > > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
>       > > ---
>       > >   docs/misc/arm/device-tree/booting.txt | 102
>       > > ++++++++++++++++++++++++++++++++++
>       > >   1 file changed, 102 insertions(+)
>       > >
>       > > diff --git a/docs/misc/arm/device-tree/booting.txt
>       > > b/docs/misc/arm/device-tree/booting.txt
>       > > index ce2d0dc..95255e5 100644
>       > > --- a/docs/misc/arm/device-tree/booting.txt
>       > > +++ b/docs/misc/arm/device-tree/booting.txt
>       > > @@ -119,3 +119,105 @@ For those you would hardcode the Xen commandline in
>       > > the DTB under
>       > >   line by writing bootargs (as for native Linux).
>       > >   A Xen-aware bootloader would set xen,xen-bootargs for Xen,
>       > > xen,dom0-bootargs
>       > >   for Dom0 and bootargs for native Linux.
>       > > +
>       > > +
>       > > +Creating DomUs directly from Xen
>       > > +================================
>       > > +
>       > > +It is possible to have Xen create other domains, in addition to dom0,
>       > > +out of the information provided via device tree. A Kernel and initrd
>       >
>       > NIT: s/Kernel/kernel/
> 
>       OK
> 
> 
>       > > +(optional) need to be specified for each guest.
>       > > +
>       > > +For each DomU to be created there needs to be one node under /chosen
>       > > +with the following properties:
>       >
>       > I think it would be better to make this domain agnostic. I.e allow Dom0 to be
>       > created the same way but still supporting the current bindings.
>       >
>       > We could differentiate Dom0 from the other with a property
>       > "xen,initial-domain". Note that I am not asking to add this property in this
>       > series. This is more a forward looking of the use of this binding.
> 
>       It sounds like a good idea, I'll add it.
> 
> 
>       > > +
>       > > +- compatible
>       > > +
>       > > +    "xen,domU"
>       >
>       > If we follow my suggestion, this would be renamed "xen,domain".
> 
>       OK
> 
> 
>       > > +
>       > > +- mem (optional)
>       >
>       > I would prefer the full name "memory".
> 
>       Yes, especially given that the corresponding xl config file option is
>       called "memory".
> 
> 
>       > > +
>       > > +    A string specifying the amount of RAM to allocate to the guest. If
>       > > +    not specified it defaults to "64M". The format of the string is the
>       > > same
>       > > +    as the one for the mem= parameter in xl config files.
>       >
>       > I don't much like default values because they are pretty arbitrary. This also
>       > does not match the default value for Dom0. Why don't you mandate the property?
> 
>       Yes, I can do that.
> 
> 
>       > I would also prefer if the size is specified in the same way number are
>       > described in Device-Tree. I.e using cells. You could impose 2 cells for the
>       > size here.
> 
>       see below
> 
> 
>       > > +
>       > > +- cpus (optional)
>       > > +
>       > > +    A string specifying the number of vcpus to allocate to the guest. If
>       > > +    not specified it defaults to "1".
>       >
>       > Same remarks as for "mem".
> 
>       I think it would be nicer if we kept the same format used for xl config
>       files for these two options. Especially given that we already have the
>       functions in the hypervisor to parse them (Xen knows how to parse
>       dom0_max_vcpus and dom0_mem for instance). It is going to be easier to
>       use and it doesn't come with a cost for Xen.
> 
>       Edgar, what do you think?
> 
> 
>       > > +
>       > > +- #address-cells and #size-cells
>       > > +
>       > > +    Both #address-cells and #size-cells need to be specified because
>       > > +    both sub-nodes (described shortly) have reg properties.
>       > > +
>       > > +Under the "xen,domU" compatible node, one or more sub-nodes are present
>       > > +for the DomU kernel and ramdisk.
>       > > +
>       > > +The kernel sub-node has the following properties:
>       > > +
>       > > +- compatible
>       > > +
>       > > +    "multiboot,domU-kernel"
>       >
>       > I don't think we need to specify a new compatible here. We could re-use
>       > "multiboot,kernel" here because the parent node will contain "xen,domU".
> 
>       Yes you are right
> 
> 
>       > > +
>       > > +- reg
>       > > +
>       > > +    Specifies the physical address of the kernel in RAM and its
>       > > +    length.
>       > > +
>       > > +- bootargs (optional)
>       > > +
>       > > +    Command line parameters for the guest kernel.
>       > > +
>       > > +The ramdisk sub-node has the following properties:
>       > > +
>       > > +- compatible
>       > > +
>       > > +    "multiboot,domU-ramdisk"
>       >
>       > Same here, we could re-use "multiboot,ramdisk".
> 
>       OK
> 
> 
>       > > +
>       > > +- reg
>       > > +
>       > > +    Specifies the physical address of the ramdisk in RAM and its
>       > > +    length.
>       >
>       > We should mention somewhere that this should be described in the /chosen node
>       > of the device-tree.
> 
>       OK
> 
>       _______________________________________________
>       Xen-devel mailing list
>       Xen-devel@lists.xenproject.org
>       https://lists.xenproject.org/mailman/listinfo/xen-devel
> 
> 
> 

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 02/15] xen/arm: move a few guest related #defines to public/arch-arm.h

[Julien Grall]

Hi,

On 03/07/2018 22:30, Stefano Stabellini wrote:
> On Tue, 3 Jul 2018, Julien Grall wrote:
>> Hi,
>>
>> On 02/07/18 22:38, Stefano Stabellini wrote:
>>> On Mon, 2 Jul 2018, Julien Grall wrote:
>>>> Hi,
>>>>
>>>> On 02/07/2018 21:37, Stefano Stabellini wrote:
>>>>> On Fri, 15 Jun 2018, Julien Grall wrote:
>>>>>> Hi Stefano,
>>>>>>
>>>>>> On 06/14/2018 10:15 PM, Stefano Stabellini wrote:
>>>>>>> On Thu, 14 Jun 2018, Julien Grall wrote:
>>>>>>>> On 13/06/18 23:15, Stefano Stabellini wrote:
>>>>>>>>> Move a few constants defined by libxl_arm.c to
>>>>>>>>> xen/include/public/arch-arm.h, so that they are together with
>>>>>>>>> the
>>>>>>>>> other
>>>>>>>>> guest related #defines such as GUEST_GICD_BASE and
>>>>>>>>> GUEST_VPL011_SPI.
>>>>>>>>> Also, this way they can be reused by hypervisor code.
>>>>>>>>
>>>>>>>> All variables moved to arch-arm.h should be prefixed with GUEST_*
>>>>>>>> to
>>>>>>>> avoid
>>>>>>>> clash with the rest of Xen.
>>>>>>>
>>>>>>> I'll do.
>>>>>>>
>>>>>>>>>
>>>>>>>>> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
>>>>>>>>> CC: wei.liu2@citrix.com
>>>>>>>>> CC: ian.jackson@eu.citrix.com
>>>>>>>>> ---
>>>>>>>>>       tools/libxl/libxl_arm.c       | 26
>>>>>>>>> --------------------------
>>>>>>>>>       xen/include/public/arch-arm.h | 26
>>>>>>>>> ++++++++++++++++++++++++++
>>>>>>>>>       2 files changed, 26 insertions(+), 26 deletions(-)
>>>>>>>>>
>>>>>>>>> diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
>>>>>>>>> index 8af9f6f..89a417f 100644
>>>>>>>>> --- a/tools/libxl/libxl_arm.c
>>>>>>>>> +++ b/tools/libxl/libxl_arm.c
>>>>>>>>> @@ -8,23 +8,6 @@
>>>>>>>>>       #include <libfdt.h>
>>>>>>>>>       #include <assert.h>
>>>>>>>>>       -/**
>>>>>>>>> - * IRQ line type.
>>>>>>>>> - * DT_IRQ_TYPE_NONE            - default, unspecified type
>>>>>>>>> - * DT_IRQ_TYPE_EDGE_RISING     - rising edge triggered
>>>>>>>>> - * DT_IRQ_TYPE_EDGE_FALLING    - falling edge triggered
>>>>>>>>> - * DT_IRQ_TYPE_EDGE_BOTH       - rising and falling edge
>>>>>>>>> triggered
>>>>>>>>> - * DT_IRQ_TYPE_LEVEL_HIGH      - high level triggered
>>>>>>>>> - * DT_IRQ_TYPE_LEVEL_LOW       - low level triggered
>>>>>>>>> - */
>>>>>>>>> -#define DT_IRQ_TYPE_NONE           0x00000000
>>>>>>>>> -#define DT_IRQ_TYPE_EDGE_RISING    0x00000001
>>>>>>>>> -#define DT_IRQ_TYPE_EDGE_FALLING   0x00000002
>>>>>>>>> -#define DT_IRQ_TYPE_EDGE_BOTH                           \
>>>>>>>>> -    (DT_IRQ_TYPE_EDGE_FALLING | DT_IRQ_TYPE_EDGE_RISING)
>>>>>>>>> -#define DT_IRQ_TYPE_LEVEL_HIGH     0x00000004
>>>>>>>>> -#define DT_IRQ_TYPE_LEVEL_LOW      0x00000008
>>>>>>>>> -
>>>>>>>>
>>>>>>>> Those defines have nothing to do with the guest itself. They are
>>>>>>>> currently
>>>>>>>> define in Xen without the DT_ prefix.
>>>>>>>
>>>>>>> Sounds like we want to get rid of the DT_IRQ_TYPE_* definitions
>>>>>>> completely, move the IRQ_TYPE_* definitions from device_tree.h to
>>>>>>> here,
>>>>>>> and start using them in tools/libxl/libxl_arm.c (which involves a
>>>>>>> renaming s/DT_IRQ_TYPE/IRQ_TYPE/g).
>>>>>>>
>>>>>>> Is that what you had in mind?
>>>>>>
>>>>>> Even if DT is Arm only today, the DT code is in common code and
>>>>>> therefore
>>>>>> header device_tree.h should contain every thing necessary to use a DT.
>>>>>>
>>>>>> If we still want to share constant with libxl then I would prefer to
>>>>>> introduce
>>>>>> a new header (similar to acpi/acconfig.h) that provide all the common
>>>>>> values.
>>>>>
>>>>> OK, I can do that. I'll introduce a new header file.
>>>>>
>>>>>
>>>>>> Note that the hypervisor one don't have the DT_ prefix because they
>>>>>> are
>>>>>> use to
>>>>>> describe IRQ for both DT and ACPI in Xen. It is not that nice, we
>>>>>> might
>>>>>> want
>>>>>> to introduce aliases in that case. So we keep DT_* in libxl.
>>>>>
>>>>> With the new header file we'll be able to reuse the same #defines in
>>>>> libxl and xen. I think it would be nicer to avoid the aliases and just
>>>>> use the regular definitions in libxl too? Changing libxl to use
>>>>> IRQ_TYPE_ directly only requires a small patch.
>>>>
>>>> I really don't want that. As I explained, IRQ_TYPE have been conveniently
>>>> chosen to avoid converting DT value to IRQ_TYPE. They are not meant to be
>>>> use
>>>> like that libxl (or any header publicly shared) and will provide more
>>>> confusion that anything for other bits than the hypervisor.
>>>
>>> I don't follow the explanation. Why would they be confusing in public
>>> headers? What is the important difference between DT_IRQ_TYPE_* and
>>> IRQ_TYPE_* that we are trying to keep? Why IRQ_TYPE_* shouldn't be used
>>> by libxl?
>>
>> Because IRQ_TYPE_* does not have the prefix DT_ in them. There are no way from
>> the name to say this is for DT only and might be misused outside of the
>> hypervisor.
>>
>> IRQ_TYPE_* used to have the prefix DT_ but it was dropped as in the hypervisor
>> they are used to represent irq type for both ACPI and DT. If we end up to
>> expose those values to libxl then we should define DT_IRQ_TYPE_* and provide
>> aliases for IRQ_TYPE_.
> 
> The irq types defined as IRQ_TYPE_* are not just used in device tree
> code but also in ACPI. This is the reason why they were originally
> renamed from DT_IRQ_TYPE_* to IRQ_TYPE_.
> 
> Libxl is also carrying a copy of the same definition as DT_IRQ_TYPE_*.
> 
> I am suggesting to remove the code duplication and use the same
> IRQ_TYPE_* #defines everywhere: DT in Xen, ACPI in Xen and DT in libxl.
I guess you read the ACPI spec and code written in Xen, right? You 
probably noticed that those values are just custom made for Xen. This is 
a pure internal decision on the value to ease the use with DT.

> 
> Your suggestion is to use DT_IRQ_TYPE_* for DT in libxl and introduce
> IRQ_TYPE_* aliases for DT and ACPI in Xen. The reason is that they are
> meant for device tree usage so it should be clear that they are for
> device tree. (I hope I got it right.)
> 
> The two goals are not mutually exclusive. I don't care about what is
> called IRQ_TYPE_* and what is called DT_IRQ_TYPE_*, I am just trying to
> use the same defines in libxl DT and Xen DT code. I think those two
> should be the same, whether they are named DT_IRQ_TYPE_* or simply
> IRQ_TYPE_*. For that, I could introduce a new header with DT_IRQ_TYPE_*
> and use it both in Xen and Libxl. I think that would satisfy your
> requirements?

I understand your goal, I just disagree on the way to do it. If you use 
IRQ_TYPE* in libxl, it will confuse the developer as those values are DT 
specific.

If you use DT_TYPE_TYPE_* everywhere in the hypervisor, it will confuse 
the developer as they are used in non-specific firmware.

> 
> It remains to be seen what to do for the ACPI defines. I would probably
> just use the DT_* defines in the ACPI code, given that now the
> DT_IRQ_TYPE_* defines become available outside of device_tree.h.
> Another option is to introduce ACPI specific aliases, such as
> ACPI_IRQ_TYPE_*. I don't particularly care.

Again, they are *not* ACPI defines. IRQ_TYPE is just the internal 
representation of the irq type in Xen. If you look at the code, the ACPI 
specific information is read and then converted it to those types. We 
took a shortcut to define them as the same values as DT IRQ type, but 
that was just laziness.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 03/15] xen/arm: extend device tree based multiboot protocol

[Julien Grall]

Hi Stefano,

On 03/07/18 23:16, Stefano Stabellini wrote:
> On Tue, 3 Jul 2018, Julien Grall wrote:
>> Hi Stefano,
>>
>> On 02/07/18 22:31, Stefano Stabellini wrote:
>>> On Thu, 14 Jun 2018, Julien Grall wrote:
>>>> On 13/06/18 23:15, Stefano Stabellini wrote:
>>>>> +
>>>>> +- cpus (optional)
>>>>> +
>>>>> +    A string specifying the number of vcpus to allocate to the guest.
>>>>> If
>>>>> +    not specified it defaults to "1".
>>>>
>>>> Same remarks as for "mem".
>>>
>>> I think it would be nicer if we kept the same format used for xl config
>>> files for these two options.Especially given that we already have the
>>> functions in the hypervisor to parse them (Xen knows how to parse
>>> dom0_max_vcpus and dom0_mem for instance). It is going to be easier to > use
>>> and it doesn't come with a cost for Xen.
>>
>> I can't see any reason to keep those 2 options similar to libxl when all the
>> rest is embracing DT format. The user would already have to spend some times
>> to look at the new bindings anyway, not using string is not going to be a big
>> difficulty.
>>
>> While I agree that Xen knows how to parse dom0_mem, dom0_max_vcpus, Xen also
>> knows how to read numbers from cells which is widely used within the
>> hypervisor. So both solution are going to be easy to use in the hypervisor.
>> However, the former is always going to be faster, less error prone than the
>> latter.
> 
> Memory and cpus are the only two VM options we have so far. Kernels and
> ramdisks are part of the multiboot protocol, they are about passing
> binaries from the bootloader to Xen in memory, rather than specifying VM
> properties. They describe something in memory, a bit like a device on
> device tree, while memory and cpus are options a bit like the existing
> bootargs. There are "mem" and "dom0_mem" options as part of the common
> bootargs but they are passed as strings, not as device tree fields.
> 
> So I don't think we have anything else "embracing DT format" yet.

I do consider multiboot protocol part of your new VM format.

> 
> For instance, if we were to implement this on x86 (which I am not), I
> would image we would be using the x86 version of multiboot to pass the
> additional kernels and ramdisks, and we would also add an extra strings
> field with the VM options, similar to the string field used as command
> line arguments.
> 
> This is to say that I don't think we necessarily have to abide to device
> tree best recommendations to pass the VM options (but we could still
> choose to). There is an argument for using the same format as per the VM
> config file to make it easier for users, but as Edgar also pointed out,
> we don't want to do that if it means adding a complex parser. There is
> also an argument for using a format that could be reused on non-device
> tree platforms.

Well, the DT format gives you so much more flexibility over string.
Let's take the example of describing a Device. With DT you could just do:

mydev {
   prop1;
   prop2;
   host-regs = <.....>;	/* Host MMIO region to map on ... */
   regs = <......> ;      /* Guest MMIO region ... */
   interrupts = <......>;
   ...
}

Let's now take your example with the string

"iomem=0x400000,0x1000 irqs=112,113"

Leaving aside the fact you assume direct mapping, I always hated that 
format in xl because it is hard to correlate with a given device. With 
the DT solution, you have everything related to a device in one place 
and can easily see if you made any mistake.

Also, the iomem solution will have some shortcoming on Arm because it 
takes a GFN and not full physical address. This does not play very well 
with multiple page granularity and I would rather avoid to spread that 
error for dom0less.

Overall, this is much easier for a user to read and write DT over any 
kind of string. It also does not add much in Xen because we already have 
all the DT helpers in hand. And ... parsing cells is a matter of few 
instructions over converting a string to number.

I understand you want to cover everyone (including x86), but it would be 
much easier to convince them to use DT. It is more powerful over 
anything else without adding extra complexity (and time) to parse 
string. I also think the DT solution would be beneficial in long term in 
addition of the current xl format.

This would help to create in similar fashion guest either from Xen or 
userspace.

> 
> This is why I think that for cpus and memory is relatively simple, I
> would just use the same format as dom0_mem and dom0_max_vcpus for
> simplicity of everybody involved (xen and the user).

You still don't explain how this is more simple...

Someone familiar with DT will expect to specify the information in a DT 
way. By that I mean describing the region in cells not in a string.

Xen also has everything to read an arbitrary number of cells.

> The other options
> we'll need are iomem, which takes an address and size, irqs that takes a
> list of integers, and dtdev which takes a device tree path.

That's what you need for your use case, thought not quite enough. You 
still have to provide node for device.

But then, how about PV drivers? How about potentially describing a 
specific memory layout?...

> We could
> make them all a single string such as:
I am afraid that a no-go for me. This is error-prone for the user, not 
readable and a nightmare to parse.

To be honest, I don't think following xl format is the right way to go. 
It is a pain to describe nicely devices or even a memory layout. The DT 
format (or any tree format) feels more suitable here.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 08/15] xen/arm: probe domU kernels and initrds

[Stefano Stabellini]

On Thu, 14 Jun 2018, Julien Grall wrote:
> Hi Stefano,
> 
> On 13/06/18 23:15, Stefano Stabellini wrote:
> > Find addresses and sizes on device tree.
> > Introduce a new boot_module_find_by_addr_and_kind function to match not
> > just on boot module kind, but also by address so that we can support
> > multiple domUs.
> > 
> > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > ---
> >   xen/arch/arm/kernel.c       | 54
> > +++++++++++++++++++++++++++++++++++++++++++++
> >   xen/arch/arm/kernel.h       |  2 ++
> >   xen/arch/arm/setup.c        | 15 +++++++++++++
> >   xen/include/asm-arm/setup.h |  2 ++
> >   4 files changed, 73 insertions(+)
> > 
> > diff --git a/xen/arch/arm/kernel.c b/xen/arch/arm/kernel.c
> > index 8fdfd91..c41092e 100644
> > --- a/xen/arch/arm/kernel.c
> > +++ b/xen/arch/arm/kernel.c
> > @@ -496,6 +496,60 @@ err:
> >       return rc;
> >   }
> >   +int kernel_probe_domU(struct kernel_info *info, struct dt_device_node
> > *domU)
> > +{
> > +    struct bootmodule *mod;
> > +    struct dt_device_node *node;
> > +    u64 kernel_addr, initrd_addr, size;
> > +    int rc;
> > +
> > +    dt_for_each_child_node(domU, node)
> > +    {
> > +        if ( dt_device_is_compatible(node, "multiboot,domU-kernel") )
> > +        {
> > +            u32 len;
> > +            const __be32 *val;
> > +            val = dt_get_property(node, "reg", &len);
> > +            dt_get_range(&val, node, &kernel_addr, &size);
> > +        }
> > +        else if ( dt_device_is_compatible(node, "multiboot,domU-ramdisk") )
> > +        {
> > +            u32 len;
> > +            const __be32 *val;
> > +            val = dt_get_property(node, "reg", &len);
> > +            dt_get_range(&val, node, &initrd_addr, &size);
> > +        }
> > +        else
> > +            continue;
> > +    }
> > +    info->kernel_bootmodule = mod = boot_module_find_by_addr_and_kind(
> > +
> > BOOTMOD_DOMU_KERNEL, kernel_addr);
> 
> This line contains hard tab.

This patch was screwed for some reason. I fixed it.


> But I don't think this will work as you expect. Imagine the kernel is the same
> for each guest. It would be fine to have the Image loaded once in memory and
> therefore specify the same physical address for all domU-kernel compatible
> node. However, the command line may be different.
> 
> So you would end up to use the wrong module here.

Yes, you are right. In order to fix the issue, I had to rework the way
boot_modules are done. In the next series, it will be possible to use
the same kernel address in multiple modules, and the right cmdline will
get retrieved.


> > +    info->initrd_bootmodule = boot_module_find_by_addr_and_kind(
> > +
> > BOOTMOD_DOMU_RAMDISK, initrd_addr);
> 
> Same here.
> 
> > +    printk("Loading DomU kernel from boot module @ %"PRIpaddr"\n",
> > +		   info->kernel_bootmodule->start);
> 
> The indentation is wrong here.
> 
> > +    if ( info->initrd_bootmodule )
> > +        printk("Loading ramdisk from boot module @ %"PRIpaddr"\n",
> > +               info->initrd_bootmodule->start);
> > +
> > +    /* if it is a gzip'ed image, 32bit or 64bit, uncompress it */
> > +    rc = kernel_decompress(mod);
> > +    if (rc < 0 && rc != -EINVAL)
> > +        return rc;
> > +
> > +#ifdef CONFIG_ARM_64
> > +    rc = kernel_zimage64_probe(info, mod->start, mod->size);
> > +    if (rc < 0)
> > +#endif
> > +        rc = kernel_uimage_probe(info, mod->start, mod->size);
> > +    if (rc < 0)
> > +        rc = kernel_zimage32_probe(info, mod->start, mod->size);
> > +    if (rc < 0)
> > +        rc = kernel_elf_probe(info, mod->start, mod->size);
> 
> Most of this code is the same as kernel_probe. How about reworking
> kernel_probe to handle any domain?

Yes, I can make it common.


> > +
> > +    return rc;
> > +}
> > +
> >   int kernel_probe(struct kernel_info *info)
> >   {
> >       struct bootmodule *mod = boot_module_find_by_kind(BOOTMOD_KERNEL);
> > diff --git a/xen/arch/arm/kernel.h b/xen/arch/arm/kernel.h
> > index 6d69509..8e1614b 100644
> > --- a/xen/arch/arm/kernel.h
> > +++ b/xen/arch/arm/kernel.h
> > @@ -64,6 +64,8 @@ struct kernel_info {
> >    */
> >   int kernel_probe(struct kernel_info *info);
> >   +int kernel_probe_domU(struct kernel_info *info, struct dt_device_node
> > *node);
> > +
> >   /*
> >    * Loads the kernel into guest RAM.
> >    *
> > diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> > index 82593c8..98bdb24 100644
> > --- a/xen/arch/arm/setup.c
> > +++ b/xen/arch/arm/setup.c
> > @@ -241,6 +241,21 @@ struct bootmodule * __init
> > boot_module_find_by_kind(bootmodule_kind kind)
> >       return NULL;
> >   }
> >   +struct bootmodule * __init
> > boot_module_find_by_addr_and_kind(bootmodule_kind kind,
> > +                                                             paddr_t start)
> > +{
> > +    struct bootmodules *mods = &bootinfo.modules;
> > +    struct bootmodule *mod;
> > +    int i;
> > +    for (i = 0 ; i < mods->nr_mods ; i++ )
> > +    {
> > +        mod = &mods->module[i];
> > +        if ( mod->kind == kind && mod->start == start )
> > +            return mod;
> > +    }
> > +    return NULL;
> > +}
> > +
> >   const char * __init boot_module_kind_as_string(bootmodule_kind kind)
> >   {
> >       switch ( kind )
> > diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
> > index 86aac0e..903782f 100644
> > --- a/xen/include/asm-arm/setup.h
> > +++ b/xen/include/asm-arm/setup.h
> > @@ -83,6 +83,8 @@ struct bootmodule *add_boot_module(bootmodule_kind kind,
> >                                      paddr_t start, paddr_t size,
> >                                      const char *cmdline);
> >   struct bootmodule *boot_module_find_by_kind(bootmodule_kind kind);
> > +struct bootmodule * __init
> > boot_module_find_by_addr_and_kind(bootmodule_kind kind,
> > +                                                             paddr_t
> > start);
> >   const char * __init boot_module_kind_as_string(bootmodule_kind kind);
> >     #endif
> > 
> 
> Cheers,
> 
> -- 
> Julien Grall
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 09/15] xen/arm: refactor construct_dom0

[Stefano Stabellini]

On Fri, 15 Jun 2018, Julien Grall wrote:
> Hi Stefano,
> 
> On 06/15/2018 12:35 AM, Stefano Stabellini wrote:
> > On Thu, 14 Jun 2018, Julien Grall wrote:
> > > On 13/06/18 23:15, Stefano Stabellini wrote:
> > > > -
> > > > -    printk("*** LOADING DOMAIN 0 ***\n");
> > > > -    if ( dom0_mem <= 0 )
> > > > -    {
> > > > -        warning_add("PLEASE SPECIFY dom0_mem PARAMETER -
> > > > USING 512M FOR
> > > > NOW\n");
> > > > -        dom0_mem = MB(512);
> > > > -    }
> > > > -
> > > > -
> > > > -    iommu_hwdom_init(d);
> > > > -
> > > > -    d->max_pages = ~0U;
> > > > -
> > > > -    kinfo.unassigned_mem = dom0_mem;
> > > > -    kinfo.d = d;
> > > > -
> > > > -    rc = kernel_probe(&kinfo);
> > > > -    if ( rc < 0 )
> > > > -        return rc;
> > > > -
> > > >    #ifdef CONFIG_ARM_64
> > > >        /* if aarch32 mode is not supported at EL1 do not
> > > > allow 32-bit domain
> > > > */
> > > > -    if ( !(cpu_has_el1_32) && kinfo.type == DOMAIN_32BIT
> > > > )
> > > > +    if ( !(cpu_has_el1_32) && kinfo->type ==
> > > > DOMAIN_32BIT )
> > > >        {
> > > >            printk("Platform does not support 32-bit
> > > > domain\n");
> > > >            return -EINVAL;
> > > >        }
> > > > -    d->arch.type = kinfo.type;
> > > 
> > > Any reason to move this out?
> > 
> > Yeah, initially I left it there but it didn't work. It needs
> > to be set
> > before calling allocate_memory() for domUs otherwise memory
> > allocations
> > fail.
> 
> Oh because allocate_domain(d) rely on is_domain_32bit, right? I
> don't much like the duplication here just because of
> prepare_dtb_domU. I am wondering if we could do:
> 
> if ( !is_hardware_domain(d) )
>   prepare_dtb_domU(...);
> else if ( acpi_disabled )
>   prepare_acpi_hwdom(...);
> else
>   prepare_dt_hwdom(....);

The few remaining things in construct_dom0 and construct_domU are
different enough that I don't think there is much gain in trying to
merge them.


> > > > +    if ( acpi_disabled )
> > > > +        rc = prepare_dtb(d, &kinfo);
> > > > +    else
> > > > +        rc = prepare_acpi(d, &kinfo);
> > > > +
> > > > +    if ( rc < 0 )
> > > > +        return rc;
> > > > +
> > > > +    discard_initial_modules();
> > > 
> > > You say "no functional change" in this patch. But this is
> > > one. The module are
> > > now discard much earlier. This imply that memory baking the
> > > Image/Initrd will
> > > be free to be re-used at any time.
> > > 
> > > I don't think this is what we want. Unless you can promise
> > > no memory is
> > > allocated in __construct_domain().
> > 
> > discard_initial_modules() will be moved later by patch #14,
> > but I think
> > it makes sense to call discard_initial_modules() after
> > __construct_domain() here.
> 
> Yeah, I noticed you moved the discard_initial_modules() later
> on. But I would like to have the series bisectable if possible
> :).

Yep, it will be done.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 09/15] xen/arm: refactor construct_dom0

[Julien Grall]

Hi Stefano,

On 05/07/2018 21:55, Stefano Stabellini wrote:
> On Fri, 15 Jun 2018, Julien Grall wrote:
>> Hi Stefano,
>>
>> On 06/15/2018 12:35 AM, Stefano Stabellini wrote:
>>> On Thu, 14 Jun 2018, Julien Grall wrote:
>>>> On 13/06/18 23:15, Stefano Stabellini wrote:
>> if ( !is_hardware_domain(d) )
>>    prepare_dtb_domU(...);
>> else if ( acpi_disabled )
>>    prepare_acpi_hwdom(...);
>> else
>>    prepare_dt_hwdom(....);
> 
> The few remaining things in construct_dom0 and construct_domU are
> different enough that I don't think there is much gain in trying to
> merge them.

Do you have a concrete example? When I looked at it the only things I 
can find different are:
    - iommu_hwdom_init() -> Most likely you would need a similar call 
somewhere for Guest DomU.
    - find_gnttab_region -> I think this could be re-ordered
    - gic_map_hwdom_extra_mappings & platform_specific_mapping -> This 
can be called before the construct.

So I don't see why we can't merge most of the code and still have few 
lines for Dom0.

Overall I would much prefer if we don't duplicate the allocation of the 
memory or anything that is not Dom0 specific.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 10/15] xen/arm: introduce construct_domU

[Stefano Stabellini]

On Thu, 14 Jun 2018, Julien Grall wrote:
> Hi,
> 
> On 13/06/18 23:15, Stefano Stabellini wrote:
> > Similar to construct_dom0, construct_domU creates a barebone DomU guest.
> > Default to 1 max vcpu and 64MB of memory if not specified otherwise.
> > 
> > The device tree node passed as argument is compatible "xen,domU", see
> > docs/misc/arm/device-tree/booting.txt.
> > 
> > Allocate all vcpus on cpu0 initially.
> 
> I don't think this comment is true. __construct_domain will allocate vCPUs in
> cycle.

I'll remove the comment


> > 
> > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > ---
> >   xen/arch/arm/domain_build.c | 37 +++++++++++++++++++++++++++++++++++++
> >   xen/include/asm-arm/setup.h |  2 ++
> >   2 files changed, 39 insertions(+)
> > 
> > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > index b31c563..02a7f94 100644
> > --- a/xen/arch/arm/domain_build.c
> > +++ b/xen/arch/arm/domain_build.c
> > @@ -2187,6 +2187,43 @@ int __init __construct_domain(struct domain *d,
> > struct kernel_info *kinfo)
> >       return 0;
> >   }
> >   +int __init construct_domU(struct domain *d, struct dt_device_node *node)
> > +{
> > +    struct kernel_info kinfo = {};
> > +    int rc;
> > +    const char *cpus = NULL, *mem = NULL;
> > +
> > +    printk("*** LOADING DOMU ***\n");
> > +
> > +    d->max_vcpus = 1;
> > +    rc = dt_property_read_string(node, "cpus", &cpus);
> > +    if ( !rc )
> > +        d->max_vcpus = simple_strtoul(cpus, &cpus, 0);
> > +
> > +    kinfo.unassigned_mem = MB(64);
> > +    rc = dt_property_read_string(node, "mem", &mem);
> > +    if ( !rc )
> > +        kinfo.unassigned_mem = parse_size_and_unit(mem, &mem);
> > +
> > +    d->vcpu = xzalloc_array(struct vcpu *, d->max_vcpus);
> > +    if ( !d->vcpu )
> > +        return -ENOMEM;;
> > +    if ( alloc_vcpu(d, 0, 0) == NULL )
> > +        return -ENOMEM;
> > +    d->max_pages = ~0U;
> > +
> > +    kinfo.d = d;
> > +
> > +    rc = kernel_probe_domU(&kinfo, node);
> > +    if ( rc < 0 )
> > +        return rc;
> > +
> > +    d->arch.type = kinfo.type;
> > +    allocate_memory(d, &kinfo);
> 
> allocate_memory() will allocate direct mapped memory but you impose a static
> memory layout for the guest. Both are not going to work very well together.
> 
> So you probably want to extend allocate_memory to support allocating memory
> for a given region.

Yes, I'll extend allocate_memory to respect the current domU memory layout.


> > +
> > +    return __construct_domain(d, &kinfo);
> > +}
> > +
> >   int __init construct_dom0(struct domain *d)
> >   {
> >       struct kernel_info kinfo = {};
> > diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
> > index 903782f..e9f9905 100644
> > --- a/xen/include/asm-arm/setup.h
> > +++ b/xen/include/asm-arm/setup.h
> > @@ -2,6 +2,7 @@
> >   #define __ARM_SETUP_H_
> >     #include <public/version.h>
> > +#include <xen/device_tree.h>
> >     #define MIN_FDT_ALIGN 8
> >   #define MAX_FDT_SIZE SZ_2M
> > @@ -71,6 +72,7 @@ void acpi_create_efi_mmap_table(struct domain *d,
> >   int acpi_make_efi_nodes(void *fdt, struct membank tbl_add[]);
> >     int construct_dom0(struct domain *d);
> > +int construct_domU(struct domain *d, struct dt_device_node *node);
> >     void discard_initial_modules(void);
> >   void dt_unreserved_regions(paddr_t s, paddr_t e,

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 11/15] xen/arm: generate a simple device tree for domUs

[Stefano Stabellini]

On Thu, 14 Jun 2018, Julien Grall wrote:
> Hi Stefano,
> 
> On 13/06/18 23:15, Stefano Stabellini wrote:
> > Introduce functions to generate a basic domU device tree, similar to the
> > existing functions in tools/libxl/libxl_arm.c.
> > 
> > Rename existing prepare_dtb to prepare_dtb_dom0 to avoid confusion.
> 
> Ah this is were the rename is. It might have make sense to do that in patch
> #9. Also, you want to name it "hwdom" as this is the preferred name nowadays.

OK


> > 
> > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > ---
> >   xen/arch/arm/domain_build.c | 195
> > +++++++++++++++++++++++++++++++++++++++++++-
> >   1 file changed, 193 insertions(+), 2 deletions(-)
> > 
> > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > index 02a7f94..b4f560f 100644
> > --- a/xen/arch/arm/domain_build.c
> > +++ b/xen/arch/arm/domain_build.c
> > @@ -1360,7 +1360,194 @@ static int handle_node(struct domain *d, struct
> > kernel_info *kinfo,
> >       return res;
> >   }
> >   -static int prepare_dtb(struct domain *d, struct kernel_info *kinfo)
> > +static int make_gic_domU_node(const struct domain *d, void *fdt, int
> > addrcells, int sizecells)
> > +{
> > +    int res = 0;
> > +    int reg_size = addrcells + sizecells;
> > +    int nr_cells = reg_size * 2;
> > +    __be32 reg[nr_cells]; > +    __be32 *cells;
> > +
> > +    res = fdt_begin_node(fdt, "interrupt-controller");
> 
> Per the DT spec, node name should contain @unit-address when a "reg" property
> present. "unit-address" been the first address of the "reg" property.
> 
> Per the DT spec, node should have @base when a "regs" property is present.

OK, I'll add it


> > +    if ( res )
> > +        return res;
> > +
> > +    res = fdt_property_cell(fdt, "#address-cells", 0);
> > +    if ( res )
> > +        return res;
> > +
> > +    res = fdt_property_cell(fdt, "#interrupt-cells", 3);
> > +    if ( res )
> > +        return res;
> > +
> > +    res = fdt_property(fdt, "interrupt-controller", NULL, 0);
> > +    if ( res )
> > +        return res;
> > +
> > +    if (gic_hw_version() == GIC_V3)
> 
> Could we use a switch please? This would allow to catch easily new hardware
> version.

Sure


> > +    {
> > +        const uint64_t gicd_base = GUEST_GICV3_GICD_BASE;
> > +        const uint64_t gicd_size = GUEST_GICV3_GICD_SIZE;
> > +        const uint64_t gicr0_base = GUEST_GICV3_GICR0_BASE;
> > +        const uint64_t gicr0_size = GUEST_GICV3_GICR0_SIZE;
> > +
> > +        res = fdt_property_string(fdt, "compatible", "arm,gic-v3");
> > +        if ( res )
> > +            return res;
> > +
> > +        cells = &reg[0];
> > +        dt_child_set_range(&cells, addrcells, sizecells, gicd_base,
> > gicd_size);
> > +        dt_child_set_range(&cells, addrcells, sizecells, gicr0_base,
> > gicr0_size);
> > +        res = fdt_property(fdt, "reg", reg, sizeof(reg));
> > +    }
> > +    else if (gic_hw_version() == GIC_V3)
> 
> I think this should be GIC_V2 here.

Yep


> > +    {
> > +        const uint64_t gicd_base = GUEST_GICD_BASE;
> > +        const uint64_t gicd_size = GUEST_GICD_SIZE;
> > +        const uint64_t gicc_base = GUEST_GICC_BASE;
> > +        const uint64_t gicc_size = GUEST_GICC_SIZE;
> > +
> > +        res = fdt_property_string(fdt, "compatible", "arm,cortex-a15-gic");
> 
> I know that we use that property in libxl. But this is a bit weird to use it
> for Arm64 guest :). It would be better to use arm,gic-400 here.

OK

> > +        if ( res )
> > +            return res;
> > +
> > +        cells = &reg[0];
> > +        dt_child_set_range(&cells, addrcells, sizecells, gicd_base,
> > gicd_size);
> > +        dt_child_set_range(&cells, addrcells, sizecells, gicc_base,
> > gicc_size);
> > +    }
> > +
> > +    res = fdt_property(fdt, "reg", reg, sizeof(reg));
> > +    if (res)
> > +        return res;
> > +
> > +    res = fdt_property_cell(fdt, "linux,phandle", PHANDLE_GIC);
> > +    if (res)
> > +        return res;
> > +
> > +    res = fdt_property_cell(fdt, "phandle", PHANDLE_GIC);
> > +    if (res)
> > +        return res;
> > +
> > +    res = fdt_end_node(fdt);
> > +
> > +    return res;
> > +}
> > +
> > +static int make_timer_domU_node(const struct domain *d, void *fdt)
> > +{
> > +    int res;
> > +    gic_interrupt_t intrs[3];
> > +
> > +    res = fdt_begin_node(fdt, "timer");
> > +    if ( res )
> > +        return res;
> > +
> > +    if (!is_64bit_domain(d))
> > +    {
> > +        res = fdt_property_string(fdt, "compatible", "arm,armv7-timer");
> > +        if ( res )
> > +            return res;
> > +    } else {
> > +        res = fdt_property_string(fdt, "compatible", "arm,armv8-timer");
> > +        if ( res )
> > +            return res;
> > +    }
> > +
> > +    set_interrupt_ppi(intrs[0], GUEST_TIMER_PHYS_S_PPI, 0xf,
> > DT_IRQ_TYPE_LEVEL_LOW);
> > +    set_interrupt_ppi(intrs[1], GUEST_TIMER_PHYS_NS_PPI, 0xf,
> > DT_IRQ_TYPE_LEVEL_LOW);
> > +    set_interrupt_ppi(intrs[2], GUEST_TIMER_VIRT_PPI, 0xf,
> > DT_IRQ_TYPE_LEVEL_LOW);
> > +
> > +    res = fdt_property(fdt, "interrupts", intrs, sizeof (intrs[0]) * 3);
> > +    if ( res )
> > +        return res;
> > +
> > +    res = fdt_property_cell(fdt, "interrupt-parent",
> > +                            PHANDLE_GIC);
> > +    if (res)
> > +        return res;
> > +
> > +    res = fdt_end_node(fdt);
> > +    return res;
> > +}
> > +
> > +#define DOMU_DTB_SIZE 4096
> 
> Is this going to be enough? Per the documentation, the maximum size of a DT is
> 2MB.

That is what we use today for the small generated DTB for ACPI dom0.
Certainly it is enough memory from my empirical tests so far.  I suggest
to keep as is in this series, then potentially change it when we
introduce larger DomU DTBs (device assignment might require it, for
example).


> > +static int prepare_dtb_domU(struct domain *d, struct kernel_info *kinfo)
> > +{
> > +    int addrcells, sizecells;
> > +    int ret;
> > +
> > +    addrcells = dt_child_n_addr_cells(dt_host);
> > +    sizecells = dt_child_n_size_cells(dt_host);
> > +
> > +    kinfo->fdt = xmalloc_bytes(DOMU_DTB_SIZE);
> > +    if ( kinfo->fdt == NULL )
> > +        return -ENOMEM;
> > +
> > +    ret = fdt_create(kinfo->fdt, DOMU_DTB_SIZE);
> > +    if ( ret < 0 )
> > +        goto err;
> > +
> > +    ret = fdt_finish_reservemap(kinfo->fdt);
> > +    if ( ret < 0 )
> > +        goto err;
> > +
> > +    ret = fdt_begin_node(kinfo->fdt, "/");
> > +    if ( ret < 0 )
> > +        goto err;
> > +
> > +    ret = fdt_property_cell(kinfo->fdt, "#address-cells", addrcells);
> > +    if ( ret )
> > +        goto err;
> > +
> > +    ret = fdt_property_cell(kinfo->fdt, "#size-cells", sizecells);
> > +    if ( ret )
> > +        goto err;
> > +
> > +    ret = make_chosen_node(kinfo);
> > +    if ( ret )
> > +        goto err;
> > +
> > +    ret = make_hypervisor_node(d, kinfo, addrcells, sizecells);
> 
> make_hypervisor_node() will allocate an PPIfor the event channel based on
> what's available. However, the PPI for the guests is static and the timer is
> allocated afterwards.
> 
> So you probably want to rework make_hypervisor_node to pass the interrupt in
> parameter.

I'll fix


> > +    if ( ret )
> > +        goto err;
> > +
> > +    ret = make_psci_node(kinfo->fdt, NULL);
> > +    if ( ret )
> > +        goto err;
> > +
> > +    ret = make_cpus_node(d, kinfo->fdt, NULL);
> > +    if ( ret )
> > +        goto err;
> > +
> > +    ret = make_memory_node(d, kinfo->fdt, addrcells, sizecells, kinfo);
> > +    if ( ret )
> > +        goto err;
> > +
> > +    ret = make_gic_domU_node(d, kinfo->fdt, addrcells, sizecells);
> > +    if ( ret )
> > +        goto err;
> > +
> > +    ret = make_timer_domU_node(d, kinfo->fdt);
> > +    if ( ret )
> > +        goto err;
> > +
> > +    ret = fdt_end_node(kinfo->fdt);
> > +    if ( ret < 0 )
> > +        goto err;
> > +
> > +    ret = fdt_finish(kinfo->fdt);
> > +    if ( ret < 0 )
> > +        goto err;
> > +
> > +    return 0;
> > +
> > +  err:
> > +    printk("Device tree generation failed (%d).\n", ret);
> > +    xfree(kinfo->fdt);
> > +    return -EINVAL;
> > +}
> > +
> > +static int prepare_dtb_dom0(struct domain *d, struct kernel_info *kinfo)
> >   {
> >       const p2m_type_t default_p2mt = p2m_mmio_direct_c;
> >       const void *fdt;
> > @@ -2221,6 +2408,10 @@ int __init construct_domU(struct domain *d, struct
> > dt_device_node *node)
> >       d->arch.type = kinfo.type;
> >       allocate_memory(d, &kinfo);
> >   +    rc = prepare_dtb_domU(d, &kinfo);
> > +    if ( rc < 0 )
> > +        return rc;
> > +
> >       return __construct_domain(d, &kinfo);
> >   }
> >   @@ -2270,7 +2461,7 @@ int __init construct_dom0(struct domain *d)
> >       d->arch.type = kinfo.type;
> >         if ( acpi_disabled )
> > -        rc = prepare_dtb(d, &kinfo);
> > +        rc = prepare_dtb_dom0(d, &kinfo);
> >       else
> >           rc = prepare_acpi(d, &kinfo);
> >   
> 
> Cheers,
> 
> -- 
> Julien Grall
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 07/15] xen/arm: increase MAX_MODULES

[Doug Goldstein]

On Wed, Jun 13, 2018 at 03:15:10PM -0700, Stefano Stabellini wrote:
> Xen boot modules need to account not just for Dom0 but also for a few
> potential DomUs, each of them coming with their own kernel and initrd.
> Increase MAX_MODULES to 32 to allow for more DomUs.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>

Reviewed-by: Doug Goldstein <cardoe@cardoe.com>

Didn't see any comment on this so far so I figured I'd comment. This is
probably the least controversial patch and could just go in today.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 07/15] xen/arm: increase MAX_MODULES

[Julien Grall]

Hi Doug,

On 06/07/18 03:10, Doug Goldstein wrote:
> On Wed, Jun 13, 2018 at 03:15:10PM -0700, Stefano Stabellini wrote:
>> Xen boot modules need to account not just for Dom0 but also for a few
>> potential DomUs, each of them coming with their own kernel and initrd.
>> Increase MAX_MODULES to 32 to allow for more DomUs.
>>
>> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> 
> Reviewed-by: Doug Goldstein <cardoe@cardoe.com>
> 
> Didn't see any comment on this so far so I figured I'd comment. This is
> probably the least controversial patch and could just go in today.

The only reason I didn't comment this is because I wanted to see how the 
rest of the series go. The patch standalone is not something that is 
useful for Xen today as we will never have more than 5 modules. So I 
would rather wait the rest of the series before merging it.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 11/15] xen/arm: generate a simple device tree for domUs

[Julien Grall]

On 06/07/18 00:59, Stefano Stabellini wrote:
> On Thu, 14 Jun 2018, Julien Grall wrote:
>>> +#define DOMU_DTB_SIZE 4096
>>
>> Is this going to be enough? Per the documentation, the maximum size of a DT is
>> 2MB.
> 
> That is what we use today for the small generated DTB for ACPI dom0.

The generated DTB for ACPI dom0 is much more smaller than this one. :)

> Certainly it is enough memory from my empirical tests so far.  I suggest
> to keep as is in this series, then potentially change it when we
> introduce larger DomU DTBs (device assignment might require it, for
> example).

Can you please add a comment on top of DOMU_DTB_SIZE then?

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 11/15] xen/arm: generate a simple device tree for domUs

[Stefano Stabellini]

On Fri, 6 Jul 2018, Julien Grall wrote:
> On 06/07/18 00:59, Stefano Stabellini wrote:
> > On Thu, 14 Jun 2018, Julien Grall wrote:
> > > > +#define DOMU_DTB_SIZE 4096
> > > 
> > > Is this going to be enough? Per the documentation, the maximum size of a
> > > DT is
> > > 2MB.
> > 
> > That is what we use today for the small generated DTB for ACPI dom0.
> 
> The generated DTB for ACPI dom0 is much more smaller than this one. :)
> 
> > Certainly it is enough memory from my empirical tests so far.  I suggest
> > to keep as is in this series, then potentially change it when we
> > introduce larger DomU DTBs (device assignment might require it, for
> > example).
> 
> Can you please add a comment on top of DOMU_DTB_SIZE then?

yes, I'll do

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 12/15] xen/arm: generate vpl011 node on device tree for domU

[Stefano Stabellini]

On Fri, 15 Jun 2018, Julien Grall wrote:
> Hi Stefano,
> 
> On 06/13/2018 11:15 PM, Stefano Stabellini wrote:
> > Introduce vpl011 support to guests started from Xen: it provides a
> > simple way to print output from a guest, as most guests come with a
> > pl011 driver. It is also able to provide a working console with
> > interrupt support.
> > 
> > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > ---
> >   xen/arch/arm/domain_build.c | 70
> > +++++++++++++++++++++++++++++++++++++++++++++
> >   1 file changed, 70 insertions(+)
> > 
> > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > index b4f560f..ff65057 100644
> > --- a/xen/arch/arm/domain_build.c
> > +++ b/xen/arch/arm/domain_build.c
> > @@ -1470,6 +1470,70 @@ static int make_timer_domU_node(const struct domain
> > *d, void *fdt)
> >       return res;
> >   }
> >   +static void set_interrupt(gic_interrupt_t *interrupt, unsigned int irq,
> 
> The definition of interrupt looks suspicious. gic_interrupt_t is defined as
> be32[3]. Here you pass a pointer, so interrupt type would be __be32 **, that
> you crudely cast to __be32* below.
> 
> Most likely you don't want to pass a pointer here and just use the type
> gic_interrupt_t. Because it is an array, then there will be no issue.

Right, I'll fix


> > +                          unsigned int cpumask, unsigned int level)
> > +{
> > +    __be32 *cells = (__be32 *) interrupt;
> 
> Explicit cast are always a bad idea. If you need one, then mostly likely you
> did something wrong :). In that case interrupt type is __be32** and you cast
> to __be32*. If you change the type as suggested above, then the cast will not
> be necessary here.

Yeah


> > +    int is_ppi = (irq < 32);
> > +
> > +    irq -= (is_ppi) ? 16: 32; /* PPIs start at 16, SPIs at 32 */
> > +
> > +    /* See linux Documentation/devictree/bindings/arm/gic.txt */
> > +    dt_set_cell(&cells, 1, is_ppi); /* is a PPI? */
> > +    dt_set_cell(&cells, 1, irq);
> > +    dt_set_cell(&cells, 1, (cpumask << 8) | level);
> > +}
> 
> We already have a function to generate PPI interrupt (set_interrupt_ppi).
> Would it be possible to extend it to support interrupt?
> 
> Most likely, you will want to use set_interrupt(...) everywhere and just drop
> set_interrupt_ppi.

Good idea, I'll remove set_interrupt_ppi

> > +
> > +#ifdef CONFIG_SBSA_VUART_CONSOLE
> > +static int make_vpl011_uart_node(const struct domain *d, void *fdt,
> > +                                 int addrcells, int sizecells)
> > +{
> > +    int res;
> > +    gic_interrupt_t intr;
> > +    int reg_size = addrcells + sizecells;
> > +    int nr_cells = reg_size;
> > +    __be32 reg[nr_cells];
> > +    __be32 *cells;
> > +
> > +    res = fdt_begin_node(fdt, "sbsa-pl011");
> > +    if (res)
> 
> Coding style:
> 
> if ( ... )
> 
> > +        return res;
> > +
> > +    res = fdt_property_string(fdt, "compatible", "arm,sbsa-uart");
> 
> To make clear, you are exposing a SBSA compatible UART and not a PL011. SBSA
> UART is a subset of PL011 r1p5. A full PL011 implementation in Xen would just
> be too difficult, so your guest may require some changes in their driver.
> 
> I think this is a small price to pay, but I wanted to make sure you don't
> expect the guest to drive the UART the same way a PL011.

I'll add a note to the commit message


> > +    if (res)
> 
> Coding style
> 
> > +        return res;
> > +
> > +    cells = &reg[0];
> > +    dt_child_set_range(&cells, addrcells, sizecells, GUEST_PL011_BASE,
> > +            GUEST_PL011_SIZE);
> 
> The indentation looks wrong here.
> 
> > +    if (res)
> 
> Coding style
> 
> > +        return res;
> > +    res = fdt_property(fdt, "reg", reg, sizeof(reg));
> > +    if (res)
> 
> Coding style
> 
> > +        return res;
> > +
> > +    set_interrupt(&intr, GUEST_VPL011_SPI, 0xf, DT_IRQ_TYPE_LEVEL_HIGH);
> > +
> > +    res = fdt_property(fdt, "interrupts", intr, sizeof (intr));
> > +    if (res)
> 
> Coding style
> 
> > +        return res;
> > +
> > +    res = fdt_property_cell(fdt, "interrupt-parent",
> > +                            PHANDLE_GIC);
> > +    if (res)
> 
> Coding style
> 
> > +        return res;
> > +
> > +    /* Use a default baud rate of 115200. */
> > +    fdt_property_u32(fdt, "current-speed", 115200);
> > +
> > +    res = fdt_end_node(fdt);
> > +    if (res)
> 
> Coding style

Fixed them all


> > +        return res;
> > +
> > +    return 0;
> > +}
> > +#endif
> > +
> >   #define DOMU_DTB_SIZE 4096
> >   static int prepare_dtb_domU(struct domain *d, struct kernel_info *kinfo)
> >   {
> > @@ -1531,6 +1595,12 @@ static int prepare_dtb_domU(struct domain *d, struct
> > kernel_info *kinfo)
> >       if ( ret )
> >           goto err;
> >   +#ifdef CONFIG_SBSA_VUART_CONSOLE.
> > +    ret = make_vpl011_uart_node(d, kinfo->fdt, addrcells, sizecells);
> 
> I would prefer if don't expose the pl011 by default to a guest and provide a
> way to enable it for a given guest

I'll add a vpl011 option


> > +    if ( ret )
> > +        goto err;
> > +#endif
> > +
> >       ret = fdt_end_node(kinfo->fdt);
> >       if ( ret < 0 )
> >           goto err;
> > 
> 
> Cheers,
> 
> -- 
> Julien Grall
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 13/15] xen/arm: Allow vpl011 to be used by DomU

[Stefano Stabellini]

On Fri, 15 Jun 2018, Julien Grall wrote:
> Hi Stefano,
> 
> On 06/13/2018 11:15 PM, Stefano Stabellini wrote:
> > Make vpl011 being able to be used without a userspace component in Dom0.
> > In that case, output is printed to the Xen serial and input is received
> > from the Xen serial one character at a time.
> > 
> > Call domain_vpl011_init during construct_domU.
> > 
> > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > ---
> >   xen/arch/arm/domain_build.c  |  9 +++-
> >   xen/arch/arm/vpl011.c        | 98
> > +++++++++++++++++++++++++++++++++-----------
> >   xen/include/asm-arm/vpl011.h |  2 +
> >   3 files changed, 84 insertions(+), 25 deletions(-)
> > 
> > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > index ff65057..97f14ca 100644
> > --- a/xen/arch/arm/domain_build.c
> > +++ b/xen/arch/arm/domain_build.c
> > @@ -2482,7 +2482,14 @@ int __init construct_domU(struct domain *d, struct
> > dt_device_node *node)
> >       if ( rc < 0 )
> >           return rc;
> >   -    return __construct_domain(d, &kinfo);
> > +    rc = __construct_domain(d, &kinfo);
> > +    if ( rc < 0 )
> > +        return rc;
> > +
> > +#ifdef CONFIG_SBSA_VUART_CONSOLE
> > +    rc = domain_vpl011_init(d, NULL);
> 
> See my remark on the previous patch about exposing vpl011 by default.

Done


> > +#endif
> > +    return rc;
> >   }
> >     int __init construct_dom0(struct domain *d)
> > diff --git a/xen/arch/arm/vpl011.c b/xen/arch/arm/vpl011.c
> > index a281eab..5f1dc7a 100644
> > --- a/xen/arch/arm/vpl011.c
> > +++ b/xen/arch/arm/vpl011.c
> > @@ -34,6 +34,8 @@
> >   #include <asm/vgic-emul.h>
> >   #include <asm/vpl011.h>
> >   +static void vpl011_data_avail(struct domain *d);
> > +
> >   /*
> >    * Since pl011 registers are 32-bit registers, all registers
> >    * are handled similarly allowing 8-bit, 16-bit and 32-bit
> > @@ -77,6 +79,29 @@ static void vpl011_update_interrupt_status(struct domain
> > *d)
> >   #endif
> >   }
> >   +void vpl011_read_char(struct domain *d, char c)
> 
> The name is slightly odd. From the name, I would expect that a character is
> returned. But in fact, you write a character you received in the ring. So a
> better name would be vpl011_rx_char.

OK


> > +{
> > +    unsigned long flags;
> > +    XENCONS_RING_IDX in_cons, in_prod;
> > +    struct xencons_interface *intf = d->arch.vpl011.ring_buf;
> > +
> > +    VPL011_LOCK(d, flags);
> > +
> > +    in_cons = intf->in_cons;
> > +    in_prod = intf->in_prod;
> > +    if (xencons_queued(in_prod, in_cons, sizeof(intf->in)) ==
> > sizeof(intf->in))
> > +    {
> > +        VPL011_UNLOCK(d, flags);
> > +        return;
> > +    }
> > +
> > +    intf->in[xencons_mask(in_prod, sizeof(intf->in))] = c;
> > +    intf->in_prod = in_prod + 1;
> > +
> > +    VPL011_UNLOCK(d, flags);
> > +    vpl011_data_avail(d);
> > +}
> > +
> >   static uint8_t vpl011_read_data(struct domain *d)
> >   {
> >       unsigned long flags;
> > @@ -166,9 +191,18 @@ static void vpl011_write_data(struct domain *d, uint8_t
> > data)
> >       struct vpl011 *vpl011 = &d->arch.vpl011;
> >       struct xencons_interface *intf = vpl011->ring_buf;
> >       XENCONS_RING_IDX out_cons, out_prod;
> > +    unsigned int fifo_level = 0;
> >         VPL011_LOCK(d, flags);
> >   +    if ( vpl011->ring_page == NULL )
> > +    {
> > +        printk("%c", data);
> > +        if (data == '\n')
> > +            printk("DOM%u: ", d->domain_id);
> > +        goto done;
> > +    }
> > +
> 
> I would rather introduce separate function to read/write data for the case
> without PV console. And use it where appropriate. This would make the code
> slightly easier to understand because "ring_page == NULL" is slightly
> untuitive.
> 
> An idea would be introduce callback and set them during the initialization of
> the vpl011 for the domain.
> 
> >       out_cons = intf->out_cons;
> >       out_prod = intf->out_prod;
> >   @@ -184,13 +218,10 @@ static void vpl011_write_data(struct domain *d,
> > uint8_t data)
> >       if ( xencons_queued(out_prod, out_cons, sizeof(intf->out)) !=
> >            sizeof (intf->out) )
> >       {
> > -        unsigned int fifo_level;
> > -
> >           intf->out[xencons_mask(out_prod, sizeof(intf->out))] = data;
> >           out_prod += 1;
> >           smp_wmb();
> >           intf->out_prod = out_prod;
> > -
> 
> Spurious change.
 
removed


> >           fifo_level = xencons_queued(out_prod, out_cons,
> > sizeof(intf->out));
> >             if ( fifo_level == sizeof(intf->out) )
> > @@ -205,14 +236,15 @@ static void vpl011_write_data(struct domain *d,
> > uint8_t data)
> >                */
> >               vpl011->uartfr |= BUSY;
> >           }
> > -
> > -        vpl011_update_tx_fifo_status(vpl011, fifo_level);
> > -
> > -        vpl011_update_interrupt_status(d);
> >       }
> >       else
> >           gprintk(XENLOG_ERR, "vpl011: Unexpected OUT ring buffer full\n");
> >   +done:
> > +    vpl011_update_tx_fifo_status(vpl011, fifo_level);
> > +
> > +    vpl011_update_interrupt_status(d);
> 
> Hmmm, now you will call vpl011_update_* in the error case when writing to the
> case. If you want to keep that, this should at least be explained in the
> commit message or probably be a separate patch.

I removed this changed and opted for separate read/write functions and a
smaller ring struct.


> > +
> >       vpl011->uartfr &= ~TXFE;
> >         VPL011_UNLOCK(d, flags);
> > @@ -462,13 +494,30 @@ int domain_vpl011_init(struct domain *d, struct
> > vpl011_init_info *info)
> >       if ( vpl011->ring_buf )
> >           return -EINVAL;
> >   -    /* Map the guest PFN to Xen address space. */
> > -    rc =  prepare_ring_for_helper(d,
> > -                                  gfn_x(info->gfn),
> > -                                  &vpl011->ring_page,
> > -                                  &vpl011->ring_buf);
> > -    if ( rc < 0 )
> > -        goto out;
> > +    if ( info != NULL )
> 
> Please document how info could be NULL here.

OK


> > +    {
> > +        /* Map the guest PFN to Xen address space. */
> > +        rc =  prepare_ring_for_helper(d,
> > +                gfn_x(info->gfn),
> > +                &vpl011->ring_page,
> > +                &vpl011->ring_buf);
> > +        if ( rc < 0 )
> > +            goto out;
> > +
> > +        rc = alloc_unbound_xen_event_channel(d, 0, info->console_domid,
> > +                vpl011_notification);
> > +        if ( rc < 0 )
> > +            goto out2;
> 
> When you move code around, you should also look at the error path. In that
> case, you are going to free a virq that does not exist (because not yet
> allocated), and if vgic_reserve_virq below fails, you will not free the event
> channel allocated.
 
I'll fix


> > +
> > +        vpl011->evtchn = info->evtchn = rc;
> > +    } else {
> > +        vpl011->ring_buf = xzalloc(struct xencons_interface);
> 
> Using ring_buf is such waste of memory. You basically only allow 1024
> character but still using 4K.
> 
> Furthermore, the way you use ring_page is really confusing. A bit more
> documentation might help. Although, this new code does not feel integrated
> with the rest of the vpl011.
> 
> It looks like you want to rework the vpl011 code to have move anything related
> to ring in separate function. Once it is done, we could then introduce new
> callback for the guest started in Xen.

I have done that now, I introduced a smaller ring struct with only the
in parts and introduced separate read_data and write_data functions.
Overall, I think is better. The price to pay is that the new read_data
is almost an exact duplicate of the existing function. They cannot be
reused because the pointer format is different, although the field names
are the same. You'll see in the next version.


> > +        if ( vpl011->ring_buf == NULL )
> > +        {
> > +            rc = -EINVAL;
> > +            goto out;
> > +        }
> > +    }
> >         rc = vgic_reserve_virq(d, GUEST_VPL011_SPI);
> >       if ( !rc )
> > @@ -477,13 +526,6 @@ int domain_vpl011_init(struct domain *d, struct
> > vpl011_init_info *info)
> >           goto out1;
> >       }
> >   -    rc = alloc_unbound_xen_event_channel(d, 0, info->console_domid,
> > -                                         vpl011_notification);
> > -    if ( rc < 0 )
> > -        goto out2;
> > -
> > -    vpl011->evtchn = info->evtchn = rc;
> > -
> >       spin_lock_init(&vpl011->lock);
> >         register_mmio_handler(d, &vpl011_mmio_handler,
> > @@ -495,7 +537,10 @@ out2:
> >       vgic_free_virq(d, GUEST_VPL011_SPI);
> >     out1:
> > -    destroy_ring_for_helper(&vpl011->ring_buf, vpl011->ring_page);
> > +    if ( vpl011->ring_page == NULL && vpl011->ring_buf != NULL )
> > +        xfree(vpl011->ring_buf);
> > +    else
> > +        destroy_ring_for_helper(&vpl011->ring_buf, vpl011->ring_page);
> >     out:
> >       return rc;
> > @@ -508,8 +553,13 @@ void domain_vpl011_deinit(struct domain *d)
> >       if ( !vpl011->ring_buf )
> >           return;
> >   -    free_xen_event_channel(d, vpl011->evtchn);
> > -    destroy_ring_for_helper(&vpl011->ring_buf, vpl011->ring_page);
> > +    if ( vpl011->ring_page == NULL && vpl011->ring_buf != NULL )
> > +    {
> > +        xfree(vpl011->ring_buf);
> > +    } else {
> 
> Coding style
> 
> }
> else
> {
 
OK


> > +        free_xen_event_channel(d, vpl011->evtchn);
> > +        destroy_ring_for_helper(&vpl011->ring_buf, vpl011->ring_page);
> > +    }
> >   }
> >     /*
> > diff --git a/xen/include/asm-arm/vpl011.h b/xen/include/asm-arm/vpl011.h
> > index db95ff8..8d9b0da 100644
> > --- a/xen/include/asm-arm/vpl011.h
> > +++ b/xen/include/asm-arm/vpl011.h
> > @@ -53,6 +53,7 @@ struct vpl011_init_info {
> >   int domain_vpl011_init(struct domain *d,
> >                          struct vpl011_init_info *info);
> >   void domain_vpl011_deinit(struct domain *d);
> > +void vpl011_read_char(struct domain *d, char c);
> >   #else
> >   static inline int domain_vpl011_init(struct domain *d,
> >                                        struct vpl011_init_info *info)
> > @@ -61,6 +62,7 @@ static inline int domain_vpl011_init(struct domain *d,
> >   }
> >     static inline void domain_vpl011_deinit(struct domain *d) { }
> > +static inline void vpl011_read_char(struct domain *d, char c) { }
> >   #endif
> >   #endif  /* _VPL011_H_ */
> >   

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 14/15] xen/arm: call construct_domU from start_xen and start DomU VMs

[Stefano Stabellini]

On Fri, 15 Jun 2018, Julien Grall wrote:
> On 06/13/2018 11:15 PM, Stefano Stabellini wrote:
> > Introduce support for the "xen,domU" compatible node on device tree.
> > Create new DomU VMs based on the information found on device tree under
> > "xen,domU".
> 
> While I like the idea of having multiple domain created by Xen, I think there
> are still few open questions here:
> 	1) The domains will be listed via "xl list". So are they still
> manageable via DOMCTL?

Yes, they are. There is an argument for making them "hidden" from
DOMCTLs, but that is not done as part of this series. Future work.


> 	2) Is it possible to restart those domains?

No they can't be restarted. For example, their original kernel is gone
from memory.


> 	3) If a domain crash, what will happen? Are they just going to 	sit
> there using resources until the platform rebooted?

The entire platform needs to be rebooted.


> 	4) How do you handle scheduling? Is it still possible to do it via
> Dom0? How about the dom0less situation?

The scheduler can be chosen via the Xen command line option. It is also
possible to do that from dom0 (if there is a dom0).


> > 
> > Introduce a simple global variable named max_init_domid to keep track of
> > the initial allocated domids.
> 
> What is the exact goal of this new variable?

The goal of this variable is to know the number of domUs allocated at
boot time by Xen. Specifically, it is used by console.c to switch the
serial input to the right domU.


> > 
> > Move the discard_initial_modules after DomUs have been built
> > 
> > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > ---
> >   xen/arch/arm/domain_build.c |  2 --
> >   xen/arch/arm/setup.c        | 35 ++++++++++++++++++++++++++++++++++-
> >   xen/include/asm-arm/setup.h |  2 ++
> >   xen/include/asm-x86/setup.h |  2 ++
> 
> You need to CC x86 maintainers for this change.

I'll add them


> >   4 files changed, 38 insertions(+), 3 deletions(-)
> > 
> > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > index 97f14ca..e2d370f 100644
> > --- a/xen/arch/arm/domain_build.c
> > +++ b/xen/arch/arm/domain_build.c
> > @@ -2545,8 +2545,6 @@ int __init construct_dom0(struct domain *d)
> >       if ( rc < 0 )
> >           return rc;
> >   -    discard_initial_modules();
> > -
> 
> Please mention this move in the commit message.

It is mentioned: "Move the discard_initial_modules after DomUs have been
built"


> >       return __construct_domain(d, &kinfo);
> >   }
> >   diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> > index 98bdb24..3723704 100644
> > --- a/xen/arch/arm/setup.c
> > +++ b/xen/arch/arm/setup.c
> > @@ -63,6 +63,8 @@ static unsigned long opt_xenheap_megabytes __initdata;
> >   integer_param("xenheap_megabytes", opt_xenheap_megabytes);
> >   #endif
> >   +domid_t __read_mostly max_init_domid = 0;
> > +
> >   static __used void init_done(void)
> >   {
> >       free_init_memory();
> > @@ -711,6 +713,8 @@ void __init start_xen(unsigned long boot_phys_offset,
> >       struct bootmodule *xen_bootmodule;
> >       struct domain *dom0;
> >       struct xen_domctl_createdomain dom0_cfg = {};
> > +    struct dt_device_node *chosen;
> > +    struct dt_device_node *node;
> >         dcache_line_bytes = read_dcache_line_bytes();
> >   @@ -860,7 +864,7 @@ void __init start_xen(unsigned long boot_phys_offset,
> >       dom0_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
> >       dom0_cfg.arch.nr_spis = gic_number_lines() - 32;
> >   -    dom0 = domain_create(0, &dom0_cfg);
> > +    dom0 = domain_create(max_init_domid++, &dom0_cfg);
> >       if ( IS_ERR(dom0) || (alloc_dom0_vcpu0(dom0) == NULL) )
> >               panic("Error creating domain 0");
> >   @@ -886,6 +890,35 @@ void __init start_xen(unsigned long boot_phys_offset,
> >         domain_unpause_by_systemcontroller(dom0);
> >   +    chosen = dt_find_node_by_name(dt_host, "chosen");
> > +    if ( chosen != NULL )
> > +    {
> > +        dt_for_each_child_node(chosen, node)
> > +        {
> > +            struct domain *d;
> > +            struct xen_domctl_createdomain d_cfg = {};
> 
> There are quite a few field in xen_domctl_createdomain that we may want to
> allow the user setting them. I am thinking of ssidref for XSM. How is this
> going to be done?

We'll introduce a separate function to initialize the
xen_domctl_createdomain fields as necessary. I don't think it is
required at the moment?


> > +
> > +            if ( !dt_device_is_compatible(node, "xen,domU") )
> > +                continue;
> > +
> > +            d_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
> 
> Any reason to impose using the native GIC here?

It is a good default. I haven't introduced an option to change the gic
version for a domU yet. It could be possible to add it in the future.


> > +            d_cfg.arch.nr_spis = gic_number_lines() - 32;
> 
> That's a bit unfortunate. So you are imposing to use 1020 IRQs and the waste
> of memory associated when only 32 SPIs is enough at the moment.

Well spotted, I'll fix


> > +
> > +            d = domain_create(max_init_domid++, &d_cfg);
> > +            if ( IS_ERR(d))
> 
> Coding style ( ... )

I'll fix


> > +                panic("Error creating domU");
> > +
> > +            d->is_privileged = 0;
> > +            d->target = NULL;
> 
> Why do you set them? They are zeroed by default.

Mostly for my own clarity and understading. I can remove them if you
prefer.


> > +
> > +            if ( construct_domU(d, node) != 0)
> 
> Coding style ( ... )

OK


> > +                printk("Could not set up DOMU guest OS");
> > +
> > +            domain_unpause_by_systemcontroller(d);
> > +        }
> > +    }
> 
> Please introduce a new function, this would avoid to increate start_xen() too
> much.

Yes, I'll do


> > +    discard_initial_modules();
> > +
> >       /* Switch on to the dynamically allocated stack for the idle vcpu
> >        * since the static one we're running on is about to be freed. */
> >       memcpy(idle_vcpu[0]->arch.cpu_info, get_cpu_info(),
> > diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
> > index e9f9905..578f3b9 100644
> > --- a/xen/include/asm-arm/setup.h
> > +++ b/xen/include/asm-arm/setup.h
> > @@ -56,6 +56,8 @@ struct bootinfo {
> >     extern struct bootinfo bootinfo;
> >   +extern domid_t max_init_domid;
> > +
> >   void arch_init_memory(void);
> >     void copy_from_paddr(void *dst, paddr_t paddr, unsigned long len);
> > diff --git a/xen/include/asm-x86/setup.h b/xen/include/asm-x86/setup.h
> > index 19232af..2fb9529 100644
> > --- a/xen/include/asm-x86/setup.h
> > +++ b/xen/include/asm-x86/setup.h
> > @@ -73,4 +73,6 @@ extern bool opt_dom0_shadow;
> >   #endif
> >   extern bool dom0_pvh;
> >   +#define max_init_domid (1)
> > +
> >   #endif
> > 
> 
> Cheers,
> 
> -- 
> Julien Grall
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 09/15] xen/arm: refactor construct_dom0

[Stefano Stabellini]

On Thu, 5 Jul 2018, Julien Grall wrote:
> Hi Stefano,
> 
> On 05/07/2018 21:55, Stefano Stabellini wrote:
> > On Fri, 15 Jun 2018, Julien Grall wrote:
> > > Hi Stefano,
> > > 
> > > On 06/15/2018 12:35 AM, Stefano Stabellini wrote:
> > > > On Thu, 14 Jun 2018, Julien Grall wrote:
> > > > > On 13/06/18 23:15, Stefano Stabellini wrote:
> > > if ( !is_hardware_domain(d) )
> > >    prepare_dtb_domU(...);
> > > else if ( acpi_disabled )
> > >    prepare_acpi_hwdom(...);
> > > else
> > >    prepare_dt_hwdom(....);
> > 
> > The few remaining things in construct_dom0 and construct_domU are
> > different enough that I don't think there is much gain in trying to
> > merge them.
> 
> Do you have a concrete example? When I looked at it the only things I can find
> different are:
>    - iommu_hwdom_init() -> Most likely you would need a similar call somewhere
> for Guest DomU.
>    - find_gnttab_region -> I think this could be re-ordered
>    - gic_map_hwdom_extra_mappings & platform_specific_mapping -> This can be
> called before the construct.
> 
> So I don't see why we can't merge most of the code and still have few lines
> for Dom0.
> 
> Overall I would much prefer if we don't duplicate the allocation of the memory
> or anything that is not Dom0 specific.

You might be right, but the series has changed significantly since the
RFC. Let's give a look at v2, you'll get a more concrete idea. Then,
I'll do any required changes in v3.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 14/15] xen/arm: call construct_domU from start_xen and start DomU VMs

[Julien Grall]

Hi Stefano,

On 07/07/18 00:11, Stefano Stabellini wrote:
> On Fri, 15 Jun 2018, Julien Grall wrote:
>> On 06/13/2018 11:15 PM, Stefano Stabellini wrote:
>>> Introduce support for the "xen,domU" compatible node on device tree.
>>> Create new DomU VMs based on the information found on device tree under
>>> "xen,domU".
>>
>> While I like the idea of having multiple domain created by Xen, I think there
>> are still few open questions here:
>> 	1) The domains will be listed via "xl list". So are they still
>> manageable via DOMCTL?
> 
> Yes, they are. There is an argument for making them "hidden" from
> DOMCTLs, but that is not done as part of this series. Future work.

What will happen with libxl today? Is the toolstack going to be confused?

> 
> 
>> 	2) Is it possible to restart those domains?
> 
> No they can't be restarted. For example, their original kernel is gone
> from memory.

So what will happen if you use "xl restart" on them?

> 
> 
>> 	3) If a domain crash, what will happen? Are they just going to 	sit
>> there using resources until the platform rebooted?
> 
> The entire platform needs to be rebooted.

That should be clarified somewhere in the documentation.

> 
> 
>> 	4) How do you handle scheduling? Is it still possible to do it via
>> Dom0? How about the dom0less situation?
> 
> The scheduler can be chosen via the Xen command line option. It is also
> possible to do that from dom0 (if there is a dom0).

Can you explain how the vCPUs are going to get pinned via the command line?

> 
> 
>>>
>>> Introduce a simple global variable named max_init_domid to keep track of
>>> the initial allocated domids.
>>
>> What is the exact goal of this new variable?
> 
> The goal of this variable is to know the number of domUs allocated at
> boot time by Xen. Specifically, it is used by console.c to switch the
> serial input to the right domU.

I don't think this is necessary. I will reply on the second version.

> 
> 
>>>
>>> Move the discard_initial_modules after DomUs have been built
>>>
>>> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
>>> ---
>>>    xen/arch/arm/domain_build.c |  2 --
>>>    xen/arch/arm/setup.c        | 35 ++++++++++++++++++++++++++++++++++-
>>>    xen/include/asm-arm/setup.h |  2 ++
>>>    xen/include/asm-x86/setup.h |  2 ++
>>
>> You need to CC x86 maintainers for this change.
> 
> I'll add them
> 
> 
>>>    4 files changed, 38 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>>> index 97f14ca..e2d370f 100644
>>> --- a/xen/arch/arm/domain_build.c
>>> +++ b/xen/arch/arm/domain_build.c
>>> @@ -2545,8 +2545,6 @@ int __init construct_dom0(struct domain *d)
>>>        if ( rc < 0 )
>>>            return rc;
>>>    -    discard_initial_modules();
>>> -
>>
>> Please mention this move in the commit message.
> 
> It is mentioned: "Move the discard_initial_modules after DomUs have been
> built"

I missed that sorry.

> 
> 
>>>        return __construct_domain(d, &kinfo);
>>>    }
>>>    diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
>>> index 98bdb24..3723704 100644
>>> --- a/xen/arch/arm/setup.c
>>> +++ b/xen/arch/arm/setup.c
>>> @@ -63,6 +63,8 @@ static unsigned long opt_xenheap_megabytes __initdata;
>>>    integer_param("xenheap_megabytes", opt_xenheap_megabytes);
>>>    #endif
>>>    +domid_t __read_mostly max_init_domid = 0;
>>> +
>>>    static __used void init_done(void)
>>>    {
>>>        free_init_memory();
>>> @@ -711,6 +713,8 @@ void __init start_xen(unsigned long boot_phys_offset,
>>>        struct bootmodule *xen_bootmodule;
>>>        struct domain *dom0;
>>>        struct xen_domctl_createdomain dom0_cfg = {};
>>> +    struct dt_device_node *chosen;
>>> +    struct dt_device_node *node;
>>>          dcache_line_bytes = read_dcache_line_bytes();
>>>    @@ -860,7 +864,7 @@ void __init start_xen(unsigned long boot_phys_offset,
>>>        dom0_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
>>>        dom0_cfg.arch.nr_spis = gic_number_lines() - 32;
>>>    -    dom0 = domain_create(0, &dom0_cfg);
>>> +    dom0 = domain_create(max_init_domid++, &dom0_cfg);
>>>        if ( IS_ERR(dom0) || (alloc_dom0_vcpu0(dom0) == NULL) )
>>>                panic("Error creating domain 0");
>>>    @@ -886,6 +890,35 @@ void __init start_xen(unsigned long boot_phys_offset,
>>>          domain_unpause_by_systemcontroller(dom0);
>>>    +    chosen = dt_find_node_by_name(dt_host, "chosen");
>>> +    if ( chosen != NULL )
>>> +    {
>>> +        dt_for_each_child_node(chosen, node)
>>> +        {
>>> +            struct domain *d;
>>> +            struct xen_domctl_createdomain d_cfg = {};
>>
>> There are quite a few field in xen_domctl_createdomain that we may want to
>> allow the user setting them. I am thinking of ssidref for XSM. How is this
>> going to be done?
> 
> We'll introduce a separate function to initialize the
> xen_domctl_createdomain fields as necessary. I don't think it is
> required at the moment?

I didn't ask the implementation but how this is going to fit together.

>>> +
>>> +            if ( !dt_device_is_compatible(node, "xen,domU") )
>>> +                continue;
>>> +
>>> +            d_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
>>
>> Any reason to impose using the native GIC here?
> 
> It is a good default. I haven't introduced an option to change the gic
> version for a domU yet. It could be possible to add it in the future.

Please document it in somewhere.

>>> +
>>> +            d = domain_create(max_init_domid++, &d_cfg);
>>> +            if ( IS_ERR(d))
>>
>> Coding style ( ... )
> 
> I'll fix
> 
> 
>>> +                panic("Error creating domU");
>>> +
>>> +            d->is_privileged = 0;
>>> +            d->target = NULL;
>>
>> Why do you set them? They are zeroed by default.
> 
> Mostly for my own clarity and understading. I can remove them if you
> prefer.

I would rather remove it because it give the feeling the other fields 
may not be zeroed.

>>> +
>>> +            if ( construct_domU(d, node) != 0)
>>
>> Coding style ( ... )
> 
> OK
> 
> 
>>> +                printk("Could not set up DOMU guest OS");
>>> +
>>> +            domain_unpause_by_systemcontroller(d);
>>> +        }
>>> +    }
>>
>> Please introduce a new function, this would avoid to increate start_xen() too
>> much.
> 
> Yes, I'll do
> 
> 
>>> +    discard_initial_modules();
>>> +
>>>        /* Switch on to the dynamically allocated stack for the idle vcpu
>>>         * since the static one we're running on is about to be freed. */
>>>        memcpy(idle_vcpu[0]->arch.cpu_info, get_cpu_info(),
>>> diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h
>>> index e9f9905..578f3b9 100644
>>> --- a/xen/include/asm-arm/setup.h
>>> +++ b/xen/include/asm-arm/setup.h
>>> @@ -56,6 +56,8 @@ struct bootinfo {
>>>      extern struct bootinfo bootinfo;
>>>    +extern domid_t max_init_domid;
>>> +
>>>    void arch_init_memory(void);
>>>      void copy_from_paddr(void *dst, paddr_t paddr, unsigned long len);
>>> diff --git a/xen/include/asm-x86/setup.h b/xen/include/asm-x86/setup.h
>>> index 19232af..2fb9529 100644
>>> --- a/xen/include/asm-x86/setup.h
>>> +++ b/xen/include/asm-x86/setup.h
>>> @@ -73,4 +73,6 @@ extern bool opt_dom0_shadow;
>>>    #endif
>>>    extern bool dom0_pvh;
>>>    +#define max_init_domid (1)
>>> +
>>>    #endif

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 14/15] xen/arm: call construct_domU from start_xen and start DomU VMs

[Stefano Stabellini]

On Mon, 9 Jul 2018, Julien Grall wrote:
> Hi Stefano,
> 
> On 07/07/18 00:11, Stefano Stabellini wrote:
> > On Fri, 15 Jun 2018, Julien Grall wrote:
> > > On 06/13/2018 11:15 PM, Stefano Stabellini wrote:
> > > > Introduce support for the "xen,domU" compatible node on device tree.
> > > > Create new DomU VMs based on the information found on device tree under
> > > > "xen,domU".
> > > 
> > > While I like the idea of having multiple domain created by Xen, I think
> > > there
> > > are still few open questions here:
> > > 	1) The domains will be listed via "xl list". So are they still
> > > manageable via DOMCTL?
> > 
> > Yes, they are. There is an argument for making them "hidden" from
> > DOMCTLs, but that is not done as part of this series. Future work.
> 
> What will happen with libxl today? Is the toolstack going to be confused?

The toolstack can list the running domains without problems with `xl
list' (although they have no name). Also, xl vcpu-pin works fine.
However, some operations might not work correctly though. For instance,
`xl destroy' cannot completely get rid of the domain. I'll add info
about these limitations to a separate dom0less document (as you also
suggested below), because it is not part of the multiboot spec.


> > 
> > 
> > > 	2) Is it possible to restart those domains?
> > 
> > No they can't be restarted. For example, their original kernel is gone
> > from memory.
> 
> So what will happen if you use "xl restart" on them?

Do you mean `xl reboot'?  The command executes but nothing happens.


> > 
> > 
> > > 	3) If a domain crash, what will happen? Are they just going to 	sit
> > > there using resources until the platform rebooted?
> > 
> > The entire platform needs to be rebooted.
> 
> That should be clarified somewhere in the documentation.

Yes, you are right. I'll add this to the new dom0less doc. 


> > 
> > 
> > > 	4) How do you handle scheduling? Is it still possible to do it via
> > > Dom0? How about the dom0less situation?
> > 
> > The scheduler can be chosen via the Xen command line option. It is also
> > possible to do that from dom0 (if there is a dom0).
> 
> Can you explain how the vCPUs are going to get pinned via the command line?

Today, only dom0 vCPUs can get automatically pinned with a Xen command
line option. However, `xl vcpu-pin' in dom0 works with other domains
started by Xen at boot, and the NULL scheduler doesn't require pinning.
FYI for my own usage, I plan to use the NULL scheduler.


> > 
> > 
> > > > 
> > > > Introduce a simple global variable named max_init_domid to keep track of
> > > > the initial allocated domids.
> > > 
> > > What is the exact goal of this new variable?
> > 
> > The goal of this variable is to know the number of domUs allocated at
> > boot time by Xen. Specifically, it is used by console.c to switch the
> > serial input to the right domU.
> 
> I don't think this is necessary. I will reply on the second version.

OK


> > 
> > 
> > > > 
> > > > Move the discard_initial_modules after DomUs have been built
> > > > 
> > > > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > > > ---
> > > >    xen/arch/arm/domain_build.c |  2 --
> > > >    xen/arch/arm/setup.c        | 35 ++++++++++++++++++++++++++++++++++-
> > > >    xen/include/asm-arm/setup.h |  2 ++
> > > >    xen/include/asm-x86/setup.h |  2 ++
> > > 
> > > You need to CC x86 maintainers for this change.
> > 
> > I'll add them
> > 
> > 
> > > >    4 files changed, 38 insertions(+), 3 deletions(-)
> > > > 
> > > > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > > > index 97f14ca..e2d370f 100644
> > > > --- a/xen/arch/arm/domain_build.c
> > > > +++ b/xen/arch/arm/domain_build.c
> > > > @@ -2545,8 +2545,6 @@ int __init construct_dom0(struct domain *d)
> > > >        if ( rc < 0 )
> > > >            return rc;
> > > >    -    discard_initial_modules();
> > > > -
> > > 
> > > Please mention this move in the commit message.
> > 
> > It is mentioned: "Move the discard_initial_modules after DomUs have been
> > built"
> 
> I missed that sorry.
> 
> > 
> > 
> > > >        return __construct_domain(d, &kinfo);
> > > >    }
> > > >    diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> > > > index 98bdb24..3723704 100644
> > > > --- a/xen/arch/arm/setup.c
> > > > +++ b/xen/arch/arm/setup.c
> > > > @@ -63,6 +63,8 @@ static unsigned long opt_xenheap_megabytes __initdata;
> > > >    integer_param("xenheap_megabytes", opt_xenheap_megabytes);
> > > >    #endif
> > > >    +domid_t __read_mostly max_init_domid = 0;
> > > > +
> > > >    static __used void init_done(void)
> > > >    {
> > > >        free_init_memory();
> > > > @@ -711,6 +713,8 @@ void __init start_xen(unsigned long
> > > > boot_phys_offset,
> > > >        struct bootmodule *xen_bootmodule;
> > > >        struct domain *dom0;
> > > >        struct xen_domctl_createdomain dom0_cfg = {};
> > > > +    struct dt_device_node *chosen;
> > > > +    struct dt_device_node *node;
> > > >          dcache_line_bytes = read_dcache_line_bytes();
> > > >    @@ -860,7 +864,7 @@ void __init start_xen(unsigned long
> > > > boot_phys_offset,
> > > >        dom0_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
> > > >        dom0_cfg.arch.nr_spis = gic_number_lines() - 32;
> > > >    -    dom0 = domain_create(0, &dom0_cfg);
> > > > +    dom0 = domain_create(max_init_domid++, &dom0_cfg);
> > > >        if ( IS_ERR(dom0) || (alloc_dom0_vcpu0(dom0) == NULL) )
> > > >                panic("Error creating domain 0");
> > > >    @@ -886,6 +890,35 @@ void __init start_xen(unsigned long
> > > > boot_phys_offset,
> > > >          domain_unpause_by_systemcontroller(dom0);
> > > >    +    chosen = dt_find_node_by_name(dt_host, "chosen");
> > > > +    if ( chosen != NULL )
> > > > +    {
> > > > +        dt_for_each_child_node(chosen, node)
> > > > +        {
> > > > +            struct domain *d;
> > > > +            struct xen_domctl_createdomain d_cfg = {};
> > > 
> > > There are quite a few field in xen_domctl_createdomain that we may want to
> > > allow the user setting them. I am thinking of ssidref for XSM. How is this
> > > going to be done?
> > 
> > We'll introduce a separate function to initialize the
> > xen_domctl_createdomain fields as necessary. I don't think it is
> > required at the moment?
> 
> I didn't ask the implementation but how this is going to fit together.
> 
> > > > +
> > > > +            if ( !dt_device_is_compatible(node, "xen,domU") )
> > > > +                continue;
> > > > +
> > > > +            d_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
> > > 
> > > Any reason to impose using the native GIC here?
> > 
> > It is a good default. I haven't introduced an option to change the gic
> > version for a domU yet. It could be possible to add it in the future.
> 
> Please document it in somewhere.

Yes, I'll also add this to the separate dom0less document.


> > > > +
> > > > +            d = domain_create(max_init_domid++, &d_cfg);
> > > > +            if ( IS_ERR(d))
> > > 
> > > Coding style ( ... )
> > 
> > I'll fix
> > 
> > 
> > > > +                panic("Error creating domU");
> > > > +
> > > > +            d->is_privileged = 0;
> > > > +            d->target = NULL;
> > > 
> > > Why do you set them? They are zeroed by default.
> > 
> > Mostly for my own clarity and understading. I can remove them if you
> > prefer.
> 
> I would rather remove it because it give the feeling the other fields may not
> be zeroed.

OK

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 14/15] xen/arm: call construct_domU from start_xen and start DomU VMs

[Julien Grall]

Hi,

On 09/07/2018 21:59, Stefano Stabellini wrote:
> On Mon, 9 Jul 2018, Julien Grall wrote:
>> Hi Stefano,
>>
>> On 07/07/18 00:11, Stefano Stabellini wrote:
>>> On Fri, 15 Jun 2018, Julien Grall wrote:
>>>> On 06/13/2018 11:15 PM, Stefano Stabellini wrote:
>>>>> Introduce support for the "xen,domU" compatible node on device tree.
>>>>> Create new DomU VMs based on the information found on device tree under
>>>>> "xen,domU".
>>>>
>>>> While I like the idea of having multiple domain created by Xen, I think
>>>> there
>>>> are still few open questions here:
>>>> 	1) The domains will be listed via "xl list". So are they still
>>>> manageable via DOMCTL?
>>>
>>> Yes, they are. There is an argument for making them "hidden" from
>>> DOMCTLs, but that is not done as part of this series. Future work.
>>
>> What will happen with libxl today? Is the toolstack going to be confused?
> 
> The toolstack can list the running domains without problems with `xl
> list' (although they have no name). Also, xl vcpu-pin works fine.
> However, some operations might not work correctly though. For instance,
> `xl destroy' cannot completely get rid of the domain. I'll add info
> about these limitations to a separate dom0less document (as you also
> suggested below), because it is not part of the multiboot spec.
> 
> 
>>>
>>>
>>>> 	2) Is it possible to restart those domains?
>>>
>>> No they can't be restarted. For example, their original kernel is gone
>>> from memory.
>>
>> So what will happen if you use "xl restart" on them?
> 
> Do you mean `xl reboot'?  The command executes but nothing happens.
> 
> 
>>>
>>>
>>>> 	3) If a domain crash, what will happen? Are they just going to 	sit
>>>> there using resources until the platform rebooted?
>>>
>>> The entire platform needs to be rebooted.
>>
>> That should be clarified somewhere in the documentation.
> 
> Yes, you are right. I'll add this to the new dom0less doc.
> 
> 
>>>
>>>
>>>> 	4) How do you handle scheduling? Is it still possible to do it via
>>>> Dom0? How about the dom0less situation?
>>>
>>> The scheduler can be chosen via the Xen command line option. It is also
>>> possible to do that from dom0 (if there is a dom0).
>>
>> Can you explain how the vCPUs are going to get pinned via the command line?
> 
> Today, only dom0 vCPUs can get automatically pinned with a Xen command
> line option. However, `xl vcpu-pin' in dom0 works with other domains
> started by Xen at boot, and the NULL scheduler doesn't require pinning.
> FYI for my own usage, I plan to use the NULL scheduler.

Well your series is called "Dom0less", so using Dom0 to pin the vCPU 
does not look right to me.

But even with NULL scheduler you may want to pin a vCPU to a given pCPU. 
Imagine a big.LITTLE environment. However, NULL scheduler is not 
supported, so it feels a little odd that there are no way to configure 
guest DomU without Dom0 in play.

So there is some clarification needed here.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 14/15] xen/arm: call construct_domU from start_xen and start DomU VMs

[Stefano Stabellini]

On Mon, 9 Jul 2018, Julien Grall wrote:
> Hi,
> 
> On 09/07/2018 21:59, Stefano Stabellini wrote:
> > On Mon, 9 Jul 2018, Julien Grall wrote:
> > > Hi Stefano,
> > > 
> > > On 07/07/18 00:11, Stefano Stabellini wrote:
> > > > On Fri, 15 Jun 2018, Julien Grall wrote:
> > > > > On 06/13/2018 11:15 PM, Stefano Stabellini wrote:
> > > > > > Introduce support for the "xen,domU" compatible node on device tree.
> > > > > > Create new DomU VMs based on the information found on device tree
> > > > > > under
> > > > > > "xen,domU".
> > > > > 
> > > > > While I like the idea of having multiple domain created by Xen, I
> > > > > think
> > > > > there
> > > > > are still few open questions here:
> > > > > 	1) The domains will be listed via "xl list". So are they still
> > > > > manageable via DOMCTL?
> > > > 
> > > > Yes, they are. There is an argument for making them "hidden" from
> > > > DOMCTLs, but that is not done as part of this series. Future work.
> > > 
> > > What will happen with libxl today? Is the toolstack going to be confused?
> > 
> > The toolstack can list the running domains without problems with `xl
> > list' (although they have no name). Also, xl vcpu-pin works fine.
> > However, some operations might not work correctly though. For instance,
> > `xl destroy' cannot completely get rid of the domain. I'll add info
> > about these limitations to a separate dom0less document (as you also
> > suggested below), because it is not part of the multiboot spec.
> > 
> > 
> > > > 
> > > > 
> > > > > 	2) Is it possible to restart those domains?
> > > > 
> > > > No they can't be restarted. For example, their original kernel is gone
> > > > from memory.
> > > 
> > > So what will happen if you use "xl restart" on them?
> > 
> > Do you mean `xl reboot'?  The command executes but nothing happens.
> > 
> > 
> > > > 
> > > > 
> > > > > 	3) If a domain crash, what will happen? Are they just going to
> > > > > sit
> > > > > there using resources until the platform rebooted?
> > > > 
> > > > The entire platform needs to be rebooted.
> > > 
> > > That should be clarified somewhere in the documentation.
> > 
> > Yes, you are right. I'll add this to the new dom0less doc.
> > 
> > 
> > > > 
> > > > 
> > > > > 	4) How do you handle scheduling? Is it still possible to do it
> > > > > via
> > > > > Dom0? How about the dom0less situation?
> > > > 
> > > > The scheduler can be chosen via the Xen command line option. It is also
> > > > possible to do that from dom0 (if there is a dom0).
> > > 
> > > Can you explain how the vCPUs are going to get pinned via the command
> > > line?
> > 
> > Today, only dom0 vCPUs can get automatically pinned with a Xen command
> > line option. However, `xl vcpu-pin' in dom0 works with other domains
> > started by Xen at boot, and the NULL scheduler doesn't require pinning.
> > FYI for my own usage, I plan to use the NULL scheduler.
> 
> Well your series is called "Dom0less", so using Dom0 to pin the vCPU does not
> look right to me.
> 
> But even with NULL scheduler you may want to pin a vCPU to a given pCPU.
> Imagine a big.LITTLE environment. However, NULL scheduler is not supported, so
> it feels a little odd that there are no way to configure guest DomU without
> Dom0 in play.
> 
> So there is some clarification needed here.

I'll add a note about this limitation to the dom0less doc I am writing.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel