* [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo
@ 2023-04-11 19:16 Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 01/17] xen/arm/device: Remove __init from function type Vikram Garhwal
` (34 more replies)
0 siblings, 35 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel
Cc: sstabellini, Vikram Garhwal, Julien Grall, Bertrand Marquis,
Volodymyr Babchuk, Andrew Cooper, George Dunlap, Jan Beulich,
Wei Liu, Rahul Singh, Paul Durrant, Roger Pau Monné,
Anthony PERARD, Juergen Gross
Hi,
The main purpose of this series to address first part of the dynamic programming
i.e. making Xen aware of new device tree node which means updating the dt_host
with overlay node information. Here we are adding/removing node from dt_host,
and checking/set IOMMU and IRQ permission but never mapping them to any domain.
Right now, mapping/Un-mapping will happen only when a new domU is
created/destroyed using "xl create".
For adding a node using dynamic programming:
1. flatten device tree overlay node will be added to a fdt
2. Updated fdt will be unflattened to a new dt_host_new
3. Extract the newly added node information from dt_host_new
4. Add the added node under correct parent in original dt_host.
For removing a node:
1. Find the node with given path.
2. Check if the node is used by any of domus. Removes the node only when
it's not used by any domain.
3. Removes IRQ permissions and MMIO access.
5. Find the node in dt_host and delete the device node entry from dt_host.
6. Free the overlay_tracker entry which means free dt_host_new also(created
in adding node step).
To map IOREQ and IOMMU during runtime, there will be another small series after
this one where we will do the actual IOMMU and IRQ mapping to a running domain
and will call unmap_mmio_regions() to remove the mapping.
Change Log:
v4 -> v5:
Split patch 01/16 to two patches. One with function type changes and another
with changes inside unflatten_device_tree().
Change dt_overlay xl command to dt-overlay.
Protect overlay functionality with CONFIG(arm).
Fix rwlock issues.
Move include "device_tree.h" to c file where arch_cpu_init() is called and
forward declare dt_device_node. This was done to avoid circular deps b/w
device_tree.h and rwlock.h
Address Michal's comment on coding style.
v3 -> v4:
Add support for adding node's children.
Add rwlock to dt_host functions.
Corrected fdt size issue when applying overlay into it.
Add memory allocation fail handling for unflatten_device_tree().
Changed xl overlay to xl dt_overlay.
Correct commit messages.
Addressed code issue from v3 review.
v2 -> v3:
Moved overlay functionalities to dt_overlay.c file.
Renamed XEN_SYSCTL_overlay to XEN_SYSCTL_dt_overlay.
Add dt_* prefix to overlay_add/remove_nodes.
Added dtdevs_lock to protect iommu_add_dt_device().
For iommu, moved spin_lock to caller.
Address code issue from v2 review.
v1 -> v2:
Add support for multiple node addition/removal using dtbo.
Replaced fpga-add and fpga-remove with one hypercall overlay_op.
Moved common domain_build.c function to device.c
Add OVERLAY_DTB configuration.
Renamed overlay_get_target() to fdt_overlay_get_target().
Split remove_device patch into two patches.
Moved overlay_add/remove code to sysctl and changed it from domctl to sysctl.
Added all overlay code under CONFIG_OVERLAY_DTB
Renamed all tool domains fpga function to overlay
Addressed code issues from v1 review.
Regards,
Vikram
Vikram Garhwal (17):
xen/arm/device: Remove __init from function type
common/device_tree: change __unflatten_device_tree()
xen/arm: Add CONFIG_OVERLAY_DTB
libfdt: Keep fdt functions after init for CONFIG_OVERLAY_DTB.
libfdt: overlay: change overlay_get_target()
xen/device-tree: Add device_tree_find_node_by_path() to find nodes in
device tree
xen/smmu: Add remove_device callback for smmu_iommu ops
xen/iommu: Move spin_lock from iommu_dt_device_is_assigned to caller
xen/iommu: protect iommu_add_dt_device() with dtdevs_lock
xen/iommu: Introduce iommu_remove_dt_device()
asm/smp.h: Fix circular dependency for device_tree.h and rwlock.h
common/device_tree: Add rwlock for dt_host
xen/arm: Implement device tree node removal functionalities
xen/arm: Implement device tree node addition functionalities
tools/libs/ctrl: Implement new xc interfaces for dt overlay
tools/libs/light: Implement new libxl functions for device tree
overlay ops
tools/xl: Add new xl command overlay for device tree overlay support
SUPPORT.md | 6 +
tools/include/libxl.h | 11 +
tools/include/xenctrl.h | 5 +
tools/libs/ctrl/Makefile.common | 1 +
tools/libs/ctrl/xc_dt_overlay.c | 48 ++
tools/libs/light/Makefile | 3 +
tools/libs/light/libxl_dt_overlay.c | 71 ++
tools/xl/xl.h | 1 +
tools/xl/xl_cmdtable.c | 6 +
tools/xl/xl_vmcontrol.c | 52 ++
xen/arch/arm/Kconfig | 5 +
xen/arch/arm/device.c | 145 ++++
xen/arch/arm/domain_build.c | 142 ----
xen/arch/arm/include/asm/domain_build.h | 2 -
xen/arch/arm/include/asm/setup.h | 6 +
xen/arch/arm/include/asm/smp.h | 3 +-
xen/arch/arm/smpboot.c | 1 +
xen/arch/arm/sysctl.c | 16 +-
xen/common/Makefile | 1 +
xen/common/device_tree.c | 30 +-
xen/common/dt_overlay.c | 897 ++++++++++++++++++++++++
xen/common/libfdt/Makefile | 4 +
xen/common/libfdt/fdt_overlay.c | 29 +-
xen/common/libfdt/version.lds | 1 +
xen/drivers/passthrough/arm/smmu.c | 56 ++
xen/drivers/passthrough/device_tree.c | 109 ++-
xen/include/public/sysctl.h | 24 +
xen/include/xen/device_tree.h | 28 +-
xen/include/xen/dt_overlay.h | 59 ++
xen/include/xen/iommu.h | 2 +
xen/include/xen/libfdt/libfdt.h | 18 +
31 files changed, 1595 insertions(+), 187 deletions(-)
create mode 100644 tools/libs/ctrl/xc_dt_overlay.c
create mode 100644 tools/libs/light/libxl_dt_overlay.c
create mode 100644 xen/common/dt_overlay.c
create mode 100644 xen/include/xen/dt_overlay.h
--
2.17.1
^ permalink raw reply [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 01/17] xen/arm/device: Remove __init from function type
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-13 9:19 ` Michal Orzel
2023-04-11 19:16 ` [XEN][PATCH v5 02/17] common/device_tree: change __unflatten_device_tree() Vikram Garhwal
` (33 subsequent siblings)
34 siblings, 1 reply; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel
Cc: sstabellini, Vikram Garhwal, Julien Grall, Bertrand Marquis,
Volodymyr Babchuk
Remove __init from following function to access during runtime:
1. map_irq_to_domain()
2. handle_device_interrupt()
3. map_range_to_domain()
4. unflatten_dt_node()
5. unflatten_device_tree()
Move map_irq_to_domain() prototype from domain_build.h to setup.h.
To avoid the breaking the build, following changes are also done:
1. Move map_irq_to_domain(), handle_device_interrupt() and map_range_to_domain() to
device.c. After removing __init type, these functions are not specific to
domain building, so moving them out of domain_build.c to device.c.
2. Remove static type from handle_device_interrupt().
Overall, these changes are done to support the dynamic programming of a nodes
where an overlay node will be added to fdt and unflattened node will be added to
dt_host. Furthermore, IRQ and mmio mapping will be done for the added node.
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
xen/arch/arm/device.c | 145 ++++++++++++++++++++++++
xen/arch/arm/domain_build.c | 142 -----------------------
xen/arch/arm/include/asm/domain_build.h | 2 -
xen/arch/arm/include/asm/setup.h | 6 +
xen/common/device_tree.c | 16 +--
5 files changed, 159 insertions(+), 152 deletions(-)
diff --git a/xen/arch/arm/device.c b/xen/arch/arm/device.c
index ca8539dee5..fec6e29c42 100644
--- a/xen/arch/arm/device.c
+++ b/xen/arch/arm/device.c
@@ -12,6 +12,9 @@
#include <xen/errno.h>
#include <xen/init.h>
#include <xen/lib.h>
+#include <xen/iocap.h>
+#include <asm/domain_build.h>
+#include <asm/setup.h>
extern const struct device_desc _sdevice[], _edevice[];
extern const struct acpi_device_desc _asdevice[], _aedevice[];
@@ -75,6 +78,148 @@ enum device_class device_get_class(const struct dt_device_node *dev)
return DEVICE_UNKNOWN;
}
+int map_irq_to_domain(struct domain *d, unsigned int irq,
+ bool need_mapping, const char *devname)
+{
+ int res;
+
+ res = irq_permit_access(d, irq);
+ if ( res )
+ {
+ printk(XENLOG_ERR "Unable to permit to dom%u access to IRQ %u\n",
+ d->domain_id, irq);
+ return res;
+ }
+
+ if ( need_mapping )
+ {
+ /*
+ * Checking the return of vgic_reserve_virq is not
+ * necessary. It should not fail except when we try to map
+ * the IRQ twice. This can legitimately happen if the IRQ is shared
+ */
+ vgic_reserve_virq(d, irq);
+
+ res = route_irq_to_guest(d, irq, irq, devname);
+ if ( res < 0 )
+ {
+ printk(XENLOG_ERR "Unable to map IRQ%"PRId32" to dom%d\n",
+ irq, d->domain_id);
+ return res;
+ }
+ }
+
+ dt_dprintk(" - IRQ: %u\n", irq);
+ return 0;
+}
+
+int map_range_to_domain(const struct dt_device_node *dev,
+ u64 addr, u64 len, void *data)
+{
+ struct map_range_data *mr_data = data;
+ struct domain *d = mr_data->d;
+ int res;
+
+ /*
+ * reserved-memory regions are RAM carved out for a special purpose.
+ * They are not MMIO and therefore a domain should not be able to
+ * manage them via the IOMEM interface.
+ */
+ if ( strncasecmp(dt_node_full_name(dev), "/reserved-memory/",
+ strlen("/reserved-memory/")) != 0 )
+ {
+ res = iomem_permit_access(d, paddr_to_pfn(addr),
+ paddr_to_pfn(PAGE_ALIGN(addr + len - 1)));
+ if ( res )
+ {
+ printk(XENLOG_ERR "Unable to permit to dom%d access to"
+ " 0x%"PRIx64" - 0x%"PRIx64"\n",
+ d->domain_id,
+ addr & PAGE_MASK, PAGE_ALIGN(addr + len) - 1);
+ return res;
+ }
+ }
+
+ if ( !mr_data->skip_mapping )
+ {
+ res = map_regions_p2mt(d,
+ gaddr_to_gfn(addr),
+ PFN_UP(len),
+ maddr_to_mfn(addr),
+ mr_data->p2mt);
+
+ if ( res < 0 )
+ {
+ printk(XENLOG_ERR "Unable to map 0x%"PRIx64
+ " - 0x%"PRIx64" in domain %d\n",
+ addr & PAGE_MASK, PAGE_ALIGN(addr + len) - 1,
+ d->domain_id);
+ return res;
+ }
+ }
+
+ dt_dprintk(" - MMIO: %010"PRIx64" - %010"PRIx64" P2MType=%x\n",
+ addr, addr + len, mr_data->p2mt);
+
+ return 0;
+}
+
+/*
+ * handle_device_interrupts retrieves the interrupts configuration from
+ * a device tree node and maps those interrupts to the target domain.
+ *
+ * Returns:
+ * < 0 error
+ * 0 success
+ */
+int handle_device_interrupts(struct domain *d,
+ struct dt_device_node *dev,
+ bool need_mapping)
+{
+ unsigned int i, nirq;
+ int res;
+ struct dt_raw_irq rirq;
+
+ nirq = dt_number_of_irq(dev);
+
+ /* Give permission and map IRQs */
+ for ( i = 0; i < nirq; i++ )
+ {
+ res = dt_device_get_raw_irq(dev, i, &rirq);
+ if ( res )
+ {
+ printk(XENLOG_ERR "Unable to retrieve irq %u for %s\n",
+ i, dt_node_full_name(dev));
+ return res;
+ }
+
+ /*
+ * Don't map IRQ that have no physical meaning
+ * ie: IRQ whose controller is not the GIC
+ */
+ if ( rirq.controller != dt_interrupt_controller )
+ {
+ dt_dprintk("irq %u not connected to primary controller. Connected to %s\n",
+ i, dt_node_full_name(rirq.controller));
+ continue;
+ }
+
+ res = platform_get_irq(dev, i);
+ if ( res < 0 )
+ {
+ printk(XENLOG_ERR "Unable to get irq %u for %s\n",
+ i, dt_node_full_name(dev));
+ return res;
+ }
+
+ res = map_irq_to_domain(d, res, need_mapping, dt_node_name(dev));
+ if ( res )
+ return res;
+ }
+
+ return 0;
+}
+
/*
* Local variables:
* mode: C
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 4f9d4f9d88..6ab18c53ab 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2256,41 +2256,6 @@ int __init make_chosen_node(const struct kernel_info *kinfo)
return res;
}
-int __init map_irq_to_domain(struct domain *d, unsigned int irq,
- bool need_mapping, const char *devname)
-{
- int res;
-
- res = irq_permit_access(d, irq);
- if ( res )
- {
- printk(XENLOG_ERR "Unable to permit to dom%u access to IRQ %u\n",
- d->domain_id, irq);
- return res;
- }
-
- if ( need_mapping )
- {
- /*
- * Checking the return of vgic_reserve_virq is not
- * necessary. It should not fail except when we try to map
- * the IRQ twice. This can legitimately happen if the IRQ is shared
- */
- vgic_reserve_virq(d, irq);
-
- res = route_irq_to_guest(d, irq, irq, devname);
- if ( res < 0 )
- {
- printk(XENLOG_ERR "Unable to map IRQ%"PRId32" to dom%d\n",
- irq, d->domain_id);
- return res;
- }
- }
-
- dt_dprintk(" - IRQ: %u\n", irq);
- return 0;
-}
-
static int __init map_dt_irq_to_domain(const struct dt_device_node *dev,
const struct dt_irq *dt_irq,
void *data)
@@ -2322,57 +2287,6 @@ static int __init map_dt_irq_to_domain(const struct dt_device_node *dev,
return 0;
}
-int __init map_range_to_domain(const struct dt_device_node *dev,
- u64 addr, u64 len, void *data)
-{
- struct map_range_data *mr_data = data;
- struct domain *d = mr_data->d;
- int res;
-
- /*
- * reserved-memory regions are RAM carved out for a special purpose.
- * They are not MMIO and therefore a domain should not be able to
- * manage them via the IOMEM interface.
- */
- if ( strncasecmp(dt_node_full_name(dev), "/reserved-memory/",
- strlen("/reserved-memory/")) != 0 )
- {
- res = iomem_permit_access(d, paddr_to_pfn(addr),
- paddr_to_pfn(PAGE_ALIGN(addr + len - 1)));
- if ( res )
- {
- printk(XENLOG_ERR "Unable to permit to dom%d access to"
- " 0x%"PRIx64" - 0x%"PRIx64"\n",
- d->domain_id,
- addr & PAGE_MASK, PAGE_ALIGN(addr + len) - 1);
- return res;
- }
- }
-
- if ( !mr_data->skip_mapping )
- {
- res = map_regions_p2mt(d,
- gaddr_to_gfn(addr),
- PFN_UP(len),
- maddr_to_mfn(addr),
- mr_data->p2mt);
-
- if ( res < 0 )
- {
- printk(XENLOG_ERR "Unable to map 0x%"PRIx64
- " - 0x%"PRIx64" in domain %d\n",
- addr & PAGE_MASK, PAGE_ALIGN(addr + len) - 1,
- d->domain_id);
- return res;
- }
- }
-
- dt_dprintk(" - MMIO: %010"PRIx64" - %010"PRIx64" P2MType=%x\n",
- addr, addr + len, mr_data->p2mt);
-
- return 0;
-}
-
/*
* For a node which describes a discoverable bus (such as a PCI bus)
* then we may need to perform additional mappings in order to make
@@ -2400,62 +2314,6 @@ static int __init map_device_children(const struct dt_device_node *dev,
return 0;
}
-/*
- * handle_device_interrupts retrieves the interrupts configuration from
- * a device tree node and maps those interrupts to the target domain.
- *
- * Returns:
- * < 0 error
- * 0 success
- */
-static int __init handle_device_interrupts(struct domain *d,
- struct dt_device_node *dev,
- bool need_mapping)
-{
- unsigned int i, nirq;
- int res;
- struct dt_raw_irq rirq;
-
- nirq = dt_number_of_irq(dev);
-
- /* Give permission and map IRQs */
- for ( i = 0; i < nirq; i++ )
- {
- res = dt_device_get_raw_irq(dev, i, &rirq);
- if ( res )
- {
- printk(XENLOG_ERR "Unable to retrieve irq %u for %s\n",
- i, dt_node_full_name(dev));
- return res;
- }
-
- /*
- * Don't map IRQ that have no physical meaning
- * ie: IRQ whose controller is not the GIC
- */
- if ( rirq.controller != dt_interrupt_controller )
- {
- dt_dprintk("irq %u not connected to primary controller. Connected to %s\n",
- i, dt_node_full_name(rirq.controller));
- continue;
- }
-
- res = platform_get_irq(dev, i);
- if ( res < 0 )
- {
- printk(XENLOG_ERR "Unable to get irq %u for %s\n",
- i, dt_node_full_name(dev));
- return res;
- }
-
- res = map_irq_to_domain(d, res, need_mapping, dt_node_name(dev));
- if ( res )
- return res;
- }
-
- return 0;
-}
-
/*
* For a given device node:
* - Give permission to the guest to manage IRQ and MMIO range
diff --git a/xen/arch/arm/include/asm/domain_build.h b/xen/arch/arm/include/asm/domain_build.h
index 34ceddc995..b9329c9ee0 100644
--- a/xen/arch/arm/include/asm/domain_build.h
+++ b/xen/arch/arm/include/asm/domain_build.h
@@ -4,8 +4,6 @@
#include <xen/sched.h>
#include <asm/kernel.h>
-int map_irq_to_domain(struct domain *d, unsigned int irq,
- bool need_mapping, const char *devname);
int make_chosen_node(const struct kernel_info *kinfo);
void evtchn_allocate(struct domain *d);
diff --git a/xen/arch/arm/include/asm/setup.h b/xen/arch/arm/include/asm/setup.h
index a926f30a2b..1d636e8a4a 100644
--- a/xen/arch/arm/include/asm/setup.h
+++ b/xen/arch/arm/include/asm/setup.h
@@ -163,9 +163,15 @@ void device_tree_get_reg(const __be32 **cell, u32 address_cells,
u32 device_tree_get_u32(const void *fdt, int node,
const char *prop_name, u32 dflt);
+int handle_device_interrupts(struct domain *d, struct dt_device_node *dev,
+ bool need_mapping);
+
int map_range_to_domain(const struct dt_device_node *dev,
u64 addr, u64 len, void *data);
+int map_irq_to_domain(struct domain *d, unsigned int irq,
+ bool need_mapping, const char *devname);
+
extern const char __ro_after_init_start[], __ro_after_init_end[];
struct init_info
diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index 6c9712ab7b..aed38ff63c 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -1811,12 +1811,12 @@ int dt_count_phandle_with_args(const struct dt_device_node *np,
* @allnextpp: pointer to ->allnext from last allocated device_node
* @fpsize: Size of the node path up at the current depth.
*/
-static unsigned long __init unflatten_dt_node(const void *fdt,
- unsigned long mem,
- unsigned long *p,
- struct dt_device_node *dad,
- struct dt_device_node ***allnextpp,
- unsigned long fpsize)
+static unsigned long unflatten_dt_node(const void *fdt,
+ unsigned long mem,
+ unsigned long *p,
+ struct dt_device_node *dad,
+ struct dt_device_node ***allnextpp,
+ unsigned long fpsize)
{
struct dt_device_node *np;
struct dt_property *pp, **prev_pp = NULL;
@@ -2056,8 +2056,8 @@ static unsigned long __init unflatten_dt_node(const void *fdt,
* @fdt: The fdt to expand
* @mynodes: The device_node tree created by the call
*/
-static void __init __unflatten_device_tree(const void *fdt,
- struct dt_device_node **mynodes)
+static void __unflatten_device_tree(const void *fdt,
+ struct dt_device_node **mynodes)
{
unsigned long start, mem, size;
struct dt_device_node **allnextp = mynodes;
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 02/17] common/device_tree: change __unflatten_device_tree()
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 01/17] xen/arm/device: Remove __init from function type Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-13 9:45 ` Michal Orzel
2023-04-13 10:03 ` Julien Grall
2023-04-11 19:16 ` [XEN][PATCH v5 03/17] xen/arm: Add CONFIG_OVERLAY_DTB Vikram Garhwal
` (32 subsequent siblings)
34 siblings, 2 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel; +Cc: sstabellini, Vikram Garhwal, Julien Grall
Following changes are done to __unflatten_device_tree():
1. __unflatten_device_tree() is renamed to unflatten_device_tree().
2. Remove static function type.
3. Add handling of memory allocation. This will be useful in dynamic node
programming when we unflatten the dt during runtime memory allocation
can fail.
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
xen/common/device_tree.c | 10 ++++++----
xen/include/xen/device_tree.h | 5 +++++
2 files changed, 11 insertions(+), 4 deletions(-)
diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index aed38ff63c..bf847b2584 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -2047,7 +2047,7 @@ static unsigned long unflatten_dt_node(const void *fdt,
}
/**
- * __unflatten_device_tree - create tree of device_nodes from flat blob
+ * unflatten_device_tree - create tree of device_nodes from flat blob
*
* unflattens a device-tree, creating the
* tree of struct device_node. It also fills the "name" and "type"
@@ -2056,8 +2056,7 @@ static unsigned long unflatten_dt_node(const void *fdt,
* @fdt: The fdt to expand
* @mynodes: The device_node tree created by the call
*/
-static void __unflatten_device_tree(const void *fdt,
- struct dt_device_node **mynodes)
+void unflatten_device_tree(const void *fdt, struct dt_device_node **mynodes)
{
unsigned long start, mem, size;
struct dt_device_node **allnextp = mynodes;
@@ -2079,6 +2078,9 @@ static void __unflatten_device_tree(const void *fdt,
/* Allocate memory for the expanded device tree */
mem = (unsigned long)_xmalloc (size + 4, __alignof__(struct dt_device_node));
+ if ( !mem )
+ panic("Cannot allocate memory for unflatten device tree\n");
+
((__be32 *)mem)[size / 4] = cpu_to_be32(0xdeadbeef);
dt_dprintk(" unflattening %lx...\n", mem);
@@ -2179,7 +2181,7 @@ dt_find_interrupt_controller(const struct dt_device_match *matches)
void __init dt_unflatten_host_device_tree(void)
{
- __unflatten_device_tree(device_tree_flattened, &dt_host);
+ unflatten_device_tree(device_tree_flattened, &dt_host);
dt_alias_scan();
}
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index 19a74909ce..58ac12abe3 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -178,6 +178,11 @@ int device_tree_for_each_node(const void *fdt, int node,
*/
void dt_unflatten_host_device_tree(void);
+/**
+ * unflatten any device tree.
+ */
+void unflatten_device_tree(const void *fdt, struct dt_device_node **mynodes);
+
/**
* IRQ translation callback
* TODO: For the moment we assume that we only have ONE
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 03/17] xen/arm: Add CONFIG_OVERLAY_DTB
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 01/17] xen/arm/device: Remove __init from function type Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 02/17] common/device_tree: change __unflatten_device_tree() Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-13 9:58 ` Michal Orzel
2023-04-11 19:16 ` [XEN][PATCH v5 04/17] libfdt: Keep fdt functions after init for CONFIG_OVERLAY_DTB Vikram Garhwal
` (31 subsequent siblings)
34 siblings, 1 reply; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel
Cc: sstabellini, Vikram Garhwal, Andrew Cooper, George Dunlap,
Jan Beulich, Julien Grall, Wei Liu, Bertrand Marquis,
Volodymyr Babchuk
Introduce a config option where the user can enable support for adding/removing
device tree nodes using a device tree binary overlay.
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
SUPPORT.md | 6 ++++++
xen/arch/arm/Kconfig | 5 +++++
2 files changed, 11 insertions(+)
diff --git a/SUPPORT.md b/SUPPORT.md
index aa1940e55f..0a31f40af4 100644
--- a/SUPPORT.md
+++ b/SUPPORT.md
@@ -822,6 +822,12 @@ No support for QEMU backends in a 16K or 64K domain.
Status: Supported
+### Device Tree Overlays
+
+Add/Remove device tree nodes using a device tree overlay binary(.dtbo).
+
+ Status: Supported for ARM
+
### ARM: Guest ACPI support
Status: Supported
diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index 239d3aed3c..1fe3d698a5 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -53,6 +53,11 @@ config HAS_ITS
bool "GICv3 ITS MSI controller support (UNSUPPORTED)" if UNSUPPORTED
depends on GICV3 && !NEW_VGIC && !ARM_32
+config OVERLAY_DTB
+ bool "DTB overlay support (UNSUPPORTED)" if UNSUPPORTED
+ help
+ Dynamic addition/removal of Xen device tree nodes using a dtbo.
+
config HVM
def_bool y
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 04/17] libfdt: Keep fdt functions after init for CONFIG_OVERLAY_DTB.
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (2 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 03/17] xen/arm: Add CONFIG_OVERLAY_DTB Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 05/17] libfdt: overlay: change overlay_get_target() Vikram Garhwal
` (30 subsequent siblings)
34 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel; +Cc: sstabellini, Vikram Garhwal, Julien Grall
This is done to access fdt library function which are required for adding device
tree overlay nodes for dynamic programming of nodes.
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
Acked-by: Julien Grall <jgrall@amazon.com>
---
xen/common/libfdt/Makefile | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/xen/common/libfdt/Makefile b/xen/common/libfdt/Makefile
index 75aaefa2e3..d50487aa6e 100644
--- a/xen/common/libfdt/Makefile
+++ b/xen/common/libfdt/Makefile
@@ -1,7 +1,11 @@
include $(src)/Makefile.libfdt
SECTIONS := text data $(SPECIAL_DATA_SECTIONS)
+
+# For CONFIG_OVERLAY_DTB, libfdt functionalities will be needed during runtime.
+ifneq ($(CONFIG_OVERLAY_DTB),y)
OBJCOPYFLAGS := $(foreach s,$(SECTIONS),--rename-section .$(s)=.init.$(s))
+endif
obj-y += libfdt.o
nocov-y += libfdt.o
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 05/17] libfdt: overlay: change overlay_get_target()
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (3 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 04/17] libfdt: Keep fdt functions after init for CONFIG_OVERLAY_DTB Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-13 13:11 ` Michal Orzel
2023-04-11 19:16 ` [XEN][PATCH v5 06/17] xen/device-tree: Add device_tree_find_node_by_path() to find nodes in device tree Vikram Garhwal
` (29 subsequent siblings)
34 siblings, 1 reply; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel
Cc: sstabellini, Vikram Garhwal, Julien Grall, Vikram Garhwal, David Gibson
Rename overlay_get_target() to fdt_overlay_target_offset() and remove static
function type.
This is done to get the target path for the overlay nodes which is very useful
in many cases. For example, Xen hypervisor needs it when applying overlays
because Xen needs to do further processing of the overlay nodes, e.g. mapping of
resources(IRQs and IOMMUs) to other VMs, creation of SMMU pagetables, etc.
Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
Message-Id: <1637204036-382159-2-git-send-email-fnu.vikram@xilinx.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Origin: https://github.com/dgibson/dtc 45f3d1a095dd
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
xen/common/libfdt/fdt_overlay.c | 29 +++++++----------------------
xen/common/libfdt/version.lds | 1 +
xen/include/xen/libfdt/libfdt.h | 18 ++++++++++++++++++
3 files changed, 26 insertions(+), 22 deletions(-)
diff --git a/xen/common/libfdt/fdt_overlay.c b/xen/common/libfdt/fdt_overlay.c
index 7b95e2b639..acf0c4c2a6 100644
--- a/xen/common/libfdt/fdt_overlay.c
+++ b/xen/common/libfdt/fdt_overlay.c
@@ -41,37 +41,22 @@ static uint32_t overlay_get_target_phandle(const void *fdto, int fragment)
return fdt32_to_cpu(*val);
}
-/**
- * overlay_get_target - retrieves the offset of a fragment's target
- * @fdt: Base device tree blob
- * @fdto: Device tree overlay blob
- * @fragment: node offset of the fragment in the overlay
- * @pathp: pointer which receives the path of the target (or NULL)
- *
- * overlay_get_target() retrieves the target offset in the base
- * device tree of a fragment, no matter how the actual targeting is
- * done (through a phandle or a path)
- *
- * returns:
- * the targeted node offset in the base device tree
- * Negative error code on error
- */
-static int overlay_get_target(const void *fdt, const void *fdto,
- int fragment, char const **pathp)
+int fdt_overlay_target_offset(const void *fdt, const void *fdto,
+ int fragment_offset, char const **pathp)
{
uint32_t phandle;
const char *path = NULL;
int path_len = 0, ret;
/* Try first to do a phandle based lookup */
- phandle = overlay_get_target_phandle(fdto, fragment);
+ phandle = overlay_get_target_phandle(fdto, fragment_offset);
if (phandle == (uint32_t)-1)
return -FDT_ERR_BADPHANDLE;
/* no phandle, try path */
if (!phandle) {
/* And then a path based lookup */
- path = fdt_getprop(fdto, fragment, "target-path", &path_len);
+ path = fdt_getprop(fdto, fragment_offset, "target-path", &path_len);
if (path)
ret = fdt_path_offset(fdt, path);
else
@@ -638,7 +623,7 @@ static int overlay_merge(void *fdt, void *fdto)
if (overlay < 0)
return overlay;
- target = overlay_get_target(fdt, fdto, fragment, NULL);
+ target = fdt_overlay_target_offset(fdt, fdto, fragment, NULL);
if (target < 0)
return target;
@@ -781,7 +766,7 @@ static int overlay_symbol_update(void *fdt, void *fdto)
return -FDT_ERR_BADOVERLAY;
/* get the target of the fragment */
- ret = overlay_get_target(fdt, fdto, fragment, &target_path);
+ ret = fdt_overlay_target_offset(fdt, fdto, fragment, &target_path);
if (ret < 0)
return ret;
target = ret;
@@ -803,7 +788,7 @@ static int overlay_symbol_update(void *fdt, void *fdto)
if (!target_path) {
/* again in case setprop_placeholder changed it */
- ret = overlay_get_target(fdt, fdto, fragment, &target_path);
+ ret = fdt_overlay_target_offset(fdt, fdto, fragment, &target_path);
if (ret < 0)
return ret;
target = ret;
diff --git a/xen/common/libfdt/version.lds b/xen/common/libfdt/version.lds
index 7ab85f1d9d..cbce5d4a8b 100644
--- a/xen/common/libfdt/version.lds
+++ b/xen/common/libfdt/version.lds
@@ -77,6 +77,7 @@ LIBFDT_1.2 {
fdt_appendprop_addrrange;
fdt_setprop_inplace_namelen_partial;
fdt_create_with_flags;
+ fdt_overlay_target_offset;
local:
*;
};
diff --git a/xen/include/xen/libfdt/libfdt.h b/xen/include/xen/libfdt/libfdt.h
index c71689e2be..fabddbee8c 100644
--- a/xen/include/xen/libfdt/libfdt.h
+++ b/xen/include/xen/libfdt/libfdt.h
@@ -2109,6 +2109,24 @@ int fdt_del_node(void *fdt, int nodeoffset);
*/
int fdt_overlay_apply(void *fdt, void *fdto);
+/**
+ * fdt_overlay_target_offset - retrieves the offset of a fragment's target
+ * @fdt: Base device tree blob
+ * @fdto: Device tree overlay blob
+ * @fragment_offset: node offset of the fragment in the overlay
+ * @pathp: pointer which receives the path of the target (or NULL)
+ *
+ * fdt_overlay_target_offset() retrieves the target offset in the base
+ * device tree of a fragment, no matter how the actual targeting is
+ * done (through a phandle or a path)
+ *
+ * returns:
+ * the targeted node offset in the base device tree
+ * Negative error code on error
+ */
+int fdt_overlay_target_offset(const void *fdt, const void *fdto,
+ int fragment_offset, char const **pathp);
+
/**********************************************************************/
/* Debugging / informational functions */
/**********************************************************************/
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 06/17] xen/device-tree: Add device_tree_find_node_by_path() to find nodes in device tree
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (4 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 05/17] libfdt: overlay: change overlay_get_target() Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-13 13:40 ` Michal Orzel
2023-04-11 19:16 ` [XEN][PATCH v5 07/17] xen/smmu: Add remove_device callback for smmu_iommu ops Vikram Garhwal
` (28 subsequent siblings)
34 siblings, 1 reply; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel; +Cc: sstabellini, Vikram Garhwal, Julien Grall
Add device_tree_find_node_by_path() to find a matching node with path for a
dt_device_node.
Reason behind this function:
Each time overlay nodes are added using .dtbo, a new fdt(memcpy of
device_tree_flattened) is created and updated with overlay nodes. This
updated fdt is further unflattened to a dt_host_new. Next, we need to find
the overlay nodes in dt_host_new, find the overlay node's parent in dt_host
and add the nodes as child under their parent in the dt_host. Thus we need
this function to search for node in different unflattened device trees.
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
xen/common/device_tree.c | 5 +++--
xen/include/xen/device_tree.h | 17 +++++++++++++++--
2 files changed, 18 insertions(+), 4 deletions(-)
diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index bf847b2584..507b4ac5b6 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -358,11 +358,12 @@ struct dt_device_node *dt_find_node_by_type(struct dt_device_node *from,
return np;
}
-struct dt_device_node *dt_find_node_by_path(const char *path)
+struct dt_device_node *device_tree_find_node_by_path(struct dt_device_node *dt,
+ const char *path)
{
struct dt_device_node *np;
- dt_for_each_device_node(dt_host, np)
+ dt_for_each_device_node(dt, np)
if ( np->full_name && (dt_node_cmp(np->full_name, path) == 0) )
break;
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index 58ac12abe3..998f972ebc 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -534,13 +534,26 @@ struct dt_device_node *dt_find_node_by_type(struct dt_device_node *from,
struct dt_device_node *dt_find_node_by_alias(const char *alias);
/**
- * dt_find_node_by_path - Find a node matching a full DT path
+ * device_tree_find_node_by_path - Generic function to find a node matching the
+ * full DT path for any given unflatten device tree
+ * @dt_node: The device tree to search
* @path: The full path to match
*
* Returns a node pointer.
*/
-struct dt_device_node *dt_find_node_by_path(const char *path);
+struct dt_device_node *device_tree_find_node_by_path(struct dt_device_node *dt,
+ const char *path);
+/**
+ * dt_find_node_by_path - Find a node matching a full DT path in dt_host
+ * @path: The full path to match
+ *
+ * Returns a node pointer.
+ */
+static inline struct dt_device_node *dt_find_node_by_path(const char *path)
+{
+ return device_tree_find_node_by_path(dt_host, path);
+}
/**
* dt_find_node_by_gpath - Same as dt_find_node_by_path but retrieve the
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 07/17] xen/smmu: Add remove_device callback for smmu_iommu ops
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (5 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 06/17] xen/device-tree: Add device_tree_find_node_by_path() to find nodes in device tree Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-17 10:24 ` Michal Orzel
2023-04-11 19:16 ` [XEN][PATCH v5 08/17] xen/iommu: Move spin_lock from iommu_dt_device_is_assigned to caller Vikram Garhwal
` (27 subsequent siblings)
34 siblings, 1 reply; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel
Cc: sstabellini, Vikram Garhwal, Julien Grall, Rahul Singh,
Bertrand Marquis, Volodymyr Babchuk
Add remove_device callback for removing the device entry from smmu-master using
following steps:
1. Find if SMMU master exists for the device node.
2. Remove the SMMU master
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
---
xen/drivers/passthrough/arm/smmu.c | 56 ++++++++++++++++++++++++++++++
1 file changed, 56 insertions(+)
diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c
index 0a514821b3..14e15f1bc6 100644
--- a/xen/drivers/passthrough/arm/smmu.c
+++ b/xen/drivers/passthrough/arm/smmu.c
@@ -816,6 +816,19 @@ static int insert_smmu_master(struct arm_smmu_device *smmu,
return 0;
}
+static int remove_smmu_master(struct arm_smmu_device *smmu,
+ struct arm_smmu_master *master)
+{
+ if (!smmu->masters.rb_node) {
+ ASSERT_UNREACHABLE();
+ return -ENOENT;
+ }
+
+ rb_erase(&master->node, &smmu->masters);
+
+ return 0;
+}
+
static int arm_smmu_dt_add_device_legacy(struct arm_smmu_device *smmu,
struct device *dev,
struct iommu_fwspec *fwspec)
@@ -853,6 +866,32 @@ static int arm_smmu_dt_add_device_legacy(struct arm_smmu_device *smmu,
return insert_smmu_master(smmu, master);
}
+static int arm_smmu_dt_remove_device_legacy(struct arm_smmu_device *smmu,
+ struct device *dev)
+{
+ struct arm_smmu_master *master;
+ struct device_node *dev_node = dev_get_dev_node(dev);
+ int ret;
+
+ master = find_smmu_master(smmu, dev_node);
+ if (master == NULL) {
+ dev_err(dev,
+ "No registrations found for master device %s\n",
+ dev_node->name);
+ return -EINVAL;
+ }
+
+ ret = remove_smmu_master(smmu, master);
+
+ if (ret)
+ return ret;
+
+ dev_node->is_protected = false;
+
+ kfree(master);
+ return 0;
+}
+
static int register_smmu_master(struct arm_smmu_device *smmu,
struct device *dev,
struct of_phandle_args *masterspec)
@@ -876,6 +915,22 @@ static int register_smmu_master(struct arm_smmu_device *smmu,
fwspec);
}
+static int arm_smmu_dt_remove_device_generic(u8 devfn, struct device *dev)
+{
+ struct arm_smmu_device *smmu;
+ struct iommu_fwspec *fwspec;
+
+ fwspec = dev_iommu_fwspec_get(dev);
+ if (fwspec == NULL)
+ return -ENXIO;
+
+ smmu = find_smmu(fwspec->iommu_dev);
+ if (smmu == NULL)
+ return -ENXIO;
+
+ return arm_smmu_dt_remove_device_legacy(smmu, dev);
+}
+
static int arm_smmu_dt_add_device_generic(u8 devfn, struct device *dev)
{
struct arm_smmu_device *smmu;
@@ -2858,6 +2913,7 @@ static const struct iommu_ops arm_smmu_iommu_ops = {
.init = arm_smmu_iommu_domain_init,
.hwdom_init = arch_iommu_hwdom_init,
.add_device = arm_smmu_dt_add_device_generic,
+ .remove_device = arm_smmu_dt_remove_device_generic,
.teardown = arm_smmu_iommu_domain_teardown,
.iotlb_flush = arm_smmu_iotlb_flush,
.assign_device = arm_smmu_assign_dev,
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 08/17] xen/iommu: Move spin_lock from iommu_dt_device_is_assigned to caller
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (6 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 07/17] xen/smmu: Add remove_device callback for smmu_iommu ops Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-17 9:14 ` Michal Orzel
2023-04-11 19:16 ` [XEN][PATCH v5 09/17] xen/iommu: protect iommu_add_dt_device() with dtdevs_lock Vikram Garhwal
` (26 subsequent siblings)
34 siblings, 1 reply; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel; +Cc: sstabellini, Vikram Garhwal, Julien Grall
Rename iommu_dt_device_is_assigned() to iommu_dt_device_is_assigned_lock().
Moving spin_lock to caller was done to prevent the concurrent access to
iommu_dt_device_is_assigned while doing add/remove/assign/deassign.
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
---
xen/drivers/passthrough/device_tree.c | 23 +++++++++++++++++++----
1 file changed, 19 insertions(+), 4 deletions(-)
diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c
index 1c32d7b50c..bb4cf7784d 100644
--- a/xen/drivers/passthrough/device_tree.c
+++ b/xen/drivers/passthrough/device_tree.c
@@ -83,16 +83,15 @@ fail:
return rc;
}
-static bool_t iommu_dt_device_is_assigned(const struct dt_device_node *dev)
+static bool_t
+ iommu_dt_device_is_assigned_locked(const struct dt_device_node *dev)
{
bool_t assigned = 0;
if ( !dt_device_is_protected(dev) )
return 0;
- spin_lock(&dtdevs_lock);
assigned = !list_empty(&dev->domain_list);
- spin_unlock(&dtdevs_lock);
return assigned;
}
@@ -213,27 +212,43 @@ int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d,
if ( (d && d->is_dying) || domctl->u.assign_device.flags )
break;
+ spin_lock(&dtdevs_lock);
+
ret = dt_find_node_by_gpath(domctl->u.assign_device.u.dt.path,
domctl->u.assign_device.u.dt.size,
&dev);
if ( ret )
+ {
+ spin_unlock(&dtdevs_lock);
+
break;
+ }
ret = xsm_assign_dtdevice(XSM_HOOK, d, dt_node_full_name(dev));
if ( ret )
+ {
+ spin_unlock(&dtdevs_lock);
+
break;
+ }
if ( domctl->cmd == XEN_DOMCTL_test_assign_device )
{
- if ( iommu_dt_device_is_assigned(dev) )
+
+ if ( iommu_dt_device_is_assigned_locked(dev) )
{
printk(XENLOG_G_ERR "%s already assigned.\n",
dt_node_full_name(dev));
ret = -EINVAL;
}
+
+ spin_unlock(&dtdevs_lock);
+
break;
}
+ spin_unlock(&dtdevs_lock);
+
if ( d == dom_io )
return -EINVAL;
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 09/17] xen/iommu: protect iommu_add_dt_device() with dtdevs_lock
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (7 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 08/17] xen/iommu: Move spin_lock from iommu_dt_device_is_assigned to caller Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-17 10:35 ` Michal Orzel
2023-04-11 19:16 ` [XEN][PATCH v5 10/17] xen/iommu: Introduce iommu_remove_dt_device() Vikram Garhwal
` (25 subsequent siblings)
34 siblings, 1 reply; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel; +Cc: sstabellini, Vikram Garhwal, Julien Grall
Protect iommu_add_dt_device() with dtdevs_lock to prevent concurrent access add.
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
---
xen/drivers/passthrough/device_tree.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c
index bb4cf7784d..457df333a0 100644
--- a/xen/drivers/passthrough/device_tree.c
+++ b/xen/drivers/passthrough/device_tree.c
@@ -146,6 +146,8 @@ int iommu_add_dt_device(struct dt_device_node *np)
if ( dev_iommu_fwspec_get(dev) )
return 0;
+ spin_lock(&dtdevs_lock);
+
/*
* According to the Documentation/devicetree/bindings/iommu/iommu.txt
* from Linux.
@@ -158,7 +160,10 @@ int iommu_add_dt_device(struct dt_device_node *np)
* these callback implemented.
*/
if ( !ops->add_device || !ops->dt_xlate )
- return -EINVAL;
+ {
+ rc = -EINVAL;
+ goto fail;
+ }
if ( !dt_device_is_available(iommu_spec.np) )
break;
@@ -189,6 +194,8 @@ int iommu_add_dt_device(struct dt_device_node *np)
if ( rc < 0 )
iommu_fwspec_free(dev);
+fail:
+ spin_unlock(&dtdevs_lock);
return rc;
}
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 10/17] xen/iommu: Introduce iommu_remove_dt_device()
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (8 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 09/17] xen/iommu: protect iommu_add_dt_device() with dtdevs_lock Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-17 10:49 ` Michal Orzel
2023-04-11 19:16 ` [XEN][PATCH v5 11/17] asm/smp.h: Fix circular dependency for device_tree.h and rwlock.h Vikram Garhwal
` (24 subsequent siblings)
34 siblings, 1 reply; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel
Cc: sstabellini, Vikram Garhwal, Julien Grall, Jan Beulich,
Paul Durrant, Roger Pau Monné
Remove master device from the IOMMU. This will be helpful when removing the
overlay nodes using dynamic programming during run time.
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
xen/drivers/passthrough/device_tree.c | 38 +++++++++++++++++++++++++++
xen/include/xen/iommu.h | 2 ++
2 files changed, 40 insertions(+)
diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c
index 457df333a0..a77a217f3d 100644
--- a/xen/drivers/passthrough/device_tree.c
+++ b/xen/drivers/passthrough/device_tree.c
@@ -126,6 +126,44 @@ int iommu_release_dt_devices(struct domain *d)
return 0;
}
+int iommu_remove_dt_device(struct dt_device_node *np)
+{
+ const struct iommu_ops *ops = iommu_get_ops();
+ struct device *dev = dt_to_dev(np);
+ int rc;
+
+ if ( !ops )
+ return -EOPNOTSUPP;
+
+ spin_lock(&dtdevs_lock);
+
+ if ( iommu_dt_device_is_assigned_locked(np) )
+ {
+ rc = -EBUSY;
+ goto fail;
+ }
+
+ /*
+ * The driver which supports generic IOMMU DT bindings must have
+ * these callback implemented.
+ */
+ if ( !ops->remove_device )
+ {
+ rc = -EOPNOTSUPP;
+ goto fail;
+ }
+
+ /* Remove master device from the IOMMU if latter is present and available. */
+ rc = ops->remove_device(0, dev);
+
+ if ( !rc )
+ iommu_fwspec_free(dev);
+
+fail:
+ spin_unlock(&dtdevs_lock);
+ return rc;
+}
+
int iommu_add_dt_device(struct dt_device_node *np)
{
const struct iommu_ops *ops = iommu_get_ops();
diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
index 405db59971..079c06321e 100644
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -218,6 +218,8 @@ int iommu_deassign_dt_device(struct domain *d, struct dt_device_node *dev);
int iommu_dt_domain_init(struct domain *d);
int iommu_release_dt_devices(struct domain *d);
+int iommu_remove_dt_device(struct dt_device_node *np);
+
/*
* Helper to add master device to the IOMMU using generic IOMMU DT bindings.
*
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 11/17] asm/smp.h: Fix circular dependency for device_tree.h and rwlock.h
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (9 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 10/17] xen/iommu: Introduce iommu_remove_dt_device() Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-14 1:50 ` Henry Wang
2023-04-17 9:41 ` Michal Orzel
2023-04-11 19:16 ` [XEN][PATCH v5 12/17] common/device_tree: Add rwlock for dt_host Vikram Garhwal
` (23 subsequent siblings)
34 siblings, 2 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel
Cc: sstabellini, Vikram Garhwal, Julien Grall, Bertrand Marquis,
Volodymyr Babchuk
Dynamic programming ops will modify the dt_host and there might be other
function which are browsing the dt_host at the same time. To avoid the race
conditions, adding rwlock for browsing the dt_host. But adding rwlock in
device_tree.h causes following circular dependency:
device_tree.h->rwlock.h->smp.h->asm/smp.h->device_tree.h
To fix this, removed the "#include <xen/device_tree.h> and forward declared
"struct dt_device_node".
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
xen/arch/arm/include/asm/smp.h | 3 ++-
xen/arch/arm/smpboot.c | 1 +
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/xen/arch/arm/include/asm/smp.h b/xen/arch/arm/include/asm/smp.h
index 8133d5c295..afe6129276 100644
--- a/xen/arch/arm/include/asm/smp.h
+++ b/xen/arch/arm/include/asm/smp.h
@@ -3,13 +3,14 @@
#ifndef __ASSEMBLY__
#include <xen/cpumask.h>
-#include <xen/device_tree.h>
#include <asm/current.h>
#endif
DECLARE_PER_CPU(cpumask_var_t, cpu_sibling_mask);
DECLARE_PER_CPU(cpumask_var_t, cpu_core_mask);
+struct dt_device_node;
+
#define cpu_is_offline(cpu) unlikely(!cpu_online(cpu))
#define smp_processor_id() get_processor_id()
diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
index 412ae22869..336a7d418b 100644
--- a/xen/arch/arm/smpboot.c
+++ b/xen/arch/arm/smpboot.c
@@ -11,6 +11,7 @@
#include <xen/cpumask.h>
#include <xen/delay.h>
#include <xen/domain_page.h>
+#include <xen/device_tree.h>
#include <xen/errno.h>
#include <xen/init.h>
#include <xen/mm.h>
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 12/17] common/device_tree: Add rwlock for dt_host
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (10 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 11/17] asm/smp.h: Fix circular dependency for device_tree.h and rwlock.h Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-14 2:09 ` Henry Wang
2023-04-11 19:16 ` [XEN][PATCH v5 13/17] xen/arm: Implement device tree node removal functionalities Vikram Garhwal
` (22 subsequent siblings)
34 siblings, 1 reply; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel; +Cc: sstabellini, Vikram Garhwal, Julien Grall
Dynamic programming ops will modify the dt_host and there might be other
function which are browsing the dt_host at the same time. To avoid the race
conditions, adding rwlock for browsing the dt_host during runtime.
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
xen/common/device_tree.c | 3 +++
xen/drivers/passthrough/device_tree.c | 39 +++++++++++++++++++++++++++
xen/include/xen/device_tree.h | 6 +++++
3 files changed, 48 insertions(+)
diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index 507b4ac5b6..b330e6a013 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -2098,6 +2098,9 @@ void unflatten_device_tree(const void *fdt, struct dt_device_node **mynodes)
*allnextp = NULL;
dt_dprintk(" <- unflatten_device_tree()\n");
+
+ /* Init r/w lock for host device tree. */
+ rwlock_init(&dt_host->lock);
}
static void dt_alias_add(struct dt_alias_prop *ap,
diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c
index a77a217f3d..74f994b9da 100644
--- a/xen/drivers/passthrough/device_tree.c
+++ b/xen/drivers/passthrough/device_tree.c
@@ -112,6 +112,8 @@ int iommu_release_dt_devices(struct domain *d)
if ( !is_iommu_enabled(d) )
return 0;
+ read_lock(&dt_host->lock);
+
list_for_each_entry_safe(dev, _dev, &hd->dt_devices, domain_list)
{
rc = iommu_deassign_dt_device(d, dev);
@@ -119,10 +121,14 @@ int iommu_release_dt_devices(struct domain *d)
{
dprintk(XENLOG_ERR, "Failed to deassign %s in domain %u\n",
dt_node_full_name(dev), d->domain_id);
+
+ read_unlock(&dt_host->lock);
return rc;
}
}
+ read_unlock(&dt_host->lock);
+
return 0;
}
@@ -259,11 +265,15 @@ int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d,
spin_lock(&dtdevs_lock);
+ read_lock(&dt_host->lock);
+
ret = dt_find_node_by_gpath(domctl->u.assign_device.u.dt.path,
domctl->u.assign_device.u.dt.size,
&dev);
if ( ret )
{
+ read_unlock(&dt_host->lock);
+
spin_unlock(&dtdevs_lock);
break;
@@ -272,6 +282,8 @@ int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d,
ret = xsm_assign_dtdevice(XSM_HOOK, d, dt_node_full_name(dev));
if ( ret )
{
+ read_unlock(&dt_host->lock);
+
spin_unlock(&dtdevs_lock);
break;
@@ -287,6 +299,8 @@ int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d,
ret = -EINVAL;
}
+ read_unlock(&dt_host->lock);
+
spin_unlock(&dtdevs_lock);
break;
@@ -295,22 +309,31 @@ int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d,
spin_unlock(&dtdevs_lock);
if ( d == dom_io )
+ {
+ read_unlock(&dt_host->lock);
return -EINVAL;
+ }
ret = iommu_add_dt_device(dev);
if ( ret < 0 )
{
printk(XENLOG_G_ERR "Failed to add %s to the IOMMU\n",
dt_node_full_name(dev));
+
+ read_unlock(&dt_host->lock);
break;
}
ret = iommu_assign_dt_device(d, dev);
if ( ret )
+ {
printk(XENLOG_G_ERR "XEN_DOMCTL_assign_dt_device: assign \"%s\""
" to dom%u failed (%d)\n",
dt_node_full_name(dev), d->domain_id, ret);
+ }
+
+ read_unlock(&dt_host->lock);
break;
case XEN_DOMCTL_deassign_device:
@@ -322,25 +345,41 @@ int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d,
if ( domctl->u.assign_device.flags )
break;
+ read_lock(&dt_host->lock);
+
ret = dt_find_node_by_gpath(domctl->u.assign_device.u.dt.path,
domctl->u.assign_device.u.dt.size,
&dev);
if ( ret )
+ {
+ read_unlock(&dt_host->lock);
break;
+ }
ret = xsm_deassign_dtdevice(XSM_HOOK, d, dt_node_full_name(dev));
+
if ( ret )
+ {
+ read_unlock(&dt_host->lock);
break;
+ }
if ( d == dom_io )
+ {
+ read_unlock(&dt_host->lock);
return -EINVAL;
+ }
ret = iommu_deassign_dt_device(d, dev);
if ( ret )
+ {
printk(XENLOG_G_ERR "XEN_DOMCTL_assign_dt_device: assign \"%s\""
" to dom%u failed (%d)\n",
dt_node_full_name(dev), d->domain_id, ret);
+ }
+
+ read_unlock(&dt_host->lock);
break;
default:
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index 998f972ebc..b7272bbccc 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -18,6 +18,7 @@
#include <xen/string.h>
#include <xen/types.h>
#include <xen/list.h>
+#include <xen/rwlock.h>
#define DEVICE_TREE_MAX_DEPTH 16
@@ -106,6 +107,11 @@ struct dt_device_node {
struct list_head domain_list;
struct device dev;
+
+ /*
+ * Lock that protects r/w updates to unflattened device tree i.e. dt_host.
+ */
+ rwlock_t lock;
};
#define dt_to_dev(dt_node) (&(dt_node)->dev)
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 13/17] xen/arm: Implement device tree node removal functionalities
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (11 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 12/17] common/device_tree: Add rwlock for dt_host Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-17 8:08 ` Jan Beulich
2023-04-17 15:06 ` Michal Orzel
2023-04-11 19:16 ` [XEN][PATCH v5 14/17] xen/arm: Implement device tree node addition functionalities Vikram Garhwal
` (21 subsequent siblings)
34 siblings, 2 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel
Cc: sstabellini, Vikram Garhwal, Julien Grall, Bertrand Marquis,
Volodymyr Babchuk, Andrew Cooper, George Dunlap, Jan Beulich,
Wei Liu
Introduce sysctl XEN_SYSCTL_dt_overlay to remove device-tree nodes added using
device tree overlay.
xl dt-overlay remove file.dtbo:
Removes all the nodes in a given dtbo.
First, removes IRQ permissions and MMIO accesses. Next, it finds the nodes
in dt_host and delete the device node entries from dt_host.
The nodes get removed only if it is not used by any of dom0 or domio.
Also, added overlay_track struct to keep the track of added node through device
tree overlay. overlay_track has dt_host_new which is unflattened form of updated
fdt and name of overlay nodes. When a node is removed, we also free the memory
used by overlay_track for the particular overlay node.
Nested overlay removal is supported in sequential manner only i.e. if
overlay_child nests under overlay_parent, it is assumed that user first removes
overlay_child and then removes overlay_parent.
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
xen/arch/arm/sysctl.c | 16 +-
xen/common/Makefile | 1 +
xen/common/dt_overlay.c | 415 +++++++++++++++++++++++++++++++++++
xen/include/public/sysctl.h | 24 ++
xen/include/xen/dt_overlay.h | 59 +++++
5 files changed, 514 insertions(+), 1 deletion(-)
create mode 100644 xen/common/dt_overlay.c
create mode 100644 xen/include/xen/dt_overlay.h
diff --git a/xen/arch/arm/sysctl.c b/xen/arch/arm/sysctl.c
index b0a78a8b10..672db61650 100644
--- a/xen/arch/arm/sysctl.c
+++ b/xen/arch/arm/sysctl.c
@@ -12,6 +12,7 @@
#include <xen/errno.h>
#include <xen/hypercall.h>
#include <public/sysctl.h>
+#include <xen/dt_overlay.h>
void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
{
@@ -21,7 +22,20 @@ void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
long arch_do_sysctl(struct xen_sysctl *sysctl,
XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl)
{
- return -ENOSYS;
+ long ret = 0;
+
+ switch ( sysctl->cmd )
+ {
+ case XEN_SYSCTL_dt_overlay:
+ ret = dt_sysctl(&sysctl->u.dt_overlay);
+ break;
+
+ default:
+ ret = -ENOSYS;
+ break;
+ }
+
+ return ret;
}
/*
diff --git a/xen/common/Makefile b/xen/common/Makefile
index 46049eac35..be78c9a8c2 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -8,6 +8,7 @@ obj-$(CONFIG_DEBUG_TRACE) += debugtrace.o
obj-$(CONFIG_HAS_DEVICE_TREE) += device_tree.o
obj-$(CONFIG_IOREQ_SERVER) += dm.o
obj-y += domain.o
+obj-$(CONFIG_OVERLAY_DTB) += dt_overlay.o
obj-y += event_2l.o
obj-y += event_channel.o
obj-y += event_fifo.o
diff --git a/xen/common/dt_overlay.c b/xen/common/dt_overlay.c
new file mode 100644
index 0000000000..516e8010c5
--- /dev/null
+++ b/xen/common/dt_overlay.c
@@ -0,0 +1,415 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0
+ *
+ * xen/common/dt_overlay.c
+ *
+ * Device tree overlay support in Xen.
+ *
+ * Copyright (C) 2023, Advanced Micro Devices, Inc. All Rights Reserved.
+ * Written by Vikram Garhwal <vikram.garhwal@amd.com>
+ *
+ */
+#include <xen/iocap.h>
+#include <xen/xmalloc.h>
+#include <asm/domain_build.h>
+#include <xen/dt_overlay.h>
+#include <xen/guest_access.h>
+
+static LIST_HEAD(overlay_tracker);
+static DEFINE_SPINLOCK(overlay_lock);
+
+/* Find last descendants of the device_node. */
+static struct dt_device_node *find_last_descendants_node(
+ struct dt_device_node *device_node)
+{
+ struct dt_device_node *child_node;
+
+ for ( child_node = device_node->child; child_node->sibling != NULL;
+ child_node = child_node->sibling )
+ {
+ }
+
+ /* If last child_node also have children. */
+ if ( child_node->child )
+ child_node = find_last_descendants_node(child_node);
+
+ return child_node;
+}
+
+static int dt_overlay_remove_node(struct dt_device_node *device_node)
+{
+ struct dt_device_node *np;
+ struct dt_device_node *parent_node;
+ struct dt_device_node *device_node_last_descendant = device_node->child;
+
+ parent_node = device_node->parent;
+
+ if ( parent_node == NULL )
+ {
+ dt_dprintk("%s's parent node not found\n", device_node->name);
+ return -EFAULT;
+ }
+
+ np = parent_node->child;
+
+ if ( np == NULL )
+ {
+ dt_dprintk("parent node %s's not found\n", parent_node->name);
+ return -EFAULT;
+ }
+
+ /* If node to be removed is only child node or first child. */
+ if ( !dt_node_cmp(np->full_name, device_node->full_name) )
+ {
+ parent_node->child = np->sibling;
+
+ /*
+ * Iterate over all child nodes of device_node. Given that we are
+ * removing parent node, we need to remove all it's descendants too.
+ */
+ if ( device_node_last_descendant )
+ {
+ device_node_last_descendant =
+ find_last_descendants_node(device_node);
+ parent_node->allnext = device_node_last_descendant->allnext;
+ }
+ else
+ parent_node->allnext = np->allnext;
+
+ return 0;
+ }
+
+ for ( np = parent_node->child; np->sibling != NULL; np = np->sibling )
+ {
+ if ( !dt_node_cmp(np->sibling->full_name, device_node->full_name) )
+ {
+ /* Found the node. Now we remove it. */
+ np->sibling = np->sibling->sibling;
+
+ if ( np->child )
+ np = find_last_descendants_node(np);
+
+ /*
+ * Iterate over all child nodes of device_node. Given that we are
+ * removing parent node, we need to remove all it's descendants too.
+ */
+ if ( device_node_last_descendant )
+ device_node_last_descendant =
+ find_last_descendants_node(device_node);
+
+ if ( device_node_last_descendant )
+ np->allnext = device_node_last_descendant->allnext;
+ else
+ np->allnext = np->allnext->allnext;
+
+ break;
+ }
+ }
+
+ return 0;
+}
+
+/* Basic sanity check for the dtbo tool stack provided to Xen. */
+static int check_overlay_fdt(const void *overlay_fdt, uint32_t overlay_fdt_size)
+{
+ if ( (fdt_totalsize(overlay_fdt) != overlay_fdt_size) ||
+ fdt_check_header(overlay_fdt) )
+ {
+ printk(XENLOG_ERR "The overlay FDT is not a valid Flat Device Tree\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+/* Count number of nodes till one level of __overlay__ tag. */
+static unsigned int overlay_node_count(void *fdto)
+{
+ unsigned int num_overlay_nodes = 0;
+ int fragment;
+
+ fdt_for_each_subnode(fragment, fdto, 0)
+ {
+ int subnode;
+ int overlay;
+
+ overlay = fdt_subnode_offset(fdto, fragment, "__overlay__");
+
+ /*
+ * overlay value can be < 0. But fdt_for_each_subnode() loop checks for
+ * overlay >= 0. So, no need for a overlay>=0 check here.
+ */
+ fdt_for_each_subnode(subnode, fdto, overlay)
+ {
+ num_overlay_nodes++;
+ }
+ }
+
+ return num_overlay_nodes;
+}
+
+static int handle_remove_irq_iommu(struct dt_device_node *device_node)
+{
+ int rc = 0;
+ struct domain *d = hardware_domain;
+ domid_t domid;
+ unsigned int naddr, len;
+ unsigned int i, nirq;
+ uint64_t addr, size;
+
+ domid = dt_device_used_by(device_node);
+
+ dt_dprintk("Checking if node %s is used by any domain\n",
+ device_node->full_name);
+
+ /* Remove the node iff it's assigned to domain 0 or domain io. */
+ if ( domid != 0 && domid != DOMID_IO )
+ {
+ printk(XENLOG_ERR "Device %s as it is being used by domain %d. Removing nodes failed\n",
+ device_node->full_name, domid);
+ return -EINVAL;
+ }
+
+ dt_dprintk("Removing node: %s\n", device_node->full_name);
+
+ nirq = dt_number_of_irq(device_node);
+
+ /* Remove IRQ permission */
+ for ( i = 0; i < nirq; i++ )
+ {
+ rc = platform_get_irq(device_node, i);;
+
+ if ( irq_access_permitted(d, rc) == false )
+ {
+ printk(XENLOG_ERR "IRQ %d is not routed to domain %d\n", rc,
+ domid);
+ return -EINVAL;
+ }
+ /*
+ * TODO: We don't handle shared IRQs for now. So, it is assumed that
+ * the IRQs was not shared with another devices.
+ */
+ rc = irq_deny_access(d, rc);
+ if ( rc )
+ {
+ printk(XENLOG_ERR "unable to revoke access for irq %u for %s\n",
+ i, device_node->full_name);
+ return rc;
+ }
+ }
+
+ /* Check if iommu property exists. */
+ if ( dt_get_property(device_node, "iommus", &len) )
+ {
+ rc = iommu_remove_dt_device(device_node);
+ if ( rc != 0 && rc != -ENXIO )
+ return rc;
+ }
+
+ naddr = dt_number_of_address(device_node);
+
+ /* Remove mmio access. */
+ for ( i = 0; i < naddr; i++ )
+ {
+ rc = dt_device_get_address(device_node, i, &addr, &size);
+ if ( rc )
+ {
+ printk(XENLOG_ERR "Unable to retrieve address %u for %s\n",
+ i, dt_node_full_name(device_node));
+ return rc;
+ }
+
+ rc = iomem_deny_access(d, paddr_to_pfn(addr),
+ paddr_to_pfn(PAGE_ALIGN(addr + size - 1)));
+ if ( rc )
+ {
+ printk(XENLOG_ERR "Unable to remove dom%d access to"
+ " 0x%"PRIx64" - 0x%"PRIx64"\n",
+ d->domain_id,
+ addr & PAGE_MASK, PAGE_ALIGN(addr + size) - 1);
+ return rc;
+ }
+
+ }
+
+ return rc;
+}
+
+/* Removes all descendants of the given node. */
+static int remove_all_descendant_nodes(struct dt_device_node *device_node)
+{
+ int rc = 0;
+ struct dt_device_node *child_node;
+
+ for ( child_node = device_node->child; child_node != NULL;
+ child_node = child_node->sibling )
+ {
+ if ( child_node->child )
+ remove_all_descendant_nodes(child_node);
+
+ rc = handle_remove_irq_iommu(child_node);
+ if ( rc )
+ return rc;
+ }
+
+ return rc;
+}
+
+/* Remove nodes from dt_host. */
+static int remove_nodes(const struct overlay_track *tracker)
+{
+ int rc = 0;
+ struct dt_device_node *overlay_node;
+ unsigned int j;
+
+ for ( j = 0; j < tracker->num_nodes; j++ )
+ {
+ overlay_node = (struct dt_device_node *)tracker->nodes_address[j];
+ if ( overlay_node == NULL )
+ {
+ printk(XENLOG_ERR "Device %s is not present in the tree. Removing nodes failed\n",
+ overlay_node->full_name);
+ return -EINVAL;
+ }
+
+ rc = remove_all_descendant_nodes(overlay_node);
+
+ /* All children nodes are unmapped. Now remove the node itself. */
+ rc = handle_remove_irq_iommu(overlay_node);
+ if ( rc )
+ return rc;
+
+ read_lock(&dt_host->lock);
+
+ rc = dt_overlay_remove_node(overlay_node);
+ if ( rc )
+ {
+ read_unlock(&dt_host->lock);
+
+ return rc;
+ }
+
+ read_unlock(&dt_host->lock);
+ }
+
+ return rc;
+}
+
+/*
+ * First finds the device node to remove. Check if the device is being used by
+ * any dom and finally remove it from dt_host. IOMMU is already being taken care
+ * while destroying the domain.
+ */
+static long handle_remove_overlay_nodes(void *overlay_fdt,
+ uint32_t overlay_fdt_size)
+{
+ int rc = 0;
+ struct overlay_track *entry, *temp, *track;
+ bool found_entry = false;
+
+ rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
+ if ( rc )
+ return rc;
+
+ if ( overlay_node_count(overlay_fdt) == 0 )
+ return -ENOMEM;
+
+ spin_lock(&overlay_lock);
+
+ /*
+ * First check if dtbo is correct i.e. it should one of the dtbo which was
+ * used when dynamically adding the node.
+ * Limitation: Cases with same node names but different property are not
+ * supported currently. We are relying on user to provide the same dtbo
+ * as it was used when adding the nodes.
+ */
+ list_for_each_entry_safe( entry, temp, &overlay_tracker, entry )
+ {
+ if ( memcmp(entry->overlay_fdt, overlay_fdt, overlay_fdt_size) == 0 )
+ {
+ track = entry;
+ found_entry = true;
+ break;
+ }
+ }
+
+ if ( found_entry == false )
+ {
+ rc = -EINVAL;
+
+ printk(XENLOG_ERR "Cannot find any matching tracker with input dtbo."
+ " Removing nodes is supported for only prior added dtbo. Please"
+ " provide a valid dtbo which was used to add the nodes.\n");
+ goto out;
+
+ }
+
+ rc = remove_nodes(entry);
+
+ if ( rc )
+ {
+ printk(XENLOG_ERR "Removing node failed\n");
+ goto out;
+ }
+
+ list_del(&entry->entry);
+
+ xfree(entry->dt_host_new);
+ xfree(entry->fdt);
+ xfree(entry->overlay_fdt);
+
+ xfree(entry->nodes_address);
+
+ xfree(entry);
+
+out:
+ spin_unlock(&overlay_lock);
+ return rc;
+}
+
+long dt_sysctl(struct xen_sysctl_dt_overlay *op)
+{
+ long ret;
+ void *overlay_fdt;
+
+ if ( op->overlay_fdt_size <= 0 || op->overlay_fdt_size > KB(500) )
+ return -EINVAL;
+
+ overlay_fdt = xmalloc_bytes(op->overlay_fdt_size);
+
+ if ( overlay_fdt == NULL )
+ return -ENOMEM;
+
+ ret = copy_from_guest(overlay_fdt, op->overlay_fdt, op->overlay_fdt_size);
+ if ( ret )
+ {
+ gprintk(XENLOG_ERR, "copy from guest failed\n");
+ xfree(overlay_fdt);
+
+ return -EFAULT;
+ }
+
+ switch ( op->overlay_op )
+ {
+ case XEN_SYSCTL_DT_OVERLAY_REMOVE:
+ ret = handle_remove_overlay_nodes(overlay_fdt, op->overlay_fdt_size);
+ xfree(overlay_fdt);
+
+ break;
+
+ default:
+ xfree(overlay_fdt);
+ break;
+ }
+
+ return ret;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
index 2b24d6bfd0..1158c1efb3 100644
--- a/xen/include/public/sysctl.h
+++ b/xen/include/public/sysctl.h
@@ -1057,6 +1057,25 @@ typedef struct xen_sysctl_cpu_policy xen_sysctl_cpu_policy_t;
DEFINE_XEN_GUEST_HANDLE(xen_sysctl_cpu_policy_t);
#endif
+#if defined(__arm__) || defined (__aarch64__)
+#define XEN_SYSCTL_DT_OVERLAY_ADD 1
+#define XEN_SYSCTL_DT_OVERLAY_REMOVE 2
+
+/*
+ * XEN_SYSCTL_dt_overlay
+ * Performs addition/removal of device tree nodes under parent node using dtbo.
+ * This does in three steps:
+ * - Adds/Removes the nodes from dt_host.
+ * - Adds/Removes IRQ permission for the nodes.
+ * - Adds/Removes MMIO accesses.
+ */
+struct xen_sysctl_dt_overlay {
+ XEN_GUEST_HANDLE_64(void) overlay_fdt; /* IN: overlay fdt. */
+ uint32_t overlay_fdt_size; /* IN: Overlay dtb size. */
+ uint8_t overlay_op; /* IN: Add or remove. */
+};
+#endif
+
struct xen_sysctl {
uint32_t cmd;
#define XEN_SYSCTL_readconsole 1
@@ -1087,6 +1106,7 @@ struct xen_sysctl {
#define XEN_SYSCTL_livepatch_op 27
/* #define XEN_SYSCTL_set_parameter 28 */
#define XEN_SYSCTL_get_cpu_policy 29
+#define XEN_SYSCTL_dt_overlay 30
uint32_t interface_version; /* XEN_SYSCTL_INTERFACE_VERSION */
union {
struct xen_sysctl_readconsole readconsole;
@@ -1117,6 +1137,10 @@ struct xen_sysctl {
#if defined(__i386__) || defined(__x86_64__)
struct xen_sysctl_cpu_policy cpu_policy;
#endif
+
+#if defined(__arm__) || defined (__aarch64__)
+ struct xen_sysctl_dt_overlay dt_overlay;
+#endif
uint8_t pad[128];
} u;
};
diff --git a/xen/include/xen/dt_overlay.h b/xen/include/xen/dt_overlay.h
new file mode 100644
index 0000000000..2cd975a070
--- /dev/null
+++ b/xen/include/xen/dt_overlay.h
@@ -0,0 +1,59 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0
+ *
+ * xen/dt_overlay.h
+ *
+ * Device tree overlay support in Xen.
+ *
+ * Copyright (c) 2022 AMD Inc.
+ * Written by Vikram Garhwal <vikram.garhwal@amd.com>
+ *
+ */
+#ifndef __XEN_DT_SYSCTL_H__
+#define __XEN_DT_SYSCTL_H__
+
+#include <xen/list.h>
+#include <xen/libfdt/libfdt.h>
+#include <xen/device_tree.h>
+#include <xen/rangeset.h>
+
+/*
+ * overlay_node_track describes information about added nodes through dtbo.
+ * @entry: List pointer.
+ * @dt_host_new: Pointer to the updated dt_host_new unflattened 'updated fdt'.
+ * @fdt: Stores the fdt.
+ * @nodes_fullname: Stores the full name of nodes.
+ * @nodes_irq: Stores the IRQ added from overlay dtb.
+ * @node_num_irq: Stores num of IRQ for each node in overlay dtb.
+ * @num_nodes: Stores total number of nodes in overlay dtb.
+ */
+struct overlay_track {
+ struct list_head entry;
+ struct dt_device_node *dt_host_new;
+ void *fdt;
+ void *overlay_fdt;
+ unsigned long *nodes_address;
+ unsigned int num_nodes;
+};
+
+struct xen_sysctl_dt_overlay;
+
+#ifdef CONFIG_OVERLAY_DTB
+long dt_sysctl(struct xen_sysctl_dt_overlay *op);
+#else
+static inline long dt_sysctl(struct xen_sysctl_dt_overlay *op)
+{
+ return -ENOSYS;
+}
+#endif
+#endif
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 14/17] xen/arm: Implement device tree node addition functionalities
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (12 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 13/17] xen/arm: Implement device tree node removal functionalities Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-18 9:03 ` Michal Orzel
2023-04-11 19:16 ` [XEN][PATCH v5 15/17] tools/libs/ctrl: Implement new xc interfaces for dt overlay Vikram Garhwal
` (20 subsequent siblings)
34 siblings, 1 reply; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel
Cc: sstabellini, Vikram Garhwal, Andrew Cooper, George Dunlap,
Jan Beulich, Julien Grall, Wei Liu
Update sysctl XEN_SYSCTL_dt_overlay to enable support for dtbo nodes addition
using device tree overlay.
xl dt-overlay add file.dtbo:
Each time overlay nodes are added using .dtbo, a new fdt(memcpy of
device_tree_flattened) is created and updated with overlay nodes. This
updated fdt is further unflattened to a dt_host_new. Next, it checks if any
of the overlay nodes already exists in the dt_host. If overlay nodes doesn't
exist then find the overlay nodes in dt_host_new, find the overlay node's
parent in dt_host and add the nodes as child under their parent in the
dt_host. The node is attached as the last node under target parent.
Finally, add IRQs, add device to IOMMUs, set permissions and map MMIO for the
overlay node.
When a node is added using overlay, a new entry is allocated in the
overlay_track to keep the track of memory allocation due to addition of overlay
node. This is helpful for freeing the memory allocated when a device tree node
is removed.
The main purpose of this to address first part of dynamic programming i.e.
making xen aware of new device tree node which means updating the dt_host with
overlay node information. Here we are adding/removing node from dt_host, and
checking/setting IOMMU and IRQ permission but never mapping them to any domain.
Right now, mapping/Un-mapping will happen only when a new domU is
created/destroyed using "xl create".
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
xen/common/dt_overlay.c | 482 ++++++++++++++++++++++++++++++++++++++++
1 file changed, 482 insertions(+)
diff --git a/xen/common/dt_overlay.c b/xen/common/dt_overlay.c
index 516e8010c5..3344bad313 100644
--- a/xen/common/dt_overlay.c
+++ b/xen/common/dt_overlay.c
@@ -36,6 +36,25 @@ static struct dt_device_node *find_last_descendants_node(
return child_node;
}
+/*
+ * Returns next node to the input node. If node has children then return
+ * last descendant's next node.
+*/
+static struct dt_device_node *dt_find_next_node(struct dt_device_node *dt,
+ const struct dt_device_node *node)
+{
+ struct dt_device_node *np;
+
+ dt_for_each_device_node(dt, np)
+ if ( np == node )
+ break;
+
+ if ( np->child )
+ np = find_last_descendants_node(np);
+
+ return np->allnext;
+}
+
static int dt_overlay_remove_node(struct dt_device_node *device_node)
{
struct dt_device_node *np;
@@ -109,6 +128,72 @@ static int dt_overlay_remove_node(struct dt_device_node *device_node)
return 0;
}
+static int dt_overlay_add_node(struct dt_device_node *device_node,
+ const char *parent_node_path)
+{
+ struct dt_device_node *parent_node;
+ struct dt_device_node *np, *np_last_descendant;
+ struct dt_device_node *next_node;
+ struct dt_device_node *device_node_last_descendant;
+
+ parent_node = dt_find_node_by_path(parent_node_path);
+
+ if ( parent_node == NULL )
+ {
+ dt_dprintk("Node not found. Overlay node will not be added\n");
+ return -EINVAL;
+ }
+
+ /* If parent has no child. */
+ if ( parent_node->child == NULL )
+ {
+ next_node = parent_node->allnext;
+ device_node->parent = parent_node;
+ parent_node->allnext = device_node;
+ parent_node->child = device_node;
+ }
+ else
+ {
+ /* If parent has at least one child node.
+ * Iterate to the last child node of parent.
+ */
+ for ( np = parent_node->child; np->sibling != NULL; np = np->sibling );
+
+ /* Iterate over all child nodes of np node. */
+ if ( np->child )
+ {
+ np_last_descendant = find_last_descendants_node(np);
+
+ next_node = np_last_descendant->allnext;
+ np_last_descendant->allnext = device_node;
+ }
+ else
+ {
+ next_node = np->allnext;
+ np->allnext = device_node;
+ }
+
+ device_node->parent = parent_node;
+ np->sibling = device_node;
+ np->sibling->sibling = NULL;
+ }
+
+ /* Iterate over all child nodes of device_node to add children too. */
+ if ( device_node->child )
+ {
+ device_node_last_descendant = find_last_descendants_node(device_node);
+ /* Plug next_node at the end of last children of device_node. */
+ device_node_last_descendant->allnext = next_node;
+ }
+ else
+ {
+ /* Now plug next_node at the end of device_node. */
+ device_node->allnext = next_node;
+ }
+
+ return 0;
+}
+
/* Basic sanity check for the dtbo tool stack provided to Xen. */
static int check_overlay_fdt(const void *overlay_fdt, uint32_t overlay_fdt_size)
{
@@ -148,6 +233,79 @@ static unsigned int overlay_node_count(void *fdto)
return num_overlay_nodes;
}
+/*
+ * overlay_get_nodes_info will get full name with path for all the nodes which
+ * are in one level of __overlay__ tag. This is useful when checking node for
+ * duplication i.e. dtbo tries to add nodes which already exists in device tree.
+ */
+static int overlay_get_nodes_info(const void *fdto, char ***nodes_full_path,
+ unsigned int num_overlay_nodes)
+{
+ int fragment;
+
+ *nodes_full_path = xzalloc_bytes(num_overlay_nodes * sizeof(char *));
+
+ if ( *nodes_full_path == NULL )
+ return -ENOMEM;
+
+ fdt_for_each_subnode(fragment, fdto, 0)
+ {
+ int target;
+ int overlay;
+ int subnode;
+ const char *target_path;
+
+ target = fdt_overlay_target_offset(device_tree_flattened, fdto,
+ fragment, &target_path);
+ if ( target < 0 )
+ return target;
+
+ overlay = fdt_subnode_offset(fdto, fragment, "__overlay__");
+
+ /*
+ * overlay value can be < 0. But fdt_for_each_subnode() loop checks for
+ * overlay >= 0. So, no need for a overlay>=0 check here.
+ */
+ fdt_for_each_subnode(subnode, fdto, overlay)
+ {
+ const char *node_name = NULL;
+ int node_name_len;
+ unsigned int target_path_len = strlen(target_path);
+ unsigned int node_full_name_len;
+ unsigned int node_num = 0;
+
+ node_name = fdt_get_name(fdto, subnode, &node_name_len);
+
+ if ( node_name == NULL )
+ return node_name_len;
+
+ /*
+ * Magic number 2 is for adding '/'. This is done to keep the
+ * node_full_name in the correct full node name format.
+ */
+ node_full_name_len = target_path_len + node_name_len + 2;
+
+ (*nodes_full_path)[node_num] = xmalloc_bytes(node_full_name_len);
+
+ if ( (*nodes_full_path)[node_num] == NULL )
+ return -ENOMEM;
+
+ memcpy((*nodes_full_path)[node_num], target_path, target_path_len);
+
+ (*nodes_full_path)[node_num][target_path_len] = '/';
+
+ memcpy((*nodes_full_path)[node_num] + target_path_len + 1,
+ node_name, node_name_len);
+
+ (*nodes_full_path)[node_num][node_full_name_len - 1] = '\0';
+
+ node_num++;
+ }
+ }
+
+ return 0;
+}
+
static int handle_remove_irq_iommu(struct dt_device_node *device_node)
{
int rc = 0;
@@ -367,6 +525,322 @@ out:
return rc;
}
+/*
+ * Handles IRQ and IOMMU mapping for the overlay_node and all descendants of the
+ * overlay_nodes.
+ */
+static int handle_add_irq_iommu(struct domain *d,
+ struct dt_device_node *overlay_node)
+{
+ int rc;
+ unsigned int naddr, i, len;
+ uint64_t addr, size;
+ struct dt_device_node *np;
+
+ /* First let's handle the interrupts. */
+ rc = handle_device_interrupts(d, overlay_node, false);
+ if ( rc < 0 )
+ {
+ printk(XENLOG_ERR "Interrupt failed\n");
+ return rc;
+ }
+
+ /* Check if iommu property exists. */
+ if ( dt_get_property(overlay_node, "iommus", &len) )
+ {
+
+ /* Add device to IOMMUs. */
+ rc = iommu_add_dt_device(overlay_node);
+ if ( rc < 0 )
+ {
+ printk(XENLOG_ERR "Failed to add %s to the IOMMU\n",
+ dt_node_full_name(overlay_node));
+ return rc;
+ }
+ }
+
+ /* Set permissions. */
+ naddr = dt_number_of_address(overlay_node);
+
+ dt_dprintk("%s passthrough = %d naddr = %u\n",
+ dt_node_full_name(overlay_node), false, naddr);
+
+ /* Give permission for map MMIOs */
+ for ( i = 0; i < naddr; i++ )
+ {
+ /*
+ * For now, we skip_mapping which means it will only permit iomem access
+ * to hardware_domain using iomem_permit_access() but will never map as
+ * map_range_p2mt() will not be called.
+ */
+ struct map_range_data mr_data = { .d = d,
+ .p2mt = p2m_mmio_direct_c,
+ .skip_mapping = true };
+
+ rc = dt_device_get_address(overlay_node, i, &addr, &size);
+ if ( rc )
+ {
+ printk(XENLOG_ERR "Unable to retrieve address %u for %s\n",
+ i, dt_node_full_name(overlay_node));
+ return rc;
+ }
+
+ rc = map_range_to_domain(overlay_node, addr, size, &mr_data);
+ if ( rc )
+ return rc;
+ }
+
+ /* Map IRQ and IOMMU for overlay_node's children. */
+ for ( np = overlay_node->child; np != NULL; np = np->sibling)
+ {
+ rc = handle_add_irq_iommu(d, np);
+ if ( rc )
+ {
+ printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n");
+ return rc;
+ }
+ }
+
+ return rc;
+}
+
+/*
+ * Adds device tree nodes under target node.
+ * We use tr->dt_host_new to unflatten the updated device_tree_flattened. This
+ * is done to avoid the removal of device_tree generation, iomem regions mapping
+ * to hardware domain done by handle_node().
+ */
+static long handle_add_overlay_nodes(void *overlay_fdt,
+ uint32_t overlay_fdt_size)
+{
+ int rc, j, i;
+ struct dt_device_node *overlay_node, *prev_node, *next_node;
+ struct domain *d = hardware_domain;
+ struct overlay_track *tr = NULL;
+ char **nodes_full_path = NULL;
+ unsigned int new_fdt_size;
+
+ tr = xzalloc(struct overlay_track);
+ if ( tr == NULL )
+ return -ENOMEM;
+
+ new_fdt_size = fdt_totalsize(device_tree_flattened) +
+ fdt_totalsize(overlay_fdt);
+
+ tr->fdt = xzalloc_bytes(new_fdt_size);
+ if ( tr->fdt == NULL )
+ {
+ xfree(tr);
+ return -ENOMEM;
+ }
+
+ tr->num_nodes = overlay_node_count(overlay_fdt);
+ if ( tr->num_nodes == 0 )
+ {
+ xfree(tr->fdt);
+ xfree(tr);
+ return -ENOMEM;
+ }
+
+ tr->nodes_address = xzalloc_bytes(tr->num_nodes * sizeof(unsigned long));
+ if ( tr->nodes_address == NULL )
+ {
+ xfree(tr->fdt);
+ xfree(tr);
+ return -ENOMEM;
+ }
+
+ rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
+ if ( rc )
+ {
+ xfree(tr->nodes_address);
+ xfree(tr->fdt);
+ xfree(tr);
+ return rc;
+ }
+
+ /*
+ * Keep a copy of overlay_fdt as fdt_overlay_apply will change the input
+ * overlay's content(magic) when applying overlay.
+ */
+ tr->overlay_fdt = xzalloc_bytes(overlay_fdt_size);
+ if ( tr->overlay_fdt == NULL )
+ {
+ xfree(tr->nodes_address);
+ xfree(tr->fdt);
+ xfree(tr);
+ return -ENOMEM;
+ }
+
+ memcpy(tr->overlay_fdt, overlay_fdt, overlay_fdt_size);
+
+ spin_lock(&overlay_lock);
+
+ memcpy(tr->fdt, device_tree_flattened,
+ fdt_totalsize(device_tree_flattened));
+
+ /* Open tr->fdt with more space to accommodate the overlay_fdt. */
+ fdt_open_into(tr->fdt, tr->fdt, new_fdt_size);
+
+ /*
+ * overlay_get_nodes_info is called to get the node information from dtbo.
+ * This is done before fdt_overlay_apply() because the overlay apply will
+ * erase the magic of overlay_fdt.
+ */
+ rc = overlay_get_nodes_info(overlay_fdt, &nodes_full_path,
+ tr->num_nodes);
+ if ( rc )
+ {
+ printk(XENLOG_ERR "Getting nodes information failed with error %d\n",
+ rc);
+ goto err;
+ }
+
+ rc = fdt_overlay_apply(tr->fdt, overlay_fdt);
+ if ( rc )
+ {
+ printk(XENLOG_ERR "Adding overlay node failed with error %d\n", rc);
+ goto err;
+ }
+
+ /*
+ * Check if any of the node already exists in dt_host. If node already exits
+ * we can return here as this overlay_fdt is not suitable for overlay ops.
+ */
+ for ( j = 0; j < tr->num_nodes; j++ )
+ {
+ overlay_node = dt_find_node_by_path(nodes_full_path[j]);
+ if ( overlay_node != NULL )
+ {
+ printk(XENLOG_ERR "node %s exists in device tree\n",
+ nodes_full_path[j]);
+ rc = -EINVAL;
+ goto err;
+ }
+ }
+
+ /* Unflatten the tr->fdt into a new dt_host. */
+ unflatten_device_tree(tr->fdt, &tr->dt_host_new);
+
+ for ( j = 0; j < tr->num_nodes; j++ )
+ {
+ dt_dprintk("Adding node: %s\n", nodes_full_path[j]);
+
+ /* Find the newly added node in tr->dt_host_new by it's full path. */
+ overlay_node = device_tree_find_node_by_path(tr->dt_host_new,
+ nodes_full_path[j]);
+ if ( overlay_node == NULL )
+ {
+ dt_dprintk("%s node not found\n", nodes_full_path[j]);
+ rc = -EFAULT;
+ goto remove_node;
+ }
+
+ /*
+ * Find previous and next node to overlay_node in dt_host_new. We will
+ * need these nodes to fix the dt_host_new mapping. When overlay_node is
+ * take out of dt_host_new tree and added to dt_host, link between
+ * previous node and next_node is broken. We will need to refresh
+ * dt_host_new with correct linking for any other overlay nodes
+ * extraction in future.
+ */
+ dt_for_each_device_node(tr->dt_host_new, prev_node)
+ if ( prev_node->allnext == overlay_node )
+ break;
+
+ next_node = dt_find_next_node(tr->dt_host_new, overlay_node);
+
+ read_lock(&dt_host->lock);
+
+ /* Add the node to dt_host. */
+ rc = dt_overlay_add_node(overlay_node, overlay_node->parent->full_name);
+ if ( rc )
+ {
+ read_unlock(&dt_host->lock);
+
+ /* Node not added in dt_host. */
+ goto remove_node;
+ }
+
+ read_unlock(&dt_host->lock);
+
+ prev_node->allnext = next_node;
+
+ overlay_node = dt_find_node_by_path(overlay_node->full_name);
+ if ( overlay_node == NULL )
+ {
+ /* Sanity check. But code will never come here. */
+ ASSERT_UNREACHABLE();
+ goto remove_node;
+ }
+
+ rc = handle_add_irq_iommu(d, overlay_node);
+ if ( rc )
+ {
+ printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n");
+ return rc;
+ }
+
+ /* Keep overlay_node address in tracker. */
+ tr->nodes_address[j] = (unsigned long)overlay_node;
+ }
+
+ INIT_LIST_HEAD(&tr->entry);
+ list_add_tail(&tr->entry, &overlay_tracker);
+
+ spin_unlock(&overlay_lock);
+
+ if ( nodes_full_path != NULL )
+ {
+ for ( i = 0; i < tr->num_nodes && nodes_full_path[i] != NULL;
+ i++ )
+ {
+ xfree(nodes_full_path[i]);
+ }
+ xfree(nodes_full_path);
+ }
+
+ return rc;
+
+/*
+ * Failure case. We need to remove the nodes, free tracker(if tr exists) and
+ * tr->dt_host_new.
+ */
+remove_node:
+ tr->num_nodes = j;
+ rc = remove_nodes(tr);
+
+ if ( rc )
+ {
+ /* If removing node fails, this may cause memory leaks. */
+ printk(XENLOG_ERR "Removing node failed.\n");
+ spin_unlock(&overlay_lock);
+ return rc;
+ }
+
+err:
+ spin_unlock(&overlay_lock);
+
+ xfree(tr->dt_host_new);
+ xfree(tr->fdt);
+ xfree(tr->overlay_fdt);
+ xfree(tr->nodes_address);
+
+ if ( nodes_full_path != NULL )
+ {
+ for ( i = 0; i < tr->num_nodes && nodes_full_path[i] != NULL;
+ i++ )
+ {
+ xfree(nodes_full_path[i]);
+ }
+ xfree(nodes_full_path);
+ }
+
+ xfree(tr);
+
+ return rc;
+}
+
long dt_sysctl(struct xen_sysctl_dt_overlay *op)
{
long ret;
@@ -391,6 +865,14 @@ long dt_sysctl(struct xen_sysctl_dt_overlay *op)
switch ( op->overlay_op )
{
+ case XEN_SYSCTL_DT_OVERLAY_ADD:
+ ret = handle_add_overlay_nodes(overlay_fdt, op->overlay_fdt_size);
+
+ if ( ret )
+ xfree(overlay_fdt);
+
+ break;
+
case XEN_SYSCTL_DT_OVERLAY_REMOVE:
ret = handle_remove_overlay_nodes(overlay_fdt, op->overlay_fdt_size);
xfree(overlay_fdt);
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 15/17] tools/libs/ctrl: Implement new xc interfaces for dt overlay
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (13 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 14/17] xen/arm: Implement device tree node addition functionalities Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 16/17] tools/libs/light: Implement new libxl functions for device tree overlay ops Vikram Garhwal
` (19 subsequent siblings)
34 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel
Cc: sstabellini, Vikram Garhwal, Wei Liu, Anthony PERARD, Juergen Gross
xc_dt_overlay() sends the device tree binary overlay, size of .dtbo and overlay
operation type i.e. add or remove to xen.
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
tools/include/xenctrl.h | 5 ++++
tools/libs/ctrl/Makefile.common | 1 +
tools/libs/ctrl/xc_dt_overlay.c | 48 +++++++++++++++++++++++++++++++++
3 files changed, 54 insertions(+)
create mode 100644 tools/libs/ctrl/xc_dt_overlay.c
diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h
index 05967ecc92..b932589c8d 100644
--- a/tools/include/xenctrl.h
+++ b/tools/include/xenctrl.h
@@ -2637,6 +2637,11 @@ int xc_livepatch_replace(xc_interface *xch, char *name, uint32_t timeout, uint32
int xc_domain_cacheflush(xc_interface *xch, uint32_t domid,
xen_pfn_t start_pfn, xen_pfn_t nr_pfns);
+#if defined(__arm__) || defined(__aarch64__)
+int xc_dt_overlay(xc_interface *xch, void *overlay_fdt,
+ uint32_t overlay_fdt_size, uint8_t overlay_op);
+#endif
+
/* Compat shims */
#include "xenctrl_compat.h"
diff --git a/tools/libs/ctrl/Makefile.common b/tools/libs/ctrl/Makefile.common
index 0a09c28fd3..247afbe5f9 100644
--- a/tools/libs/ctrl/Makefile.common
+++ b/tools/libs/ctrl/Makefile.common
@@ -24,6 +24,7 @@ OBJS-y += xc_hcall_buf.o
OBJS-y += xc_foreign_memory.o
OBJS-y += xc_kexec.o
OBJS-y += xc_resource.o
+OBJS-$(CONFIG_ARM) += xc_dt_overlay.o
OBJS-$(CONFIG_X86) += xc_psr.o
OBJS-$(CONFIG_X86) += xc_pagetab.o
OBJS-$(CONFIG_Linux) += xc_linux.o
diff --git a/tools/libs/ctrl/xc_dt_overlay.c b/tools/libs/ctrl/xc_dt_overlay.c
new file mode 100644
index 0000000000..202fc906f4
--- /dev/null
+++ b/tools/libs/ctrl/xc_dt_overlay.c
@@ -0,0 +1,48 @@
+/*
+ *
+ * Device Tree Overlay functions.
+ * Copyright (C) 2021 Xilinx Inc.
+ * Author Vikram Garhwal <fnu.vikram@xilinx.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation;
+ * version 2.1 of the License.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "xc_private.h"
+
+int xc_dt_overlay(xc_interface *xch, void *overlay_fdt,
+ uint32_t overlay_fdt_size, uint8_t overlay_op)
+{
+ int err;
+ DECLARE_SYSCTL;
+
+ DECLARE_HYPERCALL_BOUNCE(overlay_fdt, overlay_fdt_size,
+ XC_HYPERCALL_BUFFER_BOUNCE_IN);
+
+ if ( (err = xc_hypercall_bounce_pre(xch, overlay_fdt)) )
+ goto err;
+
+ sysctl.cmd = XEN_SYSCTL_dt_overlay;
+ sysctl.u.dt_overlay.overlay_op = overlay_op;
+ sysctl.u.dt_overlay.overlay_fdt_size = overlay_fdt_size;
+
+ set_xen_guest_handle(sysctl.u.dt_overlay.overlay_fdt, overlay_fdt);
+
+ if ( (err = do_sysctl(xch, &sysctl)) != 0 )
+ PERROR("%s failed", __func__);
+
+err:
+ xc_hypercall_bounce_post(xch, overlay_fdt);
+
+ return err;
+}
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 16/17] tools/libs/light: Implement new libxl functions for device tree overlay ops
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (14 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 15/17] tools/libs/ctrl: Implement new xc interfaces for dt overlay Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 17/17] tools/xl: Add new xl command overlay for device tree overlay support Vikram Garhwal
` (18 subsequent siblings)
34 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel
Cc: sstabellini, Vikram Garhwal, Wei Liu, Anthony PERARD, Juergen Gross
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
tools/include/libxl.h | 11 +++++
tools/libs/light/Makefile | 3 ++
tools/libs/light/libxl_dt_overlay.c | 71 +++++++++++++++++++++++++++++
3 files changed, 85 insertions(+)
create mode 100644 tools/libs/light/libxl_dt_overlay.c
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index cfa1a19131..1c5e8abaae 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -250,6 +250,12 @@
*/
#define LIBXL_HAVE_DEVICETREE_PASSTHROUGH 1
+#if defined(__arm__) || defined(__aarch64__)
+/**
+ * This means Device Tree Overlay is supported.
+ */
+#define LIBXL_HAVE_DT_OVERLAY 1
+#endif
/*
* libxl_domain_build_info has device_model_user to specify the user to
* run the device model with. See docs/misc/qemu-deprivilege.txt.
@@ -2453,6 +2459,11 @@ libxl_device_pci *libxl_device_pci_list(libxl_ctx *ctx, uint32_t domid,
int *num);
void libxl_device_pci_list_free(libxl_device_pci* list, int num);
+#if defined(__arm__) || defined(__aarch64__)
+int libxl_dt_overlay(libxl_ctx *ctx, void *overlay,
+ uint32_t overlay_size, uint8_t overlay_op);
+#endif
+
/*
* Turns the current process into a backend device service daemon
* for a driver domain.
diff --git a/tools/libs/light/Makefile b/tools/libs/light/Makefile
index 96daeabc47..563a1e8d0a 100644
--- a/tools/libs/light/Makefile
+++ b/tools/libs/light/Makefile
@@ -112,6 +112,9 @@ OBJS-y += _libxl_types.o
OBJS-y += libxl_flask.o
OBJS-y += _libxl_types_internal.o
+# Device tree overlay is enabled only for ARM architecture.
+OBJS-$(CONFIG_ARM) += libxl_dt_overlay.o
+
ifeq ($(CONFIG_LIBNL),y)
CFLAGS_LIBXL += $(LIBNL3_CFLAGS)
endif
diff --git a/tools/libs/light/libxl_dt_overlay.c b/tools/libs/light/libxl_dt_overlay.c
new file mode 100644
index 0000000000..a6c709a6dc
--- /dev/null
+++ b/tools/libs/light/libxl_dt_overlay.c
@@ -0,0 +1,71 @@
+/*
+ * Copyright (C) 2021 Xilinx Inc.
+ * Author Vikram Garhwal <fnu.vikram@xilinx.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as published
+ * by the Free Software Foundation; version 2.1 only. with the special
+ * exception on linking described in file LICENSE.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU Lesser General Public License for more details.
+ */
+
+#include "libxl_osdeps.h" /* must come before any other headers */
+#include "libxl_internal.h"
+#include <libfdt.h>
+#include <xenctrl.h>
+
+static int check_overlay_fdt(libxl__gc *gc, void *fdt, size_t size)
+{
+ int r;
+
+ if (fdt_magic(fdt) != FDT_MAGIC) {
+ LOG(ERROR, "Overlay FDT is not a valid Flat Device Tree");
+ return ERROR_FAIL;
+ }
+
+ r = fdt_check_header(fdt);
+ if (r) {
+ LOG(ERROR, "Failed to check the overlay FDT (%d)", r);
+ return ERROR_FAIL;
+ }
+
+ if (fdt_totalsize(fdt) > size) {
+ LOG(ERROR, "Overlay FDT totalsize is too big");
+ return ERROR_FAIL;
+ }
+
+ return 0;
+}
+
+int libxl_dt_overlay(libxl_ctx *ctx, void *overlay_dt, uint32_t overlay_dt_size,
+ uint8_t overlay_op)
+{
+ int rc;
+ int r;
+ GC_INIT(ctx);
+
+ if (check_overlay_fdt(gc, overlay_dt, overlay_dt_size)) {
+ LOG(ERROR, "Overlay DTB check failed");
+ rc = ERROR_FAIL;
+ goto out;
+ } else {
+ LOG(DEBUG, "Overlay DTB check passed");
+ rc = 0;
+ }
+
+ r = xc_dt_overlay(ctx->xch, overlay_dt, overlay_dt_size, overlay_op);
+
+ if (r) {
+ LOG(ERROR, "%s: Adding/Removing overlay dtb failed.", __func__);
+ rc = ERROR_FAIL;
+ }
+
+out:
+ GC_FREE;
+ return rc;
+}
+
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 17/17] tools/xl: Add new xl command overlay for device tree overlay support
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (15 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 16/17] tools/libs/light: Implement new libxl functions for device tree overlay ops Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (17 subsequent siblings)
34 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel; +Cc: sstabellini, Vikram Garhwal, Wei Liu, Anthony PERARD
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
tools/xl/xl.h | 1 +
tools/xl/xl_cmdtable.c | 6 +++++
tools/xl/xl_vmcontrol.c | 52 +++++++++++++++++++++++++++++++++++++++++
3 files changed, 59 insertions(+)
diff --git a/tools/xl/xl.h b/tools/xl/xl.h
index 72538d6a81..a923daccd3 100644
--- a/tools/xl/xl.h
+++ b/tools/xl/xl.h
@@ -138,6 +138,7 @@ int main_shutdown(int argc, char **argv);
int main_reboot(int argc, char **argv);
int main_list(int argc, char **argv);
int main_vm_list(int argc, char **argv);
+int main_dt_overlay(int argc, char **argv);
int main_create(int argc, char **argv);
int main_config_update(int argc, char **argv);
int main_button_press(int argc, char **argv);
diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index ccf4d83584..db0acff62a 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -630,6 +630,12 @@ const struct cmd_spec cmd_table[] = {
"Issue a qemu monitor command to the device model of a domain",
"<Domain> <Command>",
},
+ { "dt-overlay",
+ &main_dt_overlay, 0, 1,
+ "Add/Remove a device tree overlay",
+ "add/remove <.dtbo>"
+ "-h print this help\n"
+ },
};
const int cmdtable_len = ARRAY_SIZE(cmd_table);
diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c
index 5518c78dc6..de56e00d8b 100644
--- a/tools/xl/xl_vmcontrol.c
+++ b/tools/xl/xl_vmcontrol.c
@@ -1265,6 +1265,58 @@ int main_create(int argc, char **argv)
return 0;
}
+int main_dt_overlay(int argc, char **argv)
+{
+ const char *overlay_ops = NULL;
+ const char *overlay_config_file = NULL;
+ void *overlay_dtb = NULL;
+ int rc;
+ uint8_t op;
+ int overlay_dtb_size = 0;
+ const int overlay_add_op = 1;
+ const int overlay_remove_op = 2;
+
+ if (argc < 2) {
+ help("dt_overlay");
+ return EXIT_FAILURE;
+ }
+
+ overlay_ops = argv[1];
+ overlay_config_file = argv[2];
+
+ if (strcmp(overlay_ops, "add") == 0)
+ op = overlay_add_op;
+ else if (strcmp(overlay_ops, "remove") == 0)
+ op = overlay_remove_op;
+ else {
+ fprintf(stderr, "Invalid dt overlay operation\n");
+ return EXIT_FAILURE;
+ }
+
+ if (overlay_config_file) {
+ rc = libxl_read_file_contents(ctx, overlay_config_file,
+ &overlay_dtb, &overlay_dtb_size);
+
+ if (rc) {
+ fprintf(stderr, "failed to read the overlay device tree file %s\n",
+ overlay_config_file);
+ free(overlay_dtb);
+ return ERROR_FAIL;
+ }
+ } else {
+ fprintf(stderr, "overlay dtbo file not provided\n");
+ return ERROR_FAIL;
+ }
+
+ rc = libxl_dt_overlay(ctx, overlay_dtb, overlay_dtb_size, op);
+
+ free(overlay_dtb);
+
+ if (rc)
+ return EXIT_FAILURE;
+
+ return rc;
+}
/*
* Local variables:
* mode: C
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (16 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 17/17] tools/xl: Add new xl command overlay for device tree overlay support Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 01/17] xen/arm/device: Remove __init from function type Vikram Garhwal
` (16 subsequent siblings)
34 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel
Cc: sstabellini, Vikram Garhwal, Julien Grall, Bertrand Marquis,
Volodymyr Babchuk, Andrew Cooper, George Dunlap, Jan Beulich,
Wei Liu, Rahul Singh, Paul Durrant, Roger Pau Monné,
Anthony PERARD, Juergen Gross
Hi,
The main purpose of this series to address first part of the dynamic programming
i.e. making Xen aware of new device tree node which means updating the dt_host
with overlay node information. Here we are adding/removing node from dt_host,
and checking/set IOMMU and IRQ permission but never mapping them to any domain.
Right now, mapping/Un-mapping will happen only when a new domU is
created/destroyed using "xl create".
For adding a node using dynamic programming:
1. flatten device tree overlay node will be added to a fdt
2. Updated fdt will be unflattened to a new dt_host_new
3. Extract the newly added node information from dt_host_new
4. Add the added node under correct parent in original dt_host.
For removing a node:
1. Find the node with given path.
2. Check if the node is used by any of domus. Removes the node only when
it's not used by any domain.
3. Removes IRQ permissions and MMIO access.
5. Find the node in dt_host and delete the device node entry from dt_host.
6. Free the overlay_tracker entry which means free dt_host_new also(created
in adding node step).
To map IOREQ and IOMMU during runtime, there will be another small series after
this one where we will do the actual IOMMU and IRQ mapping to a running domain
and will call unmap_mmio_regions() to remove the mapping.
Change Log:
v4 -> v5:
Split patch 01/16 to two patches. One with function type changes and another
with changes inside unflatten_device_tree().
Change dt_overlay xl command to dt-overlay.
Protect overlay functionality with CONFIG(arm).
Fix rwlock issues.
Move include "device_tree.h" to c file where arch_cpu_init() is called and
forward declare dt_device_node. This was done to avoid circular deps b/w
device_tree.h and rwlock.h
Address Michal's comment on coding style.
v3 -> v4:
Add support for adding node's children.
Add rwlock to dt_host functions.
Corrected fdt size issue when applying overlay into it.
Add memory allocation fail handling for unflatten_device_tree().
Changed xl overlay to xl dt_overlay.
Correct commit messages.
Addressed code issue from v3 review.
v2 -> v3:
Moved overlay functionalities to dt_overlay.c file.
Renamed XEN_SYSCTL_overlay to XEN_SYSCTL_dt_overlay.
Add dt_* prefix to overlay_add/remove_nodes.
Added dtdevs_lock to protect iommu_add_dt_device().
For iommu, moved spin_lock to caller.
Address code issue from v2 review.
v1 -> v2:
Add support for multiple node addition/removal using dtbo.
Replaced fpga-add and fpga-remove with one hypercall overlay_op.
Moved common domain_build.c function to device.c
Add OVERLAY_DTB configuration.
Renamed overlay_get_target() to fdt_overlay_get_target().
Split remove_device patch into two patches.
Moved overlay_add/remove code to sysctl and changed it from domctl to sysctl.
Added all overlay code under CONFIG_OVERLAY_DTB
Renamed all tool domains fpga function to overlay
Addressed code issues from v1 review.
Regards,
Vikram
Vikram Garhwal (17):
xen/arm/device: Remove __init from function type
common/device_tree: change __unflatten_device_tree()
xen/arm: Add CONFIG_OVERLAY_DTB
libfdt: Keep fdt functions after init for CONFIG_OVERLAY_DTB.
libfdt: overlay: change overlay_get_target()
xen/device-tree: Add device_tree_find_node_by_path() to find nodes in
device tree
xen/smmu: Add remove_device callback for smmu_iommu ops
xen/iommu: Move spin_lock from iommu_dt_device_is_assigned to caller
xen/iommu: protect iommu_add_dt_device() with dtdevs_lock
xen/iommu: Introduce iommu_remove_dt_device()
asm/smp.h: Fix circular dependency for device_tree.h and rwlock.h
common/device_tree: Add rwlock for dt_host
xen/arm: Implement device tree node removal functionalities
xen/arm: Implement device tree node addition functionalities
tools/libs/ctrl: Implement new xc interfaces for dt overlay
tools/libs/light: Implement new libxl functions for device tree
overlay ops
tools/xl: Add new xl command overlay for device tree overlay support
SUPPORT.md | 6 +
tools/include/libxl.h | 11 +
tools/include/xenctrl.h | 5 +
tools/libs/ctrl/Makefile.common | 1 +
tools/libs/ctrl/xc_dt_overlay.c | 48 ++
tools/libs/light/Makefile | 3 +
tools/libs/light/libxl_dt_overlay.c | 71 ++
tools/xl/xl.h | 1 +
tools/xl/xl_cmdtable.c | 6 +
tools/xl/xl_vmcontrol.c | 52 ++
xen/arch/arm/Kconfig | 5 +
xen/arch/arm/device.c | 145 ++++
xen/arch/arm/domain_build.c | 142 ----
xen/arch/arm/include/asm/domain_build.h | 2 -
xen/arch/arm/include/asm/setup.h | 6 +
xen/arch/arm/include/asm/smp.h | 3 +-
xen/arch/arm/smpboot.c | 1 +
xen/arch/arm/sysctl.c | 16 +-
xen/common/Makefile | 1 +
xen/common/device_tree.c | 30 +-
xen/common/dt_overlay.c | 897 ++++++++++++++++++++++++
xen/common/libfdt/Makefile | 4 +
xen/common/libfdt/fdt_overlay.c | 29 +-
xen/common/libfdt/version.lds | 1 +
xen/drivers/passthrough/arm/smmu.c | 56 ++
xen/drivers/passthrough/device_tree.c | 109 ++-
xen/include/public/sysctl.h | 24 +
xen/include/xen/device_tree.h | 28 +-
xen/include/xen/dt_overlay.h | 59 ++
xen/include/xen/iommu.h | 2 +
xen/include/xen/libfdt/libfdt.h | 18 +
31 files changed, 1595 insertions(+), 187 deletions(-)
create mode 100644 tools/libs/ctrl/xc_dt_overlay.c
create mode 100644 tools/libs/light/libxl_dt_overlay.c
create mode 100644 xen/common/dt_overlay.c
create mode 100644 xen/include/xen/dt_overlay.h
--
2.17.1
^ permalink raw reply [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 01/17] xen/arm/device: Remove __init from function type
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (17 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 02/17] common/device_tree: change __unflatten_device_tree() Vikram Garhwal
` (15 subsequent siblings)
34 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel
Cc: sstabellini, Vikram Garhwal, Julien Grall, Bertrand Marquis,
Volodymyr Babchuk
Remove __init from following function to access during runtime:
1. map_irq_to_domain()
2. handle_device_interrupt()
3. map_range_to_domain()
4. unflatten_dt_node()
5. unflatten_device_tree()
Move map_irq_to_domain() prototype from domain_build.h to setup.h.
To avoid the breaking the build, following changes are also done:
1. Move map_irq_to_domain(), handle_device_interrupt() and map_range_to_domain() to
device.c. After removing __init type, these functions are not specific to
domain building, so moving them out of domain_build.c to device.c.
2. Remove static type from handle_device_interrupt().
Overall, these changes are done to support the dynamic programming of a nodes
where an overlay node will be added to fdt and unflattened node will be added to
dt_host. Furthermore, IRQ and mmio mapping will be done for the added node.
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
xen/arch/arm/device.c | 145 ++++++++++++++++++++++++
xen/arch/arm/domain_build.c | 142 -----------------------
xen/arch/arm/include/asm/domain_build.h | 2 -
xen/arch/arm/include/asm/setup.h | 6 +
xen/common/device_tree.c | 16 +--
5 files changed, 159 insertions(+), 152 deletions(-)
diff --git a/xen/arch/arm/device.c b/xen/arch/arm/device.c
index ca8539dee5..fec6e29c42 100644
--- a/xen/arch/arm/device.c
+++ b/xen/arch/arm/device.c
@@ -12,6 +12,9 @@
#include <xen/errno.h>
#include <xen/init.h>
#include <xen/lib.h>
+#include <xen/iocap.h>
+#include <asm/domain_build.h>
+#include <asm/setup.h>
extern const struct device_desc _sdevice[], _edevice[];
extern const struct acpi_device_desc _asdevice[], _aedevice[];
@@ -75,6 +78,148 @@ enum device_class device_get_class(const struct dt_device_node *dev)
return DEVICE_UNKNOWN;
}
+int map_irq_to_domain(struct domain *d, unsigned int irq,
+ bool need_mapping, const char *devname)
+{
+ int res;
+
+ res = irq_permit_access(d, irq);
+ if ( res )
+ {
+ printk(XENLOG_ERR "Unable to permit to dom%u access to IRQ %u\n",
+ d->domain_id, irq);
+ return res;
+ }
+
+ if ( need_mapping )
+ {
+ /*
+ * Checking the return of vgic_reserve_virq is not
+ * necessary. It should not fail except when we try to map
+ * the IRQ twice. This can legitimately happen if the IRQ is shared
+ */
+ vgic_reserve_virq(d, irq);
+
+ res = route_irq_to_guest(d, irq, irq, devname);
+ if ( res < 0 )
+ {
+ printk(XENLOG_ERR "Unable to map IRQ%"PRId32" to dom%d\n",
+ irq, d->domain_id);
+ return res;
+ }
+ }
+
+ dt_dprintk(" - IRQ: %u\n", irq);
+ return 0;
+}
+
+int map_range_to_domain(const struct dt_device_node *dev,
+ u64 addr, u64 len, void *data)
+{
+ struct map_range_data *mr_data = data;
+ struct domain *d = mr_data->d;
+ int res;
+
+ /*
+ * reserved-memory regions are RAM carved out for a special purpose.
+ * They are not MMIO and therefore a domain should not be able to
+ * manage them via the IOMEM interface.
+ */
+ if ( strncasecmp(dt_node_full_name(dev), "/reserved-memory/",
+ strlen("/reserved-memory/")) != 0 )
+ {
+ res = iomem_permit_access(d, paddr_to_pfn(addr),
+ paddr_to_pfn(PAGE_ALIGN(addr + len - 1)));
+ if ( res )
+ {
+ printk(XENLOG_ERR "Unable to permit to dom%d access to"
+ " 0x%"PRIx64" - 0x%"PRIx64"\n",
+ d->domain_id,
+ addr & PAGE_MASK, PAGE_ALIGN(addr + len) - 1);
+ return res;
+ }
+ }
+
+ if ( !mr_data->skip_mapping )
+ {
+ res = map_regions_p2mt(d,
+ gaddr_to_gfn(addr),
+ PFN_UP(len),
+ maddr_to_mfn(addr),
+ mr_data->p2mt);
+
+ if ( res < 0 )
+ {
+ printk(XENLOG_ERR "Unable to map 0x%"PRIx64
+ " - 0x%"PRIx64" in domain %d\n",
+ addr & PAGE_MASK, PAGE_ALIGN(addr + len) - 1,
+ d->domain_id);
+ return res;
+ }
+ }
+
+ dt_dprintk(" - MMIO: %010"PRIx64" - %010"PRIx64" P2MType=%x\n",
+ addr, addr + len, mr_data->p2mt);
+
+ return 0;
+}
+
+/*
+ * handle_device_interrupts retrieves the interrupts configuration from
+ * a device tree node and maps those interrupts to the target domain.
+ *
+ * Returns:
+ * < 0 error
+ * 0 success
+ */
+int handle_device_interrupts(struct domain *d,
+ struct dt_device_node *dev,
+ bool need_mapping)
+{
+ unsigned int i, nirq;
+ int res;
+ struct dt_raw_irq rirq;
+
+ nirq = dt_number_of_irq(dev);
+
+ /* Give permission and map IRQs */
+ for ( i = 0; i < nirq; i++ )
+ {
+ res = dt_device_get_raw_irq(dev, i, &rirq);
+ if ( res )
+ {
+ printk(XENLOG_ERR "Unable to retrieve irq %u for %s\n",
+ i, dt_node_full_name(dev));
+ return res;
+ }
+
+ /*
+ * Don't map IRQ that have no physical meaning
+ * ie: IRQ whose controller is not the GIC
+ */
+ if ( rirq.controller != dt_interrupt_controller )
+ {
+ dt_dprintk("irq %u not connected to primary controller. Connected to %s\n",
+ i, dt_node_full_name(rirq.controller));
+ continue;
+ }
+
+ res = platform_get_irq(dev, i);
+ if ( res < 0 )
+ {
+ printk(XENLOG_ERR "Unable to get irq %u for %s\n",
+ i, dt_node_full_name(dev));
+ return res;
+ }
+
+ res = map_irq_to_domain(d, res, need_mapping, dt_node_name(dev));
+ if ( res )
+ return res;
+ }
+
+ return 0;
+}
+
/*
* Local variables:
* mode: C
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 4f9d4f9d88..6ab18c53ab 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2256,41 +2256,6 @@ int __init make_chosen_node(const struct kernel_info *kinfo)
return res;
}
-int __init map_irq_to_domain(struct domain *d, unsigned int irq,
- bool need_mapping, const char *devname)
-{
- int res;
-
- res = irq_permit_access(d, irq);
- if ( res )
- {
- printk(XENLOG_ERR "Unable to permit to dom%u access to IRQ %u\n",
- d->domain_id, irq);
- return res;
- }
-
- if ( need_mapping )
- {
- /*
- * Checking the return of vgic_reserve_virq is not
- * necessary. It should not fail except when we try to map
- * the IRQ twice. This can legitimately happen if the IRQ is shared
- */
- vgic_reserve_virq(d, irq);
-
- res = route_irq_to_guest(d, irq, irq, devname);
- if ( res < 0 )
- {
- printk(XENLOG_ERR "Unable to map IRQ%"PRId32" to dom%d\n",
- irq, d->domain_id);
- return res;
- }
- }
-
- dt_dprintk(" - IRQ: %u\n", irq);
- return 0;
-}
-
static int __init map_dt_irq_to_domain(const struct dt_device_node *dev,
const struct dt_irq *dt_irq,
void *data)
@@ -2322,57 +2287,6 @@ static int __init map_dt_irq_to_domain(const struct dt_device_node *dev,
return 0;
}
-int __init map_range_to_domain(const struct dt_device_node *dev,
- u64 addr, u64 len, void *data)
-{
- struct map_range_data *mr_data = data;
- struct domain *d = mr_data->d;
- int res;
-
- /*
- * reserved-memory regions are RAM carved out for a special purpose.
- * They are not MMIO and therefore a domain should not be able to
- * manage them via the IOMEM interface.
- */
- if ( strncasecmp(dt_node_full_name(dev), "/reserved-memory/",
- strlen("/reserved-memory/")) != 0 )
- {
- res = iomem_permit_access(d, paddr_to_pfn(addr),
- paddr_to_pfn(PAGE_ALIGN(addr + len - 1)));
- if ( res )
- {
- printk(XENLOG_ERR "Unable to permit to dom%d access to"
- " 0x%"PRIx64" - 0x%"PRIx64"\n",
- d->domain_id,
- addr & PAGE_MASK, PAGE_ALIGN(addr + len) - 1);
- return res;
- }
- }
-
- if ( !mr_data->skip_mapping )
- {
- res = map_regions_p2mt(d,
- gaddr_to_gfn(addr),
- PFN_UP(len),
- maddr_to_mfn(addr),
- mr_data->p2mt);
-
- if ( res < 0 )
- {
- printk(XENLOG_ERR "Unable to map 0x%"PRIx64
- " - 0x%"PRIx64" in domain %d\n",
- addr & PAGE_MASK, PAGE_ALIGN(addr + len) - 1,
- d->domain_id);
- return res;
- }
- }
-
- dt_dprintk(" - MMIO: %010"PRIx64" - %010"PRIx64" P2MType=%x\n",
- addr, addr + len, mr_data->p2mt);
-
- return 0;
-}
-
/*
* For a node which describes a discoverable bus (such as a PCI bus)
* then we may need to perform additional mappings in order to make
@@ -2400,62 +2314,6 @@ static int __init map_device_children(const struct dt_device_node *dev,
return 0;
}
-/*
- * handle_device_interrupts retrieves the interrupts configuration from
- * a device tree node and maps those interrupts to the target domain.
- *
- * Returns:
- * < 0 error
- * 0 success
- */
-static int __init handle_device_interrupts(struct domain *d,
- struct dt_device_node *dev,
- bool need_mapping)
-{
- unsigned int i, nirq;
- int res;
- struct dt_raw_irq rirq;
-
- nirq = dt_number_of_irq(dev);
-
- /* Give permission and map IRQs */
- for ( i = 0; i < nirq; i++ )
- {
- res = dt_device_get_raw_irq(dev, i, &rirq);
- if ( res )
- {
- printk(XENLOG_ERR "Unable to retrieve irq %u for %s\n",
- i, dt_node_full_name(dev));
- return res;
- }
-
- /*
- * Don't map IRQ that have no physical meaning
- * ie: IRQ whose controller is not the GIC
- */
- if ( rirq.controller != dt_interrupt_controller )
- {
- dt_dprintk("irq %u not connected to primary controller. Connected to %s\n",
- i, dt_node_full_name(rirq.controller));
- continue;
- }
-
- res = platform_get_irq(dev, i);
- if ( res < 0 )
- {
- printk(XENLOG_ERR "Unable to get irq %u for %s\n",
- i, dt_node_full_name(dev));
- return res;
- }
-
- res = map_irq_to_domain(d, res, need_mapping, dt_node_name(dev));
- if ( res )
- return res;
- }
-
- return 0;
-}
-
/*
* For a given device node:
* - Give permission to the guest to manage IRQ and MMIO range
diff --git a/xen/arch/arm/include/asm/domain_build.h b/xen/arch/arm/include/asm/domain_build.h
index 34ceddc995..b9329c9ee0 100644
--- a/xen/arch/arm/include/asm/domain_build.h
+++ b/xen/arch/arm/include/asm/domain_build.h
@@ -4,8 +4,6 @@
#include <xen/sched.h>
#include <asm/kernel.h>
-int map_irq_to_domain(struct domain *d, unsigned int irq,
- bool need_mapping, const char *devname);
int make_chosen_node(const struct kernel_info *kinfo);
void evtchn_allocate(struct domain *d);
diff --git a/xen/arch/arm/include/asm/setup.h b/xen/arch/arm/include/asm/setup.h
index a926f30a2b..1d636e8a4a 100644
--- a/xen/arch/arm/include/asm/setup.h
+++ b/xen/arch/arm/include/asm/setup.h
@@ -163,9 +163,15 @@ void device_tree_get_reg(const __be32 **cell, u32 address_cells,
u32 device_tree_get_u32(const void *fdt, int node,
const char *prop_name, u32 dflt);
+int handle_device_interrupts(struct domain *d, struct dt_device_node *dev,
+ bool need_mapping);
+
int map_range_to_domain(const struct dt_device_node *dev,
u64 addr, u64 len, void *data);
+int map_irq_to_domain(struct domain *d, unsigned int irq,
+ bool need_mapping, const char *devname);
+
extern const char __ro_after_init_start[], __ro_after_init_end[];
struct init_info
diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index 6c9712ab7b..aed38ff63c 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -1811,12 +1811,12 @@ int dt_count_phandle_with_args(const struct dt_device_node *np,
* @allnextpp: pointer to ->allnext from last allocated device_node
* @fpsize: Size of the node path up at the current depth.
*/
-static unsigned long __init unflatten_dt_node(const void *fdt,
- unsigned long mem,
- unsigned long *p,
- struct dt_device_node *dad,
- struct dt_device_node ***allnextpp,
- unsigned long fpsize)
+static unsigned long unflatten_dt_node(const void *fdt,
+ unsigned long mem,
+ unsigned long *p,
+ struct dt_device_node *dad,
+ struct dt_device_node ***allnextpp,
+ unsigned long fpsize)
{
struct dt_device_node *np;
struct dt_property *pp, **prev_pp = NULL;
@@ -2056,8 +2056,8 @@ static unsigned long __init unflatten_dt_node(const void *fdt,
* @fdt: The fdt to expand
* @mynodes: The device_node tree created by the call
*/
-static void __init __unflatten_device_tree(const void *fdt,
- struct dt_device_node **mynodes)
+static void __unflatten_device_tree(const void *fdt,
+ struct dt_device_node **mynodes)
{
unsigned long start, mem, size;
struct dt_device_node **allnextp = mynodes;
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 02/17] common/device_tree: change __unflatten_device_tree()
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (18 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 01/17] xen/arm/device: Remove __init from function type Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 03/17] xen/arm: Add CONFIG_OVERLAY_DTB Vikram Garhwal
` (14 subsequent siblings)
34 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel; +Cc: sstabellini, Vikram Garhwal, Julien Grall
Following changes are done to __unflatten_device_tree():
1. __unflatten_device_tree() is renamed to unflatten_device_tree().
2. Remove static function type.
3. Add handling of memory allocation. This will be useful in dynamic node
programming when we unflatten the dt during runtime memory allocation
can fail.
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
xen/common/device_tree.c | 10 ++++++----
xen/include/xen/device_tree.h | 5 +++++
2 files changed, 11 insertions(+), 4 deletions(-)
diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index aed38ff63c..bf847b2584 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -2047,7 +2047,7 @@ static unsigned long unflatten_dt_node(const void *fdt,
}
/**
- * __unflatten_device_tree - create tree of device_nodes from flat blob
+ * unflatten_device_tree - create tree of device_nodes from flat blob
*
* unflattens a device-tree, creating the
* tree of struct device_node. It also fills the "name" and "type"
@@ -2056,8 +2056,7 @@ static unsigned long unflatten_dt_node(const void *fdt,
* @fdt: The fdt to expand
* @mynodes: The device_node tree created by the call
*/
-static void __unflatten_device_tree(const void *fdt,
- struct dt_device_node **mynodes)
+void unflatten_device_tree(const void *fdt, struct dt_device_node **mynodes)
{
unsigned long start, mem, size;
struct dt_device_node **allnextp = mynodes;
@@ -2079,6 +2078,9 @@ static void __unflatten_device_tree(const void *fdt,
/* Allocate memory for the expanded device tree */
mem = (unsigned long)_xmalloc (size + 4, __alignof__(struct dt_device_node));
+ if ( !mem )
+ panic("Cannot allocate memory for unflatten device tree\n");
+
((__be32 *)mem)[size / 4] = cpu_to_be32(0xdeadbeef);
dt_dprintk(" unflattening %lx...\n", mem);
@@ -2179,7 +2181,7 @@ dt_find_interrupt_controller(const struct dt_device_match *matches)
void __init dt_unflatten_host_device_tree(void)
{
- __unflatten_device_tree(device_tree_flattened, &dt_host);
+ unflatten_device_tree(device_tree_flattened, &dt_host);
dt_alias_scan();
}
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index 19a74909ce..58ac12abe3 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -178,6 +178,11 @@ int device_tree_for_each_node(const void *fdt, int node,
*/
void dt_unflatten_host_device_tree(void);
+/**
+ * unflatten any device tree.
+ */
+void unflatten_device_tree(const void *fdt, struct dt_device_node **mynodes);
+
/**
* IRQ translation callback
* TODO: For the moment we assume that we only have ONE
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 03/17] xen/arm: Add CONFIG_OVERLAY_DTB
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (19 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 02/17] common/device_tree: change __unflatten_device_tree() Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 04/17] libfdt: Keep fdt functions after init for CONFIG_OVERLAY_DTB Vikram Garhwal
` (13 subsequent siblings)
34 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel
Cc: sstabellini, Vikram Garhwal, Andrew Cooper, George Dunlap,
Jan Beulich, Julien Grall, Wei Liu, Bertrand Marquis,
Volodymyr Babchuk
Introduce a config option where the user can enable support for adding/removing
device tree nodes using a device tree binary overlay.
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
SUPPORT.md | 6 ++++++
xen/arch/arm/Kconfig | 5 +++++
2 files changed, 11 insertions(+)
diff --git a/SUPPORT.md b/SUPPORT.md
index aa1940e55f..0a31f40af4 100644
--- a/SUPPORT.md
+++ b/SUPPORT.md
@@ -822,6 +822,12 @@ No support for QEMU backends in a 16K or 64K domain.
Status: Supported
+### Device Tree Overlays
+
+Add/Remove device tree nodes using a device tree overlay binary(.dtbo).
+
+ Status: Supported for ARM
+
### ARM: Guest ACPI support
Status: Supported
diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index 239d3aed3c..1fe3d698a5 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -53,6 +53,11 @@ config HAS_ITS
bool "GICv3 ITS MSI controller support (UNSUPPORTED)" if UNSUPPORTED
depends on GICV3 && !NEW_VGIC && !ARM_32
+config OVERLAY_DTB
+ bool "DTB overlay support (UNSUPPORTED)" if UNSUPPORTED
+ help
+ Dynamic addition/removal of Xen device tree nodes using a dtbo.
+
config HVM
def_bool y
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 04/17] libfdt: Keep fdt functions after init for CONFIG_OVERLAY_DTB.
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (20 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 03/17] xen/arm: Add CONFIG_OVERLAY_DTB Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 05/17] libfdt: overlay: change overlay_get_target() Vikram Garhwal
` (12 subsequent siblings)
34 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel; +Cc: sstabellini, Vikram Garhwal, Julien Grall
This is done to access fdt library function which are required for adding device
tree overlay nodes for dynamic programming of nodes.
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
Acked-by: Julien Grall <jgrall@amazon.com>
---
xen/common/libfdt/Makefile | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/xen/common/libfdt/Makefile b/xen/common/libfdt/Makefile
index 75aaefa2e3..d50487aa6e 100644
--- a/xen/common/libfdt/Makefile
+++ b/xen/common/libfdt/Makefile
@@ -1,7 +1,11 @@
include $(src)/Makefile.libfdt
SECTIONS := text data $(SPECIAL_DATA_SECTIONS)
+
+# For CONFIG_OVERLAY_DTB, libfdt functionalities will be needed during runtime.
+ifneq ($(CONFIG_OVERLAY_DTB),y)
OBJCOPYFLAGS := $(foreach s,$(SECTIONS),--rename-section .$(s)=.init.$(s))
+endif
obj-y += libfdt.o
nocov-y += libfdt.o
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 05/17] libfdt: overlay: change overlay_get_target()
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (21 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 04/17] libfdt: Keep fdt functions after init for CONFIG_OVERLAY_DTB Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 06/17] xen/device-tree: Add device_tree_find_node_by_path() to find nodes in device tree Vikram Garhwal
` (11 subsequent siblings)
34 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel
Cc: sstabellini, Vikram Garhwal, Julien Grall, Vikram Garhwal, David Gibson
Rename overlay_get_target() to fdt_overlay_target_offset() and remove static
function type.
This is done to get the target path for the overlay nodes which is very useful
in many cases. For example, Xen hypervisor needs it when applying overlays
because Xen needs to do further processing of the overlay nodes, e.g. mapping of
resources(IRQs and IOMMUs) to other VMs, creation of SMMU pagetables, etc.
Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
Message-Id: <1637204036-382159-2-git-send-email-fnu.vikram@xilinx.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Origin: https://github.com/dgibson/dtc 45f3d1a095dd
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
xen/common/libfdt/fdt_overlay.c | 29 +++++++----------------------
xen/common/libfdt/version.lds | 1 +
xen/include/xen/libfdt/libfdt.h | 18 ++++++++++++++++++
3 files changed, 26 insertions(+), 22 deletions(-)
diff --git a/xen/common/libfdt/fdt_overlay.c b/xen/common/libfdt/fdt_overlay.c
index 7b95e2b639..acf0c4c2a6 100644
--- a/xen/common/libfdt/fdt_overlay.c
+++ b/xen/common/libfdt/fdt_overlay.c
@@ -41,37 +41,22 @@ static uint32_t overlay_get_target_phandle(const void *fdto, int fragment)
return fdt32_to_cpu(*val);
}
-/**
- * overlay_get_target - retrieves the offset of a fragment's target
- * @fdt: Base device tree blob
- * @fdto: Device tree overlay blob
- * @fragment: node offset of the fragment in the overlay
- * @pathp: pointer which receives the path of the target (or NULL)
- *
- * overlay_get_target() retrieves the target offset in the base
- * device tree of a fragment, no matter how the actual targeting is
- * done (through a phandle or a path)
- *
- * returns:
- * the targeted node offset in the base device tree
- * Negative error code on error
- */
-static int overlay_get_target(const void *fdt, const void *fdto,
- int fragment, char const **pathp)
+int fdt_overlay_target_offset(const void *fdt, const void *fdto,
+ int fragment_offset, char const **pathp)
{
uint32_t phandle;
const char *path = NULL;
int path_len = 0, ret;
/* Try first to do a phandle based lookup */
- phandle = overlay_get_target_phandle(fdto, fragment);
+ phandle = overlay_get_target_phandle(fdto, fragment_offset);
if (phandle == (uint32_t)-1)
return -FDT_ERR_BADPHANDLE;
/* no phandle, try path */
if (!phandle) {
/* And then a path based lookup */
- path = fdt_getprop(fdto, fragment, "target-path", &path_len);
+ path = fdt_getprop(fdto, fragment_offset, "target-path", &path_len);
if (path)
ret = fdt_path_offset(fdt, path);
else
@@ -638,7 +623,7 @@ static int overlay_merge(void *fdt, void *fdto)
if (overlay < 0)
return overlay;
- target = overlay_get_target(fdt, fdto, fragment, NULL);
+ target = fdt_overlay_target_offset(fdt, fdto, fragment, NULL);
if (target < 0)
return target;
@@ -781,7 +766,7 @@ static int overlay_symbol_update(void *fdt, void *fdto)
return -FDT_ERR_BADOVERLAY;
/* get the target of the fragment */
- ret = overlay_get_target(fdt, fdto, fragment, &target_path);
+ ret = fdt_overlay_target_offset(fdt, fdto, fragment, &target_path);
if (ret < 0)
return ret;
target = ret;
@@ -803,7 +788,7 @@ static int overlay_symbol_update(void *fdt, void *fdto)
if (!target_path) {
/* again in case setprop_placeholder changed it */
- ret = overlay_get_target(fdt, fdto, fragment, &target_path);
+ ret = fdt_overlay_target_offset(fdt, fdto, fragment, &target_path);
if (ret < 0)
return ret;
target = ret;
diff --git a/xen/common/libfdt/version.lds b/xen/common/libfdt/version.lds
index 7ab85f1d9d..cbce5d4a8b 100644
--- a/xen/common/libfdt/version.lds
+++ b/xen/common/libfdt/version.lds
@@ -77,6 +77,7 @@ LIBFDT_1.2 {
fdt_appendprop_addrrange;
fdt_setprop_inplace_namelen_partial;
fdt_create_with_flags;
+ fdt_overlay_target_offset;
local:
*;
};
diff --git a/xen/include/xen/libfdt/libfdt.h b/xen/include/xen/libfdt/libfdt.h
index c71689e2be..fabddbee8c 100644
--- a/xen/include/xen/libfdt/libfdt.h
+++ b/xen/include/xen/libfdt/libfdt.h
@@ -2109,6 +2109,24 @@ int fdt_del_node(void *fdt, int nodeoffset);
*/
int fdt_overlay_apply(void *fdt, void *fdto);
+/**
+ * fdt_overlay_target_offset - retrieves the offset of a fragment's target
+ * @fdt: Base device tree blob
+ * @fdto: Device tree overlay blob
+ * @fragment_offset: node offset of the fragment in the overlay
+ * @pathp: pointer which receives the path of the target (or NULL)
+ *
+ * fdt_overlay_target_offset() retrieves the target offset in the base
+ * device tree of a fragment, no matter how the actual targeting is
+ * done (through a phandle or a path)
+ *
+ * returns:
+ * the targeted node offset in the base device tree
+ * Negative error code on error
+ */
+int fdt_overlay_target_offset(const void *fdt, const void *fdto,
+ int fragment_offset, char const **pathp);
+
/**********************************************************************/
/* Debugging / informational functions */
/**********************************************************************/
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 06/17] xen/device-tree: Add device_tree_find_node_by_path() to find nodes in device tree
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (22 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 05/17] libfdt: overlay: change overlay_get_target() Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 07/17] xen/smmu: Add remove_device callback for smmu_iommu ops Vikram Garhwal
` (10 subsequent siblings)
34 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel; +Cc: sstabellini, Vikram Garhwal, Julien Grall
Add device_tree_find_node_by_path() to find a matching node with path for a
dt_device_node.
Reason behind this function:
Each time overlay nodes are added using .dtbo, a new fdt(memcpy of
device_tree_flattened) is created and updated with overlay nodes. This
updated fdt is further unflattened to a dt_host_new. Next, we need to find
the overlay nodes in dt_host_new, find the overlay node's parent in dt_host
and add the nodes as child under their parent in the dt_host. Thus we need
this function to search for node in different unflattened device trees.
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
xen/common/device_tree.c | 5 +++--
xen/include/xen/device_tree.h | 17 +++++++++++++++--
2 files changed, 18 insertions(+), 4 deletions(-)
diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index bf847b2584..507b4ac5b6 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -358,11 +358,12 @@ struct dt_device_node *dt_find_node_by_type(struct dt_device_node *from,
return np;
}
-struct dt_device_node *dt_find_node_by_path(const char *path)
+struct dt_device_node *device_tree_find_node_by_path(struct dt_device_node *dt,
+ const char *path)
{
struct dt_device_node *np;
- dt_for_each_device_node(dt_host, np)
+ dt_for_each_device_node(dt, np)
if ( np->full_name && (dt_node_cmp(np->full_name, path) == 0) )
break;
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index 58ac12abe3..998f972ebc 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -534,13 +534,26 @@ struct dt_device_node *dt_find_node_by_type(struct dt_device_node *from,
struct dt_device_node *dt_find_node_by_alias(const char *alias);
/**
- * dt_find_node_by_path - Find a node matching a full DT path
+ * device_tree_find_node_by_path - Generic function to find a node matching the
+ * full DT path for any given unflatten device tree
+ * @dt_node: The device tree to search
* @path: The full path to match
*
* Returns a node pointer.
*/
-struct dt_device_node *dt_find_node_by_path(const char *path);
+struct dt_device_node *device_tree_find_node_by_path(struct dt_device_node *dt,
+ const char *path);
+/**
+ * dt_find_node_by_path - Find a node matching a full DT path in dt_host
+ * @path: The full path to match
+ *
+ * Returns a node pointer.
+ */
+static inline struct dt_device_node *dt_find_node_by_path(const char *path)
+{
+ return device_tree_find_node_by_path(dt_host, path);
+}
/**
* dt_find_node_by_gpath - Same as dt_find_node_by_path but retrieve the
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 07/17] xen/smmu: Add remove_device callback for smmu_iommu ops
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (23 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 06/17] xen/device-tree: Add device_tree_find_node_by_path() to find nodes in device tree Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 08/17] xen/iommu: Move spin_lock from iommu_dt_device_is_assigned to caller Vikram Garhwal
` (9 subsequent siblings)
34 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel
Cc: sstabellini, Vikram Garhwal, Julien Grall, Rahul Singh,
Bertrand Marquis, Volodymyr Babchuk
Add remove_device callback for removing the device entry from smmu-master using
following steps:
1. Find if SMMU master exists for the device node.
2. Remove the SMMU master
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
---
xen/drivers/passthrough/arm/smmu.c | 56 ++++++++++++++++++++++++++++++
1 file changed, 56 insertions(+)
diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c
index 0a514821b3..14e15f1bc6 100644
--- a/xen/drivers/passthrough/arm/smmu.c
+++ b/xen/drivers/passthrough/arm/smmu.c
@@ -816,6 +816,19 @@ static int insert_smmu_master(struct arm_smmu_device *smmu,
return 0;
}
+static int remove_smmu_master(struct arm_smmu_device *smmu,
+ struct arm_smmu_master *master)
+{
+ if (!smmu->masters.rb_node) {
+ ASSERT_UNREACHABLE();
+ return -ENOENT;
+ }
+
+ rb_erase(&master->node, &smmu->masters);
+
+ return 0;
+}
+
static int arm_smmu_dt_add_device_legacy(struct arm_smmu_device *smmu,
struct device *dev,
struct iommu_fwspec *fwspec)
@@ -853,6 +866,32 @@ static int arm_smmu_dt_add_device_legacy(struct arm_smmu_device *smmu,
return insert_smmu_master(smmu, master);
}
+static int arm_smmu_dt_remove_device_legacy(struct arm_smmu_device *smmu,
+ struct device *dev)
+{
+ struct arm_smmu_master *master;
+ struct device_node *dev_node = dev_get_dev_node(dev);
+ int ret;
+
+ master = find_smmu_master(smmu, dev_node);
+ if (master == NULL) {
+ dev_err(dev,
+ "No registrations found for master device %s\n",
+ dev_node->name);
+ return -EINVAL;
+ }
+
+ ret = remove_smmu_master(smmu, master);
+
+ if (ret)
+ return ret;
+
+ dev_node->is_protected = false;
+
+ kfree(master);
+ return 0;
+}
+
static int register_smmu_master(struct arm_smmu_device *smmu,
struct device *dev,
struct of_phandle_args *masterspec)
@@ -876,6 +915,22 @@ static int register_smmu_master(struct arm_smmu_device *smmu,
fwspec);
}
+static int arm_smmu_dt_remove_device_generic(u8 devfn, struct device *dev)
+{
+ struct arm_smmu_device *smmu;
+ struct iommu_fwspec *fwspec;
+
+ fwspec = dev_iommu_fwspec_get(dev);
+ if (fwspec == NULL)
+ return -ENXIO;
+
+ smmu = find_smmu(fwspec->iommu_dev);
+ if (smmu == NULL)
+ return -ENXIO;
+
+ return arm_smmu_dt_remove_device_legacy(smmu, dev);
+}
+
static int arm_smmu_dt_add_device_generic(u8 devfn, struct device *dev)
{
struct arm_smmu_device *smmu;
@@ -2858,6 +2913,7 @@ static const struct iommu_ops arm_smmu_iommu_ops = {
.init = arm_smmu_iommu_domain_init,
.hwdom_init = arch_iommu_hwdom_init,
.add_device = arm_smmu_dt_add_device_generic,
+ .remove_device = arm_smmu_dt_remove_device_generic,
.teardown = arm_smmu_iommu_domain_teardown,
.iotlb_flush = arm_smmu_iotlb_flush,
.assign_device = arm_smmu_assign_dev,
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 08/17] xen/iommu: Move spin_lock from iommu_dt_device_is_assigned to caller
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (24 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 07/17] xen/smmu: Add remove_device callback for smmu_iommu ops Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 09/17] xen/iommu: protect iommu_add_dt_device() with dtdevs_lock Vikram Garhwal
` (8 subsequent siblings)
34 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel; +Cc: sstabellini, Vikram Garhwal, Julien Grall
Rename iommu_dt_device_is_assigned() to iommu_dt_device_is_assigned_lock().
Moving spin_lock to caller was done to prevent the concurrent access to
iommu_dt_device_is_assigned while doing add/remove/assign/deassign.
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
---
xen/drivers/passthrough/device_tree.c | 23 +++++++++++++++++++----
1 file changed, 19 insertions(+), 4 deletions(-)
diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c
index 1c32d7b50c..bb4cf7784d 100644
--- a/xen/drivers/passthrough/device_tree.c
+++ b/xen/drivers/passthrough/device_tree.c
@@ -83,16 +83,15 @@ fail:
return rc;
}
-static bool_t iommu_dt_device_is_assigned(const struct dt_device_node *dev)
+static bool_t
+ iommu_dt_device_is_assigned_locked(const struct dt_device_node *dev)
{
bool_t assigned = 0;
if ( !dt_device_is_protected(dev) )
return 0;
- spin_lock(&dtdevs_lock);
assigned = !list_empty(&dev->domain_list);
- spin_unlock(&dtdevs_lock);
return assigned;
}
@@ -213,27 +212,43 @@ int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d,
if ( (d && d->is_dying) || domctl->u.assign_device.flags )
break;
+ spin_lock(&dtdevs_lock);
+
ret = dt_find_node_by_gpath(domctl->u.assign_device.u.dt.path,
domctl->u.assign_device.u.dt.size,
&dev);
if ( ret )
+ {
+ spin_unlock(&dtdevs_lock);
+
break;
+ }
ret = xsm_assign_dtdevice(XSM_HOOK, d, dt_node_full_name(dev));
if ( ret )
+ {
+ spin_unlock(&dtdevs_lock);
+
break;
+ }
if ( domctl->cmd == XEN_DOMCTL_test_assign_device )
{
- if ( iommu_dt_device_is_assigned(dev) )
+
+ if ( iommu_dt_device_is_assigned_locked(dev) )
{
printk(XENLOG_G_ERR "%s already assigned.\n",
dt_node_full_name(dev));
ret = -EINVAL;
}
+
+ spin_unlock(&dtdevs_lock);
+
break;
}
+ spin_unlock(&dtdevs_lock);
+
if ( d == dom_io )
return -EINVAL;
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 09/17] xen/iommu: protect iommu_add_dt_device() with dtdevs_lock
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (25 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 08/17] xen/iommu: Move spin_lock from iommu_dt_device_is_assigned to caller Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 10/17] xen/iommu: Introduce iommu_remove_dt_device() Vikram Garhwal
` (7 subsequent siblings)
34 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel; +Cc: sstabellini, Vikram Garhwal, Julien Grall
Protect iommu_add_dt_device() with dtdevs_lock to prevent concurrent access add.
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
---
xen/drivers/passthrough/device_tree.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c
index bb4cf7784d..457df333a0 100644
--- a/xen/drivers/passthrough/device_tree.c
+++ b/xen/drivers/passthrough/device_tree.c
@@ -146,6 +146,8 @@ int iommu_add_dt_device(struct dt_device_node *np)
if ( dev_iommu_fwspec_get(dev) )
return 0;
+ spin_lock(&dtdevs_lock);
+
/*
* According to the Documentation/devicetree/bindings/iommu/iommu.txt
* from Linux.
@@ -158,7 +160,10 @@ int iommu_add_dt_device(struct dt_device_node *np)
* these callback implemented.
*/
if ( !ops->add_device || !ops->dt_xlate )
- return -EINVAL;
+ {
+ rc = -EINVAL;
+ goto fail;
+ }
if ( !dt_device_is_available(iommu_spec.np) )
break;
@@ -189,6 +194,8 @@ int iommu_add_dt_device(struct dt_device_node *np)
if ( rc < 0 )
iommu_fwspec_free(dev);
+fail:
+ spin_unlock(&dtdevs_lock);
return rc;
}
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 10/17] xen/iommu: Introduce iommu_remove_dt_device()
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (26 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 09/17] xen/iommu: protect iommu_add_dt_device() with dtdevs_lock Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 11/17] asm/smp.h: Fix circular dependency for device_tree.h and rwlock.h Vikram Garhwal
` (6 subsequent siblings)
34 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel
Cc: sstabellini, Vikram Garhwal, Julien Grall, Jan Beulich,
Paul Durrant, Roger Pau Monné
Remove master device from the IOMMU. This will be helpful when removing the
overlay nodes using dynamic programming during run time.
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
xen/drivers/passthrough/device_tree.c | 38 +++++++++++++++++++++++++++
xen/include/xen/iommu.h | 2 ++
2 files changed, 40 insertions(+)
diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c
index 457df333a0..a77a217f3d 100644
--- a/xen/drivers/passthrough/device_tree.c
+++ b/xen/drivers/passthrough/device_tree.c
@@ -126,6 +126,44 @@ int iommu_release_dt_devices(struct domain *d)
return 0;
}
+int iommu_remove_dt_device(struct dt_device_node *np)
+{
+ const struct iommu_ops *ops = iommu_get_ops();
+ struct device *dev = dt_to_dev(np);
+ int rc;
+
+ if ( !ops )
+ return -EOPNOTSUPP;
+
+ spin_lock(&dtdevs_lock);
+
+ if ( iommu_dt_device_is_assigned_locked(np) )
+ {
+ rc = -EBUSY;
+ goto fail;
+ }
+
+ /*
+ * The driver which supports generic IOMMU DT bindings must have
+ * these callback implemented.
+ */
+ if ( !ops->remove_device )
+ {
+ rc = -EOPNOTSUPP;
+ goto fail;
+ }
+
+ /* Remove master device from the IOMMU if latter is present and available. */
+ rc = ops->remove_device(0, dev);
+
+ if ( !rc )
+ iommu_fwspec_free(dev);
+
+fail:
+ spin_unlock(&dtdevs_lock);
+ return rc;
+}
+
int iommu_add_dt_device(struct dt_device_node *np)
{
const struct iommu_ops *ops = iommu_get_ops();
diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
index 405db59971..079c06321e 100644
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -218,6 +218,8 @@ int iommu_deassign_dt_device(struct domain *d, struct dt_device_node *dev);
int iommu_dt_domain_init(struct domain *d);
int iommu_release_dt_devices(struct domain *d);
+int iommu_remove_dt_device(struct dt_device_node *np);
+
/*
* Helper to add master device to the IOMMU using generic IOMMU DT bindings.
*
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 11/17] asm/smp.h: Fix circular dependency for device_tree.h and rwlock.h
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (27 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 10/17] xen/iommu: Introduce iommu_remove_dt_device() Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 12/17] common/device_tree: Add rwlock for dt_host Vikram Garhwal
` (5 subsequent siblings)
34 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel
Cc: sstabellini, Vikram Garhwal, Julien Grall, Bertrand Marquis,
Volodymyr Babchuk
Dynamic programming ops will modify the dt_host and there might be other
function which are browsing the dt_host at the same time. To avoid the race
conditions, adding rwlock for browsing the dt_host. But adding rwlock in
device_tree.h causes following circular dependency:
device_tree.h->rwlock.h->smp.h->asm/smp.h->device_tree.h
To fix this, removed the "#include <xen/device_tree.h> and forward declared
"struct dt_device_node".
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
xen/arch/arm/include/asm/smp.h | 3 ++-
xen/arch/arm/smpboot.c | 1 +
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/xen/arch/arm/include/asm/smp.h b/xen/arch/arm/include/asm/smp.h
index 8133d5c295..afe6129276 100644
--- a/xen/arch/arm/include/asm/smp.h
+++ b/xen/arch/arm/include/asm/smp.h
@@ -3,13 +3,14 @@
#ifndef __ASSEMBLY__
#include <xen/cpumask.h>
-#include <xen/device_tree.h>
#include <asm/current.h>
#endif
DECLARE_PER_CPU(cpumask_var_t, cpu_sibling_mask);
DECLARE_PER_CPU(cpumask_var_t, cpu_core_mask);
+struct dt_device_node;
+
#define cpu_is_offline(cpu) unlikely(!cpu_online(cpu))
#define smp_processor_id() get_processor_id()
diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
index 412ae22869..336a7d418b 100644
--- a/xen/arch/arm/smpboot.c
+++ b/xen/arch/arm/smpboot.c
@@ -11,6 +11,7 @@
#include <xen/cpumask.h>
#include <xen/delay.h>
#include <xen/domain_page.h>
+#include <xen/device_tree.h>
#include <xen/errno.h>
#include <xen/init.h>
#include <xen/mm.h>
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 12/17] common/device_tree: Add rwlock for dt_host
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (28 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 11/17] asm/smp.h: Fix circular dependency for device_tree.h and rwlock.h Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 13/17] xen/arm: Implement device tree node removal functionalities Vikram Garhwal
` (4 subsequent siblings)
34 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel; +Cc: sstabellini, Vikram Garhwal, Julien Grall
Dynamic programming ops will modify the dt_host and there might be other
function which are browsing the dt_host at the same time. To avoid the race
conditions, adding rwlock for browsing the dt_host during runtime.
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
xen/common/device_tree.c | 3 +++
xen/drivers/passthrough/device_tree.c | 39 +++++++++++++++++++++++++++
xen/include/xen/device_tree.h | 6 +++++
3 files changed, 48 insertions(+)
diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index 507b4ac5b6..b330e6a013 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -2098,6 +2098,9 @@ void unflatten_device_tree(const void *fdt, struct dt_device_node **mynodes)
*allnextp = NULL;
dt_dprintk(" <- unflatten_device_tree()\n");
+
+ /* Init r/w lock for host device tree. */
+ rwlock_init(&dt_host->lock);
}
static void dt_alias_add(struct dt_alias_prop *ap,
diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c
index a77a217f3d..74f994b9da 100644
--- a/xen/drivers/passthrough/device_tree.c
+++ b/xen/drivers/passthrough/device_tree.c
@@ -112,6 +112,8 @@ int iommu_release_dt_devices(struct domain *d)
if ( !is_iommu_enabled(d) )
return 0;
+ read_lock(&dt_host->lock);
+
list_for_each_entry_safe(dev, _dev, &hd->dt_devices, domain_list)
{
rc = iommu_deassign_dt_device(d, dev);
@@ -119,10 +121,14 @@ int iommu_release_dt_devices(struct domain *d)
{
dprintk(XENLOG_ERR, "Failed to deassign %s in domain %u\n",
dt_node_full_name(dev), d->domain_id);
+
+ read_unlock(&dt_host->lock);
return rc;
}
}
+ read_unlock(&dt_host->lock);
+
return 0;
}
@@ -259,11 +265,15 @@ int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d,
spin_lock(&dtdevs_lock);
+ read_lock(&dt_host->lock);
+
ret = dt_find_node_by_gpath(domctl->u.assign_device.u.dt.path,
domctl->u.assign_device.u.dt.size,
&dev);
if ( ret )
{
+ read_unlock(&dt_host->lock);
+
spin_unlock(&dtdevs_lock);
break;
@@ -272,6 +282,8 @@ int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d,
ret = xsm_assign_dtdevice(XSM_HOOK, d, dt_node_full_name(dev));
if ( ret )
{
+ read_unlock(&dt_host->lock);
+
spin_unlock(&dtdevs_lock);
break;
@@ -287,6 +299,8 @@ int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d,
ret = -EINVAL;
}
+ read_unlock(&dt_host->lock);
+
spin_unlock(&dtdevs_lock);
break;
@@ -295,22 +309,31 @@ int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d,
spin_unlock(&dtdevs_lock);
if ( d == dom_io )
+ {
+ read_unlock(&dt_host->lock);
return -EINVAL;
+ }
ret = iommu_add_dt_device(dev);
if ( ret < 0 )
{
printk(XENLOG_G_ERR "Failed to add %s to the IOMMU\n",
dt_node_full_name(dev));
+
+ read_unlock(&dt_host->lock);
break;
}
ret = iommu_assign_dt_device(d, dev);
if ( ret )
+ {
printk(XENLOG_G_ERR "XEN_DOMCTL_assign_dt_device: assign \"%s\""
" to dom%u failed (%d)\n",
dt_node_full_name(dev), d->domain_id, ret);
+ }
+
+ read_unlock(&dt_host->lock);
break;
case XEN_DOMCTL_deassign_device:
@@ -322,25 +345,41 @@ int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d,
if ( domctl->u.assign_device.flags )
break;
+ read_lock(&dt_host->lock);
+
ret = dt_find_node_by_gpath(domctl->u.assign_device.u.dt.path,
domctl->u.assign_device.u.dt.size,
&dev);
if ( ret )
+ {
+ read_unlock(&dt_host->lock);
break;
+ }
ret = xsm_deassign_dtdevice(XSM_HOOK, d, dt_node_full_name(dev));
+
if ( ret )
+ {
+ read_unlock(&dt_host->lock);
break;
+ }
if ( d == dom_io )
+ {
+ read_unlock(&dt_host->lock);
return -EINVAL;
+ }
ret = iommu_deassign_dt_device(d, dev);
if ( ret )
+ {
printk(XENLOG_G_ERR "XEN_DOMCTL_assign_dt_device: assign \"%s\""
" to dom%u failed (%d)\n",
dt_node_full_name(dev), d->domain_id, ret);
+ }
+
+ read_unlock(&dt_host->lock);
break;
default:
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index 998f972ebc..b7272bbccc 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -18,6 +18,7 @@
#include <xen/string.h>
#include <xen/types.h>
#include <xen/list.h>
+#include <xen/rwlock.h>
#define DEVICE_TREE_MAX_DEPTH 16
@@ -106,6 +107,11 @@ struct dt_device_node {
struct list_head domain_list;
struct device dev;
+
+ /*
+ * Lock that protects r/w updates to unflattened device tree i.e. dt_host.
+ */
+ rwlock_t lock;
};
#define dt_to_dev(dt_node) (&(dt_node)->dev)
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 13/17] xen/arm: Implement device tree node removal functionalities
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (29 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 12/17] common/device_tree: Add rwlock for dt_host Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 14/17] xen/arm: Implement device tree node addition functionalities Vikram Garhwal
` (3 subsequent siblings)
34 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel
Cc: sstabellini, Vikram Garhwal, Julien Grall, Bertrand Marquis,
Volodymyr Babchuk, Andrew Cooper, George Dunlap, Jan Beulich,
Wei Liu
Introduce sysctl XEN_SYSCTL_dt_overlay to remove device-tree nodes added using
device tree overlay.
xl dt-overlay remove file.dtbo:
Removes all the nodes in a given dtbo.
First, removes IRQ permissions and MMIO accesses. Next, it finds the nodes
in dt_host and delete the device node entries from dt_host.
The nodes get removed only if it is not used by any of dom0 or domio.
Also, added overlay_track struct to keep the track of added node through device
tree overlay. overlay_track has dt_host_new which is unflattened form of updated
fdt and name of overlay nodes. When a node is removed, we also free the memory
used by overlay_track for the particular overlay node.
Nested overlay removal is supported in sequential manner only i.e. if
overlay_child nests under overlay_parent, it is assumed that user first removes
overlay_child and then removes overlay_parent.
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
xen/arch/arm/sysctl.c | 16 +-
xen/common/Makefile | 1 +
xen/common/dt_overlay.c | 415 +++++++++++++++++++++++++++++++++++
xen/include/public/sysctl.h | 24 ++
xen/include/xen/dt_overlay.h | 59 +++++
5 files changed, 514 insertions(+), 1 deletion(-)
create mode 100644 xen/common/dt_overlay.c
create mode 100644 xen/include/xen/dt_overlay.h
diff --git a/xen/arch/arm/sysctl.c b/xen/arch/arm/sysctl.c
index b0a78a8b10..672db61650 100644
--- a/xen/arch/arm/sysctl.c
+++ b/xen/arch/arm/sysctl.c
@@ -12,6 +12,7 @@
#include <xen/errno.h>
#include <xen/hypercall.h>
#include <public/sysctl.h>
+#include <xen/dt_overlay.h>
void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
{
@@ -21,7 +22,20 @@ void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
long arch_do_sysctl(struct xen_sysctl *sysctl,
XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl)
{
- return -ENOSYS;
+ long ret = 0;
+
+ switch ( sysctl->cmd )
+ {
+ case XEN_SYSCTL_dt_overlay:
+ ret = dt_sysctl(&sysctl->u.dt_overlay);
+ break;
+
+ default:
+ ret = -ENOSYS;
+ break;
+ }
+
+ return ret;
}
/*
diff --git a/xen/common/Makefile b/xen/common/Makefile
index 46049eac35..be78c9a8c2 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -8,6 +8,7 @@ obj-$(CONFIG_DEBUG_TRACE) += debugtrace.o
obj-$(CONFIG_HAS_DEVICE_TREE) += device_tree.o
obj-$(CONFIG_IOREQ_SERVER) += dm.o
obj-y += domain.o
+obj-$(CONFIG_OVERLAY_DTB) += dt_overlay.o
obj-y += event_2l.o
obj-y += event_channel.o
obj-y += event_fifo.o
diff --git a/xen/common/dt_overlay.c b/xen/common/dt_overlay.c
new file mode 100644
index 0000000000..516e8010c5
--- /dev/null
+++ b/xen/common/dt_overlay.c
@@ -0,0 +1,415 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0
+ *
+ * xen/common/dt_overlay.c
+ *
+ * Device tree overlay support in Xen.
+ *
+ * Copyright (C) 2023, Advanced Micro Devices, Inc. All Rights Reserved.
+ * Written by Vikram Garhwal <vikram.garhwal@amd.com>
+ *
+ */
+#include <xen/iocap.h>
+#include <xen/xmalloc.h>
+#include <asm/domain_build.h>
+#include <xen/dt_overlay.h>
+#include <xen/guest_access.h>
+
+static LIST_HEAD(overlay_tracker);
+static DEFINE_SPINLOCK(overlay_lock);
+
+/* Find last descendants of the device_node. */
+static struct dt_device_node *find_last_descendants_node(
+ struct dt_device_node *device_node)
+{
+ struct dt_device_node *child_node;
+
+ for ( child_node = device_node->child; child_node->sibling != NULL;
+ child_node = child_node->sibling )
+ {
+ }
+
+ /* If last child_node also have children. */
+ if ( child_node->child )
+ child_node = find_last_descendants_node(child_node);
+
+ return child_node;
+}
+
+static int dt_overlay_remove_node(struct dt_device_node *device_node)
+{
+ struct dt_device_node *np;
+ struct dt_device_node *parent_node;
+ struct dt_device_node *device_node_last_descendant = device_node->child;
+
+ parent_node = device_node->parent;
+
+ if ( parent_node == NULL )
+ {
+ dt_dprintk("%s's parent node not found\n", device_node->name);
+ return -EFAULT;
+ }
+
+ np = parent_node->child;
+
+ if ( np == NULL )
+ {
+ dt_dprintk("parent node %s's not found\n", parent_node->name);
+ return -EFAULT;
+ }
+
+ /* If node to be removed is only child node or first child. */
+ if ( !dt_node_cmp(np->full_name, device_node->full_name) )
+ {
+ parent_node->child = np->sibling;
+
+ /*
+ * Iterate over all child nodes of device_node. Given that we are
+ * removing parent node, we need to remove all it's descendants too.
+ */
+ if ( device_node_last_descendant )
+ {
+ device_node_last_descendant =
+ find_last_descendants_node(device_node);
+ parent_node->allnext = device_node_last_descendant->allnext;
+ }
+ else
+ parent_node->allnext = np->allnext;
+
+ return 0;
+ }
+
+ for ( np = parent_node->child; np->sibling != NULL; np = np->sibling )
+ {
+ if ( !dt_node_cmp(np->sibling->full_name, device_node->full_name) )
+ {
+ /* Found the node. Now we remove it. */
+ np->sibling = np->sibling->sibling;
+
+ if ( np->child )
+ np = find_last_descendants_node(np);
+
+ /*
+ * Iterate over all child nodes of device_node. Given that we are
+ * removing parent node, we need to remove all it's descendants too.
+ */
+ if ( device_node_last_descendant )
+ device_node_last_descendant =
+ find_last_descendants_node(device_node);
+
+ if ( device_node_last_descendant )
+ np->allnext = device_node_last_descendant->allnext;
+ else
+ np->allnext = np->allnext->allnext;
+
+ break;
+ }
+ }
+
+ return 0;
+}
+
+/* Basic sanity check for the dtbo tool stack provided to Xen. */
+static int check_overlay_fdt(const void *overlay_fdt, uint32_t overlay_fdt_size)
+{
+ if ( (fdt_totalsize(overlay_fdt) != overlay_fdt_size) ||
+ fdt_check_header(overlay_fdt) )
+ {
+ printk(XENLOG_ERR "The overlay FDT is not a valid Flat Device Tree\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+/* Count number of nodes till one level of __overlay__ tag. */
+static unsigned int overlay_node_count(void *fdto)
+{
+ unsigned int num_overlay_nodes = 0;
+ int fragment;
+
+ fdt_for_each_subnode(fragment, fdto, 0)
+ {
+ int subnode;
+ int overlay;
+
+ overlay = fdt_subnode_offset(fdto, fragment, "__overlay__");
+
+ /*
+ * overlay value can be < 0. But fdt_for_each_subnode() loop checks for
+ * overlay >= 0. So, no need for a overlay>=0 check here.
+ */
+ fdt_for_each_subnode(subnode, fdto, overlay)
+ {
+ num_overlay_nodes++;
+ }
+ }
+
+ return num_overlay_nodes;
+}
+
+static int handle_remove_irq_iommu(struct dt_device_node *device_node)
+{
+ int rc = 0;
+ struct domain *d = hardware_domain;
+ domid_t domid;
+ unsigned int naddr, len;
+ unsigned int i, nirq;
+ uint64_t addr, size;
+
+ domid = dt_device_used_by(device_node);
+
+ dt_dprintk("Checking if node %s is used by any domain\n",
+ device_node->full_name);
+
+ /* Remove the node iff it's assigned to domain 0 or domain io. */
+ if ( domid != 0 && domid != DOMID_IO )
+ {
+ printk(XENLOG_ERR "Device %s as it is being used by domain %d. Removing nodes failed\n",
+ device_node->full_name, domid);
+ return -EINVAL;
+ }
+
+ dt_dprintk("Removing node: %s\n", device_node->full_name);
+
+ nirq = dt_number_of_irq(device_node);
+
+ /* Remove IRQ permission */
+ for ( i = 0; i < nirq; i++ )
+ {
+ rc = platform_get_irq(device_node, i);;
+
+ if ( irq_access_permitted(d, rc) == false )
+ {
+ printk(XENLOG_ERR "IRQ %d is not routed to domain %d\n", rc,
+ domid);
+ return -EINVAL;
+ }
+ /*
+ * TODO: We don't handle shared IRQs for now. So, it is assumed that
+ * the IRQs was not shared with another devices.
+ */
+ rc = irq_deny_access(d, rc);
+ if ( rc )
+ {
+ printk(XENLOG_ERR "unable to revoke access for irq %u for %s\n",
+ i, device_node->full_name);
+ return rc;
+ }
+ }
+
+ /* Check if iommu property exists. */
+ if ( dt_get_property(device_node, "iommus", &len) )
+ {
+ rc = iommu_remove_dt_device(device_node);
+ if ( rc != 0 && rc != -ENXIO )
+ return rc;
+ }
+
+ naddr = dt_number_of_address(device_node);
+
+ /* Remove mmio access. */
+ for ( i = 0; i < naddr; i++ )
+ {
+ rc = dt_device_get_address(device_node, i, &addr, &size);
+ if ( rc )
+ {
+ printk(XENLOG_ERR "Unable to retrieve address %u for %s\n",
+ i, dt_node_full_name(device_node));
+ return rc;
+ }
+
+ rc = iomem_deny_access(d, paddr_to_pfn(addr),
+ paddr_to_pfn(PAGE_ALIGN(addr + size - 1)));
+ if ( rc )
+ {
+ printk(XENLOG_ERR "Unable to remove dom%d access to"
+ " 0x%"PRIx64" - 0x%"PRIx64"\n",
+ d->domain_id,
+ addr & PAGE_MASK, PAGE_ALIGN(addr + size) - 1);
+ return rc;
+ }
+
+ }
+
+ return rc;
+}
+
+/* Removes all descendants of the given node. */
+static int remove_all_descendant_nodes(struct dt_device_node *device_node)
+{
+ int rc = 0;
+ struct dt_device_node *child_node;
+
+ for ( child_node = device_node->child; child_node != NULL;
+ child_node = child_node->sibling )
+ {
+ if ( child_node->child )
+ remove_all_descendant_nodes(child_node);
+
+ rc = handle_remove_irq_iommu(child_node);
+ if ( rc )
+ return rc;
+ }
+
+ return rc;
+}
+
+/* Remove nodes from dt_host. */
+static int remove_nodes(const struct overlay_track *tracker)
+{
+ int rc = 0;
+ struct dt_device_node *overlay_node;
+ unsigned int j;
+
+ for ( j = 0; j < tracker->num_nodes; j++ )
+ {
+ overlay_node = (struct dt_device_node *)tracker->nodes_address[j];
+ if ( overlay_node == NULL )
+ {
+ printk(XENLOG_ERR "Device %s is not present in the tree. Removing nodes failed\n",
+ overlay_node->full_name);
+ return -EINVAL;
+ }
+
+ rc = remove_all_descendant_nodes(overlay_node);
+
+ /* All children nodes are unmapped. Now remove the node itself. */
+ rc = handle_remove_irq_iommu(overlay_node);
+ if ( rc )
+ return rc;
+
+ read_lock(&dt_host->lock);
+
+ rc = dt_overlay_remove_node(overlay_node);
+ if ( rc )
+ {
+ read_unlock(&dt_host->lock);
+
+ return rc;
+ }
+
+ read_unlock(&dt_host->lock);
+ }
+
+ return rc;
+}
+
+/*
+ * First finds the device node to remove. Check if the device is being used by
+ * any dom and finally remove it from dt_host. IOMMU is already being taken care
+ * while destroying the domain.
+ */
+static long handle_remove_overlay_nodes(void *overlay_fdt,
+ uint32_t overlay_fdt_size)
+{
+ int rc = 0;
+ struct overlay_track *entry, *temp, *track;
+ bool found_entry = false;
+
+ rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
+ if ( rc )
+ return rc;
+
+ if ( overlay_node_count(overlay_fdt) == 0 )
+ return -ENOMEM;
+
+ spin_lock(&overlay_lock);
+
+ /*
+ * First check if dtbo is correct i.e. it should one of the dtbo which was
+ * used when dynamically adding the node.
+ * Limitation: Cases with same node names but different property are not
+ * supported currently. We are relying on user to provide the same dtbo
+ * as it was used when adding the nodes.
+ */
+ list_for_each_entry_safe( entry, temp, &overlay_tracker, entry )
+ {
+ if ( memcmp(entry->overlay_fdt, overlay_fdt, overlay_fdt_size) == 0 )
+ {
+ track = entry;
+ found_entry = true;
+ break;
+ }
+ }
+
+ if ( found_entry == false )
+ {
+ rc = -EINVAL;
+
+ printk(XENLOG_ERR "Cannot find any matching tracker with input dtbo."
+ " Removing nodes is supported for only prior added dtbo. Please"
+ " provide a valid dtbo which was used to add the nodes.\n");
+ goto out;
+
+ }
+
+ rc = remove_nodes(entry);
+
+ if ( rc )
+ {
+ printk(XENLOG_ERR "Removing node failed\n");
+ goto out;
+ }
+
+ list_del(&entry->entry);
+
+ xfree(entry->dt_host_new);
+ xfree(entry->fdt);
+ xfree(entry->overlay_fdt);
+
+ xfree(entry->nodes_address);
+
+ xfree(entry);
+
+out:
+ spin_unlock(&overlay_lock);
+ return rc;
+}
+
+long dt_sysctl(struct xen_sysctl_dt_overlay *op)
+{
+ long ret;
+ void *overlay_fdt;
+
+ if ( op->overlay_fdt_size <= 0 || op->overlay_fdt_size > KB(500) )
+ return -EINVAL;
+
+ overlay_fdt = xmalloc_bytes(op->overlay_fdt_size);
+
+ if ( overlay_fdt == NULL )
+ return -ENOMEM;
+
+ ret = copy_from_guest(overlay_fdt, op->overlay_fdt, op->overlay_fdt_size);
+ if ( ret )
+ {
+ gprintk(XENLOG_ERR, "copy from guest failed\n");
+ xfree(overlay_fdt);
+
+ return -EFAULT;
+ }
+
+ switch ( op->overlay_op )
+ {
+ case XEN_SYSCTL_DT_OVERLAY_REMOVE:
+ ret = handle_remove_overlay_nodes(overlay_fdt, op->overlay_fdt_size);
+ xfree(overlay_fdt);
+
+ break;
+
+ default:
+ xfree(overlay_fdt);
+ break;
+ }
+
+ return ret;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
index 2b24d6bfd0..1158c1efb3 100644
--- a/xen/include/public/sysctl.h
+++ b/xen/include/public/sysctl.h
@@ -1057,6 +1057,25 @@ typedef struct xen_sysctl_cpu_policy xen_sysctl_cpu_policy_t;
DEFINE_XEN_GUEST_HANDLE(xen_sysctl_cpu_policy_t);
#endif
+#if defined(__arm__) || defined (__aarch64__)
+#define XEN_SYSCTL_DT_OVERLAY_ADD 1
+#define XEN_SYSCTL_DT_OVERLAY_REMOVE 2
+
+/*
+ * XEN_SYSCTL_dt_overlay
+ * Performs addition/removal of device tree nodes under parent node using dtbo.
+ * This does in three steps:
+ * - Adds/Removes the nodes from dt_host.
+ * - Adds/Removes IRQ permission for the nodes.
+ * - Adds/Removes MMIO accesses.
+ */
+struct xen_sysctl_dt_overlay {
+ XEN_GUEST_HANDLE_64(void) overlay_fdt; /* IN: overlay fdt. */
+ uint32_t overlay_fdt_size; /* IN: Overlay dtb size. */
+ uint8_t overlay_op; /* IN: Add or remove. */
+};
+#endif
+
struct xen_sysctl {
uint32_t cmd;
#define XEN_SYSCTL_readconsole 1
@@ -1087,6 +1106,7 @@ struct xen_sysctl {
#define XEN_SYSCTL_livepatch_op 27
/* #define XEN_SYSCTL_set_parameter 28 */
#define XEN_SYSCTL_get_cpu_policy 29
+#define XEN_SYSCTL_dt_overlay 30
uint32_t interface_version; /* XEN_SYSCTL_INTERFACE_VERSION */
union {
struct xen_sysctl_readconsole readconsole;
@@ -1117,6 +1137,10 @@ struct xen_sysctl {
#if defined(__i386__) || defined(__x86_64__)
struct xen_sysctl_cpu_policy cpu_policy;
#endif
+
+#if defined(__arm__) || defined (__aarch64__)
+ struct xen_sysctl_dt_overlay dt_overlay;
+#endif
uint8_t pad[128];
} u;
};
diff --git a/xen/include/xen/dt_overlay.h b/xen/include/xen/dt_overlay.h
new file mode 100644
index 0000000000..2cd975a070
--- /dev/null
+++ b/xen/include/xen/dt_overlay.h
@@ -0,0 +1,59 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0
+ *
+ * xen/dt_overlay.h
+ *
+ * Device tree overlay support in Xen.
+ *
+ * Copyright (c) 2022 AMD Inc.
+ * Written by Vikram Garhwal <vikram.garhwal@amd.com>
+ *
+ */
+#ifndef __XEN_DT_SYSCTL_H__
+#define __XEN_DT_SYSCTL_H__
+
+#include <xen/list.h>
+#include <xen/libfdt/libfdt.h>
+#include <xen/device_tree.h>
+#include <xen/rangeset.h>
+
+/*
+ * overlay_node_track describes information about added nodes through dtbo.
+ * @entry: List pointer.
+ * @dt_host_new: Pointer to the updated dt_host_new unflattened 'updated fdt'.
+ * @fdt: Stores the fdt.
+ * @nodes_fullname: Stores the full name of nodes.
+ * @nodes_irq: Stores the IRQ added from overlay dtb.
+ * @node_num_irq: Stores num of IRQ for each node in overlay dtb.
+ * @num_nodes: Stores total number of nodes in overlay dtb.
+ */
+struct overlay_track {
+ struct list_head entry;
+ struct dt_device_node *dt_host_new;
+ void *fdt;
+ void *overlay_fdt;
+ unsigned long *nodes_address;
+ unsigned int num_nodes;
+};
+
+struct xen_sysctl_dt_overlay;
+
+#ifdef CONFIG_OVERLAY_DTB
+long dt_sysctl(struct xen_sysctl_dt_overlay *op);
+#else
+static inline long dt_sysctl(struct xen_sysctl_dt_overlay *op)
+{
+ return -ENOSYS;
+}
+#endif
+#endif
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 14/17] xen/arm: Implement device tree node addition functionalities
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (30 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 13/17] xen/arm: Implement device tree node removal functionalities Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 15/17] tools/libs/ctrl: Implement new xc interfaces for dt overlay Vikram Garhwal
` (2 subsequent siblings)
34 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel
Cc: sstabellini, Vikram Garhwal, Andrew Cooper, George Dunlap,
Jan Beulich, Julien Grall, Wei Liu
Update sysctl XEN_SYSCTL_dt_overlay to enable support for dtbo nodes addition
using device tree overlay.
xl dt-overlay add file.dtbo:
Each time overlay nodes are added using .dtbo, a new fdt(memcpy of
device_tree_flattened) is created and updated with overlay nodes. This
updated fdt is further unflattened to a dt_host_new. Next, it checks if any
of the overlay nodes already exists in the dt_host. If overlay nodes doesn't
exist then find the overlay nodes in dt_host_new, find the overlay node's
parent in dt_host and add the nodes as child under their parent in the
dt_host. The node is attached as the last node under target parent.
Finally, add IRQs, add device to IOMMUs, set permissions and map MMIO for the
overlay node.
When a node is added using overlay, a new entry is allocated in the
overlay_track to keep the track of memory allocation due to addition of overlay
node. This is helpful for freeing the memory allocated when a device tree node
is removed.
The main purpose of this to address first part of dynamic programming i.e.
making xen aware of new device tree node which means updating the dt_host with
overlay node information. Here we are adding/removing node from dt_host, and
checking/setting IOMMU and IRQ permission but never mapping them to any domain.
Right now, mapping/Un-mapping will happen only when a new domU is
created/destroyed using "xl create".
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
xen/common/dt_overlay.c | 482 ++++++++++++++++++++++++++++++++++++++++
1 file changed, 482 insertions(+)
diff --git a/xen/common/dt_overlay.c b/xen/common/dt_overlay.c
index 516e8010c5..3344bad313 100644
--- a/xen/common/dt_overlay.c
+++ b/xen/common/dt_overlay.c
@@ -36,6 +36,25 @@ static struct dt_device_node *find_last_descendants_node(
return child_node;
}
+/*
+ * Returns next node to the input node. If node has children then return
+ * last descendant's next node.
+*/
+static struct dt_device_node *dt_find_next_node(struct dt_device_node *dt,
+ const struct dt_device_node *node)
+{
+ struct dt_device_node *np;
+
+ dt_for_each_device_node(dt, np)
+ if ( np == node )
+ break;
+
+ if ( np->child )
+ np = find_last_descendants_node(np);
+
+ return np->allnext;
+}
+
static int dt_overlay_remove_node(struct dt_device_node *device_node)
{
struct dt_device_node *np;
@@ -109,6 +128,72 @@ static int dt_overlay_remove_node(struct dt_device_node *device_node)
return 0;
}
+static int dt_overlay_add_node(struct dt_device_node *device_node,
+ const char *parent_node_path)
+{
+ struct dt_device_node *parent_node;
+ struct dt_device_node *np, *np_last_descendant;
+ struct dt_device_node *next_node;
+ struct dt_device_node *device_node_last_descendant;
+
+ parent_node = dt_find_node_by_path(parent_node_path);
+
+ if ( parent_node == NULL )
+ {
+ dt_dprintk("Node not found. Overlay node will not be added\n");
+ return -EINVAL;
+ }
+
+ /* If parent has no child. */
+ if ( parent_node->child == NULL )
+ {
+ next_node = parent_node->allnext;
+ device_node->parent = parent_node;
+ parent_node->allnext = device_node;
+ parent_node->child = device_node;
+ }
+ else
+ {
+ /* If parent has at least one child node.
+ * Iterate to the last child node of parent.
+ */
+ for ( np = parent_node->child; np->sibling != NULL; np = np->sibling );
+
+ /* Iterate over all child nodes of np node. */
+ if ( np->child )
+ {
+ np_last_descendant = find_last_descendants_node(np);
+
+ next_node = np_last_descendant->allnext;
+ np_last_descendant->allnext = device_node;
+ }
+ else
+ {
+ next_node = np->allnext;
+ np->allnext = device_node;
+ }
+
+ device_node->parent = parent_node;
+ np->sibling = device_node;
+ np->sibling->sibling = NULL;
+ }
+
+ /* Iterate over all child nodes of device_node to add children too. */
+ if ( device_node->child )
+ {
+ device_node_last_descendant = find_last_descendants_node(device_node);
+ /* Plug next_node at the end of last children of device_node. */
+ device_node_last_descendant->allnext = next_node;
+ }
+ else
+ {
+ /* Now plug next_node at the end of device_node. */
+ device_node->allnext = next_node;
+ }
+
+ return 0;
+}
+
/* Basic sanity check for the dtbo tool stack provided to Xen. */
static int check_overlay_fdt(const void *overlay_fdt, uint32_t overlay_fdt_size)
{
@@ -148,6 +233,79 @@ static unsigned int overlay_node_count(void *fdto)
return num_overlay_nodes;
}
+/*
+ * overlay_get_nodes_info will get full name with path for all the nodes which
+ * are in one level of __overlay__ tag. This is useful when checking node for
+ * duplication i.e. dtbo tries to add nodes which already exists in device tree.
+ */
+static int overlay_get_nodes_info(const void *fdto, char ***nodes_full_path,
+ unsigned int num_overlay_nodes)
+{
+ int fragment;
+
+ *nodes_full_path = xzalloc_bytes(num_overlay_nodes * sizeof(char *));
+
+ if ( *nodes_full_path == NULL )
+ return -ENOMEM;
+
+ fdt_for_each_subnode(fragment, fdto, 0)
+ {
+ int target;
+ int overlay;
+ int subnode;
+ const char *target_path;
+
+ target = fdt_overlay_target_offset(device_tree_flattened, fdto,
+ fragment, &target_path);
+ if ( target < 0 )
+ return target;
+
+ overlay = fdt_subnode_offset(fdto, fragment, "__overlay__");
+
+ /*
+ * overlay value can be < 0. But fdt_for_each_subnode() loop checks for
+ * overlay >= 0. So, no need for a overlay>=0 check here.
+ */
+ fdt_for_each_subnode(subnode, fdto, overlay)
+ {
+ const char *node_name = NULL;
+ int node_name_len;
+ unsigned int target_path_len = strlen(target_path);
+ unsigned int node_full_name_len;
+ unsigned int node_num = 0;
+
+ node_name = fdt_get_name(fdto, subnode, &node_name_len);
+
+ if ( node_name == NULL )
+ return node_name_len;
+
+ /*
+ * Magic number 2 is for adding '/'. This is done to keep the
+ * node_full_name in the correct full node name format.
+ */
+ node_full_name_len = target_path_len + node_name_len + 2;
+
+ (*nodes_full_path)[node_num] = xmalloc_bytes(node_full_name_len);
+
+ if ( (*nodes_full_path)[node_num] == NULL )
+ return -ENOMEM;
+
+ memcpy((*nodes_full_path)[node_num], target_path, target_path_len);
+
+ (*nodes_full_path)[node_num][target_path_len] = '/';
+
+ memcpy((*nodes_full_path)[node_num] + target_path_len + 1,
+ node_name, node_name_len);
+
+ (*nodes_full_path)[node_num][node_full_name_len - 1] = '\0';
+
+ node_num++;
+ }
+ }
+
+ return 0;
+}
+
static int handle_remove_irq_iommu(struct dt_device_node *device_node)
{
int rc = 0;
@@ -367,6 +525,322 @@ out:
return rc;
}
+/*
+ * Handles IRQ and IOMMU mapping for the overlay_node and all descendants of the
+ * overlay_nodes.
+ */
+static int handle_add_irq_iommu(struct domain *d,
+ struct dt_device_node *overlay_node)
+{
+ int rc;
+ unsigned int naddr, i, len;
+ uint64_t addr, size;
+ struct dt_device_node *np;
+
+ /* First let's handle the interrupts. */
+ rc = handle_device_interrupts(d, overlay_node, false);
+ if ( rc < 0 )
+ {
+ printk(XENLOG_ERR "Interrupt failed\n");
+ return rc;
+ }
+
+ /* Check if iommu property exists. */
+ if ( dt_get_property(overlay_node, "iommus", &len) )
+ {
+
+ /* Add device to IOMMUs. */
+ rc = iommu_add_dt_device(overlay_node);
+ if ( rc < 0 )
+ {
+ printk(XENLOG_ERR "Failed to add %s to the IOMMU\n",
+ dt_node_full_name(overlay_node));
+ return rc;
+ }
+ }
+
+ /* Set permissions. */
+ naddr = dt_number_of_address(overlay_node);
+
+ dt_dprintk("%s passthrough = %d naddr = %u\n",
+ dt_node_full_name(overlay_node), false, naddr);
+
+ /* Give permission for map MMIOs */
+ for ( i = 0; i < naddr; i++ )
+ {
+ /*
+ * For now, we skip_mapping which means it will only permit iomem access
+ * to hardware_domain using iomem_permit_access() but will never map as
+ * map_range_p2mt() will not be called.
+ */
+ struct map_range_data mr_data = { .d = d,
+ .p2mt = p2m_mmio_direct_c,
+ .skip_mapping = true };
+
+ rc = dt_device_get_address(overlay_node, i, &addr, &size);
+ if ( rc )
+ {
+ printk(XENLOG_ERR "Unable to retrieve address %u for %s\n",
+ i, dt_node_full_name(overlay_node));
+ return rc;
+ }
+
+ rc = map_range_to_domain(overlay_node, addr, size, &mr_data);
+ if ( rc )
+ return rc;
+ }
+
+ /* Map IRQ and IOMMU for overlay_node's children. */
+ for ( np = overlay_node->child; np != NULL; np = np->sibling)
+ {
+ rc = handle_add_irq_iommu(d, np);
+ if ( rc )
+ {
+ printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n");
+ return rc;
+ }
+ }
+
+ return rc;
+}
+
+/*
+ * Adds device tree nodes under target node.
+ * We use tr->dt_host_new to unflatten the updated device_tree_flattened. This
+ * is done to avoid the removal of device_tree generation, iomem regions mapping
+ * to hardware domain done by handle_node().
+ */
+static long handle_add_overlay_nodes(void *overlay_fdt,
+ uint32_t overlay_fdt_size)
+{
+ int rc, j, i;
+ struct dt_device_node *overlay_node, *prev_node, *next_node;
+ struct domain *d = hardware_domain;
+ struct overlay_track *tr = NULL;
+ char **nodes_full_path = NULL;
+ unsigned int new_fdt_size;
+
+ tr = xzalloc(struct overlay_track);
+ if ( tr == NULL )
+ return -ENOMEM;
+
+ new_fdt_size = fdt_totalsize(device_tree_flattened) +
+ fdt_totalsize(overlay_fdt);
+
+ tr->fdt = xzalloc_bytes(new_fdt_size);
+ if ( tr->fdt == NULL )
+ {
+ xfree(tr);
+ return -ENOMEM;
+ }
+
+ tr->num_nodes = overlay_node_count(overlay_fdt);
+ if ( tr->num_nodes == 0 )
+ {
+ xfree(tr->fdt);
+ xfree(tr);
+ return -ENOMEM;
+ }
+
+ tr->nodes_address = xzalloc_bytes(tr->num_nodes * sizeof(unsigned long));
+ if ( tr->nodes_address == NULL )
+ {
+ xfree(tr->fdt);
+ xfree(tr);
+ return -ENOMEM;
+ }
+
+ rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
+ if ( rc )
+ {
+ xfree(tr->nodes_address);
+ xfree(tr->fdt);
+ xfree(tr);
+ return rc;
+ }
+
+ /*
+ * Keep a copy of overlay_fdt as fdt_overlay_apply will change the input
+ * overlay's content(magic) when applying overlay.
+ */
+ tr->overlay_fdt = xzalloc_bytes(overlay_fdt_size);
+ if ( tr->overlay_fdt == NULL )
+ {
+ xfree(tr->nodes_address);
+ xfree(tr->fdt);
+ xfree(tr);
+ return -ENOMEM;
+ }
+
+ memcpy(tr->overlay_fdt, overlay_fdt, overlay_fdt_size);
+
+ spin_lock(&overlay_lock);
+
+ memcpy(tr->fdt, device_tree_flattened,
+ fdt_totalsize(device_tree_flattened));
+
+ /* Open tr->fdt with more space to accommodate the overlay_fdt. */
+ fdt_open_into(tr->fdt, tr->fdt, new_fdt_size);
+
+ /*
+ * overlay_get_nodes_info is called to get the node information from dtbo.
+ * This is done before fdt_overlay_apply() because the overlay apply will
+ * erase the magic of overlay_fdt.
+ */
+ rc = overlay_get_nodes_info(overlay_fdt, &nodes_full_path,
+ tr->num_nodes);
+ if ( rc )
+ {
+ printk(XENLOG_ERR "Getting nodes information failed with error %d\n",
+ rc);
+ goto err;
+ }
+
+ rc = fdt_overlay_apply(tr->fdt, overlay_fdt);
+ if ( rc )
+ {
+ printk(XENLOG_ERR "Adding overlay node failed with error %d\n", rc);
+ goto err;
+ }
+
+ /*
+ * Check if any of the node already exists in dt_host. If node already exits
+ * we can return here as this overlay_fdt is not suitable for overlay ops.
+ */
+ for ( j = 0; j < tr->num_nodes; j++ )
+ {
+ overlay_node = dt_find_node_by_path(nodes_full_path[j]);
+ if ( overlay_node != NULL )
+ {
+ printk(XENLOG_ERR "node %s exists in device tree\n",
+ nodes_full_path[j]);
+ rc = -EINVAL;
+ goto err;
+ }
+ }
+
+ /* Unflatten the tr->fdt into a new dt_host. */
+ unflatten_device_tree(tr->fdt, &tr->dt_host_new);
+
+ for ( j = 0; j < tr->num_nodes; j++ )
+ {
+ dt_dprintk("Adding node: %s\n", nodes_full_path[j]);
+
+ /* Find the newly added node in tr->dt_host_new by it's full path. */
+ overlay_node = device_tree_find_node_by_path(tr->dt_host_new,
+ nodes_full_path[j]);
+ if ( overlay_node == NULL )
+ {
+ dt_dprintk("%s node not found\n", nodes_full_path[j]);
+ rc = -EFAULT;
+ goto remove_node;
+ }
+
+ /*
+ * Find previous and next node to overlay_node in dt_host_new. We will
+ * need these nodes to fix the dt_host_new mapping. When overlay_node is
+ * take out of dt_host_new tree and added to dt_host, link between
+ * previous node and next_node is broken. We will need to refresh
+ * dt_host_new with correct linking for any other overlay nodes
+ * extraction in future.
+ */
+ dt_for_each_device_node(tr->dt_host_new, prev_node)
+ if ( prev_node->allnext == overlay_node )
+ break;
+
+ next_node = dt_find_next_node(tr->dt_host_new, overlay_node);
+
+ read_lock(&dt_host->lock);
+
+ /* Add the node to dt_host. */
+ rc = dt_overlay_add_node(overlay_node, overlay_node->parent->full_name);
+ if ( rc )
+ {
+ read_unlock(&dt_host->lock);
+
+ /* Node not added in dt_host. */
+ goto remove_node;
+ }
+
+ read_unlock(&dt_host->lock);
+
+ prev_node->allnext = next_node;
+
+ overlay_node = dt_find_node_by_path(overlay_node->full_name);
+ if ( overlay_node == NULL )
+ {
+ /* Sanity check. But code will never come here. */
+ ASSERT_UNREACHABLE();
+ goto remove_node;
+ }
+
+ rc = handle_add_irq_iommu(d, overlay_node);
+ if ( rc )
+ {
+ printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n");
+ return rc;
+ }
+
+ /* Keep overlay_node address in tracker. */
+ tr->nodes_address[j] = (unsigned long)overlay_node;
+ }
+
+ INIT_LIST_HEAD(&tr->entry);
+ list_add_tail(&tr->entry, &overlay_tracker);
+
+ spin_unlock(&overlay_lock);
+
+ if ( nodes_full_path != NULL )
+ {
+ for ( i = 0; i < tr->num_nodes && nodes_full_path[i] != NULL;
+ i++ )
+ {
+ xfree(nodes_full_path[i]);
+ }
+ xfree(nodes_full_path);
+ }
+
+ return rc;
+
+/*
+ * Failure case. We need to remove the nodes, free tracker(if tr exists) and
+ * tr->dt_host_new.
+ */
+remove_node:
+ tr->num_nodes = j;
+ rc = remove_nodes(tr);
+
+ if ( rc )
+ {
+ /* If removing node fails, this may cause memory leaks. */
+ printk(XENLOG_ERR "Removing node failed.\n");
+ spin_unlock(&overlay_lock);
+ return rc;
+ }
+
+err:
+ spin_unlock(&overlay_lock);
+
+ xfree(tr->dt_host_new);
+ xfree(tr->fdt);
+ xfree(tr->overlay_fdt);
+ xfree(tr->nodes_address);
+
+ if ( nodes_full_path != NULL )
+ {
+ for ( i = 0; i < tr->num_nodes && nodes_full_path[i] != NULL;
+ i++ )
+ {
+ xfree(nodes_full_path[i]);
+ }
+ xfree(nodes_full_path);
+ }
+
+ xfree(tr);
+
+ return rc;
+}
+
long dt_sysctl(struct xen_sysctl_dt_overlay *op)
{
long ret;
@@ -391,6 +865,14 @@ long dt_sysctl(struct xen_sysctl_dt_overlay *op)
switch ( op->overlay_op )
{
+ case XEN_SYSCTL_DT_OVERLAY_ADD:
+ ret = handle_add_overlay_nodes(overlay_fdt, op->overlay_fdt_size);
+
+ if ( ret )
+ xfree(overlay_fdt);
+
+ break;
+
case XEN_SYSCTL_DT_OVERLAY_REMOVE:
ret = handle_remove_overlay_nodes(overlay_fdt, op->overlay_fdt_size);
xfree(overlay_fdt);
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 15/17] tools/libs/ctrl: Implement new xc interfaces for dt overlay
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (31 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 14/17] xen/arm: Implement device tree node addition functionalities Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 16/17] tools/libs/light: Implement new libxl functions for device tree overlay ops Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 17/17] tools/xl: Add new xl command overlay for device tree overlay support Vikram Garhwal
34 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel
Cc: sstabellini, Vikram Garhwal, Wei Liu, Anthony PERARD, Juergen Gross
xc_dt_overlay() sends the device tree binary overlay, size of .dtbo and overlay
operation type i.e. add or remove to xen.
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
tools/include/xenctrl.h | 5 ++++
tools/libs/ctrl/Makefile.common | 1 +
tools/libs/ctrl/xc_dt_overlay.c | 48 +++++++++++++++++++++++++++++++++
3 files changed, 54 insertions(+)
create mode 100644 tools/libs/ctrl/xc_dt_overlay.c
diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h
index 05967ecc92..b932589c8d 100644
--- a/tools/include/xenctrl.h
+++ b/tools/include/xenctrl.h
@@ -2637,6 +2637,11 @@ int xc_livepatch_replace(xc_interface *xch, char *name, uint32_t timeout, uint32
int xc_domain_cacheflush(xc_interface *xch, uint32_t domid,
xen_pfn_t start_pfn, xen_pfn_t nr_pfns);
+#if defined(__arm__) || defined(__aarch64__)
+int xc_dt_overlay(xc_interface *xch, void *overlay_fdt,
+ uint32_t overlay_fdt_size, uint8_t overlay_op);
+#endif
+
/* Compat shims */
#include "xenctrl_compat.h"
diff --git a/tools/libs/ctrl/Makefile.common b/tools/libs/ctrl/Makefile.common
index 0a09c28fd3..247afbe5f9 100644
--- a/tools/libs/ctrl/Makefile.common
+++ b/tools/libs/ctrl/Makefile.common
@@ -24,6 +24,7 @@ OBJS-y += xc_hcall_buf.o
OBJS-y += xc_foreign_memory.o
OBJS-y += xc_kexec.o
OBJS-y += xc_resource.o
+OBJS-$(CONFIG_ARM) += xc_dt_overlay.o
OBJS-$(CONFIG_X86) += xc_psr.o
OBJS-$(CONFIG_X86) += xc_pagetab.o
OBJS-$(CONFIG_Linux) += xc_linux.o
diff --git a/tools/libs/ctrl/xc_dt_overlay.c b/tools/libs/ctrl/xc_dt_overlay.c
new file mode 100644
index 0000000000..202fc906f4
--- /dev/null
+++ b/tools/libs/ctrl/xc_dt_overlay.c
@@ -0,0 +1,48 @@
+/*
+ *
+ * Device Tree Overlay functions.
+ * Copyright (C) 2021 Xilinx Inc.
+ * Author Vikram Garhwal <fnu.vikram@xilinx.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation;
+ * version 2.1 of the License.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "xc_private.h"
+
+int xc_dt_overlay(xc_interface *xch, void *overlay_fdt,
+ uint32_t overlay_fdt_size, uint8_t overlay_op)
+{
+ int err;
+ DECLARE_SYSCTL;
+
+ DECLARE_HYPERCALL_BOUNCE(overlay_fdt, overlay_fdt_size,
+ XC_HYPERCALL_BUFFER_BOUNCE_IN);
+
+ if ( (err = xc_hypercall_bounce_pre(xch, overlay_fdt)) )
+ goto err;
+
+ sysctl.cmd = XEN_SYSCTL_dt_overlay;
+ sysctl.u.dt_overlay.overlay_op = overlay_op;
+ sysctl.u.dt_overlay.overlay_fdt_size = overlay_fdt_size;
+
+ set_xen_guest_handle(sysctl.u.dt_overlay.overlay_fdt, overlay_fdt);
+
+ if ( (err = do_sysctl(xch, &sysctl)) != 0 )
+ PERROR("%s failed", __func__);
+
+err:
+ xc_hypercall_bounce_post(xch, overlay_fdt);
+
+ return err;
+}
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 16/17] tools/libs/light: Implement new libxl functions for device tree overlay ops
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (32 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 15/17] tools/libs/ctrl: Implement new xc interfaces for dt overlay Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 17/17] tools/xl: Add new xl command overlay for device tree overlay support Vikram Garhwal
34 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel
Cc: sstabellini, Vikram Garhwal, Wei Liu, Anthony PERARD, Juergen Gross
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
tools/include/libxl.h | 11 +++++
tools/libs/light/Makefile | 3 ++
tools/libs/light/libxl_dt_overlay.c | 71 +++++++++++++++++++++++++++++
3 files changed, 85 insertions(+)
create mode 100644 tools/libs/light/libxl_dt_overlay.c
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index cfa1a19131..1c5e8abaae 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -250,6 +250,12 @@
*/
#define LIBXL_HAVE_DEVICETREE_PASSTHROUGH 1
+#if defined(__arm__) || defined(__aarch64__)
+/**
+ * This means Device Tree Overlay is supported.
+ */
+#define LIBXL_HAVE_DT_OVERLAY 1
+#endif
/*
* libxl_domain_build_info has device_model_user to specify the user to
* run the device model with. See docs/misc/qemu-deprivilege.txt.
@@ -2453,6 +2459,11 @@ libxl_device_pci *libxl_device_pci_list(libxl_ctx *ctx, uint32_t domid,
int *num);
void libxl_device_pci_list_free(libxl_device_pci* list, int num);
+#if defined(__arm__) || defined(__aarch64__)
+int libxl_dt_overlay(libxl_ctx *ctx, void *overlay,
+ uint32_t overlay_size, uint8_t overlay_op);
+#endif
+
/*
* Turns the current process into a backend device service daemon
* for a driver domain.
diff --git a/tools/libs/light/Makefile b/tools/libs/light/Makefile
index 96daeabc47..563a1e8d0a 100644
--- a/tools/libs/light/Makefile
+++ b/tools/libs/light/Makefile
@@ -112,6 +112,9 @@ OBJS-y += _libxl_types.o
OBJS-y += libxl_flask.o
OBJS-y += _libxl_types_internal.o
+# Device tree overlay is enabled only for ARM architecture.
+OBJS-$(CONFIG_ARM) += libxl_dt_overlay.o
+
ifeq ($(CONFIG_LIBNL),y)
CFLAGS_LIBXL += $(LIBNL3_CFLAGS)
endif
diff --git a/tools/libs/light/libxl_dt_overlay.c b/tools/libs/light/libxl_dt_overlay.c
new file mode 100644
index 0000000000..a6c709a6dc
--- /dev/null
+++ b/tools/libs/light/libxl_dt_overlay.c
@@ -0,0 +1,71 @@
+/*
+ * Copyright (C) 2021 Xilinx Inc.
+ * Author Vikram Garhwal <fnu.vikram@xilinx.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as published
+ * by the Free Software Foundation; version 2.1 only. with the special
+ * exception on linking described in file LICENSE.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU Lesser General Public License for more details.
+ */
+
+#include "libxl_osdeps.h" /* must come before any other headers */
+#include "libxl_internal.h"
+#include <libfdt.h>
+#include <xenctrl.h>
+
+static int check_overlay_fdt(libxl__gc *gc, void *fdt, size_t size)
+{
+ int r;
+
+ if (fdt_magic(fdt) != FDT_MAGIC) {
+ LOG(ERROR, "Overlay FDT is not a valid Flat Device Tree");
+ return ERROR_FAIL;
+ }
+
+ r = fdt_check_header(fdt);
+ if (r) {
+ LOG(ERROR, "Failed to check the overlay FDT (%d)", r);
+ return ERROR_FAIL;
+ }
+
+ if (fdt_totalsize(fdt) > size) {
+ LOG(ERROR, "Overlay FDT totalsize is too big");
+ return ERROR_FAIL;
+ }
+
+ return 0;
+}
+
+int libxl_dt_overlay(libxl_ctx *ctx, void *overlay_dt, uint32_t overlay_dt_size,
+ uint8_t overlay_op)
+{
+ int rc;
+ int r;
+ GC_INIT(ctx);
+
+ if (check_overlay_fdt(gc, overlay_dt, overlay_dt_size)) {
+ LOG(ERROR, "Overlay DTB check failed");
+ rc = ERROR_FAIL;
+ goto out;
+ } else {
+ LOG(DEBUG, "Overlay DTB check passed");
+ rc = 0;
+ }
+
+ r = xc_dt_overlay(ctx->xch, overlay_dt, overlay_dt_size, overlay_op);
+
+ if (r) {
+ LOG(ERROR, "%s: Adding/Removing overlay dtb failed.", __func__);
+ rc = ERROR_FAIL;
+ }
+
+out:
+ GC_FREE;
+ return rc;
+}
+
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* [XEN][PATCH v5 17/17] tools/xl: Add new xl command overlay for device tree overlay support
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
` (33 preceding siblings ...)
2023-04-11 19:16 ` [XEN][PATCH v5 16/17] tools/libs/light: Implement new libxl functions for device tree overlay ops Vikram Garhwal
@ 2023-04-11 19:16 ` Vikram Garhwal
34 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-11 19:16 UTC (permalink / raw)
To: xen-devel; +Cc: sstabellini, Vikram Garhwal, Wei Liu, Anthony PERARD
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
tools/xl/xl.h | 1 +
tools/xl/xl_cmdtable.c | 6 +++++
tools/xl/xl_vmcontrol.c | 52 +++++++++++++++++++++++++++++++++++++++++
3 files changed, 59 insertions(+)
diff --git a/tools/xl/xl.h b/tools/xl/xl.h
index 72538d6a81..a923daccd3 100644
--- a/tools/xl/xl.h
+++ b/tools/xl/xl.h
@@ -138,6 +138,7 @@ int main_shutdown(int argc, char **argv);
int main_reboot(int argc, char **argv);
int main_list(int argc, char **argv);
int main_vm_list(int argc, char **argv);
+int main_dt_overlay(int argc, char **argv);
int main_create(int argc, char **argv);
int main_config_update(int argc, char **argv);
int main_button_press(int argc, char **argv);
diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index ccf4d83584..db0acff62a 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -630,6 +630,12 @@ const struct cmd_spec cmd_table[] = {
"Issue a qemu monitor command to the device model of a domain",
"<Domain> <Command>",
},
+ { "dt-overlay",
+ &main_dt_overlay, 0, 1,
+ "Add/Remove a device tree overlay",
+ "add/remove <.dtbo>"
+ "-h print this help\n"
+ },
};
const int cmdtable_len = ARRAY_SIZE(cmd_table);
diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c
index 5518c78dc6..de56e00d8b 100644
--- a/tools/xl/xl_vmcontrol.c
+++ b/tools/xl/xl_vmcontrol.c
@@ -1265,6 +1265,58 @@ int main_create(int argc, char **argv)
return 0;
}
+int main_dt_overlay(int argc, char **argv)
+{
+ const char *overlay_ops = NULL;
+ const char *overlay_config_file = NULL;
+ void *overlay_dtb = NULL;
+ int rc;
+ uint8_t op;
+ int overlay_dtb_size = 0;
+ const int overlay_add_op = 1;
+ const int overlay_remove_op = 2;
+
+ if (argc < 2) {
+ help("dt_overlay");
+ return EXIT_FAILURE;
+ }
+
+ overlay_ops = argv[1];
+ overlay_config_file = argv[2];
+
+ if (strcmp(overlay_ops, "add") == 0)
+ op = overlay_add_op;
+ else if (strcmp(overlay_ops, "remove") == 0)
+ op = overlay_remove_op;
+ else {
+ fprintf(stderr, "Invalid dt overlay operation\n");
+ return EXIT_FAILURE;
+ }
+
+ if (overlay_config_file) {
+ rc = libxl_read_file_contents(ctx, overlay_config_file,
+ &overlay_dtb, &overlay_dtb_size);
+
+ if (rc) {
+ fprintf(stderr, "failed to read the overlay device tree file %s\n",
+ overlay_config_file);
+ free(overlay_dtb);
+ return ERROR_FAIL;
+ }
+ } else {
+ fprintf(stderr, "overlay dtbo file not provided\n");
+ return ERROR_FAIL;
+ }
+
+ rc = libxl_dt_overlay(ctx, overlay_dtb, overlay_dtb_size, op);
+
+ free(overlay_dtb);
+
+ if (rc)
+ return EXIT_FAILURE;
+
+ return rc;
+}
/*
* Local variables:
* mode: C
--
2.17.1
^ permalink raw reply related [flat|nested] 61+ messages in thread
* Re: [XEN][PATCH v5 01/17] xen/arm/device: Remove __init from function type
2023-04-11 19:16 ` [XEN][PATCH v5 01/17] xen/arm/device: Remove __init from function type Vikram Garhwal
@ 2023-04-13 9:19 ` Michal Orzel
2023-04-14 17:28 ` Vikram Garhwal
0 siblings, 1 reply; 61+ messages in thread
From: Michal Orzel @ 2023-04-13 9:19 UTC (permalink / raw)
To: Vikram Garhwal, xen-devel
Cc: sstabellini, Julien Grall, Bertrand Marquis, Volodymyr Babchuk
Hi Vikram,
On 11/04/2023 21:16, Vikram Garhwal wrote:
>
>
> Remove __init from following function to access during runtime:
> 1. map_irq_to_domain()
> 2. handle_device_interrupt()
s/interrupt/interrupts/ since there is no handle_device_interrupt() function.
> 3. map_range_to_domain()
> 4. unflatten_dt_node()
> 5. unflatten_device_tree()
>
> Move map_irq_to_domain() prototype from domain_build.h to setup.h.
>
> To avoid the breaking the build, following changes are also done:
'avoid breaking' instead of 'avoid the breaking'.
> 1. Move map_irq_to_domain(), handle_device_interrupt() and map_range_to_domain() to
s/interrupt/interrupts/
> device.c. After removing __init type, these functions are not specific to
> domain building, so moving them out of domain_build.c to device.c.
> 2. Remove static type from handle_device_interrupt().
>
> Overall, these changes are done to support the dynamic programming of a nodes
> where an overlay node will be added to fdt and unflattened node will be added to
> dt_host. Furthermore, IRQ and mmio mapping will be done for the added node.
>
> Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
> ---
> xen/arch/arm/device.c | 145 ++++++++++++++++++++++++
> xen/arch/arm/domain_build.c | 142 -----------------------
> xen/arch/arm/include/asm/domain_build.h | 2 -
> xen/arch/arm/include/asm/setup.h | 6 +
> xen/common/device_tree.c | 16 +--
> 5 files changed, 159 insertions(+), 152 deletions(-)
>
> diff --git a/xen/arch/arm/device.c b/xen/arch/arm/device.c
> index ca8539dee5..fec6e29c42 100644
> --- a/xen/arch/arm/device.c
> +++ b/xen/arch/arm/device.c
> @@ -12,6 +12,9 @@
> #include <xen/errno.h>
> #include <xen/init.h>
> #include <xen/lib.h>
> +#include <xen/iocap.h>
> +#include <asm/domain_build.h>
I can't see why do we need to include this header.
> +#include <asm/setup.h>
You should keep the alphabetical order so:
- iocap goes after init.h
- setup.h goes after device.h
Apart from that, it looks ok so:
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
~Michal
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [XEN][PATCH v5 02/17] common/device_tree: change __unflatten_device_tree()
2023-04-11 19:16 ` [XEN][PATCH v5 02/17] common/device_tree: change __unflatten_device_tree() Vikram Garhwal
@ 2023-04-13 9:45 ` Michal Orzel
2023-04-13 9:53 ` Julien Grall
2023-04-13 10:03 ` Julien Grall
1 sibling, 1 reply; 61+ messages in thread
From: Michal Orzel @ 2023-04-13 9:45 UTC (permalink / raw)
To: Vikram Garhwal, xen-devel; +Cc: sstabellini, Julien Grall
Hi Vikram,
On 11/04/2023 21:16, Vikram Garhwal wrote:
>
>
> Following changes are done to __unflatten_device_tree():
> 1. __unflatten_device_tree() is renamed to unflatten_device_tree().
> 2. Remove static function type.
I think there was no need to touch this function in patch 1 if you are modifying it here
additionally in a separate patch.
> 3. Add handling of memory allocation. This will be useful in dynamic node
> programming when we unflatten the dt during runtime memory allocation
> can fail.
Didn't we say that checking if the memory allocation failed or not should be done
as a separate patch (i.e. a prerequisite to your series) as part of hardening?
In any case (depending on the maintainers vote), the change itself looks ok, so:
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
~Michal
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [XEN][PATCH v5 02/17] common/device_tree: change __unflatten_device_tree()
2023-04-13 9:45 ` Michal Orzel
@ 2023-04-13 9:53 ` Julien Grall
0 siblings, 0 replies; 61+ messages in thread
From: Julien Grall @ 2023-04-13 9:53 UTC (permalink / raw)
To: Michal Orzel, Vikram Garhwal, xen-devel; +Cc: sstabellini
Hi,
On 13/04/2023 10:45, Michal Orzel wrote:
> On 11/04/2023 21:16, Vikram Garhwal wrote:
>>
>>
>> Following changes are done to __unflatten_device_tree():
>> 1. __unflatten_device_tree() is renamed to unflatten_device_tree().
>> 2. Remove static function type.
> I think there was no need to touch this function in patch 1 if you are modifying it here
> additionally in a separate patch.
>
>> 3. Add handling of memory allocation. This will be useful in dynamic node
>> programming when we unflatten the dt during runtime memory allocation
>> can fail.
> Didn't we say that checking if the memory allocation failed or not should be done
> as a separate patch (i.e. a prerequisite to your series) as part of hardening?
>
> In any case (depending on the maintainers vote), the change itself looks ok, so:
Yes it should be separate because this is something we may want to
consider to backport.
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [XEN][PATCH v5 03/17] xen/arm: Add CONFIG_OVERLAY_DTB
2023-04-11 19:16 ` [XEN][PATCH v5 03/17] xen/arm: Add CONFIG_OVERLAY_DTB Vikram Garhwal
@ 2023-04-13 9:58 ` Michal Orzel
2023-04-13 19:27 ` Vikram Garhwal
0 siblings, 1 reply; 61+ messages in thread
From: Michal Orzel @ 2023-04-13 9:58 UTC (permalink / raw)
To: Vikram Garhwal, xen-devel
Cc: sstabellini, Andrew Cooper, George Dunlap, Jan Beulich,
Julien Grall, Wei Liu, Bertrand Marquis, Volodymyr Babchuk
Hi Vikram,
On 11/04/2023 21:16, Vikram Garhwal wrote:
>
>
> Introduce a config option where the user can enable support for adding/removing
> device tree nodes using a device tree binary overlay.
>
> Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
> ---
> SUPPORT.md | 6 ++++++
> xen/arch/arm/Kconfig | 5 +++++
> 2 files changed, 11 insertions(+)
>
> diff --git a/SUPPORT.md b/SUPPORT.md
> index aa1940e55f..0a31f40af4 100644
> --- a/SUPPORT.md
> +++ b/SUPPORT.md
> @@ -822,6 +822,12 @@ No support for QEMU backends in a 16K or 64K domain.
>
> Status: Supported
>
> +### Device Tree Overlays
> +
> +Add/Remove device tree nodes using a device tree overlay binary(.dtbo).
> +
> + Status: Supported for ARM
Hmm, so here you say supported but in Kconfig - unsupported.
I think, this should be:
Status, ARM: Tech Preview
or Experimental
> +
> ### ARM: Guest ACPI support
>
> Status: Supported
> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
> index 239d3aed3c..1fe3d698a5 100644
> --- a/xen/arch/arm/Kconfig
> +++ b/xen/arch/arm/Kconfig
> @@ -53,6 +53,11 @@ config HAS_ITS
> bool "GICv3 ITS MSI controller support (UNSUPPORTED)" if UNSUPPORTED
> depends on GICV3 && !NEW_VGIC && !ARM_32
>
> +config OVERLAY_DTB
> + bool "DTB overlay support (UNSUPPORTED)" if UNSUPPORTED
> + help
> + Dynamic addition/removal of Xen device tree nodes using a dtbo.
> +
> config HVM
> def_bool y
>
> --
> 2.17.1
>
>
~Michal
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [XEN][PATCH v5 02/17] common/device_tree: change __unflatten_device_tree()
2023-04-11 19:16 ` [XEN][PATCH v5 02/17] common/device_tree: change __unflatten_device_tree() Vikram Garhwal
2023-04-13 9:45 ` Michal Orzel
@ 2023-04-13 10:03 ` Julien Grall
2023-04-14 17:51 ` Vikram Garhwal
1 sibling, 1 reply; 61+ messages in thread
From: Julien Grall @ 2023-04-13 10:03 UTC (permalink / raw)
To: Vikram Garhwal, xen-devel; +Cc: sstabellini
Hi,
On 11/04/2023 20:16, Vikram Garhwal wrote:
> Following changes are done to __unflatten_device_tree():
> 1. __unflatten_device_tree() is renamed to unflatten_device_tree().
> 2. Remove static function type.
> 3. Add handling of memory allocation. This will be useful in dynamic node
> programming when we unflatten the dt during runtime memory allocation
> can fail.
>
> Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
> ---
> xen/common/device_tree.c | 10 ++++++----
> xen/include/xen/device_tree.h | 5 +++++
> 2 files changed, 11 insertions(+), 4 deletions(-)
>
> diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
> index aed38ff63c..bf847b2584 100644
> --- a/xen/common/device_tree.c
> +++ b/xen/common/device_tree.c
> @@ -2047,7 +2047,7 @@ static unsigned long unflatten_dt_node(const void *fdt,
> }
>
> /**
> - * __unflatten_device_tree - create tree of device_nodes from flat blob
> + * unflatten_device_tree - create tree of device_nodes from flat blob
> *
> * unflattens a device-tree, creating the
> * tree of struct device_node. It also fills the "name" and "type"
> @@ -2056,8 +2056,7 @@ static unsigned long unflatten_dt_node(const void *fdt,
> * @fdt: The fdt to expand
> * @mynodes: The device_node tree created by the call
> */
> -static void __unflatten_device_tree(const void *fdt,
> - struct dt_device_node **mynodes)
> +void unflatten_device_tree(const void *fdt, struct dt_device_node **mynodes)
> {
> unsigned long start, mem, size;
> struct dt_device_node **allnextp = mynodes;
> @@ -2079,6 +2078,9 @@ static void __unflatten_device_tree(const void *fdt,
> /* Allocate memory for the expanded device tree */
> mem = (unsigned long)_xmalloc (size + 4, __alignof__(struct dt_device_node));
>
> + if ( !mem )
> + panic("Cannot allocate memory for unflatten device tree\n");
After your series, unflatten_device_tree() will be called after boot, so
we should not unconditionally called panic(). Instead,
unflatten_device_tree() should return an error and let the caller decide
what to do.
I suggest to read misc/xen-error-handling.txt to understand when to use
panic()/BUG() & co. For...
> +
> ((__be32 *)mem)[size / 4] = cpu_to_be32(0xdeadbeef);
>
> dt_dprintk(" unflattening %lx...\n", mem);
> @@ -2179,7 +2181,7 @@ dt_find_interrupt_controller(const struct dt_device_match *matches)
>
> void __init dt_unflatten_host_device_tree(void)
> {
> - __unflatten_device_tree(device_tree_flattened, &dt_host);
> + unflatten_device_tree(device_tree_flattened, &dt_host);
... this caller this should be a panic() (this is OK here because it is
boot code).
But for your new caller, you should properly report the error back the
toolstack.
Also, unflatten_dt_node() (called by __unflatten_device_tree()) seems to
have some failure cases. Can you explain why they are not properly
propagated in your case? Are you trusting the device-tree to always be
valid?
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [XEN][PATCH v5 05/17] libfdt: overlay: change overlay_get_target()
2023-04-11 19:16 ` [XEN][PATCH v5 05/17] libfdt: overlay: change overlay_get_target() Vikram Garhwal
@ 2023-04-13 13:11 ` Michal Orzel
2023-04-14 17:54 ` Vikram Garhwal
0 siblings, 1 reply; 61+ messages in thread
From: Michal Orzel @ 2023-04-13 13:11 UTC (permalink / raw)
To: Vikram Garhwal, xen-devel
Cc: sstabellini, Julien Grall, Vikram Garhwal, David Gibson
Hi Vikram,
On 11/04/2023 21:16, Vikram Garhwal wrote:
>
>
> Rename overlay_get_target() to fdt_overlay_target_offset() and remove static
> function type.
>
> This is done to get the target path for the overlay nodes which is very useful
> in many cases. For example, Xen hypervisor needs it when applying overlays
> because Xen needs to do further processing of the overlay nodes, e.g. mapping of
> resources(IRQs and IOMMUs) to other VMs, creation of SMMU pagetables, etc.
>
> Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
> Message-Id: <1637204036-382159-2-git-send-email-fnu.vikram@xilinx.com>
> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> Origin: https://github.com/dgibson/dtc 45f3d1a095dd
Wouldn't it be better to point to the main dtc repository under kernel.org rather than github?
Origin: git://git.kernel.org/pub/scm/utils/dtc/dtc.git 45f3d1a095dd
>
> Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
In any case:
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
~Michal
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [XEN][PATCH v5 06/17] xen/device-tree: Add device_tree_find_node_by_path() to find nodes in device tree
2023-04-11 19:16 ` [XEN][PATCH v5 06/17] xen/device-tree: Add device_tree_find_node_by_path() to find nodes in device tree Vikram Garhwal
@ 2023-04-13 13:40 ` Michal Orzel
2023-04-14 18:04 ` Vikram Garhwal
0 siblings, 1 reply; 61+ messages in thread
From: Michal Orzel @ 2023-04-13 13:40 UTC (permalink / raw)
To: Vikram Garhwal, xen-devel; +Cc: sstabellini, Julien Grall
Hi Vikram,
On 11/04/2023 21:16, Vikram Garhwal wrote:
>
>
> Add device_tree_find_node_by_path() to find a matching node with path for a
> dt_device_node.
>
> Reason behind this function:
> Each time overlay nodes are added using .dtbo, a new fdt(memcpy of
> device_tree_flattened) is created and updated with overlay nodes. This
> updated fdt is further unflattened to a dt_host_new. Next, we need to find
> the overlay nodes in dt_host_new, find the overlay node's parent in dt_host
> and add the nodes as child under their parent in the dt_host. Thus we need
> this function to search for node in different unflattened device trees.
You do not mention making dt_find_node_by_path() static inline.
>
> Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
> ---
> xen/common/device_tree.c | 5 +++--
> xen/include/xen/device_tree.h | 17 +++++++++++++++--
> 2 files changed, 18 insertions(+), 4 deletions(-)
>
> diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
> index bf847b2584..507b4ac5b6 100644
> --- a/xen/common/device_tree.c
> +++ b/xen/common/device_tree.c
> @@ -358,11 +358,12 @@ struct dt_device_node *dt_find_node_by_type(struct dt_device_node *from,
> return np;
> }
>
> -struct dt_device_node *dt_find_node_by_path(const char *path)
> +struct dt_device_node *device_tree_find_node_by_path(struct dt_device_node *dt,
> + const char *path)
> {
> struct dt_device_node *np;
>
> - dt_for_each_device_node(dt_host, np)
> + dt_for_each_device_node(dt, np)
> if ( np->full_name && (dt_node_cmp(np->full_name, path) == 0) )
> break;
>
> diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
> index 58ac12abe3..998f972ebc 100644
> --- a/xen/include/xen/device_tree.h
> +++ b/xen/include/xen/device_tree.h
> @@ -534,13 +534,26 @@ struct dt_device_node *dt_find_node_by_type(struct dt_device_node *from,
> struct dt_device_node *dt_find_node_by_alias(const char *alias);
>
> /**
> - * dt_find_node_by_path - Find a node matching a full DT path
> + * device_tree_find_node_by_path - Generic function to find a node matching the
> + * full DT path for any given unflatten device tree
> + * @dt_node: The device tree to search
This should be @dt to match the parameter. Also, shouldn't the description say:
"the node to start searching from"
or
"device tree root node"
FWICS, you expect to pass a root node as dt node. However, in device_tree_find_node_by_path()
you do not check if a provided node is a root node or not (e.g. no parent). Is this intended?
~Michal
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [XEN][PATCH v5 03/17] xen/arm: Add CONFIG_OVERLAY_DTB
2023-04-13 9:58 ` Michal Orzel
@ 2023-04-13 19:27 ` Vikram Garhwal
0 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-13 19:27 UTC (permalink / raw)
To: Michal Orzel, xen-devel
Cc: sstabellini, Andrew Cooper, George Dunlap, Jan Beulich,
Julien Grall, Wei Liu, Bertrand Marquis, Volodymyr Babchuk
Hi Michal,
On 4/13/23 2:58 AM, Michal Orzel wrote:
> Hi Vikram,
>
> On 11/04/2023 21:16, Vikram Garhwal wrote:
>>
>> Introduce a config option where the user can enable support for adding/removing
>> device tree nodes using a device tree binary overlay.
>>
>> Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
>> ---
>> SUPPORT.md | 6 ++++++
>> xen/arch/arm/Kconfig | 5 +++++
>> 2 files changed, 11 insertions(+)
>>
>> diff --git a/SUPPORT.md b/SUPPORT.md
>> index aa1940e55f..0a31f40af4 100644
>> --- a/SUPPORT.md
>> +++ b/SUPPORT.md
>> @@ -822,6 +822,12 @@ No support for QEMU backends in a 16K or 64K domain.
>>
>> Status: Supported
>>
>> +### Device Tree Overlays
>> +
>> +Add/Remove device tree nodes using a device tree overlay binary(.dtbo).
>> +
>> + Status: Supported for ARM
> Hmm, so here you say supported but in Kconfig - unsupported.
> I think, this should be:
> Status, ARM: Tech Preview
> or Experimental
Experimental sounds better to me.
Will update it.
>> +
>> ### ARM: Guest ACPI support
>>
>> Status: Supported
>> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
>> index 239d3aed3c..1fe3d698a5 100644
>> --- a/xen/arch/arm/Kconfig
>> +++ b/xen/arch/arm/Kconfig
>> @@ -53,6 +53,11 @@ config HAS_ITS
>> bool "GICv3 ITS MSI controller support (UNSUPPORTED)" if UNSUPPORTED
>> depends on GICV3 && !NEW_VGIC && !ARM_32
>>
>> +config OVERLAY_DTB
>> + bool "DTB overlay support (UNSUPPORTED)" if UNSUPPORTED
>> + help
>> + Dynamic addition/removal of Xen device tree nodes using a dtbo.
>> +
>> config HVM
>> def_bool y
>>
>> --
>> 2.17.1
>>
>>
> ~Michal
^ permalink raw reply [flat|nested] 61+ messages in thread
* RE: [XEN][PATCH v5 11/17] asm/smp.h: Fix circular dependency for device_tree.h and rwlock.h
2023-04-11 19:16 ` [XEN][PATCH v5 11/17] asm/smp.h: Fix circular dependency for device_tree.h and rwlock.h Vikram Garhwal
@ 2023-04-14 1:50 ` Henry Wang
2023-04-17 9:41 ` Michal Orzel
1 sibling, 0 replies; 61+ messages in thread
From: Henry Wang @ 2023-04-14 1:50 UTC (permalink / raw)
To: Vikram Garhwal, xen-devel
Cc: sstabellini, Julien Grall, Bertrand Marquis, Volodymyr Babchuk
Hi Vikram,
> -----Original Message-----
> Subject: [XEN][PATCH v5 11/17] asm/smp.h: Fix circular dependency for
> device_tree.h and rwlock.h
>
> Dynamic programming ops will modify the dt_host and there might be other
> function which are browsing the dt_host at the same time. To avoid the race
> conditions, adding rwlock for browsing the dt_host. But adding rwlock in
> device_tree.h causes following circular dependency:
> device_tree.h->rwlock.h->smp.h->asm/smp.h->device_tree.h
>
> To fix this, removed the "#include <xen/device_tree.h> and forward declared
> "struct dt_device_node".
>
> Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
Reviewed-by: Henry Wang <Henry.Wang@arm.com>
Kind regards,
Henry
^ permalink raw reply [flat|nested] 61+ messages in thread
* RE: [XEN][PATCH v5 12/17] common/device_tree: Add rwlock for dt_host
2023-04-11 19:16 ` [XEN][PATCH v5 12/17] common/device_tree: Add rwlock for dt_host Vikram Garhwal
@ 2023-04-14 2:09 ` Henry Wang
2023-04-14 17:23 ` Vikram Garhwal
0 siblings, 1 reply; 61+ messages in thread
From: Henry Wang @ 2023-04-14 2:09 UTC (permalink / raw)
To: Vikram Garhwal, xen-devel; +Cc: sstabellini, Julien Grall
Hi Vikram,
> -----Original Message-----
> Subject: [XEN][PATCH v5 12/17] common/device_tree: Add rwlock for dt_host
>
> Dynamic programming ops will modify the dt_host and there might be other
> function which are browsing the dt_host at the same time. To avoid the race
> conditions, adding rwlock for browsing the dt_host during runtime.
For clarity, could you please add a little bit more details to explain why you chose
rwlock instead of normal spinlock?
>
> Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
> ---
> xen/common/device_tree.c | 3 +++
> xen/drivers/passthrough/device_tree.c | 39 +++++++++++++++++++++++++++
> xen/include/xen/device_tree.h | 6 +++++
> 3 files changed, 48 insertions(+)
>
> if ( ret )
> + {
> printk(XENLOG_G_ERR "XEN_DOMCTL_assign_dt_device: assign
> \"%s\""
> " to dom%u failed (%d)\n",
> dt_node_full_name(dev), d->domain_id, ret);
> + }
I am not sure if it is necessary to add "{" and "}" here.
> +
> + read_unlock(&dt_host->lock);
> break;
>
> case XEN_DOMCTL_deassign_device:
> @@ -322,25 +345,41 @@ int iommu_do_dt_domctl(struct xen_domctl
> *domctl, struct domain *d,
> if ( domctl->u.assign_device.flags )
> break;
>
> + read_lock(&dt_host->lock);
> +
> ret = dt_find_node_by_gpath(domctl->u.assign_device.u.dt.path,
> domctl->u.assign_device.u.dt.size,
> &dev);
> if ( ret )
> + {
> + read_unlock(&dt_host->lock);
I think instead of adding "read_unlock" in every break and return path,
you can...
> break;
> + }
>
> ret = xsm_deassign_dtdevice(XSM_HOOK, d, dt_node_full_name(dev));
> +
> if ( ret )
> + {
> + read_unlock(&dt_host->lock);
> break;
> + }
>
> if ( d == dom_io )
> + {
> + read_unlock(&dt_host->lock);
> return -EINVAL;
...do something like:
ret = -EINVAL;
break;
here, and then add one single "read_unlock" before the "return ret;"
in the bottom of the function?
> + }
>
> ret = iommu_deassign_dt_device(d, dev);
>
> if ( ret )
> + {
> printk(XENLOG_G_ERR "XEN_DOMCTL_assign_dt_device: assign
> \"%s\""
> " to dom%u failed (%d)\n",
> dt_node_full_name(dev), d->domain_id, ret);
> + }
Same here. I am not sure if it is necessary to add "{" and "}".
Kind regards,
Henry
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [XEN][PATCH v5 12/17] common/device_tree: Add rwlock for dt_host
2023-04-14 2:09 ` Henry Wang
@ 2023-04-14 17:23 ` Vikram Garhwal
0 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-14 17:23 UTC (permalink / raw)
To: Henry Wang, xen-devel; +Cc: sstabellini, Julien Grall
Hi Henry,
On 4/13/23 7:09 PM, Henry Wang wrote:
> Hi Vikram,
>
>> -----Original Message-----
>> Subject: [XEN][PATCH v5 12/17] common/device_tree: Add rwlock for dt_host
>>
>> Dynamic programming ops will modify the dt_host and there might be other
>> function which are browsing the dt_host at the same time. To avoid the race
>> conditions, adding rwlock for browsing the dt_host during runtime.
> For clarity, could you please add a little bit more details to explain why you chose
> rwlock instead of normal spinlock?
rwlock is just to protect someone reading the dt_host while dynamic
programming is modifying the dt_host.
Initial suggestion was from Julien about adding rwlock here.
For now, dynamic programming is the dt_host writer in Xen during run
time. All other iommu passthrough function was just readers during
run-time. So, it was better to go for r/w locks here as spinlock won't
be able to difference between read and r/w access.
For next version, I will add a comment in commit message.
>> Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
>> ---
>> xen/common/device_tree.c | 3 +++
>> xen/drivers/passthrough/device_tree.c | 39 +++++++++++++++++++++++++++
>> xen/include/xen/device_tree.h | 6 +++++
>> 3 files changed, 48 insertions(+)
>>
>> if ( ret )
>> + {
>> printk(XENLOG_G_ERR "XEN_DOMCTL_assign_dt_device: assign
>> \"%s\""
>> " to dom%u failed (%d)\n",
>> dt_node_full_name(dev), d->domain_id, ret);
>> + }
> I am not sure if it is necessary to add "{" and "}" here.
You are right. Will remove it in next version.
>
>> +
>> + read_unlock(&dt_host->lock);
>> break;
>>
>> case XEN_DOMCTL_deassign_device:
>> @@ -322,25 +345,41 @@ int iommu_do_dt_domctl(struct xen_domctl
>> *domctl, struct domain *d,
>> if ( domctl->u.assign_device.flags )
>> break;
>>
>> + read_lock(&dt_host->lock);
>> +
>> ret = dt_find_node_by_gpath(domctl->u.assign_device.u.dt.path,
>> domctl->u.assign_device.u.dt.size,
>> &dev);
>> if ( ret )
>> + {
>> + read_unlock(&dt_host->lock);
> I think instead of adding "read_unlock" in every break and return path,
> you can...
>
>> break;
>> + }
>>
>> ret = xsm_deassign_dtdevice(XSM_HOOK, d, dt_node_full_name(dev));
>> +
>> if ( ret )
>> + {
>> + read_unlock(&dt_host->lock);
>> break;
>> + }
>>
>> if ( d == dom_io )
>> + {
>> + read_unlock(&dt_host->lock);
>> return -EINVAL;
> ...do something like:
>
> ret = -EINVAL;
> break;
>
> here, and then add one single "read_unlock" before the "return ret;"
> in the bottom of the function?
Will do.
>
>> + }
>>
>> ret = iommu_deassign_dt_device(d, dev);
>>
>> if ( ret )
>> + {
>> printk(XENLOG_G_ERR "XEN_DOMCTL_assign_dt_device: assign
>> \"%s\""
>> " to dom%u failed (%d)\n",
>> dt_node_full_name(dev), d->domain_id, ret);
>> + }
> Same here. I am not sure if it is necessary to add "{" and "}".
> Kind regards,
> Henry
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [XEN][PATCH v5 01/17] xen/arm/device: Remove __init from function type
2023-04-13 9:19 ` Michal Orzel
@ 2023-04-14 17:28 ` Vikram Garhwal
0 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-14 17:28 UTC (permalink / raw)
To: Michal Orzel, xen-devel
Cc: sstabellini, Julien Grall, Bertrand Marquis, Volodymyr Babchuk
Hi Michal,
On 4/13/23 2:19 AM, Michal Orzel wrote:
> Hi Vikram,
>
> On 11/04/2023 21:16, Vikram Garhwal wrote:
>>
>> Remove __init from following function to access during runtime:
>> 1. map_irq_to_domain()
>> 2. handle_device_interrupt()
> s/interrupt/interrupts/ since there is no handle_device_interrupt() function.
>
>> 3. map_range_to_domain()
>> 4. unflatten_dt_node()
>> 5. unflatten_device_tree()
>>
>> Move map_irq_to_domain() prototype from domain_build.h to setup.h.
>>
>> To avoid the breaking the build, following changes are also done:
> 'avoid breaking' instead of 'avoid the breaking'.
>
>> 1. Move map_irq_to_domain(), handle_device_interrupt() and map_range_to_domain() to
> s/interrupt/interrupts/
>
>> device.c. After removing __init type, these functions are not specific to
>> domain building, so moving them out of domain_build.c to device.c.
>> 2. Remove static type from handle_device_interrupt().
>>
>> Overall, these changes are done to support the dynamic programming of a nodes
>> where an overlay node will be added to fdt and unflattened node will be added to
>> dt_host. Furthermore, IRQ and mmio mapping will be done for the added node.
>>
>> Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
>> ---
>> xen/arch/arm/device.c | 145 ++++++++++++++++++++++++
>> xen/arch/arm/domain_build.c | 142 -----------------------
>> xen/arch/arm/include/asm/domain_build.h | 2 -
>> xen/arch/arm/include/asm/setup.h | 6 +
>> xen/common/device_tree.c | 16 +--
>> 5 files changed, 159 insertions(+), 152 deletions(-)
>>
>> diff --git a/xen/arch/arm/device.c b/xen/arch/arm/device.c
>> index ca8539dee5..fec6e29c42 100644
>> --- a/xen/arch/arm/device.c
>> +++ b/xen/arch/arm/device.c
>> @@ -12,6 +12,9 @@
>> #include <xen/errno.h>
>> #include <xen/init.h>
>> #include <xen/lib.h>
>> +#include <xen/iocap.h>
>> +#include <asm/domain_build.h>
> I can't see why do we need to include this header.
I will recheck if this is really needed.
>
>> +#include <asm/setup.h>
> You should keep the alphabetical order so:
> - iocap goes after init.h
> - setup.h goes after device.h
Will change it.
> Apart from that, it looks ok so:
Thanks!
> Reviewed-by: Michal Orzel <michal.orzel@amd.com>
>
> ~Michal
>
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [XEN][PATCH v5 02/17] common/device_tree: change __unflatten_device_tree()
2023-04-13 10:03 ` Julien Grall
@ 2023-04-14 17:51 ` Vikram Garhwal
2023-04-14 18:09 ` Julien Grall
0 siblings, 1 reply; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-14 17:51 UTC (permalink / raw)
To: Julien Grall, xen-devel; +Cc: sstabellini
Hi,
Julien & Michal, thanks for comments.
On 4/13/23 3:03 AM, Julien Grall wrote:
> Hi,
>
> On 11/04/2023 20:16, Vikram Garhwal wrote:
>> Following changes are done to __unflatten_device_tree():
>> 1. __unflatten_device_tree() is renamed to unflatten_device_tree().
>> 2. Remove static function type.
>> 3. Add handling of memory allocation. This will be useful in
>> dynamic node
>> programming when we unflatten the dt during runtime memory
>> allocation
>> can fail.
>>
>> Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
>> ---
>> xen/common/device_tree.c | 10 ++++++----
>> xen/include/xen/device_tree.h | 5 +++++
>> 2 files changed, 11 insertions(+), 4 deletions(-)
>>
>> diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
>> index aed38ff63c..bf847b2584 100644
>> --- a/xen/common/device_tree.c
>> +++ b/xen/common/device_tree.c
>> @@ -2047,7 +2047,7 @@ static unsigned long unflatten_dt_node(const
>> void *fdt,
>> }
>> /**
>> - * __unflatten_device_tree - create tree of device_nodes from flat blob
>> + * unflatten_device_tree - create tree of device_nodes from flat blob
>> *
>> * unflattens a device-tree, creating the
>> * tree of struct device_node. It also fills the "name" and "type"
>> @@ -2056,8 +2056,7 @@ static unsigned long unflatten_dt_node(const
>> void *fdt,
>> * @fdt: The fdt to expand
>> * @mynodes: The device_node tree created by the call
>> */
>> -static void __unflatten_device_tree(const void *fdt,
>> - struct dt_device_node **mynodes)
>> +void unflatten_device_tree(const void *fdt, struct dt_device_node
>> **mynodes)
>> {
>> unsigned long start, mem, size;
>> struct dt_device_node **allnextp = mynodes;
>> @@ -2079,6 +2078,9 @@ static void __unflatten_device_tree(const void
>> *fdt,
>> /* Allocate memory for the expanded device tree */
>> mem = (unsigned long)_xmalloc (size + 4, __alignof__(struct
>> dt_device_node));
>> + if ( !mem )
>> + panic("Cannot allocate memory for unflatten device tree\n");
>
> After your series, unflatten_device_tree() will be called after boot,
> so we should not unconditionally called panic(). Instead,
> unflatten_device_tree() should return an error and let the caller
> decide what to do.
Looks like i misunderstood v4 comments. Will change it to propagate an
error in case of failure here to the handle_add_overlay_nodes() caller
and that will further forward to error to toolstack.
>
> I suggest to read misc/xen-error-handling.txt to understand when to
> use panic()/BUG() & co. For...
>
>
>> +
>> ((__be32 *)mem)[size / 4] = cpu_to_be32(0xdeadbeef);
>> dt_dprintk(" unflattening %lx...\n", mem);
>> @@ -2179,7 +2181,7 @@ dt_find_interrupt_controller(const struct
>> dt_device_match *matches)
>> void __init dt_unflatten_host_device_tree(void)
>> {
>> - __unflatten_device_tree(device_tree_flattened, &dt_host);
>> + unflatten_device_tree(device_tree_flattened, &dt_host);
>
> ... this caller this should be a panic() (this is OK here because it
> is boot code).
>
> But for your new caller, you should properly report the error back the
> toolstack.
Understood, will change it in next version.
>
> Also, unflatten_dt_node() (called by __unflatten_device_tree()) seems
> to have some failure cases. Can you explain why they are not properly
> propagated in your case? Are you trusting the device-tree to always be
> valid?
for dynamic_programming, while adding node(check patch: [XEN][PATCH v5
14/17] xen/arm: Implement device tree node addition functionalities),
fdt_overlay_apply() is called before unflatten_device_tree() is called.
fdt_overlay_apply() will catch the invalid device-tree overlay nodes.
Regards,
Vikram
>
> Cheers,
>
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [XEN][PATCH v5 05/17] libfdt: overlay: change overlay_get_target()
2023-04-13 13:11 ` Michal Orzel
@ 2023-04-14 17:54 ` Vikram Garhwal
0 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-14 17:54 UTC (permalink / raw)
To: Michal Orzel, xen-devel
Cc: sstabellini, Julien Grall, Vikram Garhwal, David Gibson
Hi Michal,
On 4/13/23 6:11 AM, Michal Orzel wrote:
> Hi Vikram,
>
> On 11/04/2023 21:16, Vikram Garhwal wrote:
>>
>> Rename overlay_get_target() to fdt_overlay_target_offset() and remove static
>> function type.
>>
>> This is done to get the target path for the overlay nodes which is very useful
>> in many cases. For example, Xen hypervisor needs it when applying overlays
>> because Xen needs to do further processing of the overlay nodes, e.g. mapping of
>> resources(IRQs and IOMMUs) to other VMs, creation of SMMU pagetables, etc.
>>
>> Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
>> Message-Id: <1637204036-382159-2-git-send-email-fnu.vikram@xilinx.com>
>> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
>> Origin: https://github.com/dgibson/dtc 45f3d1a095dd
> Wouldn't it be better to point to the main dtc repository under kernel.org rather than github?
> Origin: git://git.kernel.org/pub/scm/utils/dtc/dtc.git 45f3d1a095dd
Okay, i will change it in next version. Is there any guidelines/note for
this preference? Will be helpful to me in future series.
>> Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
> In any case:
> Reviewed-by: Michal Orzel <michal.orzel@amd.com>
Thanks you!
>
> ~Michal
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [XEN][PATCH v5 06/17] xen/device-tree: Add device_tree_find_node_by_path() to find nodes in device tree
2023-04-13 13:40 ` Michal Orzel
@ 2023-04-14 18:04 ` Vikram Garhwal
0 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-14 18:04 UTC (permalink / raw)
To: Michal Orzel, xen-devel; +Cc: sstabellini, Julien Grall
Hi Michal,
On 4/13/23 6:40 AM, Michal Orzel wrote:
> Hi Vikram,
>
> On 11/04/2023 21:16, Vikram Garhwal wrote:
>>
>> Add device_tree_find_node_by_path() to find a matching node with path for a
>> dt_device_node.
>>
>> Reason behind this function:
>> Each time overlay nodes are added using .dtbo, a new fdt(memcpy of
>> device_tree_flattened) is created and updated with overlay nodes. This
>> updated fdt is further unflattened to a dt_host_new. Next, we need to find
>> the overlay nodes in dt_host_new, find the overlay node's parent in dt_host
>> and add the nodes as child under their parent in the dt_host. Thus we need
>> this function to search for node in different unflattened device trees.
> You do not mention making dt_find_node_by_path() static inline.
Will add a comment about it in v6.
>
>> Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
>> ---
>> xen/common/device_tree.c | 5 +++--
>> xen/include/xen/device_tree.h | 17 +++++++++++++++--
>> 2 files changed, 18 insertions(+), 4 deletions(-)
>>
>> diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
>> index bf847b2584..507b4ac5b6 100644
>> --- a/xen/common/device_tree.c
>> +++ b/xen/common/device_tree.c
>> @@ -358,11 +358,12 @@ struct dt_device_node *dt_find_node_by_type(struct dt_device_node *from,
>> return np;
>> }
>>
>> -struct dt_device_node *dt_find_node_by_path(const char *path)
>> +struct dt_device_node *device_tree_find_node_by_path(struct dt_device_node *dt,
>> + const char *path)
>> {
>> struct dt_device_node *np;
>>
>> - dt_for_each_device_node(dt_host, np)
>> + dt_for_each_device_node(dt, np)
>> if ( np->full_name && (dt_node_cmp(np->full_name, path) == 0) )
>> break;
>>
>> diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
>> index 58ac12abe3..998f972ebc 100644
>> --- a/xen/include/xen/device_tree.h
>> +++ b/xen/include/xen/device_tree.h
>> @@ -534,13 +534,26 @@ struct dt_device_node *dt_find_node_by_type(struct dt_device_node *from,
>> struct dt_device_node *dt_find_node_by_alias(const char *alias);
>>
>> /**
>> - * dt_find_node_by_path - Find a node matching a full DT path
>> + * device_tree_find_node_by_path - Generic function to find a node matching the
>> + * full DT path for any given unflatten device tree
>> + * @dt_node: The device tree to search
> This should be @dt to match the parameter. Also, shouldn't the description say:
> "the node to start searching from"
> or
> "device tree root node"
>
> FWICS, you expect to pass a root node as dt node. However, in device_tree_find_node_by_path()
> you do not check if a provided node is a root node or not (e.g. no parent). Is this intended?
Yeah, intent was to write a generic function where we can search from
middle of a device_tree as long as we have the start node from where to
search.
But yeah so far for dynamic programming it's been called for a root
nodes only.
>
> ~Michal
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [XEN][PATCH v5 02/17] common/device_tree: change __unflatten_device_tree()
2023-04-14 17:51 ` Vikram Garhwal
@ 2023-04-14 18:09 ` Julien Grall
2023-04-14 18:28 ` Vikram Garhwal
0 siblings, 1 reply; 61+ messages in thread
From: Julien Grall @ 2023-04-14 18:09 UTC (permalink / raw)
To: Vikram Garhwal, xen-devel; +Cc: sstabellini
Hi,
On 14/04/2023 18:51, Vikram Garhwal wrote:
> On 4/13/23 3:03 AM, Julien Grall wrote:
>> Hi,
>>
>> On 11/04/2023 20:16, Vikram Garhwal wrote:
>>> Following changes are done to __unflatten_device_tree():
>>> 1. __unflatten_device_tree() is renamed to unflatten_device_tree().
>>> 2. Remove static function type.
>>> 3. Add handling of memory allocation. This will be useful in
>>> dynamic node
>>> programming when we unflatten the dt during runtime memory
>>> allocation
>>> can fail.
>>>
>>> Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
>>> ---
>>> xen/common/device_tree.c | 10 ++++++----
>>> xen/include/xen/device_tree.h | 5 +++++
>>> 2 files changed, 11 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
>>> index aed38ff63c..bf847b2584 100644
>>> --- a/xen/common/device_tree.c
>>> +++ b/xen/common/device_tree.c
>>> @@ -2047,7 +2047,7 @@ static unsigned long unflatten_dt_node(const
>>> void *fdt,
>>> }
>>> /**
>>> - * __unflatten_device_tree - create tree of device_nodes from flat blob
>>> + * unflatten_device_tree - create tree of device_nodes from flat blob
>>> *
>>> * unflattens a device-tree, creating the
>>> * tree of struct device_node. It also fills the "name" and "type"
>>> @@ -2056,8 +2056,7 @@ static unsigned long unflatten_dt_node(const
>>> void *fdt,
>>> * @fdt: The fdt to expand
>>> * @mynodes: The device_node tree created by the call
>>> */
>>> -static void __unflatten_device_tree(const void *fdt,
>>> - struct dt_device_node **mynodes)
>>> +void unflatten_device_tree(const void *fdt, struct dt_device_node
>>> **mynodes)
>>> {
>>> unsigned long start, mem, size;
>>> struct dt_device_node **allnextp = mynodes;
>>> @@ -2079,6 +2078,9 @@ static void __unflatten_device_tree(const void
>>> *fdt,
>>> /* Allocate memory for the expanded device tree */
>>> mem = (unsigned long)_xmalloc (size + 4, __alignof__(struct
>>> dt_device_node));
>>> + if ( !mem )
>>> + panic("Cannot allocate memory for unflatten device tree\n");
>>
>> After your series, unflatten_device_tree() will be called after boot,
>> so we should not unconditionally called panic(). Instead,
>> unflatten_device_tree() should return an error and let the caller
>> decide what to do.
> Looks like i misunderstood v4 comments. Will change it to propagate an
> error in case of failure here to the handle_add_overlay_nodes() caller
> and that will further forward to error to toolstack.
>>
>> I suggest to read misc/xen-error-handling.txt to understand when to
>> use panic()/BUG() & co. For...
>>
>>
>>> +
>>> ((__be32 *)mem)[size / 4] = cpu_to_be32(0xdeadbeef);
>>> dt_dprintk(" unflattening %lx...\n", mem);
>>> @@ -2179,7 +2181,7 @@ dt_find_interrupt_controller(const struct
>>> dt_device_match *matches)
>>> void __init dt_unflatten_host_device_tree(void)
>>> {
>>> - __unflatten_device_tree(device_tree_flattened, &dt_host);
>>> + unflatten_device_tree(device_tree_flattened, &dt_host);
>>
>> ... this caller this should be a panic() (this is OK here because it
>> is boot code).
>>
>> But for your new caller, you should properly report the error back the
>> toolstack.
> Understood, will change it in next version.
>>
>> Also, unflatten_dt_node() (called by __unflatten_device_tree()) seems
>> to have some failure cases. Can you explain why they are not properly
>> propagated in your case? Are you trusting the device-tree to always be
>> valid?
> for dynamic_programming, while adding node(check patch: [XEN][PATCH v5
> 14/17] xen/arm: Implement device tree node addition functionalities),
> fdt_overlay_apply() is called before unflatten_device_tree() is called.
> fdt_overlay_apply() will catch the invalid device-tree overlay nodes.
I agree that in theory fdt_overlay_apply() will catch invalid
device-tree. However, neither of the functions are exempts of bugs and
there is no code shared between the two (they are not even coming from
the same project).
So we could end up in a situation where fdt_overlay_apply() works but
not unflatten_device_tree(). Therefore, I would rather prefer if the
latter function properly handle any errors.
Note that unflatten_dt_node() already have check the validity of the DT
and will return. We just need to make sure they are treated as error
rather than been ignored.
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [XEN][PATCH v5 02/17] common/device_tree: change __unflatten_device_tree()
2023-04-14 18:09 ` Julien Grall
@ 2023-04-14 18:28 ` Vikram Garhwal
0 siblings, 0 replies; 61+ messages in thread
From: Vikram Garhwal @ 2023-04-14 18:28 UTC (permalink / raw)
To: Julien Grall, xen-devel; +Cc: sstabellini
Hi,
On 4/14/23 11:09 AM, Julien Grall wrote:
> Hi,
>
> On 14/04/2023 18:51, Vikram Garhwal wrote:
>> On 4/13/23 3:03 AM, Julien Grall wrote:
>>> Hi,
>>>
>>> On 11/04/2023 20:16, Vikram Garhwal wrote:
>>>> Following changes are done to __unflatten_device_tree():
>>>> 1. __unflatten_device_tree() is renamed to
>>>> unflatten_device_tree().
>>>> 2. Remove static function type.
>>>> 3. Add handling of memory allocation. This will be useful in
>>>> dynamic node
>>>> programming when we unflatten the dt during runtime memory
>>>> allocation
>>>> can fail.
>>>>
>>>> Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
>>>> ---
>>>> xen/common/device_tree.c | 10 ++++++----
>>>> xen/include/xen/device_tree.h | 5 +++++
>>>> 2 files changed, 11 insertions(+), 4 deletions(-)
>>>>
>>>> diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
>>>> index aed38ff63c..bf847b2584 100644
>>>> --- a/xen/common/device_tree.c
>>>> +++ b/xen/common/device_tree.c
>>>> @@ -2047,7 +2047,7 @@ static unsigned long unflatten_dt_node(const
>>>> void *fdt,
>>>> }
>>>> /**
>>>> - * __unflatten_device_tree - create tree of device_nodes from flat
>>>> blob
>>>> + * unflatten_device_tree - create tree of device_nodes from flat blob
>>>> *
>>>> * unflattens a device-tree, creating the
>>>> * tree of struct device_node. It also fills the "name" and "type"
>>>> @@ -2056,8 +2056,7 @@ static unsigned long unflatten_dt_node(const
>>>> void *fdt,
>>>> * @fdt: The fdt to expand
>>>> * @mynodes: The device_node tree created by the call
>>>> */
>>>> -static void __unflatten_device_tree(const void *fdt,
>>>> - struct dt_device_node **mynodes)
>>>> +void unflatten_device_tree(const void *fdt, struct dt_device_node
>>>> **mynodes)
>>>> {
>>>> unsigned long start, mem, size;
>>>> struct dt_device_node **allnextp = mynodes;
>>>> @@ -2079,6 +2078,9 @@ static void __unflatten_device_tree(const
>>>> void *fdt,
>>>> /* Allocate memory for the expanded device tree */
>>>> mem = (unsigned long)_xmalloc (size + 4, __alignof__(struct
>>>> dt_device_node));
>>>> + if ( !mem )
>>>> + panic("Cannot allocate memory for unflatten device tree\n");
>>>
>>> After your series, unflatten_device_tree() will be called after
>>> boot, so we should not unconditionally called panic(). Instead,
>>> unflatten_device_tree() should return an error and let the caller
>>> decide what to do.
>> Looks like i misunderstood v4 comments. Will change it to propagate
>> an error in case of failure here to the handle_add_overlay_nodes()
>> caller and that will further forward to error to toolstack.
>>>
>>> I suggest to read misc/xen-error-handling.txt to understand when to
>>> use panic()/BUG() & co. For...
>>>
>>>
>>>> +
>>>> ((__be32 *)mem)[size / 4] = cpu_to_be32(0xdeadbeef);
>>>> dt_dprintk(" unflattening %lx...\n", mem);
>>>> @@ -2179,7 +2181,7 @@ dt_find_interrupt_controller(const struct
>>>> dt_device_match *matches)
>>>> void __init dt_unflatten_host_device_tree(void)
>>>> {
>>>> - __unflatten_device_tree(device_tree_flattened, &dt_host);
>>>> + unflatten_device_tree(device_tree_flattened, &dt_host);
>>>
>>> ... this caller this should be a panic() (this is OK here because it
>>> is boot code).
>>>
>>> But for your new caller, you should properly report the error back
>>> the toolstack.
>> Understood, will change it in next version.
>>>
>>> Also, unflatten_dt_node() (called by __unflatten_device_tree())
>>> seems to have some failure cases. Can you explain why they are not
>>> properly propagated in your case? Are you trusting the device-tree
>>> to always be valid?
>> for dynamic_programming, while adding node(check patch: [XEN][PATCH
>> v5 14/17] xen/arm: Implement device tree node addition
>> functionalities), fdt_overlay_apply() is called before
>> unflatten_device_tree() is called. fdt_overlay_apply() will catch the
>> invalid device-tree overlay nodes.
>
> I agree that in theory fdt_overlay_apply() will catch invalid
> device-tree. However, neither of the functions are exempts of bugs and
> there is no code shared between the two (they are not even coming from
> the same project).
>
> So we could end up in a situation where fdt_overlay_apply() works but
> not unflatten_device_tree(). Therefore, I would rather prefer if the
> latter function properly handle any errors.
>
> Note that unflatten_dt_node() already have check the validity of the
> DT and will return. We just need to make sure they are treated as
> error rather than been ignored.
Thanks for explanation. Will add error handling for unflatten_dt_node()
and unflatten_device_tree() for failures.
>
> Cheers,
>
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [XEN][PATCH v5 13/17] xen/arm: Implement device tree node removal functionalities
2023-04-11 19:16 ` [XEN][PATCH v5 13/17] xen/arm: Implement device tree node removal functionalities Vikram Garhwal
@ 2023-04-17 8:08 ` Jan Beulich
2023-04-17 15:06 ` Michal Orzel
1 sibling, 0 replies; 61+ messages in thread
From: Jan Beulich @ 2023-04-17 8:08 UTC (permalink / raw)
To: Vikram Garhwal
Cc: sstabellini, Julien Grall, Bertrand Marquis, Volodymyr Babchuk,
Andrew Cooper, George Dunlap, Wei Liu, xen-devel
On 11.04.2023 21:16, Vikram Garhwal wrote:
> Introduce sysctl XEN_SYSCTL_dt_overlay to remove device-tree nodes added using
> device tree overlay.
>
> xl dt-overlay remove file.dtbo:
> Removes all the nodes in a given dtbo.
> First, removes IRQ permissions and MMIO accesses. Next, it finds the nodes
> in dt_host and delete the device node entries from dt_host.
>
> The nodes get removed only if it is not used by any of dom0 or domio.
>
> Also, added overlay_track struct to keep the track of added node through device
> tree overlay. overlay_track has dt_host_new which is unflattened form of updated
> fdt and name of overlay nodes. When a node is removed, we also free the memory
> used by overlay_track for the particular overlay node.
>
> Nested overlay removal is supported in sequential manner only i.e. if
> overlay_child nests under overlay_parent, it is assumed that user first removes
> overlay_child and then removes overlay_parent.
>
> Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
> ---
> xen/arch/arm/sysctl.c | 16 +-
> xen/common/Makefile | 1 +
> xen/common/dt_overlay.c | 415 +++++++++++++++++++++++++++++++++++
> xen/include/public/sysctl.h | 24 ++
> xen/include/xen/dt_overlay.h | 59 +++++
> 5 files changed, 514 insertions(+), 1 deletion(-)
> create mode 100644 xen/common/dt_overlay.c
> create mode 100644 xen/include/xen/dt_overlay.h
Can new files please use dashes in preference to underscores in their names?
Jan
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [XEN][PATCH v5 08/17] xen/iommu: Move spin_lock from iommu_dt_device_is_assigned to caller
2023-04-11 19:16 ` [XEN][PATCH v5 08/17] xen/iommu: Move spin_lock from iommu_dt_device_is_assigned to caller Vikram Garhwal
@ 2023-04-17 9:14 ` Michal Orzel
0 siblings, 0 replies; 61+ messages in thread
From: Michal Orzel @ 2023-04-17 9:14 UTC (permalink / raw)
To: Vikram Garhwal, xen-devel; +Cc: sstabellini, Julien Grall
Hi Vikram,
It looks like you didn't take into account any of my remarks made for v4.
I will repeat them here.
On 11/04/2023 21:16, Vikram Garhwal wrote:
>
>
> Rename iommu_dt_device_is_assigned() to iommu_dt_device_is_assigned_lock().
s/lock/locked
>
> Moving spin_lock to caller was done to prevent the concurrent access to
> iommu_dt_device_is_assigned while doing add/remove/assign/deassign.
>
> Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
> Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
> ---
> xen/drivers/passthrough/device_tree.c | 23 +++++++++++++++++++----
> 1 file changed, 19 insertions(+), 4 deletions(-)
>
> diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c
> index 1c32d7b50c..bb4cf7784d 100644
> --- a/xen/drivers/passthrough/device_tree.c
> +++ b/xen/drivers/passthrough/device_tree.c
> @@ -83,16 +83,15 @@ fail:
> return rc;
> }
>
> -static bool_t iommu_dt_device_is_assigned(const struct dt_device_node *dev)
> +static bool_t
> + iommu_dt_device_is_assigned_locked(const struct dt_device_node *dev)
This should not be indented.
> {
> bool_t assigned = 0;
>
> if ( !dt_device_is_protected(dev) )
> return 0;
>
> - spin_lock(&dtdevs_lock);
> assigned = !list_empty(&dev->domain_list);
> - spin_unlock(&dtdevs_lock);
>
> return assigned;
> }
> @@ -213,27 +212,43 @@ int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d,
> if ( (d && d->is_dying) || domctl->u.assign_device.flags )
> break;
>
> + spin_lock(&dtdevs_lock);
> +
> ret = dt_find_node_by_gpath(domctl->u.assign_device.u.dt.path,
> domctl->u.assign_device.u.dt.size,
> &dev);
> if ( ret )
> + {
> + spin_unlock(&dtdevs_lock);
> +
Please, do not add blank line between spin_unlock and break. It does not improve readability.
> break;
> + }
>
> ret = xsm_assign_dtdevice(XSM_HOOK, d, dt_node_full_name(dev));
> if ( ret )
> + {
> + spin_unlock(&dtdevs_lock);
> +
same here.
~Michal
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [XEN][PATCH v5 11/17] asm/smp.h: Fix circular dependency for device_tree.h and rwlock.h
2023-04-11 19:16 ` [XEN][PATCH v5 11/17] asm/smp.h: Fix circular dependency for device_tree.h and rwlock.h Vikram Garhwal
2023-04-14 1:50 ` Henry Wang
@ 2023-04-17 9:41 ` Michal Orzel
1 sibling, 0 replies; 61+ messages in thread
From: Michal Orzel @ 2023-04-17 9:41 UTC (permalink / raw)
To: Vikram Garhwal, xen-devel
Cc: sstabellini, Julien Grall, Bertrand Marquis, Volodymyr Babchuk
Hi Vikram,
On 11/04/2023 21:16, Vikram Garhwal wrote:
>
>
> Dynamic programming ops will modify the dt_host and there might be other
> function which are browsing the dt_host at the same time. To avoid the race
> conditions, adding rwlock for browsing the dt_host. But adding rwlock in
> device_tree.h causes following circular dependency:
> device_tree.h->rwlock.h->smp.h->asm/smp.h->device_tree.h
>
> To fix this, removed the "#include <xen/device_tree.h> and forward declared
> "struct dt_device_node".
>
> Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
> ---
> xen/arch/arm/include/asm/smp.h | 3 ++-
> xen/arch/arm/smpboot.c | 1 +
> 2 files changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/xen/arch/arm/include/asm/smp.h b/xen/arch/arm/include/asm/smp.h
> index 8133d5c295..afe6129276 100644
> --- a/xen/arch/arm/include/asm/smp.h
> +++ b/xen/arch/arm/include/asm/smp.h
> @@ -3,13 +3,14 @@
>
> #ifndef __ASSEMBLY__
> #include <xen/cpumask.h>
> -#include <xen/device_tree.h>
> #include <asm/current.h>
> #endif
>
> DECLARE_PER_CPU(cpumask_var_t, cpu_sibling_mask);
> DECLARE_PER_CPU(cpumask_var_t, cpu_core_mask);
>
> +struct dt_device_node;
> +
> #define cpu_is_offline(cpu) unlikely(!cpu_online(cpu))
>
> #define smp_processor_id() get_processor_id()
> diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
> index 412ae22869..336a7d418b 100644
> --- a/xen/arch/arm/smpboot.c
> +++ b/xen/arch/arm/smpboot.c
> @@ -11,6 +11,7 @@
> #include <xen/cpumask.h>
> #include <xen/delay.h>
> #include <xen/domain_page.h>
> +#include <xen/device_tree.h>
Headers should be listed in alphabetical order, so device_tree.h goes before domain_page.h
Other than that:
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
~Michal
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [XEN][PATCH v5 07/17] xen/smmu: Add remove_device callback for smmu_iommu ops
2023-04-11 19:16 ` [XEN][PATCH v5 07/17] xen/smmu: Add remove_device callback for smmu_iommu ops Vikram Garhwal
@ 2023-04-17 10:24 ` Michal Orzel
0 siblings, 0 replies; 61+ messages in thread
From: Michal Orzel @ 2023-04-17 10:24 UTC (permalink / raw)
To: Vikram Garhwal, xen-devel
Cc: sstabellini, Julien Grall, Rahul Singh, Bertrand Marquis,
Volodymyr Babchuk
Hi Vikram,
On 11/04/2023 21:16, Vikram Garhwal wrote:
>
>
> Add remove_device callback for removing the device entry from smmu-master using
> following steps:
> 1. Find if SMMU master exists for the device node.
> 2. Remove the SMMU master
>
> Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
> Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
> ---
> xen/drivers/passthrough/arm/smmu.c | 56 ++++++++++++++++++++++++++++++
> 1 file changed, 56 insertions(+)
>
> diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c
> index 0a514821b3..14e15f1bc6 100644
> --- a/xen/drivers/passthrough/arm/smmu.c
> +++ b/xen/drivers/passthrough/arm/smmu.c
> @@ -816,6 +816,19 @@ static int insert_smmu_master(struct arm_smmu_device *smmu,
> return 0;
> }
>
> +static int remove_smmu_master(struct arm_smmu_device *smmu,
> + struct arm_smmu_master *master)
> +{
> + if (!smmu->masters.rb_node) {
> + ASSERT_UNREACHABLE();
> + return -ENOENT;
> + }
> +
> + rb_erase(&master->node, &smmu->masters);
> +
> + return 0;
> +}
> +
> static int arm_smmu_dt_add_device_legacy(struct arm_smmu_device *smmu,
> struct device *dev,
> struct iommu_fwspec *fwspec)
> @@ -853,6 +866,32 @@ static int arm_smmu_dt_add_device_legacy(struct arm_smmu_device *smmu,
> return insert_smmu_master(smmu, master);
> }
>
> +static int arm_smmu_dt_remove_device_legacy(struct arm_smmu_device *smmu,
> + struct device *dev)
> +{
> + struct arm_smmu_master *master;
> + struct device_node *dev_node = dev_get_dev_node(dev);
> + int ret;
> +
> + master = find_smmu_master(smmu, dev_node);
> + if (master == NULL) {
> + dev_err(dev,
> + "No registrations found for master device %s\n",
> + dev_node->name);
> + return -EINVAL;
> + }
> +
> + ret = remove_smmu_master(smmu, master);
This patch looks good although I remember seeing Julien advising you that it would be beneficial
for the SMMU driver itself to check if device is not currently used before you remove it (even though
you have this check in iommu_remove_dt_device(). I could not find your answer to this.
NIT: No need for a blank line here if the next instruction is checking the ret value.
With the above things clarified:
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
~Michal
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [XEN][PATCH v5 09/17] xen/iommu: protect iommu_add_dt_device() with dtdevs_lock
2023-04-11 19:16 ` [XEN][PATCH v5 09/17] xen/iommu: protect iommu_add_dt_device() with dtdevs_lock Vikram Garhwal
@ 2023-04-17 10:35 ` Michal Orzel
0 siblings, 0 replies; 61+ messages in thread
From: Michal Orzel @ 2023-04-17 10:35 UTC (permalink / raw)
To: Vikram Garhwal, xen-devel; +Cc: sstabellini, Julien Grall
Hi Vikram,
On 11/04/2023 21:16, Vikram Garhwal wrote:
>
>
> Protect iommu_add_dt_device() with dtdevs_lock to prevent concurrent access add.
>
> Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
> Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
~Michal
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [XEN][PATCH v5 10/17] xen/iommu: Introduce iommu_remove_dt_device()
2023-04-11 19:16 ` [XEN][PATCH v5 10/17] xen/iommu: Introduce iommu_remove_dt_device() Vikram Garhwal
@ 2023-04-17 10:49 ` Michal Orzel
0 siblings, 0 replies; 61+ messages in thread
From: Michal Orzel @ 2023-04-17 10:49 UTC (permalink / raw)
To: Vikram Garhwal, xen-devel
Cc: sstabellini, Julien Grall, Jan Beulich, Paul Durrant,
Roger Pau Monné
Hi Vikram,
On 11/04/2023 21:16, Vikram Garhwal wrote:
>
>
> Remove master device from the IOMMU. This will be helpful when removing the
> overlay nodes using dynamic programming during run time.
>
> Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
> ---
> xen/drivers/passthrough/device_tree.c | 38 +++++++++++++++++++++++++++
> xen/include/xen/iommu.h | 2 ++
> 2 files changed, 40 insertions(+)
>
> diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c
> index 457df333a0..a77a217f3d 100644
> --- a/xen/drivers/passthrough/device_tree.c
> +++ b/xen/drivers/passthrough/device_tree.c
> @@ -126,6 +126,44 @@ int iommu_release_dt_devices(struct domain *d)
> return 0;
> }
>
> +int iommu_remove_dt_device(struct dt_device_node *np)
> +{
> + const struct iommu_ops *ops = iommu_get_ops();
> + struct device *dev = dt_to_dev(np);
> + int rc;
> +
> + if ( !ops )
> + return -EOPNOTSUPP;
> +
> + spin_lock(&dtdevs_lock);
> +
> + if ( iommu_dt_device_is_assigned_locked(np) )
> + {
> + rc = -EBUSY;
> + goto fail;
> + }
> +
> + /*
> + * The driver which supports generic IOMMU DT bindings must have
> + * these callback implemented.
s/these/this since you are checking for a single callback.
> + */
> + if ( !ops->remove_device )
> + {
> + rc = -EOPNOTSUPP;
> + goto fail;
> + }
> +
> + /* Remove master device from the IOMMU if latter is present and available. */
80 chars length exceeded -> please fix.
Apart from that, similarly to comment inside iommu_add_dt_device(), I would also write
something that the driver is responsible for unsetting is_protected flag.
Apart from that:
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
~Michal
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [XEN][PATCH v5 13/17] xen/arm: Implement device tree node removal functionalities
2023-04-11 19:16 ` [XEN][PATCH v5 13/17] xen/arm: Implement device tree node removal functionalities Vikram Garhwal
2023-04-17 8:08 ` Jan Beulich
@ 2023-04-17 15:06 ` Michal Orzel
1 sibling, 0 replies; 61+ messages in thread
From: Michal Orzel @ 2023-04-17 15:06 UTC (permalink / raw)
To: Vikram Garhwal, xen-devel
Cc: sstabellini, Julien Grall, Bertrand Marquis, Volodymyr Babchuk,
Andrew Cooper, George Dunlap, Jan Beulich, Wei Liu
Hi Vikram,
On 11/04/2023 21:16, Vikram Garhwal wrote:
>
>
> Introduce sysctl XEN_SYSCTL_dt_overlay to remove device-tree nodes added using
> device tree overlay.
>
> xl dt-overlay remove file.dtbo:
> Removes all the nodes in a given dtbo.
> First, removes IRQ permissions and MMIO accesses. Next, it finds the nodes
> in dt_host and delete the device node entries from dt_host.
>
> The nodes get removed only if it is not used by any of dom0 or domio.
>
> Also, added overlay_track struct to keep the track of added node through device
> tree overlay. overlay_track has dt_host_new which is unflattened form of updated
> fdt and name of overlay nodes. When a node is removed, we also free the memory
> used by overlay_track for the particular overlay node.
>
> Nested overlay removal is supported in sequential manner only i.e. if
> overlay_child nests under overlay_parent, it is assumed that user first removes
> overlay_child and then removes overlay_parent.
>
> Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
> ---
> xen/arch/arm/sysctl.c | 16 +-
> xen/common/Makefile | 1 +
> xen/common/dt_overlay.c | 415 +++++++++++++++++++++++++++++++++++
> xen/include/public/sysctl.h | 24 ++
> xen/include/xen/dt_overlay.h | 59 +++++
> 5 files changed, 514 insertions(+), 1 deletion(-)
> create mode 100644 xen/common/dt_overlay.c
> create mode 100644 xen/include/xen/dt_overlay.h
>
> diff --git a/xen/arch/arm/sysctl.c b/xen/arch/arm/sysctl.c
> index b0a78a8b10..672db61650 100644
> --- a/xen/arch/arm/sysctl.c
> +++ b/xen/arch/arm/sysctl.c
> @@ -12,6 +12,7 @@
> #include <xen/errno.h>
> #include <xen/hypercall.h>
> #include <public/sysctl.h>
> +#include <xen/dt_overlay.h>
xen/ headers should be grouped together so this should be moved before public/.
>
> void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
> {
> @@ -21,7 +22,20 @@ void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
> long arch_do_sysctl(struct xen_sysctl *sysctl,
> XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl)
> {
> - return -ENOSYS;
> + long ret = 0;
> +
> + switch ( sysctl->cmd )
> + {
> + case XEN_SYSCTL_dt_overlay:
> + ret = dt_sysctl(&sysctl->u.dt_overlay);
> + break;
> +
> + default:
> + ret = -ENOSYS;
> + break;
> + }
> +
> + return ret;
> }
>
> /*
> diff --git a/xen/common/Makefile b/xen/common/Makefile
> index 46049eac35..be78c9a8c2 100644
> --- a/xen/common/Makefile
> +++ b/xen/common/Makefile
> @@ -8,6 +8,7 @@ obj-$(CONFIG_DEBUG_TRACE) += debugtrace.o
> obj-$(CONFIG_HAS_DEVICE_TREE) += device_tree.o
> obj-$(CONFIG_IOREQ_SERVER) += dm.o
> obj-y += domain.o
> +obj-$(CONFIG_OVERLAY_DTB) += dt_overlay.o
> obj-y += event_2l.o
> obj-y += event_channel.o
> obj-y += event_fifo.o
> diff --git a/xen/common/dt_overlay.c b/xen/common/dt_overlay.c
> new file mode 100644
> index 0000000000..516e8010c5
> --- /dev/null
> +++ b/xen/common/dt_overlay.c
> @@ -0,0 +1,415 @@
> +/*
> + * SPDX-License-Identifier: GPL-2.0
Our CODING_STYLE states that SPDX tag should be a single line comment at the top of the file.
> + *
> + * xen/common/dt_overlay.c
> + *
> + * Device tree overlay support in Xen.
> + *
> + * Copyright (C) 2023, Advanced Micro Devices, Inc. All Rights Reserved.
The copyright year as well as company name is different here than in the header dt_overlay.h
> + * Written by Vikram Garhwal <vikram.garhwal@amd.com>
> + *
> + */
> +#include <xen/iocap.h>
> +#include <xen/xmalloc.h>
> +#include <asm/domain_build.h>
> +#include <xen/dt_overlay.h>
> +#include <xen/guest_access.h>
Sort headers alphabetically and group them depending on xen/ or asm/.
> +
> +static LIST_HEAD(overlay_tracker);
> +static DEFINE_SPINLOCK(overlay_lock);
> +
> +/* Find last descendants of the device_node. */
> +static struct dt_device_node *find_last_descendants_node(
> + struct dt_device_node *device_node)
To correctly split arguments this should be:
static struct dt_device_node *
find_last_descendants_node(struct dt_device_node *device_node)
> +{
> + struct dt_device_node *child_node;
> +
> + for ( child_node = device_node->child; child_node->sibling != NULL;
> + child_node = child_node->sibling )
> + {
> + }
> +
> + /* If last child_node also have children. */
> + if ( child_node->child )
> + child_node = find_last_descendants_node(child_node);
> +
> + return child_node;
> +}
> +
> +static int dt_overlay_remove_node(struct dt_device_node *device_node)
> +{
> + struct dt_device_node *np;
> + struct dt_device_node *parent_node;
> + struct dt_device_node *device_node_last_descendant = device_node->child;
> +
> + parent_node = device_node->parent;
> +
> + if ( parent_node == NULL )
> + {
> + dt_dprintk("%s's parent node not found\n", device_node->name);
> + return -EFAULT;
> + }
> +
> + np = parent_node->child;
> +
> + if ( np == NULL )
> + {
> + dt_dprintk("parent node %s's not found\n", parent_node->name);
> + return -EFAULT;
> + }
> +
> + /* If node to be removed is only child node or first child. */
> + if ( !dt_node_cmp(np->full_name, device_node->full_name) )
> + {
> + parent_node->child = np->sibling;
> +
> + /*
> + * Iterate over all child nodes of device_node. Given that we are
> + * removing parent node, we need to remove all it's descendants too.
> + */
> + if ( device_node_last_descendant )
> + {
> + device_node_last_descendant =
> + find_last_descendants_node(device_node);
> + parent_node->allnext = device_node_last_descendant->allnext;
> + }
> + else
> + parent_node->allnext = np->allnext;
> +
> + return 0;
> + }
> +
> + for ( np = parent_node->child; np->sibling != NULL; np = np->sibling )
> + {
> + if ( !dt_node_cmp(np->sibling->full_name, device_node->full_name) )
> + {
> + /* Found the node. Now we remove it. */
> + np->sibling = np->sibling->sibling;
> +
> + if ( np->child )
> + np = find_last_descendants_node(np);
> +
> + /*
> + * Iterate over all child nodes of device_node. Given that we are
> + * removing parent node, we need to remove all it's descendants too.
> + */
> + if ( device_node_last_descendant )
> + device_node_last_descendant =
> + find_last_descendants_node(device_node);
> +
> + if ( device_node_last_descendant )
> + np->allnext = device_node_last_descendant->allnext;
> + else
> + np->allnext = np->allnext->allnext;
> +
> + break;
> + }
> + }
> +
> + return 0;
> +}
> +
> +/* Basic sanity check for the dtbo tool stack provided to Xen. */
> +static int check_overlay_fdt(const void *overlay_fdt, uint32_t overlay_fdt_size)
> +{
> + if ( (fdt_totalsize(overlay_fdt) != overlay_fdt_size) ||
> + fdt_check_header(overlay_fdt) )
> + {
> + printk(XENLOG_ERR "The overlay FDT is not a valid Flat Device Tree\n");
> + return -EINVAL;
> + }
> +
> + return 0;
> +}
> +
> +/* Count number of nodes till one level of __overlay__ tag. */
> +static unsigned int overlay_node_count(void *fdto)
fdto can be const. Also in other function you name the same parameter as overlay_fdt
so it would be best to stick to one naming scheme.
> +{
> + unsigned int num_overlay_nodes = 0;
> + int fragment;
> +
> + fdt_for_each_subnode(fragment, fdto, 0)
> + {
> + int subnode;
> + int overlay;
> +
> + overlay = fdt_subnode_offset(fdto, fragment, "__overlay__");
> +
> + /*
> + * overlay value can be < 0. But fdt_for_each_subnode() loop checks for
> + * overlay >= 0. So, no need for a overlay>=0 check here.
> + */
> + fdt_for_each_subnode(subnode, fdto, overlay)
> + {
> + num_overlay_nodes++;
> + }
> + }
> +
> + return num_overlay_nodes;
> +}
> +
> +static int handle_remove_irq_iommu(struct dt_device_node *device_node)
> +{
> + int rc = 0;
> + struct domain *d = hardware_domain;
> + domid_t domid;
> + unsigned int naddr, len;
> + unsigned int i, nirq;
> + uint64_t addr, size;
You could limit the scope of addr and size by moving them to for loop.
> +
> + domid = dt_device_used_by(device_node);
> +
> + dt_dprintk("Checking if node %s is used by any domain\n",
> + device_node->full_name);
> +
> + /* Remove the node iff it's assigned to domain 0 or domain io. */
s/iff/if
> + if ( domid != 0 && domid != DOMID_IO )
> + {
> + printk(XENLOG_ERR "Device %s as it is being used by domain %d. Removing nodes failed\n",
Use %u to print domid. Also s/as it is/is/
> + device_node->full_name, domid);
> + return -EINVAL;
> + }
> +
> + dt_dprintk("Removing node: %s\n", device_node->full_name);
> +
> + nirq = dt_number_of_irq(device_node);
> +
> + /* Remove IRQ permission */
> + for ( i = 0; i < nirq; i++ )
> + {
> + rc = platform_get_irq(device_node, i);;
remove extra ; at the end of line.
Also, shouldn't you first make sure that rc is >= 0 before checking access?
> +
> + if ( irq_access_permitted(d, rc) == false )
Instead of d which points to hwdom, can't you use just domid?
> + {
> + printk(XENLOG_ERR "IRQ %d is not routed to domain %d\n", rc,
%u for domid
> + domid);
> + return -EINVAL;
> + }
> + /*
> + * TODO: We don't handle shared IRQs for now. So, it is assumed that
> + * the IRQs was not shared with another devices.
> + */
> + rc = irq_deny_access(d, rc);
> + if ( rc )
> + {
> + printk(XENLOG_ERR "unable to revoke access for irq %u for %s\n",
> + i, device_node->full_name);
> + return rc;
> + }
> + }
> +
> + /* Check if iommu property exists. */
> + if ( dt_get_property(device_node, "iommus", &len) )
> + {
> + rc = iommu_remove_dt_device(device_node);
> + if ( rc != 0 && rc != -ENXIO )
> + return rc;
> + }
> +
> + naddr = dt_number_of_address(device_node);
> +
> + /* Remove mmio access. */
> + for ( i = 0; i < naddr; i++ )
> + {
> + rc = dt_device_get_address(device_node, i, &addr, &size);
> + if ( rc )
> + {
> + printk(XENLOG_ERR "Unable to retrieve address %u for %s\n",
> + i, dt_node_full_name(device_node));
> + return rc;
> + }
> +
> + rc = iomem_deny_access(d, paddr_to_pfn(addr),
> + paddr_to_pfn(PAGE_ALIGN(addr + size - 1)));
> + if ( rc )
> + {
> + printk(XENLOG_ERR "Unable to remove dom%d access to"
> + " 0x%"PRIx64" - 0x%"PRIx64"\n",
> + d->domain_id,
> + addr & PAGE_MASK, PAGE_ALIGN(addr + size) - 1);
> + return rc;
> + }
> +
> + }
> +
> + return rc;
> +}
> +
> +/* Removes all descendants of the given node. */
> +static int remove_all_descendant_nodes(struct dt_device_node *device_node)
> +{
> + int rc = 0;
> + struct dt_device_node *child_node;
> +
> + for ( child_node = device_node->child; child_node != NULL;
> + child_node = child_node->sibling )
> + {
> + if ( child_node->child )
> + remove_all_descendant_nodes(child_node);
> +
> + rc = handle_remove_irq_iommu(child_node);
> + if ( rc )
> + return rc;
> + }
> +
> + return rc;
> +}
> +
> +/* Remove nodes from dt_host. */
> +static int remove_nodes(const struct overlay_track *tracker)
> +{
> + int rc = 0;
> + struct dt_device_node *overlay_node;
> + unsigned int j;
> +
> + for ( j = 0; j < tracker->num_nodes; j++ )
> + {
> + overlay_node = (struct dt_device_node *)tracker->nodes_address[j];
> + if ( overlay_node == NULL )
> + {
> + printk(XENLOG_ERR "Device %s is not present in the tree. Removing nodes failed\n",
> + overlay_node->full_name);
> + return -EINVAL;
> + }
> +
> + rc = remove_all_descendant_nodes(overlay_node);
> +
> + /* All children nodes are unmapped. Now remove the node itself. */
> + rc = handle_remove_irq_iommu(overlay_node);
> + if ( rc )
> + return rc;
> +
> + read_lock(&dt_host->lock);
> +
> + rc = dt_overlay_remove_node(overlay_node);
> + if ( rc )
> + {
> + read_unlock(&dt_host->lock);
> +
> + return rc;
> + }
> +
> + read_unlock(&dt_host->lock);
> + }
> +
> + return rc;
> +}
> +
> +/*
> + * First finds the device node to remove. Check if the device is being used by
> + * any dom and finally remove it from dt_host. IOMMU is already being taken care
> + * while destroying the domain.
> + */
> +static long handle_remove_overlay_nodes(void *overlay_fdt,
> + uint32_t overlay_fdt_size)
> +{
> + int rc = 0;
> + struct overlay_track *entry, *temp, *track;
> + bool found_entry = false;
> +
> + rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
> + if ( rc )
> + return rc;
> +
> + if ( overlay_node_count(overlay_fdt) == 0 )
> + return -ENOMEM;
Why ENOMEM if there was no allocation attempt?
> +
> + spin_lock(&overlay_lock);
> +
> + /*
> + * First check if dtbo is correct i.e. it should one of the dtbo which was
> + * used when dynamically adding the node.
> + * Limitation: Cases with same node names but different property are not
> + * supported currently. We are relying on user to provide the same dtbo
> + * as it was used when adding the nodes.
> + */
> + list_for_each_entry_safe( entry, temp, &overlay_tracker, entry )
> + {
> + if ( memcmp(entry->overlay_fdt, overlay_fdt, overlay_fdt_size) == 0 )
> + {
> + track = entry;
> + found_entry = true;
> + break;
> + }
> + }
> +
> + if ( found_entry == false )
> + {
> + rc = -EINVAL;
> +
> + printk(XENLOG_ERR "Cannot find any matching tracker with input dtbo."
> + " Removing nodes is supported for only prior added dtbo. Please"
> + " provide a valid dtbo which was used to add the nodes.\n");
> + goto out;
> +
> + }
> +
> + rc = remove_nodes(entry);
> +
> + if ( rc )
> + {
> + printk(XENLOG_ERR "Removing node failed\n");
> + goto out;
> + }
> +
> + list_del(&entry->entry);
> +
> + xfree(entry->dt_host_new);
> + xfree(entry->fdt);
> + xfree(entry->overlay_fdt);
> +
> + xfree(entry->nodes_address);
> +
> + xfree(entry);
> +
> +out:
> + spin_unlock(&overlay_lock);
> + return rc;
> +}
> +
> +long dt_sysctl(struct xen_sysctl_dt_overlay *op)
> +{
> + long ret;
> + void *overlay_fdt;
> +
> + if ( op->overlay_fdt_size <= 0 || op->overlay_fdt_size > KB(500) )
overlay_fdt_size is uint32_t so it cannot be < 0.
> + return -EINVAL;
> +
> + overlay_fdt = xmalloc_bytes(op->overlay_fdt_size);
> +
> + if ( overlay_fdt == NULL )
> + return -ENOMEM;
> +
> + ret = copy_from_guest(overlay_fdt, op->overlay_fdt, op->overlay_fdt_size);
> + if ( ret )
> + {
> + gprintk(XENLOG_ERR, "copy from guest failed\n");
> + xfree(overlay_fdt);
> +
> + return -EFAULT;
> + }
> +
> + switch ( op->overlay_op )
> + {
> + case XEN_SYSCTL_DT_OVERLAY_REMOVE:
> + ret = handle_remove_overlay_nodes(overlay_fdt, op->overlay_fdt_size);
> + xfree(overlay_fdt);
> +
> + break;
> +
> + default:
> + xfree(overlay_fdt);
> + break;
> + }
> +
> + return ret;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
> index 2b24d6bfd0..1158c1efb3 100644
> --- a/xen/include/public/sysctl.h
> +++ b/xen/include/public/sysctl.h
> @@ -1057,6 +1057,25 @@ typedef struct xen_sysctl_cpu_policy xen_sysctl_cpu_policy_t;
> DEFINE_XEN_GUEST_HANDLE(xen_sysctl_cpu_policy_t);
> #endif
>
> +#if defined(__arm__) || defined (__aarch64__)
> +#define XEN_SYSCTL_DT_OVERLAY_ADD 1
> +#define XEN_SYSCTL_DT_OVERLAY_REMOVE 2
> +
> +/*
> + * XEN_SYSCTL_dt_overlay
> + * Performs addition/removal of device tree nodes under parent node using dtbo.
> + * This does in three steps:
This is done... would read better.
> + * - Adds/Removes the nodes from dt_host.
> + * - Adds/Removes IRQ permission for the nodes.
> + * - Adds/Removes MMIO accesses.
> + */
> +struct xen_sysctl_dt_overlay {
> + XEN_GUEST_HANDLE_64(void) overlay_fdt; /* IN: overlay fdt. */
> + uint32_t overlay_fdt_size; /* IN: Overlay dtb size. */
> + uint8_t overlay_op; /* IN: Add or remove. */
Please align comments.
Also you could move the command macros right before overlay_op to improve
readability (just an idea to consider, not a request).
> +};
> +#endif
> +
> struct xen_sysctl {
> uint32_t cmd;
> #define XEN_SYSCTL_readconsole 1
> @@ -1087,6 +1106,7 @@ struct xen_sysctl {
> #define XEN_SYSCTL_livepatch_op 27
> /* #define XEN_SYSCTL_set_parameter 28 */
> #define XEN_SYSCTL_get_cpu_policy 29
> +#define XEN_SYSCTL_dt_overlay 30
Did you check if you need to bump XEN_SYSCTL_INTERFACE_VERSION ?
> uint32_t interface_version; /* XEN_SYSCTL_INTERFACE_VERSION */
> union {
> struct xen_sysctl_readconsole readconsole;
> @@ -1117,6 +1137,10 @@ struct xen_sysctl {
> #if defined(__i386__) || defined(__x86_64__)
> struct xen_sysctl_cpu_policy cpu_policy;
> #endif
> +
> +#if defined(__arm__) || defined (__aarch64__)
> + struct xen_sysctl_dt_overlay dt_overlay;
> +#endif
> uint8_t pad[128];
> } u;
> };
> diff --git a/xen/include/xen/dt_overlay.h b/xen/include/xen/dt_overlay.h
> new file mode 100644
> index 0000000000..2cd975a070
> --- /dev/null
> +++ b/xen/include/xen/dt_overlay.h
> @@ -0,0 +1,59 @@
> +/*
> + * SPDX-License-Identifier: GPL-2.0
> + *
> + * xen/dt_overlay.h
> + *
> + * Device tree overlay support in Xen.
> + *
> + * Copyright (c) 2022 AMD Inc.
Different copyright info in comparison to source file.
> + * Written by Vikram Garhwal <vikram.garhwal@amd.com>
> + *
> + */
> +#ifndef __XEN_DT_SYSCTL_H__
SYSCTL? I think you want __XEN_DT_OVERLAY_H__
~Michal
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [XEN][PATCH v5 14/17] xen/arm: Implement device tree node addition functionalities
2023-04-11 19:16 ` [XEN][PATCH v5 14/17] xen/arm: Implement device tree node addition functionalities Vikram Garhwal
@ 2023-04-18 9:03 ` Michal Orzel
0 siblings, 0 replies; 61+ messages in thread
From: Michal Orzel @ 2023-04-18 9:03 UTC (permalink / raw)
To: Vikram Garhwal, xen-devel
Cc: sstabellini, Andrew Cooper, George Dunlap, Jan Beulich,
Julien Grall, Wei Liu
Hi Vikram,
On 11/04/2023 21:16, Vikram Garhwal wrote:
>
>
> Update sysctl XEN_SYSCTL_dt_overlay to enable support for dtbo nodes addition
> using device tree overlay.
>
> xl dt-overlay add file.dtbo:
> Each time overlay nodes are added using .dtbo, a new fdt(memcpy of
> device_tree_flattened) is created and updated with overlay nodes. This
> updated fdt is further unflattened to a dt_host_new. Next, it checks if any
> of the overlay nodes already exists in the dt_host. If overlay nodes doesn't
> exist then find the overlay nodes in dt_host_new, find the overlay node's
> parent in dt_host and add the nodes as child under their parent in the
> dt_host. The node is attached as the last node under target parent.
>
> Finally, add IRQs, add device to IOMMUs, set permissions and map MMIO for the
> overlay node.
>
> When a node is added using overlay, a new entry is allocated in the
> overlay_track to keep the track of memory allocation due to addition of overlay
> node. This is helpful for freeing the memory allocated when a device tree node
> is removed.
>
> The main purpose of this to address first part of dynamic programming i.e.
> making xen aware of new device tree node which means updating the dt_host with
> overlay node information. Here we are adding/removing node from dt_host, and
> checking/setting IOMMU and IRQ permission but never mapping them to any domain.
> Right now, mapping/Un-mapping will happen only when a new domU is
> created/destroyed using "xl create".
>
> Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
> ---
> xen/common/dt_overlay.c | 482 ++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 482 insertions(+)
>
> diff --git a/xen/common/dt_overlay.c b/xen/common/dt_overlay.c
> index 516e8010c5..3344bad313 100644
> --- a/xen/common/dt_overlay.c
> +++ b/xen/common/dt_overlay.c
> @@ -36,6 +36,25 @@ static struct dt_device_node *find_last_descendants_node(
> return child_node;
> }
>
> +/*
> + * Returns next node to the input node. If node has children then return
> + * last descendant's next node.
> +*/
> +static struct dt_device_node *dt_find_next_node(struct dt_device_node *dt,
> + const struct dt_device_node *node)
This should be:
static struct dt_device_node *
dt_find_next_node(struct dt_device_node *dt, const struct dt_device_node *node)
> +{
> + struct dt_device_node *np;
> +
> + dt_for_each_device_node(dt, np)
> + if ( np == node )
> + break;
> +
> + if ( np->child )
> + np = find_last_descendants_node(np);
> +
> + return np->allnext;
> +}
> +
> static int dt_overlay_remove_node(struct dt_device_node *device_node)
> {
> struct dt_device_node *np;
> @@ -109,6 +128,72 @@ static int dt_overlay_remove_node(struct dt_device_node *device_node)
> return 0;
> }
>
> +static int dt_overlay_add_node(struct dt_device_node *device_node,
> + const char *parent_node_path)
> +{
> + struct dt_device_node *parent_node;
> + struct dt_device_node *np, *np_last_descendant;
> + struct dt_device_node *next_node;
> + struct dt_device_node *device_node_last_descendant;
You can limit the scope of at least 3 variables above.
> +
> + parent_node = dt_find_node_by_path(parent_node_path);
> +
> + if ( parent_node == NULL )
> + {
> + dt_dprintk("Node not found. Overlay node will not be added\n");
It would help if you print the node name.
> + return -EINVAL;
> + }
> +
> + /* If parent has no child. */
> + if ( parent_node->child == NULL )
> + {
> + next_node = parent_node->allnext;
> + device_node->parent = parent_node;
> + parent_node->allnext = device_node;
> + parent_node->child = device_node;
> + }
> + else
> + {
> + /* If parent has at least one child node.
> + * Iterate to the last child node of parent.
> + */
> + for ( np = parent_node->child; np->sibling != NULL; np = np->sibling );
NIT: Your empty for() loop is not consistent. Here you use ; and somewhere else {}.
> +
> + /* Iterate over all child nodes of np node. */
> + if ( np->child )
> + {
> + np_last_descendant = find_last_descendants_node(np);
> +
> + next_node = np_last_descendant->allnext;
> + np_last_descendant->allnext = device_node;
> + }
> + else
> + {
> + next_node = np->allnext;
> + np->allnext = device_node;
> + }
> +
> + device_node->parent = parent_node;
> + np->sibling = device_node;
> + np->sibling->sibling = NULL;
> + }
> +
> + /* Iterate over all child nodes of device_node to add children too. */
> + if ( device_node->child )
> + {
> + device_node_last_descendant = find_last_descendants_node(device_node);
> + /* Plug next_node at the end of last children of device_node. */
> + device_node_last_descendant->allnext = next_node;
> + }
> + else
> + {
> + /* Now plug next_node at the end of device_node. */
> + device_node->allnext = next_node;
> + }
> +
> + return 0;
> +}
> +
> /* Basic sanity check for the dtbo tool stack provided to Xen. */
> static int check_overlay_fdt(const void *overlay_fdt, uint32_t overlay_fdt_size)
> {
> @@ -148,6 +233,79 @@ static unsigned int overlay_node_count(void *fdto)
> return num_overlay_nodes;
> }
>
> +/*
> + * overlay_get_nodes_info will get full name with path for all the nodes which
s/will get/gets
> + * are in one level of __overlay__ tag. This is useful when checking node for
> + * duplication i.e. dtbo tries to add nodes which already exists in device tree.
> + */
> +static int overlay_get_nodes_info(const void *fdto, char ***nodes_full_path,
> + unsigned int num_overlay_nodes)
> +{
> + int fragment;
> +
> + *nodes_full_path = xzalloc_bytes(num_overlay_nodes * sizeof(char *));
> +
> + if ( *nodes_full_path == NULL )
> + return -ENOMEM;
> +
> + fdt_for_each_subnode(fragment, fdto, 0)
> + {
> + int target;
> + int overlay;
> + int subnode;
> + const char *target_path;
> +
> + target = fdt_overlay_target_offset(device_tree_flattened, fdto,
> + fragment, &target_path);
Shouldn't you also check if target_path is not NULL?
I can see that it is possible for fdt_overlay_target_offset() to return >= 0 and NULL as target_path.
> + if ( target < 0 )
> + return target;
> +
> + overlay = fdt_subnode_offset(fdto, fragment, "__overlay__");
> +
> + /*
> + * overlay value can be < 0. But fdt_for_each_subnode() loop checks for
> + * overlay >= 0. So, no need for a overlay>=0 check here.
> + */
> + fdt_for_each_subnode(subnode, fdto, overlay)
> + {
> + const char *node_name = NULL;
> + int node_name_len;
> + unsigned int target_path_len = strlen(target_path);
> + unsigned int node_full_name_len;
> + unsigned int node_num = 0;
> +
> + node_name = fdt_get_name(fdto, subnode, &node_name_len);
> +
> + if ( node_name == NULL )
> + return node_name_len;
> +
> + /*
> + * Magic number 2 is for adding '/'. This is done to keep the
I think this is 2 because of adding both '/' and '\0'.
> + * node_full_name in the correct full node name format.
No such variable as node_full_name
> + */
> + node_full_name_len = target_path_len + node_name_len + 2;
> +
> + (*nodes_full_path)[node_num] = xmalloc_bytes(node_full_name_len);
> +
> + if ( (*nodes_full_path)[node_num] == NULL )
> + return -ENOMEM;
> +
> + memcpy((*nodes_full_path)[node_num], target_path, target_path_len);
> +
> + (*nodes_full_path)[node_num][target_path_len] = '/';
> +
> + memcpy((*nodes_full_path)[node_num] + target_path_len + 1,
> + node_name, node_name_len);
> +
> + (*nodes_full_path)[node_num][node_full_name_len - 1] = '\0';
> +
> + node_num++;
> + }
> + }
> +
> + return 0;
> +}
> +
> static int handle_remove_irq_iommu(struct dt_device_node *device_node)
> {
> int rc = 0;
> @@ -367,6 +525,322 @@ out:
> return rc;
> }
>
> +/*
> + * Handles IRQ and IOMMU mapping for the overlay_node and all descendants of the
> + * overlay_nodes.
s/overlay_nodes/overlay_node/ ?
> + */
> +static int handle_add_irq_iommu(struct domain *d,
> + struct dt_device_node *overlay_node)
> +{
> + int rc;
> + unsigned int naddr, i, len;
> + uint64_t addr, size;
Please limit the scope
> + struct dt_device_node *np;
> +
> + /* First let's handle the interrupts. */
> + rc = handle_device_interrupts(d, overlay_node, false);
> + if ( rc < 0 )
> + {
> + printk(XENLOG_ERR "Interrupt failed\n");
How about: "Failed to retrieve interrupts configuration"
> + return rc;
> + }
> +
> + /* Check if iommu property exists. */
> + if ( dt_get_property(overlay_node, "iommus", &len) )
> + {
> +
remove extra blank line
> + /* Add device to IOMMUs. */
> + rc = iommu_add_dt_device(overlay_node);
> + if ( rc < 0 )
> + {
> + printk(XENLOG_ERR "Failed to add %s to the IOMMU\n",
> + dt_node_full_name(overlay_node));
> + return rc;
> + }
> + }
> +
> + /* Set permissions. */
> + naddr = dt_number_of_address(overlay_node);
> +
> + dt_dprintk("%s passthrough = %d naddr = %u\n",
Why do you need to print passthrough if it is always false?
> + dt_node_full_name(overlay_node), false, naddr);
> +
> + /* Give permission for map MMIOs */
s/for/to/
> + for ( i = 0; i < naddr; i++ )
> + {
> + /*
> + * For now, we skip_mapping which means it will only permit iomem access
> + * to hardware_domain using iomem_permit_access() but will never map as
> + * map_range_p2mt() will not be called.
> + */
> + struct map_range_data mr_data = { .d = d,
> + .p2mt = p2m_mmio_direct_c,
> + .skip_mapping = true };
}; should be placed on its own line
> +
> + rc = dt_device_get_address(overlay_node, i, &addr, &size);
> + if ( rc )
> + {
> + printk(XENLOG_ERR "Unable to retrieve address %u for %s\n",
> + i, dt_node_full_name(overlay_node));
> + return rc;
> + }
> +
> + rc = map_range_to_domain(overlay_node, addr, size, &mr_data);
> + if ( rc )
> + return rc;
> + }
> +
> + /* Map IRQ and IOMMU for overlay_node's children. */
> + for ( np = overlay_node->child; np != NULL; np = np->sibling)
missing space before closing parenthesis
> + {
> + rc = handle_add_irq_iommu(d, np);
> + if ( rc )
> + {
> + printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n");
> + return rc;
> + }
> + }
> +
> + return rc;
> +}
> +
> +/*
> + * Adds device tree nodes under target node.
> + * We use tr->dt_host_new to unflatten the updated device_tree_flattened. This
> + * is done to avoid the removal of device_tree generation, iomem regions mapping
> + * to hardware domain done by handle_node().
> + */
> +static long handle_add_overlay_nodes(void *overlay_fdt,
> + uint32_t overlay_fdt_size)
> +{
> + int rc, j, i;
> + struct dt_device_node *overlay_node, *prev_node, *next_node;
Please limit the scope.
> + struct domain *d = hardware_domain;
No need for extra variable if it is only used in one place. Just use hardware_domain.
> + struct overlay_track *tr = NULL;
> + char **nodes_full_path = NULL;
> + unsigned int new_fdt_size;
> +
> + tr = xzalloc(struct overlay_track);
> + if ( tr == NULL )
> + return -ENOMEM;
> +
> + new_fdt_size = fdt_totalsize(device_tree_flattened) +
> + fdt_totalsize(overlay_fdt);
> +
> + tr->fdt = xzalloc_bytes(new_fdt_size);
> + if ( tr->fdt == NULL )
> + {
> + xfree(tr);
> + return -ENOMEM;
> + }
> +
> + tr->num_nodes = overlay_node_count(overlay_fdt);
> + if ( tr->num_nodes == 0 )
> + {
> + xfree(tr->fdt);
> + xfree(tr);
> + return -ENOMEM;
> + }
> +
> + tr->nodes_address = xzalloc_bytes(tr->num_nodes * sizeof(unsigned long));
> + if ( tr->nodes_address == NULL )
> + {
> + xfree(tr->fdt);
> + xfree(tr);
> + return -ENOMEM;
> + }
> +
> + rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size);
> + if ( rc )
> + {
> + xfree(tr->nodes_address);
> + xfree(tr->fdt);
> + xfree(tr);
> + return rc;
> + }
> +
> + /*
> + * Keep a copy of overlay_fdt as fdt_overlay_apply will change the input
> + * overlay's content(magic) when applying overlay.
> + */
> + tr->overlay_fdt = xzalloc_bytes(overlay_fdt_size);
> + if ( tr->overlay_fdt == NULL )
> + {
> + xfree(tr->nodes_address);
> + xfree(tr->fdt);
> + xfree(tr);
> + return -ENOMEM;
> + }
> +
> + memcpy(tr->overlay_fdt, overlay_fdt, overlay_fdt_size);
> +
> + spin_lock(&overlay_lock);
> +
> + memcpy(tr->fdt, device_tree_flattened,
> + fdt_totalsize(device_tree_flattened));
> +
> + /* Open tr->fdt with more space to accommodate the overlay_fdt. */
> + fdt_open_into(tr->fdt, tr->fdt, new_fdt_size);
You are not checking the return value for an error.
> +
> + /*
> + * overlay_get_nodes_info is called to get the node information from dtbo.
> + * This is done before fdt_overlay_apply() because the overlay apply will
> + * erase the magic of overlay_fdt.
> + */
> + rc = overlay_get_nodes_info(overlay_fdt, &nodes_full_path,
> + tr->num_nodes);
> + if ( rc )
> + {
> + printk(XENLOG_ERR "Getting nodes information failed with error %d\n",
> + rc);
> + goto err;
Looking at err: if you fail here, you will free dt_host_new which was not yet allocated.
> + }
> +
> + rc = fdt_overlay_apply(tr->fdt, overlay_fdt);
> + if ( rc )
> + {
> + printk(XENLOG_ERR "Adding overlay node failed with error %d\n", rc);
> + goto err;
> + }
> +
> + /*
> + * Check if any of the node already exists in dt_host. If node already exits
> + * we can return here as this overlay_fdt is not suitable for overlay ops.
> + */
> + for ( j = 0; j < tr->num_nodes; j++ )
> + {
> + overlay_node = dt_find_node_by_path(nodes_full_path[j]);
> + if ( overlay_node != NULL )
> + {
> + printk(XENLOG_ERR "node %s exists in device tree\n",
> + nodes_full_path[j]);
> + rc = -EINVAL;
> + goto err;
> + }
> + }
> +
> + /* Unflatten the tr->fdt into a new dt_host. */
> + unflatten_device_tree(tr->fdt, &tr->dt_host_new);
> +
> + for ( j = 0; j < tr->num_nodes; j++ )
> + {
> + dt_dprintk("Adding node: %s\n", nodes_full_path[j]);
> +
> + /* Find the newly added node in tr->dt_host_new by it's full path. */
> + overlay_node = device_tree_find_node_by_path(tr->dt_host_new,
> + nodes_full_path[j]);
> + if ( overlay_node == NULL )
> + {
> + dt_dprintk("%s node not found\n", nodes_full_path[j]);
> + rc = -EFAULT;
> + goto remove_node;
You assing rc and go to remove_node which will overwrite it. Please fix.
> + }
> +
> + /*
> + * Find previous and next node to overlay_node in dt_host_new. We will
> + * need these nodes to fix the dt_host_new mapping. When overlay_node is
> + * take out of dt_host_new tree and added to dt_host, link between
> + * previous node and next_node is broken. We will need to refresh
> + * dt_host_new with correct linking for any other overlay nodes
> + * extraction in future.
> + */
> + dt_for_each_device_node(tr->dt_host_new, prev_node)
> + if ( prev_node->allnext == overlay_node )
> + break;
> +
> + next_node = dt_find_next_node(tr->dt_host_new, overlay_node);
> +
> + read_lock(&dt_host->lock);
> +
> + /* Add the node to dt_host. */
> + rc = dt_overlay_add_node(overlay_node, overlay_node->parent->full_name);
> + if ( rc )
> + {
> + read_unlock(&dt_host->lock);
> +
> + /* Node not added in dt_host. */
> + goto remove_node;
> + }
> +
> + read_unlock(&dt_host->lock);
> +
> + prev_node->allnext = next_node;
> +
> + overlay_node = dt_find_node_by_path(overlay_node->full_name);
> + if ( overlay_node == NULL )
> + {
> + /* Sanity check. But code will never come here. */
> + ASSERT_UNREACHABLE();
> + goto remove_node;
> + }
> +
> + rc = handle_add_irq_iommu(d, overlay_node);
> + if ( rc )
> + {
> + printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n");
> + return rc;
> + }
> +
> + /* Keep overlay_node address in tracker. */
> + tr->nodes_address[j] = (unsigned long)overlay_node;
> + }
> +
> + INIT_LIST_HEAD(&tr->entry);
> + list_add_tail(&tr->entry, &overlay_tracker);
> +
> + spin_unlock(&overlay_lock);
> +
> + if ( nodes_full_path != NULL )
> + {
> + for ( i = 0; i < tr->num_nodes && nodes_full_path[i] != NULL;
> + i++ )
> + {
> + xfree(nodes_full_path[i]);
> + }
> + xfree(nodes_full_path);
> + }
> +
> + return rc;
> +
> +/*
> + * Failure case. We need to remove the nodes, free tracker(if tr exists) and
> + * tr->dt_host_new.
> + */
> +remove_node:
> + tr->num_nodes = j;
> + rc = remove_nodes(tr);
> +
> + if ( rc )
> + {
> + /* If removing node fails, this may cause memory leaks. */
I think we are missing information why it is ok to leak memory.
> + printk(XENLOG_ERR "Removing node failed.\n");
> + spin_unlock(&overlay_lock);
> + return rc;
> + }
> +
> +err:
> + spin_unlock(&overlay_lock);
> +
> + xfree(tr->dt_host_new);
> + xfree(tr->fdt);
> + xfree(tr->overlay_fdt);
> + xfree(tr->nodes_address);
What is the order here? This does not look like reverse one.
> +
> + if ( nodes_full_path != NULL )
> + {
> + for ( i = 0; i < tr->num_nodes && nodes_full_path[i] != NULL;
> + i++ )
> + {
> + xfree(nodes_full_path[i]);
> + }
> + xfree(nodes_full_path);
> + }
> +
> + xfree(tr);
> +
> + return rc;
> +}
> +
> long dt_sysctl(struct xen_sysctl_dt_overlay *op)
> {
> long ret;
> @@ -391,6 +865,14 @@ long dt_sysctl(struct xen_sysctl_dt_overlay *op)
>
> switch ( op->overlay_op )
> {
> + case XEN_SYSCTL_DT_OVERLAY_ADD:
> + ret = handle_add_overlay_nodes(overlay_fdt, op->overlay_fdt_size);
> +
no need for extra blank line before checking the return value.
> + if ( ret )
> + xfree(overlay_fdt);
> +
> + break;
> +
> case XEN_SYSCTL_DT_OVERLAY_REMOVE:
> ret = handle_remove_overlay_nodes(overlay_fdt, op->overlay_fdt_size);
> xfree(overlay_fdt);
> --
> 2.17.1
>
>
~Michal
^ permalink raw reply [flat|nested] 61+ messages in thread
end of thread, other threads:[~2023-04-18 9:03 UTC | newest]
Thread overview: 61+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-04-11 19:16 [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 01/17] xen/arm/device: Remove __init from function type Vikram Garhwal
2023-04-13 9:19 ` Michal Orzel
2023-04-14 17:28 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 02/17] common/device_tree: change __unflatten_device_tree() Vikram Garhwal
2023-04-13 9:45 ` Michal Orzel
2023-04-13 9:53 ` Julien Grall
2023-04-13 10:03 ` Julien Grall
2023-04-14 17:51 ` Vikram Garhwal
2023-04-14 18:09 ` Julien Grall
2023-04-14 18:28 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 03/17] xen/arm: Add CONFIG_OVERLAY_DTB Vikram Garhwal
2023-04-13 9:58 ` Michal Orzel
2023-04-13 19:27 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 04/17] libfdt: Keep fdt functions after init for CONFIG_OVERLAY_DTB Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 05/17] libfdt: overlay: change overlay_get_target() Vikram Garhwal
2023-04-13 13:11 ` Michal Orzel
2023-04-14 17:54 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 06/17] xen/device-tree: Add device_tree_find_node_by_path() to find nodes in device tree Vikram Garhwal
2023-04-13 13:40 ` Michal Orzel
2023-04-14 18:04 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 07/17] xen/smmu: Add remove_device callback for smmu_iommu ops Vikram Garhwal
2023-04-17 10:24 ` Michal Orzel
2023-04-11 19:16 ` [XEN][PATCH v5 08/17] xen/iommu: Move spin_lock from iommu_dt_device_is_assigned to caller Vikram Garhwal
2023-04-17 9:14 ` Michal Orzel
2023-04-11 19:16 ` [XEN][PATCH v5 09/17] xen/iommu: protect iommu_add_dt_device() with dtdevs_lock Vikram Garhwal
2023-04-17 10:35 ` Michal Orzel
2023-04-11 19:16 ` [XEN][PATCH v5 10/17] xen/iommu: Introduce iommu_remove_dt_device() Vikram Garhwal
2023-04-17 10:49 ` Michal Orzel
2023-04-11 19:16 ` [XEN][PATCH v5 11/17] asm/smp.h: Fix circular dependency for device_tree.h and rwlock.h Vikram Garhwal
2023-04-14 1:50 ` Henry Wang
2023-04-17 9:41 ` Michal Orzel
2023-04-11 19:16 ` [XEN][PATCH v5 12/17] common/device_tree: Add rwlock for dt_host Vikram Garhwal
2023-04-14 2:09 ` Henry Wang
2023-04-14 17:23 ` Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 13/17] xen/arm: Implement device tree node removal functionalities Vikram Garhwal
2023-04-17 8:08 ` Jan Beulich
2023-04-17 15:06 ` Michal Orzel
2023-04-11 19:16 ` [XEN][PATCH v5 14/17] xen/arm: Implement device tree node addition functionalities Vikram Garhwal
2023-04-18 9:03 ` Michal Orzel
2023-04-11 19:16 ` [XEN][PATCH v5 15/17] tools/libs/ctrl: Implement new xc interfaces for dt overlay Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 16/17] tools/libs/light: Implement new libxl functions for device tree overlay ops Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 17/17] tools/xl: Add new xl command overlay for device tree overlay support Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 00/17] dynamic node programming using overlay dtbo Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 01/17] xen/arm/device: Remove __init from function type Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 02/17] common/device_tree: change __unflatten_device_tree() Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 03/17] xen/arm: Add CONFIG_OVERLAY_DTB Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 04/17] libfdt: Keep fdt functions after init for CONFIG_OVERLAY_DTB Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 05/17] libfdt: overlay: change overlay_get_target() Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 06/17] xen/device-tree: Add device_tree_find_node_by_path() to find nodes in device tree Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 07/17] xen/smmu: Add remove_device callback for smmu_iommu ops Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 08/17] xen/iommu: Move spin_lock from iommu_dt_device_is_assigned to caller Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 09/17] xen/iommu: protect iommu_add_dt_device() with dtdevs_lock Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 10/17] xen/iommu: Introduce iommu_remove_dt_device() Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 11/17] asm/smp.h: Fix circular dependency for device_tree.h and rwlock.h Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 12/17] common/device_tree: Add rwlock for dt_host Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 13/17] xen/arm: Implement device tree node removal functionalities Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 14/17] xen/arm: Implement device tree node addition functionalities Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 15/17] tools/libs/ctrl: Implement new xc interfaces for dt overlay Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 16/17] tools/libs/light: Implement new libxl functions for device tree overlay ops Vikram Garhwal
2023-04-11 19:16 ` [XEN][PATCH v5 17/17] tools/xl: Add new xl command overlay for device tree overlay support Vikram Garhwal
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.