linuxppc-dev.lists.ozlabs.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes
@ 2018-10-16  2:37 frowand.list
  2018-10-16  2:37 ` [PATCH v4 01/18] of: overlay: add tests to validate kfrees from overlay removal frowand.list
                   ` (18 more replies)
  0 siblings, 19 replies; 33+ messages in thread
From: frowand.list @ 2018-10-16  2:37 UTC (permalink / raw)
  To: Rob Herring, Pantelis Antoniou, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Alan Tull,
	Moritz Fischer
  Cc: devicetree, linux-fpga, linuxppc-dev, linux-kernel

From: Frank Rowand <frank.rowand@sony.com>

Add checks to (1) overlay apply process and (2) memory freeing
triggered by overlay release.  The checks are intended to detect
possible memory leaks and invalid overlays.

The checks revealed bugs in existing code.  Fixed the bugs.

While fixing bugs, noted other issues, which are fixed in
separate patches.

*****  Powerpc folks: I was not able to test the patches that
*****  directly impact Powerpc systems that use dynamic
*****  devicetree.  Please review that code carefully and
*****  test.  The specific patches are: 03/16, 04/16, 07/16

FPGA folks:

  I made the validation checks that should result in an
  invalid live devicetree report "ERROR" and cause the overlay apply
  to fail.

  I made the memory leak validation tests report "WARNING" and allow
  the overlay apply to complete successfully.  Please let me know
  if you encounter the warnings.  There are at least two paths
  forward to deal with the cases that trigger the warning: (1) change
  the warning to an error and fail the overlay apply, or (2) find a
  way to detect the potential memory leaks and free the memory
  appropriately.

ALL people:

  The validations do _not_ address another major concern I have with
  releasing overlays, which is use after free errors.

Changes since v3:
  - 01/18: Add expected value of refcount for destroy cset entry error.  Also
    explain the cause of the error.

  - 09/18: for errors of an overlay changing the value of #size-cells or
    #address-cells, return -EINVAL so that overlay apply will fail
  - 09/18: for errors of an overlay changing the value of #size-cells or
    #address-cells, make the message more direct.
    Old message:
      OF: overlay: ERROR: overlay and/or live tree #size-cells invalid in node /soc/base_fpga_region
    New message:
      OF: overlay: ERROR: changing value of /soc/base_fpga_region/#size-cells not allowed

  - 13/18: Update patch comment header to state that this patch modifies the
    previous patch to not return immediately on fragment error and
    explain this is not a performance issue.
  - 13/18: remove redundant "overlay" from two error messages.  "OF: overlay:"
    is already present in pr_fmt()

Changes since v2:

  - 13/18: Use continue to reduce indentation in find_dup_cset_node_entry()
    and find_dup_cset_prop()

Changes since v1:

  - move patch 16/16 to 17/18
  - move patch 15/16 to 18/18
  - new patch 15/18
  - new patch 16/18

  - 05/18: add_changeset_node() header comment: incorrect comment for @target

  - 18/18: add same fix for of_parse_phandle_with_args()
  - 18/18: add same fix for of_parse_phandle_with_args_map()

Frank Rowand (18):
  of: overlay: add tests to validate kfrees from overlay removal
  of: overlay: add missing of_node_put() after add new node to changeset
  of: overlay: add missing of_node_get() in __of_attach_node_sysfs
  powerpc/pseries: add of_node_put() in dlpar_detach_node()
  of: overlay: use prop add changeset entry for property in new nodes
  of: overlay: do not duplicate properties from overlay for new nodes
  of: dynamic: change type of of_{at,de}tach_node() to void
  of: overlay: reorder fields in struct fragment
  of: overlay: validate overlay properties #address-cells and
    #size-cells
  of: overlay: make all pr_debug() and pr_err() messages unique
  of: overlay: test case of two fragments adding same node
  of: overlay: check prevents multiple fragments add or delete same node
  of: overlay: check prevents multiple fragments touching same property
  of: unittest: remove unused of_unittest_apply_overlay() argument
  of: overlay: set node fields from properties when add new overlay node
  of: unittest: allow base devicetree to have symbol metadata
  of: unittest: find overlays[] entry by name instead of index
  of: unittest: initialize args before calling of_*parse_*()

 arch/powerpc/platforms/pseries/dlpar.c             |  15 +-
 arch/powerpc/platforms/pseries/reconfig.c          |   6 +-
 drivers/of/dynamic.c                               |  68 +++--
 drivers/of/kobj.c                                  |   4 +-
 drivers/of/overlay.c                               | 302 ++++++++++++++++-----
 drivers/of/unittest-data/Makefile                  |   2 +
 .../of/unittest-data/overlay_bad_add_dup_node.dts  |  28 ++
 .../of/unittest-data/overlay_bad_add_dup_prop.dts  |  24 ++
 drivers/of/unittest-data/overlay_base.dts          |   1 +
 drivers/of/unittest.c                              |  96 +++++--
 include/linux/of.h                                 |  19 +-
 11 files changed, 443 insertions(+), 122 deletions(-)
 create mode 100644 drivers/of/unittest-data/overlay_bad_add_dup_node.dts
 create mode 100644 drivers/of/unittest-data/overlay_bad_add_dup_prop.dts

-- 
Frank Rowand <frank.rowand@sony.com>


^ permalink raw reply	[flat|nested] 33+ messages in thread

* [PATCH v4 01/18] of: overlay: add tests to validate kfrees from overlay removal
  2018-10-16  2:37 [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes frowand.list
@ 2018-10-16  2:37 ` frowand.list
  2018-10-17 21:30   ` Alan Tull
  2018-10-18 17:03   ` Rob Herring
  2018-10-16  2:37 ` [PATCH v4 02/18] of: overlay: add missing of_node_put() after add new node to changeset frowand.list
                   ` (17 subsequent siblings)
  18 siblings, 2 replies; 33+ messages in thread
From: frowand.list @ 2018-10-16  2:37 UTC (permalink / raw)
  To: Rob Herring, Pantelis Antoniou, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Alan Tull,
	Moritz Fischer
  Cc: devicetree, linux-fpga, linuxppc-dev, linux-kernel

From: Frank Rowand <frank.rowand@sony.com>

Add checks:
  - attempted kfree due to refcount reaching zero before overlay
    is removed
  - properties linked to an overlay node when the node is removed
  - node refcount > one during node removal in a changeset destroy,
    if the node was created by the changeset

After applying this patch, several validation warnings will be
reported from the devicetree unittest during boot due to
pre-existing devicetree bugs. The warnings will be similar to:

  OF: ERROR: of_node_release() overlay node /testcase-data/overlay-node/test-bus/test-unittest11/test-unittest111 contains unexpected properties
  OF: ERROR: memory leak - destroy cset entry: attach overlay node /testcase-data-2/substation@100/hvac-medium-2 expected refcount 1 instead of 2.  of_node_get() / of_node_put() are unbalanced for this node.

Signed-off-by: Frank Rowand <frank.rowand@sony.com>
---
Changes since v3:
  - Add expected value of refcount for destroy cset entry error.  Also
    explain the cause of the error.

 drivers/of/dynamic.c | 29 +++++++++++++++++++++++++++++
 drivers/of/overlay.c |  1 +
 include/linux/of.h   | 15 ++++++++++-----
 3 files changed, 40 insertions(+), 5 deletions(-)

diff --git a/drivers/of/dynamic.c b/drivers/of/dynamic.c
index f4f8ed9b5454..24c97b7a050f 100644
--- a/drivers/of/dynamic.c
+++ b/drivers/of/dynamic.c
@@ -330,6 +330,25 @@ void of_node_release(struct kobject *kobj)
 	if (!of_node_check_flag(node, OF_DYNAMIC))
 		return;
 
+	if (of_node_check_flag(node, OF_OVERLAY)) {
+
+		if (!of_node_check_flag(node, OF_OVERLAY_FREE_CSET)) {
+			/* premature refcount of zero, do not free memory */
+			pr_err("ERROR: memory leak %s() overlay node %pOF before free overlay changeset\n",
+			       __func__, node);
+			return;
+		}
+
+		/*
+		 * If node->properties non-empty then properties were added
+		 * to this node either by different overlay that has not
+		 * yet been removed, or by a non-overlay mechanism.
+		 */
+		if (node->properties)
+			pr_err("ERROR: %s() overlay node %pOF contains unexpected properties\n",
+			       __func__, node);
+	}
+
 	property_list_free(node->properties);
 	property_list_free(node->deadprops);
 
@@ -434,6 +453,16 @@ struct device_node *__of_node_dup(const struct device_node *np,
 
 static void __of_changeset_entry_destroy(struct of_changeset_entry *ce)
 {
+	if (ce->action == OF_RECONFIG_ATTACH_NODE &&
+	    of_node_check_flag(ce->np, OF_OVERLAY)) {
+		if (kref_read(&ce->np->kobj.kref) > 1) {
+			pr_err("ERROR: memory leak - destroy cset entry: attach overlay node %pOF expected refcount 1 instead of %d.  of_node_get() / of_node_put() are unbalanced for this node.\n",
+			       ce->np, kref_read(&ce->np->kobj.kref));
+		} else {
+			of_node_set_flag(ce->np, OF_OVERLAY_FREE_CSET);
+		}
+	}
+
 	of_node_put(ce->np);
 	list_del(&ce->node);
 	kfree(ce);
diff --git a/drivers/of/overlay.c b/drivers/of/overlay.c
index eda57ef12fd0..1176cb4b6e4e 100644
--- a/drivers/of/overlay.c
+++ b/drivers/of/overlay.c
@@ -373,6 +373,7 @@ static int add_changeset_node(struct overlay_changeset *ovcs,
 			return -ENOMEM;
 
 		tchild->parent = target_node;
+		of_node_set_flag(tchild, OF_OVERLAY);
 
 		ret = of_changeset_attach_node(&ovcs->cset, tchild);
 		if (ret)
diff --git a/include/linux/of.h b/include/linux/of.h
index 4d25e4f952d9..aa1dafaec6ae 100644
--- a/include/linux/of.h
+++ b/include/linux/of.h
@@ -138,11 +138,16 @@ static inline void of_node_put(struct device_node *node) { }
 extern struct device_node *of_stdout;
 extern raw_spinlock_t devtree_lock;
 
-/* flag descriptions (need to be visible even when !CONFIG_OF) */
-#define OF_DYNAMIC	1 /* node and properties were allocated via kmalloc */
-#define OF_DETACHED	2 /* node has been detached from the device tree */
-#define OF_POPULATED	3 /* device already created for the node */
-#define OF_POPULATED_BUS	4 /* of_platform_populate recursed to children of this node */
+/*
+ * struct device_node flag descriptions
+ * (need to be visible even when !CONFIG_OF)
+ */
+#define OF_DYNAMIC		1 /* (and properties) allocated via kmalloc */
+#define OF_DETACHED		2 /* detached from the device tree */
+#define OF_POPULATED		3 /* device already created */
+#define OF_POPULATED_BUS	4 /* platform bus created for children */
+#define OF_OVERLAY		5 /* allocated for an overlay */
+#define OF_OVERLAY_FREE_CSET	6 /* in overlay cset being freed */
 
 #define OF_BAD_ADDR	((u64)-1)
 
-- 
Frank Rowand <frank.rowand@sony.com>


^ permalink raw reply related	[flat|nested] 33+ messages in thread

* [PATCH v4 02/18] of: overlay: add missing of_node_put() after add new node to changeset
  2018-10-16  2:37 [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes frowand.list
  2018-10-16  2:37 ` [PATCH v4 01/18] of: overlay: add tests to validate kfrees from overlay removal frowand.list
@ 2018-10-16  2:37 ` frowand.list
  2018-10-18 17:05   ` Rob Herring
  2018-10-16  2:37 ` [PATCH v4 03/18] of: overlay: add missing of_node_get() in __of_attach_node_sysfs frowand.list
                   ` (16 subsequent siblings)
  18 siblings, 1 reply; 33+ messages in thread
From: frowand.list @ 2018-10-16  2:37 UTC (permalink / raw)
  To: Rob Herring, Pantelis Antoniou, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Alan Tull,
	Moritz Fischer
  Cc: devicetree, linux-fpga, linuxppc-dev, linux-kernel

From: Frank Rowand <frank.rowand@sony.com>

The refcount of a newly added overlay node decrements to one
(instead of zero) when the overlay changeset is destroyed.  This
change will cause the final decrement be to zero.

After applying this patch, new validation warnings will be
reported from the devicetree unittest during boot due to
a pre-existing devicetree bug.  The warnings will be similar to:

  OF: ERROR: memory leak of_node_release() overlay node /testcase-data/overlay-node/test-bus/test-unittest4 before free overlay changeset

This pre-existing devicetree bug will also trigger a WARN_ONCE() from
refcount_sub_and_test_checked() when an overlay changeset is
destroyed without having first been applied.  This scenario occurs
when an error in the overlay is detected during the overlay changeset
creation:

  WARNING: CPU: 0 PID: 1 at lib/refcount.c:187 refcount_sub_and_test_checked+0xa8/0xbc
  refcount_t: underflow; use-after-free.

  (unwind_backtrace) from (show_stack+0x10/0x14)
  (show_stack) from (dump_stack+0x6c/0x8c)
  (dump_stack) from (__warn+0xdc/0x104)
  (__warn) from (warn_slowpath_fmt+0x44/0x6c)
  (warn_slowpath_fmt) from (refcount_sub_and_test_checked+0xa8/0xbc)
  (refcount_sub_and_test_checked) from (kobject_put+0x24/0x208)
  (kobject_put) from (of_changeset_destroy+0x2c/0xb4)
  (of_changeset_destroy) from (free_overlay_changeset+0x1c/0x9c)
  (free_overlay_changeset) from (of_overlay_remove+0x284/0x2cc)
  (of_overlay_remove) from (of_unittest_apply_revert_overlay_check.constprop.4+0xf8/0x1e8)
  (of_unittest_apply_revert_overlay_check.constprop.4) from (of_unittest_overlay+0x960/0xed8)
  (of_unittest_overlay) from (of_unittest+0x1cc4/0x2138)
  (of_unittest) from (do_one_initcall+0x4c/0x28c)
  (do_one_initcall) from (kernel_init_freeable+0x29c/0x378)
  (kernel_init_freeable) from (kernel_init+0x8/0x110)
  (kernel_init) from (ret_from_fork+0x14/0x2c)

Signed-off-by: Frank Rowand <frank.rowand@sony.com>
---
 drivers/of/overlay.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/of/overlay.c b/drivers/of/overlay.c
index 1176cb4b6e4e..32cfee68f2e3 100644
--- a/drivers/of/overlay.c
+++ b/drivers/of/overlay.c
@@ -379,7 +379,9 @@ static int add_changeset_node(struct overlay_changeset *ovcs,
 		if (ret)
 			return ret;
 
-		return build_changeset_next_level(ovcs, tchild, node);
+		ret = build_changeset_next_level(ovcs, tchild, node);
+		of_node_put(tchild);
+		return ret;
 	}
 
 	if (node->phandle && tchild->phandle)
-- 
Frank Rowand <frank.rowand@sony.com>


^ permalink raw reply related	[flat|nested] 33+ messages in thread

* [PATCH v4 03/18] of: overlay: add missing of_node_get() in __of_attach_node_sysfs
  2018-10-16  2:37 [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes frowand.list
  2018-10-16  2:37 ` [PATCH v4 01/18] of: overlay: add tests to validate kfrees from overlay removal frowand.list
  2018-10-16  2:37 ` [PATCH v4 02/18] of: overlay: add missing of_node_put() after add new node to changeset frowand.list
@ 2018-10-16  2:37 ` frowand.list
  2018-10-16  2:37 ` [PATCH v4 04/18] powerpc/pseries: add of_node_put() in dlpar_detach_node() frowand.list
                   ` (15 subsequent siblings)
  18 siblings, 0 replies; 33+ messages in thread
From: frowand.list @ 2018-10-16  2:37 UTC (permalink / raw)
  To: Rob Herring, Pantelis Antoniou, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Alan Tull,
	Moritz Fischer
  Cc: devicetree, linux-fpga, linuxppc-dev, linux-kernel

From: Frank Rowand <frank.rowand@sony.com>

There is a matching of_node_put() in __of_detach_node_sysfs()

Remove misleading comment from function header comment for
of_detach_node().

This patch may result in memory leaks from code that directly calls
the dynamic node add and delete functions directly instead of
using changesets.

Signed-off-by: Frank Rowand <frank.rowand@sony.com>
---

This patch should result in powerpc systems that dynamically
allocate a node, then later deallocate the node to have a
memory leak when the node is deallocated.

The next patch in the series will fix the leak.

 drivers/of/dynamic.c | 3 ---
 drivers/of/kobj.c    | 4 +++-
 2 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/drivers/of/dynamic.c b/drivers/of/dynamic.c
index 24c97b7a050f..fe8816cca99b 100644
--- a/drivers/of/dynamic.c
+++ b/drivers/of/dynamic.c
@@ -272,9 +272,6 @@ void __of_detach_node(struct device_node *np)
 
 /**
  * of_detach_node() - "Unplug" a node from the device tree.
- *
- * The caller must hold a reference to the node.  The memory associated with
- * the node is not freed until its refcount goes to zero.
  */
 int of_detach_node(struct device_node *np)
 {
diff --git a/drivers/of/kobj.c b/drivers/of/kobj.c
index 7a0a18980b98..c72eef988041 100644
--- a/drivers/of/kobj.c
+++ b/drivers/of/kobj.c
@@ -133,6 +133,9 @@ int __of_attach_node_sysfs(struct device_node *np)
 	}
 	if (!name)
 		return -ENOMEM;
+
+	of_node_get(np);
+
 	rc = kobject_add(&np->kobj, parent, "%s", name);
 	kfree(name);
 	if (rc)
@@ -159,6 +162,5 @@ void __of_detach_node_sysfs(struct device_node *np)
 		kobject_del(&np->kobj);
 	}
 
-	/* finally remove the kobj_init ref */
 	of_node_put(np);
 }
-- 
Frank Rowand <frank.rowand@sony.com>


^ permalink raw reply related	[flat|nested] 33+ messages in thread

* [PATCH v4 04/18] powerpc/pseries: add of_node_put() in dlpar_detach_node()
  2018-10-16  2:37 [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes frowand.list
                   ` (2 preceding siblings ...)
  2018-10-16  2:37 ` [PATCH v4 03/18] of: overlay: add missing of_node_get() in __of_attach_node_sysfs frowand.list
@ 2018-10-16  2:37 ` frowand.list
  2018-10-18 17:09   ` Rob Herring
  2018-10-16  2:37 ` [PATCH v4 05/18] of: overlay: use prop add changeset entry for property in new nodes frowand.list
                   ` (14 subsequent siblings)
  18 siblings, 1 reply; 33+ messages in thread
From: frowand.list @ 2018-10-16  2:37 UTC (permalink / raw)
  To: Rob Herring, Pantelis Antoniou, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Alan Tull,
	Moritz Fischer
  Cc: devicetree, linux-fpga, linuxppc-dev, linux-kernel

From: Frank Rowand <frank.rowand@sony.com>

"of: overlay: add missing of_node_get() in __of_attach_node_sysfs"
added a missing of_node_get() to __of_attach_node_sysfs().  This
results in a refcount imbalance for nodes attached with
dlpar_attach_node().  The calling sequence from dlpar_attach_node()
to __of_attach_node_sysfs() is:

   dlpar_attach_node()
      of_attach_node()
         __of_attach_node_sysfs()

Signed-off-by: Frank Rowand <frank.rowand@sony.com>
---

***** UNTESTED.  I need people with the affected PowerPC systems
*****            (systems that dynamically allocate and deallocate
*****            devicetree nodes) to test this patch.

 arch/powerpc/platforms/pseries/dlpar.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/powerpc/platforms/pseries/dlpar.c b/arch/powerpc/platforms/pseries/dlpar.c
index a0b20c03f078..e3010b14aea5 100644
--- a/arch/powerpc/platforms/pseries/dlpar.c
+++ b/arch/powerpc/platforms/pseries/dlpar.c
@@ -272,6 +272,8 @@ int dlpar_detach_node(struct device_node *dn)
 	if (rc)
 		return rc;
 
+	of_node_put(dn);
+
 	return 0;
 }
 
-- 
Frank Rowand <frank.rowand@sony.com>


^ permalink raw reply related	[flat|nested] 33+ messages in thread

* [PATCH v4 05/18] of: overlay: use prop add changeset entry for property in new nodes
  2018-10-16  2:37 [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes frowand.list
                   ` (3 preceding siblings ...)
  2018-10-16  2:37 ` [PATCH v4 04/18] powerpc/pseries: add of_node_put() in dlpar_detach_node() frowand.list
@ 2018-10-16  2:37 ` frowand.list
  2018-10-16  2:37 ` [PATCH v4 06/18] of: overlay: do not duplicate properties from overlay for " frowand.list
                   ` (13 subsequent siblings)
  18 siblings, 0 replies; 33+ messages in thread
From: frowand.list @ 2018-10-16  2:37 UTC (permalink / raw)
  To: Rob Herring, Pantelis Antoniou, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Alan Tull,
	Moritz Fischer
  Cc: devicetree, linux-fpga, linuxppc-dev, linux-kernel

From: Frank Rowand <frank.rowand@sony.com>

The changeset entry 'update property' was used for new properties in
an overlay instead of 'add property'.

The decision of whether to use 'update property' was based on whether
the property already exists in the subtree where the node is being
spliced into.  At the top level of creating a changeset describing the
overlay, the target node is in the live devicetree, so checking whether
the property exists in the target node returns the correct result.
As soon as the changeset creation algorithm recurses into a new node,
the target is no longer in the live devicetree, but is instead in the
detached overlay tree, thus all properties are incorrectly found to
already exist in the target.

This fix will expose another devicetree bug that will be fixed
in the following patch in the series.

When this patch is applied the errors reported by the devictree
unittest will change, and the unittest results will change from:

   ### dt-test ### end of unittest - 210 passed, 0 failed

to

   ### dt-test ### end of unittest - 203 passed, 7 failed

Signed-off-by: Frank Rowand <frank.rowand@sony.com>
---
 drivers/of/overlay.c | 112 ++++++++++++++++++++++++++++++++++-----------------
 1 file changed, 74 insertions(+), 38 deletions(-)

diff --git a/drivers/of/overlay.c b/drivers/of/overlay.c
index 32cfee68f2e3..94740f4ee34c 100644
--- a/drivers/of/overlay.c
+++ b/drivers/of/overlay.c
@@ -24,6 +24,26 @@
 #include "of_private.h"
 
 /**
+ * struct target - info about current target node as recursing through overlay
+ * @np:			node where current level of overlay will be applied
+ * @in_livetree:	@np is a node in the live devicetree
+ *
+ * Used in the algorithm to create the portion of a changeset that describes
+ * an overlay fragment, which is a devicetree subtree.  Initially @np is a node
+ * in the live devicetree where the overlay subtree is targeted to be grafted
+ * into.  When recursing to the next level of the overlay subtree, the target
+ * also recurses to the next level of the live devicetree, as long as overlay
+ * subtree node also exists in the live devicetree.  When a node in the overlay
+ * subtree does not exist at the same level in the live devicetree, target->np
+ * points to a newly allocated node, and all subsequent targets in the subtree
+ * will be newly allocated nodes.
+ */
+struct target {
+	struct device_node *np;
+	bool in_livetree;
+};
+
+/**
  * struct fragment - info about fragment nodes in overlay expanded device tree
  * @target:	target of the overlay operation
  * @overlay:	pointer to the __overlay__ node
@@ -72,8 +92,7 @@ static int devicetree_corrupt(void)
 }
 
 static int build_changeset_next_level(struct overlay_changeset *ovcs,
-		struct device_node *target_node,
-		const struct device_node *overlay_node);
+		struct target *target, const struct device_node *overlay_node);
 
 /*
  * of_resolve_phandles() finds the largest phandle in the live tree.
@@ -257,14 +276,17 @@ static struct property *dup_and_fixup_symbol_prop(
 /**
  * add_changeset_property() - add @overlay_prop to overlay changeset
  * @ovcs:		overlay changeset
- * @target_node:	where to place @overlay_prop in live tree
+ * @target:		where @overlay_prop will be placed
  * @overlay_prop:	property to add or update, from overlay tree
  * @is_symbols_prop:	1 if @overlay_prop is from node "/__symbols__"
  *
- * If @overlay_prop does not already exist in @target_node, add changeset entry
- * to add @overlay_prop in @target_node, else add changeset entry to update
+ * If @overlay_prop does not already exist in live devicetree, add changeset
+ * entry to add @overlay_prop in @target, else add changeset entry to update
  * value of @overlay_prop.
  *
+ * @target may be either in the live devicetree or in a new subtree that
+ * is contained in the changeset.
+ *
  * Some special properties are not updated (no error returned).
  *
  * Update of property in symbols node is not allowed.
@@ -273,20 +295,22 @@ static struct property *dup_and_fixup_symbol_prop(
  * invalid @overlay.
  */
 static int add_changeset_property(struct overlay_changeset *ovcs,
-		struct device_node *target_node,
-		struct property *overlay_prop,
+		struct target *target, struct property *overlay_prop,
 		bool is_symbols_prop)
 {
 	struct property *new_prop = NULL, *prop;
 	int ret = 0;
 
-	prop = of_find_property(target_node, overlay_prop->name, NULL);
-
 	if (!of_prop_cmp(overlay_prop->name, "name") ||
 	    !of_prop_cmp(overlay_prop->name, "phandle") ||
 	    !of_prop_cmp(overlay_prop->name, "linux,phandle"))
 		return 0;
 
+	if (target->in_livetree)
+		prop = of_find_property(target->np, overlay_prop->name, NULL);
+	else
+		prop = NULL;
+
 	if (is_symbols_prop) {
 		if (prop)
 			return -EINVAL;
@@ -299,10 +323,10 @@ static int add_changeset_property(struct overlay_changeset *ovcs,
 		return -ENOMEM;
 
 	if (!prop)
-		ret = of_changeset_add_property(&ovcs->cset, target_node,
+		ret = of_changeset_add_property(&ovcs->cset, target->np,
 						new_prop);
 	else
-		ret = of_changeset_update_property(&ovcs->cset, target_node,
+		ret = of_changeset_update_property(&ovcs->cset, target->np,
 						   new_prop);
 
 	if (ret) {
@@ -315,14 +339,14 @@ static int add_changeset_property(struct overlay_changeset *ovcs,
 
 /**
  * add_changeset_node() - add @node (and children) to overlay changeset
- * @ovcs:		overlay changeset
- * @target_node:	where to place @node in live tree
- * @node:		node from within overlay device tree fragment
+ * @ovcs:	overlay changeset
+ * @target:	where @node will be placed in live tree or changeset
+ * @node:	node from within overlay device tree fragment
  *
- * If @node does not already exist in @target_node, add changeset entry
- * to add @node in @target_node.
+ * If @node does not already exist in @target, add changeset entry
+ * to add @node in @target.
  *
- * If @node already exists in @target_node, and the existing node has
+ * If @node already exists in @target, and the existing node has
  * a phandle, the overlay node is not allowed to have a phandle.
  *
  * If @node has child nodes, add the children recursively via
@@ -355,15 +379,16 @@ static int add_changeset_property(struct overlay_changeset *ovcs,
  * invalid @overlay.
  */
 static int add_changeset_node(struct overlay_changeset *ovcs,
-		struct device_node *target_node, struct device_node *node)
+		struct target *target, struct device_node *node)
 {
 	const char *node_kbasename;
 	struct device_node *tchild;
+	struct target target_child;
 	int ret = 0;
 
 	node_kbasename = kbasename(node->full_name);
 
-	for_each_child_of_node(target_node, tchild)
+	for_each_child_of_node(target->np, tchild)
 		if (!of_node_cmp(node_kbasename, kbasename(tchild->full_name)))
 			break;
 
@@ -372,22 +397,28 @@ static int add_changeset_node(struct overlay_changeset *ovcs,
 		if (!tchild)
 			return -ENOMEM;
 
-		tchild->parent = target_node;
+		tchild->parent = target->np;
 		of_node_set_flag(tchild, OF_OVERLAY);
 
 		ret = of_changeset_attach_node(&ovcs->cset, tchild);
 		if (ret)
 			return ret;
 
-		ret = build_changeset_next_level(ovcs, tchild, node);
+		target_child.np = tchild;
+		target_child.in_livetree = false;
+
+		ret = build_changeset_next_level(ovcs, &target_child, node);
 		of_node_put(tchild);
 		return ret;
 	}
 
-	if (node->phandle && tchild->phandle)
+	if (node->phandle && tchild->phandle) {
 		ret = -EINVAL;
-	else
-		ret = build_changeset_next_level(ovcs, tchild, node);
+	} else {
+		target_child.np = tchild;
+		target_child.in_livetree = target->in_livetree;
+		ret = build_changeset_next_level(ovcs, &target_child, node);
+	}
 	of_node_put(tchild);
 
 	return ret;
@@ -396,7 +427,7 @@ static int add_changeset_node(struct overlay_changeset *ovcs,
 /**
  * build_changeset_next_level() - add level of overlay changeset
  * @ovcs:		overlay changeset
- * @target_node:	where to place @overlay_node in live tree
+ * @target:		where to place @overlay_node in live tree
  * @overlay_node:	node from within an overlay device tree fragment
  *
  * Add the properties (if any) and nodes (if any) from @overlay_node to the
@@ -409,27 +440,26 @@ static int add_changeset_node(struct overlay_changeset *ovcs,
  * invalid @overlay_node.
  */
 static int build_changeset_next_level(struct overlay_changeset *ovcs,
-		struct device_node *target_node,
-		const struct device_node *overlay_node)
+		struct target *target, const struct device_node *overlay_node)
 {
 	struct device_node *child;
 	struct property *prop;
 	int ret;
 
 	for_each_property_of_node(overlay_node, prop) {
-		ret = add_changeset_property(ovcs, target_node, prop, 0);
+		ret = add_changeset_property(ovcs, target, prop, 0);
 		if (ret) {
 			pr_debug("Failed to apply prop @%pOF/%s, err=%d\n",
-				 target_node, prop->name, ret);
+				 target->np, prop->name, ret);
 			return ret;
 		}
 	}
 
 	for_each_child_of_node(overlay_node, child) {
-		ret = add_changeset_node(ovcs, target_node, child);
+		ret = add_changeset_node(ovcs, target, child);
 		if (ret) {
 			pr_debug("Failed to apply node @%pOF/%s, err=%d\n",
-				 target_node, child->name, ret);
+				 target->np, child->name, ret);
 			of_node_put(child);
 			return ret;
 		}
@@ -442,17 +472,17 @@ static int build_changeset_next_level(struct overlay_changeset *ovcs,
  * Add the properties from __overlay__ node to the @ovcs->cset changeset.
  */
 static int build_changeset_symbols_node(struct overlay_changeset *ovcs,
-		struct device_node *target_node,
+		struct target *target,
 		const struct device_node *overlay_symbols_node)
 {
 	struct property *prop;
 	int ret;
 
 	for_each_property_of_node(overlay_symbols_node, prop) {
-		ret = add_changeset_property(ovcs, target_node, prop, 1);
+		ret = add_changeset_property(ovcs, target, prop, 1);
 		if (ret) {
 			pr_debug("Failed to apply prop @%pOF/%s, err=%d\n",
-				 target_node, prop->name, ret);
+				 target->np, prop->name, ret);
 			return ret;
 		}
 	}
@@ -475,6 +505,7 @@ static int build_changeset_symbols_node(struct overlay_changeset *ovcs,
 static int build_changeset(struct overlay_changeset *ovcs)
 {
 	struct fragment *fragment;
+	struct target target;
 	int fragments_count, i, ret;
 
 	/*
@@ -489,7 +520,9 @@ static int build_changeset(struct overlay_changeset *ovcs)
 	for (i = 0; i < fragments_count; i++) {
 		fragment = &ovcs->fragments[i];
 
-		ret = build_changeset_next_level(ovcs, fragment->target,
+		target.np = fragment->target;
+		target.in_livetree = true;
+		ret = build_changeset_next_level(ovcs, &target,
 						 fragment->overlay);
 		if (ret) {
 			pr_debug("apply failed '%pOF'\n", fragment->target);
@@ -499,7 +532,10 @@ static int build_changeset(struct overlay_changeset *ovcs)
 
 	if (ovcs->symbols_fragment) {
 		fragment = &ovcs->fragments[ovcs->count - 1];
-		ret = build_changeset_symbols_node(ovcs, fragment->target,
+
+		target.np = fragment->target;
+		target.in_livetree = true;
+		ret = build_changeset_symbols_node(ovcs, &target,
 						   fragment->overlay);
 		if (ret) {
 			pr_debug("apply failed '%pOF'\n", fragment->target);
@@ -517,7 +553,7 @@ static int build_changeset(struct overlay_changeset *ovcs)
  * 1) "target" property containing the phandle of the target
  * 2) "target-path" property containing the path of the target
  */
-static struct device_node *find_target_node(struct device_node *info_node)
+static struct device_node *find_target(struct device_node *info_node)
 {
 	struct device_node *node;
 	const char *path;
@@ -623,7 +659,7 @@ static int init_overlay_changeset(struct overlay_changeset *ovcs,
 
 		fragment = &fragments[cnt];
 		fragment->overlay = overlay_node;
-		fragment->target = find_target_node(node);
+		fragment->target = find_target(node);
 		if (!fragment->target) {
 			of_node_put(fragment->overlay);
 			ret = -EINVAL;
-- 
Frank Rowand <frank.rowand@sony.com>


^ permalink raw reply related	[flat|nested] 33+ messages in thread

* [PATCH v4 06/18] of: overlay: do not duplicate properties from overlay for new nodes
  2018-10-16  2:37 [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes frowand.list
                   ` (4 preceding siblings ...)
  2018-10-16  2:37 ` [PATCH v4 05/18] of: overlay: use prop add changeset entry for property in new nodes frowand.list
@ 2018-10-16  2:37 ` frowand.list
  2018-10-16  2:37 ` [PATCH v4 07/18] of: dynamic: change type of of_{at, de}tach_node() to void frowand.list
                   ` (12 subsequent siblings)
  18 siblings, 0 replies; 33+ messages in thread
From: frowand.list @ 2018-10-16  2:37 UTC (permalink / raw)
  To: Rob Herring, Pantelis Antoniou, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Alan Tull,
	Moritz Fischer
  Cc: devicetree, linux-fpga, linuxppc-dev, linux-kernel

From: Frank Rowand <frank.rowand@sony.com>

When allocating a new node, add_changeset_node() was duplicating the
properties from the respective node in the overlay instead of
allocating a node with no properties.

When this patch is applied the errors reported by the devictree
unittest from patch "of: overlay: add tests to validate kfrees from
overlay removal" will no longer occur.  These error messages are of
the form:

   "OF: ERROR: ..."

and the unittest results will change from:

   ### dt-test ### end of unittest - 203 passed, 7 failed

to

   ### dt-test ### end of unittest - 210 passed, 0 failed

Signed-off-by: Frank Rowand <frank.rowand@sony.com>
---
 drivers/of/overlay.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/of/overlay.c b/drivers/of/overlay.c
index 94740f4ee34c..7fcf4a812d06 100644
--- a/drivers/of/overlay.c
+++ b/drivers/of/overlay.c
@@ -393,7 +393,7 @@ static int add_changeset_node(struct overlay_changeset *ovcs,
 			break;
 
 	if (!tchild) {
-		tchild = __of_node_dup(node, node_kbasename);
+		tchild = __of_node_dup(NULL, node_kbasename);
 		if (!tchild)
 			return -ENOMEM;
 
-- 
Frank Rowand <frank.rowand@sony.com>


^ permalink raw reply related	[flat|nested] 33+ messages in thread

* [PATCH v4 07/18] of: dynamic: change type of of_{at, de}tach_node() to void
  2018-10-16  2:37 [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes frowand.list
                   ` (5 preceding siblings ...)
  2018-10-16  2:37 ` [PATCH v4 06/18] of: overlay: do not duplicate properties from overlay for " frowand.list
@ 2018-10-16  2:37 ` frowand.list
  2018-10-16  2:37 ` [PATCH v4 08/18] of: overlay: reorder fields in struct fragment frowand.list
                   ` (11 subsequent siblings)
  18 siblings, 0 replies; 33+ messages in thread
From: frowand.list @ 2018-10-16  2:37 UTC (permalink / raw)
  To: Rob Herring, Pantelis Antoniou, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Alan Tull,
	Moritz Fischer
  Cc: devicetree, linux-fpga, linuxppc-dev, linux-kernel

From: Frank Rowand <frank.rowand@sony.com>

of_attach_node() and of_detach_node() always return zero, so
their return value is meaningless.  Change their type to void
and fix all callers to ignore return value.

Signed-off-by: Frank Rowand <frank.rowand@sony.com>
---
 arch/powerpc/platforms/pseries/dlpar.c    | 13 ++-----------
 arch/powerpc/platforms/pseries/reconfig.c |  6 +-----
 drivers/of/dynamic.c                      |  9 ++-------
 include/linux/of.h                        |  4 ++--
 4 files changed, 7 insertions(+), 25 deletions(-)

diff --git a/arch/powerpc/platforms/pseries/dlpar.c b/arch/powerpc/platforms/pseries/dlpar.c
index e3010b14aea5..0027eea94a8b 100644
--- a/arch/powerpc/platforms/pseries/dlpar.c
+++ b/arch/powerpc/platforms/pseries/dlpar.c
@@ -244,15 +244,9 @@ struct device_node *dlpar_configure_connector(__be32 drc_index,
 
 int dlpar_attach_node(struct device_node *dn, struct device_node *parent)
 {
-	int rc;
-
 	dn->parent = parent;
 
-	rc = of_attach_node(dn);
-	if (rc) {
-		printk(KERN_ERR "Failed to add device node %pOF\n", dn);
-		return rc;
-	}
+	of_attach_node(dn);
 
 	return 0;
 }
@@ -260,7 +254,6 @@ int dlpar_attach_node(struct device_node *dn, struct device_node *parent)
 int dlpar_detach_node(struct device_node *dn)
 {
 	struct device_node *child;
-	int rc;
 
 	child = of_get_next_child(dn, NULL);
 	while (child) {
@@ -268,9 +261,7 @@ int dlpar_detach_node(struct device_node *dn)
 		child = of_get_next_child(dn, child);
 	}
 
-	rc = of_detach_node(dn);
-	if (rc)
-		return rc;
+	of_detach_node(dn);
 
 	of_node_put(dn);
 
diff --git a/arch/powerpc/platforms/pseries/reconfig.c b/arch/powerpc/platforms/pseries/reconfig.c
index 0e0208117e77..0b72098da454 100644
--- a/arch/powerpc/platforms/pseries/reconfig.c
+++ b/arch/powerpc/platforms/pseries/reconfig.c
@@ -47,11 +47,7 @@ static int pSeries_reconfig_add_node(const char *path, struct property *proplist
 		goto out_err;
 	}
 
-	err = of_attach_node(np);
-	if (err) {
-		printk(KERN_ERR "Failed to add device node %s\n", path);
-		goto out_err;
-	}
+	of_attach_node(np);
 
 	of_node_put(np->parent);
 
diff --git a/drivers/of/dynamic.c b/drivers/of/dynamic.c
index fe8816cca99b..a94f727ec3da 100644
--- a/drivers/of/dynamic.c
+++ b/drivers/of/dynamic.c
@@ -224,7 +224,7 @@ static void __of_attach_node(struct device_node *np)
 /**
  * of_attach_node() - Plug a device node into the tree and global list.
  */
-int of_attach_node(struct device_node *np)
+void of_attach_node(struct device_node *np)
 {
 	struct of_reconfig_data rd;
 	unsigned long flags;
@@ -241,8 +241,6 @@ int of_attach_node(struct device_node *np)
 	mutex_unlock(&of_mutex);
 
 	of_reconfig_notify(OF_RECONFIG_ATTACH_NODE, &rd);
-
-	return 0;
 }
 
 void __of_detach_node(struct device_node *np)
@@ -273,11 +271,10 @@ void __of_detach_node(struct device_node *np)
 /**
  * of_detach_node() - "Unplug" a node from the device tree.
  */
-int of_detach_node(struct device_node *np)
+void of_detach_node(struct device_node *np)
 {
 	struct of_reconfig_data rd;
 	unsigned long flags;
-	int rc = 0;
 
 	memset(&rd, 0, sizeof(rd));
 	rd.dn = np;
@@ -291,8 +288,6 @@ int of_detach_node(struct device_node *np)
 	mutex_unlock(&of_mutex);
 
 	of_reconfig_notify(OF_RECONFIG_DETACH_NODE, &rd);
-
-	return rc;
 }
 EXPORT_SYMBOL_GPL(of_detach_node);
 
diff --git a/include/linux/of.h b/include/linux/of.h
index aa1dafaec6ae..72c593455019 100644
--- a/include/linux/of.h
+++ b/include/linux/of.h
@@ -406,8 +406,8 @@ extern int of_phandle_iterator_args(struct of_phandle_iterator *it,
 #define OF_RECONFIG_REMOVE_PROPERTY	0x0004
 #define OF_RECONFIG_UPDATE_PROPERTY	0x0005
 
-extern int of_attach_node(struct device_node *);
-extern int of_detach_node(struct device_node *);
+extern void of_attach_node(struct device_node *np);
+extern void of_detach_node(struct device_node *np);
 
 #define of_match_ptr(_ptr)	(_ptr)
 
-- 
Frank Rowand <frank.rowand@sony.com>


^ permalink raw reply related	[flat|nested] 33+ messages in thread

* [PATCH v4 08/18] of: overlay: reorder fields in struct fragment
  2018-10-16  2:37 [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes frowand.list
                   ` (6 preceding siblings ...)
  2018-10-16  2:37 ` [PATCH v4 07/18] of: dynamic: change type of of_{at, de}tach_node() to void frowand.list
@ 2018-10-16  2:37 ` frowand.list
  2018-10-16  2:37 ` [PATCH v4 09/18] of: overlay: validate overlay properties #address-cells and #size-cells frowand.list
                   ` (10 subsequent siblings)
  18 siblings, 0 replies; 33+ messages in thread
From: frowand.list @ 2018-10-16  2:37 UTC (permalink / raw)
  To: Rob Herring, Pantelis Antoniou, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Alan Tull,
	Moritz Fischer
  Cc: devicetree, linux-fpga, linuxppc-dev, linux-kernel

From: Frank Rowand <frank.rowand@sony.com>

Order the fields of struct fragment in the same order as
struct of_overlay_notify_data.  The order in struct fragment is
not significant.  If both structs are ordered the same then when
examining the data in a debugger or dump the human involved does
not have to remember which context they are examining.

Signed-off-by: Frank Rowand <frank.rowand@sony.com>
---
 drivers/of/overlay.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/of/overlay.c b/drivers/of/overlay.c
index 7fcf4a812d06..272a0d1a5e18 100644
--- a/drivers/of/overlay.c
+++ b/drivers/of/overlay.c
@@ -49,8 +49,8 @@ struct target {
  * @overlay:	pointer to the __overlay__ node
  */
 struct fragment {
-	struct device_node *target;
 	struct device_node *overlay;
+	struct device_node *target;
 };
 
 /**
-- 
Frank Rowand <frank.rowand@sony.com>


^ permalink raw reply related	[flat|nested] 33+ messages in thread

* [PATCH v4 09/18] of: overlay: validate overlay properties #address-cells and #size-cells
  2018-10-16  2:37 [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes frowand.list
                   ` (7 preceding siblings ...)
  2018-10-16  2:37 ` [PATCH v4 08/18] of: overlay: reorder fields in struct fragment frowand.list
@ 2018-10-16  2:37 ` frowand.list
  2018-10-18 18:13   ` Rob Herring
  2018-10-16  2:37 ` [PATCH v4 10/18] of: overlay: make all pr_debug() and pr_err() messages unique frowand.list
                   ` (9 subsequent siblings)
  18 siblings, 1 reply; 33+ messages in thread
From: frowand.list @ 2018-10-16  2:37 UTC (permalink / raw)
  To: Rob Herring, Pantelis Antoniou, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Alan Tull,
	Moritz Fischer
  Cc: devicetree, linux-fpga, linuxppc-dev, linux-kernel

From: Frank Rowand <frank.rowand@sony.com>

If overlay properties #address-cells or #size-cells are already in
the live devicetree for any given node, then the values in the
overlay must match the values in the live tree.

If the properties are already in the live tree then there is no
need to create a changeset entry to add them since they must
have the same value.  This reduces the memory used by the
changeset and eliminates a possible memory leak.  This is
verified by 12 fewer warnings during the devicetree unittest,
as the possible memory leak warnings about #address-cells and

Signed-off-by: Frank Rowand <frank.rowand@sony.com>
---
Changes since v3:
  - for errors of an overlay changing the value of #size-cells or
    #address-cells, return -EINVAL so that overlay apply will fail
  - for errors of an overlay changing the value of #size-cells or
    #address-cells, make the message more direct.
    Old message:
      OF: overlay: ERROR: overlay and/or live tree #size-cells invalid in node /soc/base_fpga_region
    New message:
      OF: overlay: ERROR: changing value of /soc/base_fpga_region/#size-cells not allowed

 drivers/of/overlay.c | 42 +++++++++++++++++++++++++++++++++++++++---
 1 file changed, 39 insertions(+), 3 deletions(-)

diff --git a/drivers/of/overlay.c b/drivers/of/overlay.c
index 272a0d1a5e18..3e1e519c12f0 100644
--- a/drivers/of/overlay.c
+++ b/drivers/of/overlay.c
@@ -287,7 +287,12 @@ static struct property *dup_and_fixup_symbol_prop(
  * @target may be either in the live devicetree or in a new subtree that
  * is contained in the changeset.
  *
- * Some special properties are not updated (no error returned).
+ * Some special properties are not added or updated (no error returned):
+ * "name", "phandle", "linux,phandle".
+ *
+ * Properties "#address-cells" and "#size-cells" are not updated if they
+ * are already in the live tree, but if present in the live tree, the values
+ * in the overlay must match the values in the live tree.
  *
  * Update of property in symbols node is not allowed.
  *
@@ -300,6 +305,7 @@ static int add_changeset_property(struct overlay_changeset *ovcs,
 {
 	struct property *new_prop = NULL, *prop;
 	int ret = 0;
+	bool check_for_non_overlay_node = false;
 
 	if (!of_prop_cmp(overlay_prop->name, "name") ||
 	    !of_prop_cmp(overlay_prop->name, "phandle") ||
@@ -322,13 +328,43 @@ static int add_changeset_property(struct overlay_changeset *ovcs,
 	if (!new_prop)
 		return -ENOMEM;
 
-	if (!prop)
+	if (!prop) {
+
+		check_for_non_overlay_node = true;
 		ret = of_changeset_add_property(&ovcs->cset, target->np,
 						new_prop);
-	else
+
+	} else if (!of_prop_cmp(prop->name, "#address-cells")) {
+
+		if (prop->length != 4 || new_prop->length != 4 ||
+		    *(u32 *)prop->value != *(u32 *)new_prop->value) {
+			pr_err("ERROR: changing value of %pOF/#address-cells is not allowed\n",
+			       target->np);
+			ret = -EINVAL;
+		}
+
+	} else if (!of_prop_cmp(prop->name, "#size-cells")) {
+
+		if (prop->length != 4 || new_prop->length != 4 ||
+		    *(u32 *)prop->value != *(u32 *)new_prop->value) {
+			pr_err("ERROR: changing value of %pOF/#size-cells is not allowed\n",
+			       target->np);
+			ret = -EINVAL;
+		}
+
+	} else {
+
+		check_for_non_overlay_node = true;
 		ret = of_changeset_update_property(&ovcs->cset, target->np,
 						   new_prop);
 
+	}
+
+	if (check_for_non_overlay_node &&
+	    !of_node_check_flag(target->np, OF_OVERLAY))
+		pr_err("WARNING: %s(), memory leak will occur if overlay removed.  Property: %pOF/%s\n",
+		       __func__, target->np, new_prop->name);
+
 	if (ret) {
 		kfree(new_prop->name);
 		kfree(new_prop->value);
-- 
Frank Rowand <frank.rowand@sony.com>


^ permalink raw reply related	[flat|nested] 33+ messages in thread

* [PATCH v4 10/18] of: overlay: make all pr_debug() and pr_err() messages unique
  2018-10-16  2:37 [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes frowand.list
                   ` (8 preceding siblings ...)
  2018-10-16  2:37 ` [PATCH v4 09/18] of: overlay: validate overlay properties #address-cells and #size-cells frowand.list
@ 2018-10-16  2:37 ` frowand.list
  2018-10-16  2:37 ` [PATCH v4 11/18] of: overlay: test case of two fragments adding same node frowand.list
                   ` (8 subsequent siblings)
  18 siblings, 0 replies; 33+ messages in thread
From: frowand.list @ 2018-10-16  2:37 UTC (permalink / raw)
  To: Rob Herring, Pantelis Antoniou, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Alan Tull,
	Moritz Fischer
  Cc: devicetree, linux-fpga, linuxppc-dev, linux-kernel

From: Frank Rowand <frank.rowand@sony.com>

Make overlay.c debug and error messages unique so that they can be
unambiguously found by grep.

Signed-off-by: Frank Rowand <frank.rowand@sony.com>
---
 drivers/of/overlay.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/drivers/of/overlay.c b/drivers/of/overlay.c
index 3e1e519c12f0..34396d6db1b3 100644
--- a/drivers/of/overlay.c
+++ b/drivers/of/overlay.c
@@ -517,7 +517,7 @@ static int build_changeset_symbols_node(struct overlay_changeset *ovcs,
 	for_each_property_of_node(overlay_symbols_node, prop) {
 		ret = add_changeset_property(ovcs, target, prop, 1);
 		if (ret) {
-			pr_debug("Failed to apply prop @%pOF/%s, err=%d\n",
+			pr_debug("Failed to apply symbols prop @%pOF/%s, err=%d\n",
 				 target->np, prop->name, ret);
 			return ret;
 		}
@@ -561,7 +561,8 @@ static int build_changeset(struct overlay_changeset *ovcs)
 		ret = build_changeset_next_level(ovcs, &target,
 						 fragment->overlay);
 		if (ret) {
-			pr_debug("apply failed '%pOF'\n", fragment->target);
+			pr_debug("fragment apply failed '%pOF'\n",
+				 fragment->target);
 			return ret;
 		}
 	}
@@ -574,7 +575,8 @@ static int build_changeset(struct overlay_changeset *ovcs)
 		ret = build_changeset_symbols_node(ovcs, &target,
 						   fragment->overlay);
 		if (ret) {
-			pr_debug("apply failed '%pOF'\n", fragment->target);
+			pr_debug("symbols fragment apply failed '%pOF'\n",
+				 fragment->target);
 			return ret;
 		}
 	}
@@ -883,7 +885,7 @@ static int of_overlay_apply(const void *fdt, struct device_node *tree,
 
 	ret = __of_changeset_apply_notify(&ovcs->cset);
 	if (ret)
-		pr_err("overlay changeset entry notify error %d\n", ret);
+		pr_err("overlay apply changeset entry notify error %d\n", ret);
 	/* notify failure is not fatal, continue */
 
 	list_add_tail(&ovcs->ovcs_list, &ovcs_list);
@@ -1142,7 +1144,7 @@ int of_overlay_remove(int *ovcs_id)
 
 	ret = __of_changeset_revert_notify(&ovcs->cset);
 	if (ret)
-		pr_err("overlay changeset entry notify error %d\n", ret);
+		pr_err("overlay remove changeset entry notify error %d\n", ret);
 	/* notify failure is not fatal, continue */
 
 	*ovcs_id = 0;
-- 
Frank Rowand <frank.rowand@sony.com>


^ permalink raw reply related	[flat|nested] 33+ messages in thread

* [PATCH v4 11/18] of: overlay: test case of two fragments adding same node
  2018-10-16  2:37 [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes frowand.list
                   ` (9 preceding siblings ...)
  2018-10-16  2:37 ` [PATCH v4 10/18] of: overlay: make all pr_debug() and pr_err() messages unique frowand.list
@ 2018-10-16  2:37 ` frowand.list
  2018-10-16  2:37 ` [PATCH v4 12/18] of: overlay: check prevents multiple fragments add or delete " frowand.list
                   ` (7 subsequent siblings)
  18 siblings, 0 replies; 33+ messages in thread
From: frowand.list @ 2018-10-16  2:37 UTC (permalink / raw)
  To: Rob Herring, Pantelis Antoniou, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Alan Tull,
	Moritz Fischer
  Cc: devicetree, linux-fpga, linuxppc-dev, linux-kernel

From: Frank Rowand <frank.rowand@sony.com>

Multiple overlay fragments adding or deleting the same node is not
supported.  An attempt to do so results in an incorrect devicetree.
The node name will be munged for the second add.

After adding this patch, the unittest messages will show:

   Duplicate name in motor-1, renamed to "controller#1"
   OF: overlay: of_overlay_apply() err=0
   ### dt-test ### of_overlay_fdt_apply() expected -22, ret=0, overlay_bad_add_dup_node
   ### dt-test ### FAIL of_unittest_overlay_high_level():2419 Adding overlay 'overlay_bad_add_dup_node' failed

   ...

   ### dt-test ### end of unittest - 210 passed, 1 failed

The incorrect (munged) node name "controller#1" can be seen in the
/proc filesystem:

   $ pwd
   /proc/device-tree/testcase-data-2/substation@100/motor-1
   $ ls
   compatible    controller    controller#1  name          phandle       spin
   $ ls controller
   power_bus
   $ ls controller#1
   power_bus_emergency

Signed-off-by: Frank Rowand <frank.rowand@sony.com>
---
 drivers/of/unittest-data/Makefile                  |  1 +
 .../of/unittest-data/overlay_bad_add_dup_node.dts  | 28 ++++++++++++++++++++++
 drivers/of/unittest.c                              |  5 ++++
 3 files changed, 34 insertions(+)
 create mode 100644 drivers/of/unittest-data/overlay_bad_add_dup_node.dts

diff --git a/drivers/of/unittest-data/Makefile b/drivers/of/unittest-data/Makefile
index 013d85e694c6..166dbdbfd1c5 100644
--- a/drivers/of/unittest-data/Makefile
+++ b/drivers/of/unittest-data/Makefile
@@ -17,6 +17,7 @@ obj-$(CONFIG_OF_OVERLAY) += overlay.dtb.o \
 			    overlay_12.dtb.o \
 			    overlay_13.dtb.o \
 			    overlay_15.dtb.o \
+			    overlay_bad_add_dup_node.dtb.o \
 			    overlay_bad_phandle.dtb.o \
 			    overlay_bad_symbol.dtb.o \
 			    overlay_base.dtb.o
diff --git a/drivers/of/unittest-data/overlay_bad_add_dup_node.dts b/drivers/of/unittest-data/overlay_bad_add_dup_node.dts
new file mode 100644
index 000000000000..145dfc3b1024
--- /dev/null
+++ b/drivers/of/unittest-data/overlay_bad_add_dup_node.dts
@@ -0,0 +1,28 @@
+// SPDX-License-Identifier: GPL-2.0
+/dts-v1/;
+/plugin/;
+
+/*
+ * &electric_1/motor-1 and &spin_ctrl_1 are the same node:
+ *   /testcase-data-2/substation@100/motor-1
+ *
+ * Thus the new node "controller" in each fragment will
+ * result in an attempt to add the same node twice.
+ * This will result in an error and the overlay apply
+ * will fail.
+ */
+
+&electric_1 {
+
+	motor-1 {
+		controller {
+			power_bus = < 0x1 0x2 >;
+		};
+	};
+};
+
+&spin_ctrl_1 {
+		controller {
+			power_bus_emergency = < 0x101 0x102 >;
+		};
+};
diff --git a/drivers/of/unittest.c b/drivers/of/unittest.c
index 722537e14848..471b8eb6e842 100644
--- a/drivers/of/unittest.c
+++ b/drivers/of/unittest.c
@@ -2147,6 +2147,7 @@ struct overlay_info {
 OVERLAY_INFO_EXTERN(overlay_12);
 OVERLAY_INFO_EXTERN(overlay_13);
 OVERLAY_INFO_EXTERN(overlay_15);
+OVERLAY_INFO_EXTERN(overlay_bad_add_dup_node);
 OVERLAY_INFO_EXTERN(overlay_bad_phandle);
 OVERLAY_INFO_EXTERN(overlay_bad_symbol);
 
@@ -2169,6 +2170,7 @@ struct overlay_info {
 	OVERLAY_INFO(overlay_12, 0),
 	OVERLAY_INFO(overlay_13, 0),
 	OVERLAY_INFO(overlay_15, 0),
+	OVERLAY_INFO(overlay_bad_add_dup_node, -EINVAL),
 	OVERLAY_INFO(overlay_bad_phandle, -EINVAL),
 	OVERLAY_INFO(overlay_bad_symbol, -EINVAL),
 	{}
@@ -2413,6 +2415,9 @@ static __init void of_unittest_overlay_high_level(void)
 	unittest(overlay_data_apply("overlay", NULL),
 		 "Adding overlay 'overlay' failed\n");
 
+	unittest(overlay_data_apply("overlay_bad_add_dup_node", NULL),
+		 "Adding overlay 'overlay_bad_add_dup_node' failed\n");
+
 	unittest(overlay_data_apply("overlay_bad_phandle", NULL),
 		 "Adding overlay 'overlay_bad_phandle' failed\n");
 
-- 
Frank Rowand <frank.rowand@sony.com>


^ permalink raw reply related	[flat|nested] 33+ messages in thread

* [PATCH v4 12/18] of: overlay: check prevents multiple fragments add or delete same node
  2018-10-16  2:37 [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes frowand.list
                   ` (10 preceding siblings ...)
  2018-10-16  2:37 ` [PATCH v4 11/18] of: overlay: test case of two fragments adding same node frowand.list
@ 2018-10-16  2:37 ` frowand.list
  2018-10-16  2:37 ` [PATCH v4 13/18] of: overlay: check prevents multiple fragments touching same property frowand.list
                   ` (6 subsequent siblings)
  18 siblings, 0 replies; 33+ messages in thread
From: frowand.list @ 2018-10-16  2:37 UTC (permalink / raw)
  To: Rob Herring, Pantelis Antoniou, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Alan Tull,
	Moritz Fischer
  Cc: devicetree, linux-fpga, linuxppc-dev, linux-kernel

From: Frank Rowand <frank.rowand@sony.com>

Multiple overlay fragments adding or deleting the same node is not
supported.  Replace code comment of such, with check to detect the
attempt and fail the overlay apply.

Devicetree unittest where multiple fragments added the same node was
added in the previous patch in the series.  After applying this patch
the unittest messages will no longer include:

   Duplicate name in motor-1, renamed to "controller#1"
   OF: overlay: of_overlay_apply() err=0
   ### dt-test ### of_overlay_fdt_apply() expected -22, ret=0, overlay_bad_add_dup_node
   ### dt-test ### FAIL of_unittest_overlay_high_level():2419 Adding overlay 'overlay_bad_add_dup_node' failed

   ...

   ### dt-test ### end of unittest - 210 passed, 1 failed

but will instead include:

   OF: overlay: ERROR: multiple overlay fragments add and/or delete node /testcase-data-2/substation@100/motor-1/controller

   ...

   ### dt-test ### end of unittest - 211 passed, 0 failed

Signed-off-by: Frank Rowand <frank.rowand@sony.com>
---

checkpatch errors "line over 80 characters" are ok, they will be
fixed later in this series

 drivers/of/overlay.c | 58 ++++++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 49 insertions(+), 9 deletions(-)

diff --git a/drivers/of/overlay.c b/drivers/of/overlay.c
index 34396d6db1b3..0dc7452e2ed5 100644
--- a/drivers/of/overlay.c
+++ b/drivers/of/overlay.c
@@ -402,14 +402,6 @@ static int add_changeset_property(struct overlay_changeset *ovcs,
  *       a live devicetree created from Open Firmware.
  *
  * NOTE_2: Multiple mods of created nodes not supported.
- *       If more than one fragment contains a node that does not already exist
- *       in the live tree, then for each fragment of_changeset_attach_node()
- *       will add a changeset entry to add the node.  When the changeset is
- *       applied, __of_attach_node() will attach the node twice (once for
- *       each fragment).  At this point the device tree will be corrupted.
- *
- *       TODO: add integrity check to ensure that multiple fragments do not
- *             create the same node.
  *
  * Returns 0 on success, -ENOMEM if memory allocation failure, or -EINVAL if
  * invalid @overlay.
@@ -527,6 +519,54 @@ static int build_changeset_symbols_node(struct overlay_changeset *ovcs,
 }
 
 /**
+ * check_changeset_dup_add_node() - changeset validation: duplicate add node
+ * @ovcs:	Overlay changeset
+ *
+ * Check changeset @ovcs->cset for multiple add node entries for the same
+ * node.
+ *
+ * Returns 0 on success, -ENOMEM if memory allocation failure, or -EINVAL if
+ * invalid overlay in @ovcs->fragments[].
+ */
+static int check_changeset_dup_add_node(struct overlay_changeset *ovcs)
+{
+	struct of_changeset_entry *ce_1, *ce_2;
+	char *fn_1, *fn_2;
+	int name_match;
+
+	list_for_each_entry(ce_1, &ovcs->cset.entries, node) {
+
+		if (ce_1->action == OF_RECONFIG_ATTACH_NODE ||
+		    ce_1->action == OF_RECONFIG_DETACH_NODE) {
+
+			ce_2 = ce_1;
+			list_for_each_entry_continue(ce_2, &ovcs->cset.entries, node) {
+				if (ce_2->action == OF_RECONFIG_ATTACH_NODE ||
+				    ce_2->action == OF_RECONFIG_DETACH_NODE) {
+					/* inexpensive name compare */
+					if (!of_node_cmp(ce_1->np->full_name,
+					    ce_2->np->full_name)) {
+						/* expensive full path name compare */
+						fn_1 = kasprintf(GFP_KERNEL, "%pOF", ce_1->np);
+						fn_2 = kasprintf(GFP_KERNEL, "%pOF", ce_2->np);
+						name_match = !strcmp(fn_1, fn_2);
+						kfree(fn_1);
+						kfree(fn_2);
+						if (name_match) {
+							pr_err("ERROR: multiple overlay fragments add and/or delete node %pOF\n",
+							       ce_1->np);
+							return -EINVAL;
+						}
+					}
+				}
+			}
+		}
+	}
+
+	return 0;
+}
+
+/**
  * build_changeset() - populate overlay changeset in @ovcs from @ovcs->fragments
  * @ovcs:	Overlay changeset
  *
@@ -581,7 +621,7 @@ static int build_changeset(struct overlay_changeset *ovcs)
 		}
 	}
 
-	return 0;
+	return check_changeset_dup_add_node(ovcs);
 }
 
 /*
-- 
Frank Rowand <frank.rowand@sony.com>


^ permalink raw reply related	[flat|nested] 33+ messages in thread

* [PATCH v4 13/18] of: overlay: check prevents multiple fragments touching same property
  2018-10-16  2:37 [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes frowand.list
                   ` (11 preceding siblings ...)
  2018-10-16  2:37 ` [PATCH v4 12/18] of: overlay: check prevents multiple fragments add or delete " frowand.list
@ 2018-10-16  2:37 ` frowand.list
  2018-10-16  2:37 ` [PATCH v4 14/18] of: unittest: remove unused of_unittest_apply_overlay() argument frowand.list
                   ` (5 subsequent siblings)
  18 siblings, 0 replies; 33+ messages in thread
From: frowand.list @ 2018-10-16  2:37 UTC (permalink / raw)
  To: Rob Herring, Pantelis Antoniou, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Alan Tull,
	Moritz Fischer
  Cc: devicetree, linux-fpga, linuxppc-dev, linux-kernel

From: Frank Rowand <frank.rowand@sony.com>

Add test case of two fragments updating the same property.  After
adding the test case, the system hangs at end of boot, after
after slub stack dumps from kfree() in crypto modprobe code.

Multiple overlay fragments adding, modifying, or deleting the same
property is not supported.  Add check to detect the attempt and fail
the overlay apply.

Before this patch, the first fragment error would terminate
processing.  Allow fragment checking to proceed and report all
of the fragment errors before terminating the overlay apply. This
is not a hot path, thus not a performance issue (the error is not
transient and requires fixing the overlay before attempting to
apply it again).

After applying this patch, the devicetree unittest messages will
include:

   OF: overlay: ERROR: multiple fragments add, update, and/or delete property /testcase-data-2/substation@100/motor-1/rpm_avail

   ...

   ### dt-test ### end of unittest - 212 passed, 0 failed

The check to detect two fragments updating the same property is
folded into the patch that created the test case to maintain
bisectability.

Signed-off-by: Frank Rowand <frank.rowand@sony.com>
---
Changes since v3:
  - Update patch comment header to state that this patch modifies the
    previous patch to not return immediately on fragment error and
    explain this is not a performance issue.
  - remove redundant "overlay" from two error messages.  "OF: overlay:"
    is already present in pr_fmt()

 drivers/of/overlay.c                               | 118 ++++++++++++++-------
 drivers/of/unittest-data/Makefile                  |   1 +
 .../of/unittest-data/overlay_bad_add_dup_prop.dts  |  24 +++++
 drivers/of/unittest-data/overlay_base.dts          |   1 +
 drivers/of/unittest.c                              |   5 +
 5 files changed, 112 insertions(+), 37 deletions(-)
 create mode 100644 drivers/of/unittest-data/overlay_bad_add_dup_prop.dts

diff --git a/drivers/of/overlay.c b/drivers/of/overlay.c
index 0dc7452e2ed5..01afd22566ed 100644
--- a/drivers/of/overlay.c
+++ b/drivers/of/overlay.c
@@ -518,52 +518,96 @@ static int build_changeset_symbols_node(struct overlay_changeset *ovcs,
 	return 0;
 }
 
+static int find_dup_cset_node_entry(struct overlay_changeset *ovcs,
+		struct of_changeset_entry *ce_1)
+{
+	struct of_changeset_entry *ce_2;
+	char *fn_1, *fn_2;
+	int node_path_match;
+
+	if (ce_1->action != OF_RECONFIG_ATTACH_NODE &&
+	    ce_1->action != OF_RECONFIG_DETACH_NODE)
+		return 0;
+
+	ce_2 = ce_1;
+	list_for_each_entry_continue(ce_2, &ovcs->cset.entries, node) {
+		if ((ce_2->action != OF_RECONFIG_ATTACH_NODE &&
+		     ce_2->action != OF_RECONFIG_DETACH_NODE) ||
+		    of_node_cmp(ce_1->np->full_name, ce_2->np->full_name))
+			continue;
+
+		fn_1 = kasprintf(GFP_KERNEL, "%pOF", ce_1->np);
+		fn_2 = kasprintf(GFP_KERNEL, "%pOF", ce_2->np);
+		node_path_match = !strcmp(fn_1, fn_2);
+		kfree(fn_1);
+		kfree(fn_2);
+		if (node_path_match) {
+			pr_err("ERROR: multiple fragments add and/or delete node %pOF\n",
+			       ce_1->np);
+			return -EINVAL;
+		}
+	}
+
+	return 0;
+}
+
+static int find_dup_cset_prop(struct overlay_changeset *ovcs,
+		struct of_changeset_entry *ce_1)
+{
+	struct of_changeset_entry *ce_2;
+	char *fn_1, *fn_2;
+	int node_path_match;
+
+	if (ce_1->action != OF_RECONFIG_ADD_PROPERTY &&
+	    ce_1->action != OF_RECONFIG_REMOVE_PROPERTY &&
+	    ce_1->action != OF_RECONFIG_UPDATE_PROPERTY)
+		return 0;
+
+	ce_2 = ce_1;
+	list_for_each_entry_continue(ce_2, &ovcs->cset.entries, node) {
+		if ((ce_2->action != OF_RECONFIG_ADD_PROPERTY &&
+		     ce_2->action != OF_RECONFIG_REMOVE_PROPERTY &&
+		     ce_2->action != OF_RECONFIG_UPDATE_PROPERTY) ||
+		    of_node_cmp(ce_1->np->full_name, ce_2->np->full_name))
+			continue;
+
+		fn_1 = kasprintf(GFP_KERNEL, "%pOF", ce_1->np);
+		fn_2 = kasprintf(GFP_KERNEL, "%pOF", ce_2->np);
+		node_path_match = !strcmp(fn_1, fn_2);
+		kfree(fn_1);
+		kfree(fn_2);
+		if (node_path_match &&
+		    !of_prop_cmp(ce_1->prop->name, ce_2->prop->name)) {
+			pr_err("ERROR: multiple fragments add, update, and/or delete property %pOF/%s\n",
+			       ce_1->np, ce_1->prop->name);
+			return -EINVAL;
+		}
+	}
+
+	return 0;
+}
+
 /**
- * check_changeset_dup_add_node() - changeset validation: duplicate add node
+ * changeset_dup_entry_check() - check for duplicate entries
  * @ovcs:	Overlay changeset
  *
- * Check changeset @ovcs->cset for multiple add node entries for the same
- * node.
+ * Check changeset @ovcs->cset for multiple {add or delete} node entries for
+ * the same node or duplicate {add, delete, or update} properties entries
+ * for the same property.
  *
- * Returns 0 on success, -ENOMEM if memory allocation failure, or -EINVAL if
- * invalid overlay in @ovcs->fragments[].
+ * Returns 0 on success, or -EINVAL if duplicate changeset entry found.
  */
-static int check_changeset_dup_add_node(struct overlay_changeset *ovcs)
+static int changeset_dup_entry_check(struct overlay_changeset *ovcs)
 {
-	struct of_changeset_entry *ce_1, *ce_2;
-	char *fn_1, *fn_2;
-	int name_match;
+	struct of_changeset_entry *ce_1;
+	int dup_entry = 0;
 
 	list_for_each_entry(ce_1, &ovcs->cset.entries, node) {
-
-		if (ce_1->action == OF_RECONFIG_ATTACH_NODE ||
-		    ce_1->action == OF_RECONFIG_DETACH_NODE) {
-
-			ce_2 = ce_1;
-			list_for_each_entry_continue(ce_2, &ovcs->cset.entries, node) {
-				if (ce_2->action == OF_RECONFIG_ATTACH_NODE ||
-				    ce_2->action == OF_RECONFIG_DETACH_NODE) {
-					/* inexpensive name compare */
-					if (!of_node_cmp(ce_1->np->full_name,
-					    ce_2->np->full_name)) {
-						/* expensive full path name compare */
-						fn_1 = kasprintf(GFP_KERNEL, "%pOF", ce_1->np);
-						fn_2 = kasprintf(GFP_KERNEL, "%pOF", ce_2->np);
-						name_match = !strcmp(fn_1, fn_2);
-						kfree(fn_1);
-						kfree(fn_2);
-						if (name_match) {
-							pr_err("ERROR: multiple overlay fragments add and/or delete node %pOF\n",
-							       ce_1->np);
-							return -EINVAL;
-						}
-					}
-				}
-			}
-		}
+		dup_entry |= find_dup_cset_node_entry(ovcs, ce_1);
+		dup_entry |= find_dup_cset_prop(ovcs, ce_1);
 	}
 
-	return 0;
+	return dup_entry ? -EINVAL : 0;
 }
 
 /**
@@ -621,7 +665,7 @@ static int build_changeset(struct overlay_changeset *ovcs)
 		}
 	}
 
-	return check_changeset_dup_add_node(ovcs);
+	return changeset_dup_entry_check(ovcs);
 }
 
 /*
diff --git a/drivers/of/unittest-data/Makefile b/drivers/of/unittest-data/Makefile
index 166dbdbfd1c5..9b6807065827 100644
--- a/drivers/of/unittest-data/Makefile
+++ b/drivers/of/unittest-data/Makefile
@@ -18,6 +18,7 @@ obj-$(CONFIG_OF_OVERLAY) += overlay.dtb.o \
 			    overlay_13.dtb.o \
 			    overlay_15.dtb.o \
 			    overlay_bad_add_dup_node.dtb.o \
+			    overlay_bad_add_dup_prop.dtb.o \
 			    overlay_bad_phandle.dtb.o \
 			    overlay_bad_symbol.dtb.o \
 			    overlay_base.dtb.o
diff --git a/drivers/of/unittest-data/overlay_bad_add_dup_prop.dts b/drivers/of/unittest-data/overlay_bad_add_dup_prop.dts
new file mode 100644
index 000000000000..c190da54f175
--- /dev/null
+++ b/drivers/of/unittest-data/overlay_bad_add_dup_prop.dts
@@ -0,0 +1,24 @@
+// SPDX-License-Identifier: GPL-2.0
+/dts-v1/;
+/plugin/;
+
+/*
+ * &electric_1/motor-1 and &spin_ctrl_1 are the same node:
+ *   /testcase-data-2/substation@100/motor-1
+ *
+ * Thus the property "rpm_avail" in each fragment will
+ * result in an attempt to update the same property twice.
+ * This will result in an error and the overlay apply
+ * will fail.
+ */
+
+&electric_1 {
+
+	motor-1 {
+		rpm_avail = < 100 >;
+	};
+};
+
+&spin_ctrl_1 {
+		rpm_avail = < 100 200 >;
+};
diff --git a/drivers/of/unittest-data/overlay_base.dts b/drivers/of/unittest-data/overlay_base.dts
index 820b79ca378a..99ab9d12d00b 100644
--- a/drivers/of/unittest-data/overlay_base.dts
+++ b/drivers/of/unittest-data/overlay_base.dts
@@ -30,6 +30,7 @@
 			spin_ctrl_1: motor-1 {
 				compatible = "ot,ferris-wheel-motor";
 				spin = "clockwise";
+				rpm_avail = < 50 >;
 			};
 
 			spin_ctrl_2: motor-8 {
diff --git a/drivers/of/unittest.c b/drivers/of/unittest.c
index 471b8eb6e842..efd9c947f192 100644
--- a/drivers/of/unittest.c
+++ b/drivers/of/unittest.c
@@ -2148,6 +2148,7 @@ struct overlay_info {
 OVERLAY_INFO_EXTERN(overlay_13);
 OVERLAY_INFO_EXTERN(overlay_15);
 OVERLAY_INFO_EXTERN(overlay_bad_add_dup_node);
+OVERLAY_INFO_EXTERN(overlay_bad_add_dup_prop);
 OVERLAY_INFO_EXTERN(overlay_bad_phandle);
 OVERLAY_INFO_EXTERN(overlay_bad_symbol);
 
@@ -2171,6 +2172,7 @@ struct overlay_info {
 	OVERLAY_INFO(overlay_13, 0),
 	OVERLAY_INFO(overlay_15, 0),
 	OVERLAY_INFO(overlay_bad_add_dup_node, -EINVAL),
+	OVERLAY_INFO(overlay_bad_add_dup_prop, -EINVAL),
 	OVERLAY_INFO(overlay_bad_phandle, -EINVAL),
 	OVERLAY_INFO(overlay_bad_symbol, -EINVAL),
 	{}
@@ -2418,6 +2420,9 @@ static __init void of_unittest_overlay_high_level(void)
 	unittest(overlay_data_apply("overlay_bad_add_dup_node", NULL),
 		 "Adding overlay 'overlay_bad_add_dup_node' failed\n");
 
+	unittest(overlay_data_apply("overlay_bad_add_dup_prop", NULL),
+		 "Adding overlay 'overlay_bad_add_dup_prop' failed\n");
+
 	unittest(overlay_data_apply("overlay_bad_phandle", NULL),
 		 "Adding overlay 'overlay_bad_phandle' failed\n");
 
-- 
Frank Rowand <frank.rowand@sony.com>


^ permalink raw reply related	[flat|nested] 33+ messages in thread

* [PATCH v4 14/18] of: unittest: remove unused of_unittest_apply_overlay() argument
  2018-10-16  2:37 [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes frowand.list
                   ` (12 preceding siblings ...)
  2018-10-16  2:37 ` [PATCH v4 13/18] of: overlay: check prevents multiple fragments touching same property frowand.list
@ 2018-10-16  2:37 ` frowand.list
  2018-10-16  2:37 ` [PATCH v4 15/18] of: overlay: set node fields from properties when add new overlay node frowand.list
                   ` (4 subsequent siblings)
  18 siblings, 0 replies; 33+ messages in thread
From: frowand.list @ 2018-10-16  2:37 UTC (permalink / raw)
  To: Rob Herring, Pantelis Antoniou, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Alan Tull,
	Moritz Fischer
  Cc: devicetree, linux-fpga, linuxppc-dev, linux-kernel

From: Frank Rowand <frank.rowand@sony.com>

Argument unittest_nr is not used in of_unittest_apply_overlay(),
remove it.

Signed-off-by: Frank Rowand <frank.rowand@sony.com>
---
 drivers/of/unittest.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/drivers/of/unittest.c b/drivers/of/unittest.c
index efd9c947f192..6d80f474c8f2 100644
--- a/drivers/of/unittest.c
+++ b/drivers/of/unittest.c
@@ -1419,8 +1419,7 @@ static void of_unittest_destroy_tracked_overlays(void)
 	} while (defers > 0);
 }
 
-static int __init of_unittest_apply_overlay(int overlay_nr, int unittest_nr,
-		int *overlay_id)
+static int __init of_unittest_apply_overlay(int overlay_nr, int *overlay_id)
 {
 	const char *overlay_name;
 
@@ -1453,7 +1452,7 @@ static int __init of_unittest_apply_overlay_check(int overlay_nr,
 	}
 
 	ovcs_id = 0;
-	ret = of_unittest_apply_overlay(overlay_nr, unittest_nr, &ovcs_id);
+	ret = of_unittest_apply_overlay(overlay_nr, &ovcs_id);
 	if (ret != 0) {
 		/* of_unittest_apply_overlay already called unittest() */
 		return ret;
@@ -1489,7 +1488,7 @@ static int __init of_unittest_apply_revert_overlay_check(int overlay_nr,
 
 	/* apply the overlay */
 	ovcs_id = 0;
-	ret = of_unittest_apply_overlay(overlay_nr, unittest_nr, &ovcs_id);
+	ret = of_unittest_apply_overlay(overlay_nr, &ovcs_id);
 	if (ret != 0) {
 		/* of_unittest_apply_overlay already called unittest() */
 		return ret;
-- 
Frank Rowand <frank.rowand@sony.com>


^ permalink raw reply related	[flat|nested] 33+ messages in thread

* [PATCH v4 15/18] of: overlay: set node fields from properties when add new overlay node
  2018-10-16  2:37 [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes frowand.list
                   ` (13 preceding siblings ...)
  2018-10-16  2:37 ` [PATCH v4 14/18] of: unittest: remove unused of_unittest_apply_overlay() argument frowand.list
@ 2018-10-16  2:37 ` frowand.list
  2018-10-16  2:37 ` [PATCH v4 16/18] of: unittest: allow base devicetree to have symbol metadata frowand.list
                   ` (3 subsequent siblings)
  18 siblings, 0 replies; 33+ messages in thread
From: frowand.list @ 2018-10-16  2:37 UTC (permalink / raw)
  To: Rob Herring, Pantelis Antoniou, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Alan Tull,
	Moritz Fischer
  Cc: devicetree, linux-fpga, linuxppc-dev, linux-kernel

From: Frank Rowand <frank.rowand@sony.com>

Overlay nodes added by add_changeset_node() do not have the node
fields name, phandle, and type set.

The node passed to __of_attach_node() when the add node changeset
entry is processed does not contain any properties.  The node's
properties are located in add property changeset entries that will
be processed after the add node changeset is applied.

Set the node's fields in the node contained in the add node
changeset entry and do not set them to incorrect values in
add_changeset_node().

A visible symptom that is fixed by this patch is the names of nodes
added by overlays that have an entry in /sys/bus/platform/drivers/*/
will contain the unit-address but the node-name will be <NULL>,  for
example, "fc4ab000.<NULL>".  After applying the patch the name, in
this example, for node restart@fc4ab000 is "fc4ab000.restart".

Signed-off-by: Frank Rowand <frank.rowand@sony.com>
---
 drivers/of/dynamic.c | 27 ++++++++++++++++++---------
 drivers/of/overlay.c | 29 ++++++++++++++++++++++++-----
 2 files changed, 42 insertions(+), 14 deletions(-)

diff --git a/drivers/of/dynamic.c b/drivers/of/dynamic.c
index a94f727ec3da..a9f5d5fb3f25 100644
--- a/drivers/of/dynamic.c
+++ b/drivers/of/dynamic.c
@@ -205,15 +205,24 @@ static void __of_attach_node(struct device_node *np)
 	const __be32 *phandle;
 	int sz;
 
-	np->name = __of_get_property(np, "name", NULL) ? : "<NULL>";
-	np->type = __of_get_property(np, "device_type", NULL) ? : "<NULL>";
-
-	phandle = __of_get_property(np, "phandle", &sz);
-	if (!phandle)
-		phandle = __of_get_property(np, "linux,phandle", &sz);
-	if (IS_ENABLED(CONFIG_PPC_PSERIES) && !phandle)
-		phandle = __of_get_property(np, "ibm,phandle", &sz);
-	np->phandle = (phandle && (sz >= 4)) ? be32_to_cpup(phandle) : 0;
+	if (!of_node_check_flag(np, OF_OVERLAY)) {
+		np->name = __of_get_property(np, "name", NULL);
+		np->type = __of_get_property(np, "device_type", NULL);
+		if (!np->name)
+			np->name = "<NULL>";
+		if (!np->type)
+			np->type = "<NULL>";
+
+		phandle = __of_get_property(np, "phandle", &sz);
+		if (!phandle)
+			phandle = __of_get_property(np, "linux,phandle", &sz);
+		if (IS_ENABLED(CONFIG_PPC_PSERIES) && !phandle)
+			phandle = __of_get_property(np, "ibm,phandle", &sz);
+		if (phandle && (sz >= 4))
+			np->phandle = be32_to_cpup(phandle);
+		else
+			np->phandle = 0;
+	}
 
 	np->child = NULL;
 	np->sibling = np->parent->child;
diff --git a/drivers/of/overlay.c b/drivers/of/overlay.c
index 01afd22566ed..d011177e5aaa 100644
--- a/drivers/of/overlay.c
+++ b/drivers/of/overlay.c
@@ -307,10 +307,11 @@ static int add_changeset_property(struct overlay_changeset *ovcs,
 	int ret = 0;
 	bool check_for_non_overlay_node = false;
 
-	if (!of_prop_cmp(overlay_prop->name, "name") ||
-	    !of_prop_cmp(overlay_prop->name, "phandle") ||
-	    !of_prop_cmp(overlay_prop->name, "linux,phandle"))
-		return 0;
+	if (target->in_livetree)
+		if (!of_prop_cmp(overlay_prop->name, "name") ||
+		    !of_prop_cmp(overlay_prop->name, "phandle") ||
+		    !of_prop_cmp(overlay_prop->name, "linux,phandle"))
+			return 0;
 
 	if (target->in_livetree)
 		prop = of_find_property(target->np, overlay_prop->name, NULL);
@@ -331,6 +332,10 @@ static int add_changeset_property(struct overlay_changeset *ovcs,
 	if (!prop) {
 
 		check_for_non_overlay_node = true;
+		if (!target->in_livetree) {
+			new_prop->next = target->np->deadprops;
+			target->np->deadprops = new_prop;
+		}
 		ret = of_changeset_add_property(&ovcs->cset, target->np,
 						new_prop);
 
@@ -410,9 +415,10 @@ static int add_changeset_node(struct overlay_changeset *ovcs,
 		struct target *target, struct device_node *node)
 {
 	const char *node_kbasename;
+	const __be32 *phandle;
 	struct device_node *tchild;
 	struct target target_child;
-	int ret = 0;
+	int ret = 0, size;
 
 	node_kbasename = kbasename(node->full_name);
 
@@ -426,6 +432,19 @@ static int add_changeset_node(struct overlay_changeset *ovcs,
 			return -ENOMEM;
 
 		tchild->parent = target->np;
+		tchild->name = __of_get_property(node, "name", NULL);
+		tchild->type = __of_get_property(node, "device_type", NULL);
+
+		if (!tchild->name)
+			tchild->name = "<NULL>";
+		if (!tchild->type)
+			tchild->type = "<NULL>";
+
+		/* ignore obsolete "linux,phandle" */
+		phandle = __of_get_property(node, "phandle", &size);
+		if (phandle && (size == 4))
+			tchild->phandle = be32_to_cpup(phandle);
+
 		of_node_set_flag(tchild, OF_OVERLAY);
 
 		ret = of_changeset_attach_node(&ovcs->cset, tchild);
-- 
Frank Rowand <frank.rowand@sony.com>


^ permalink raw reply related	[flat|nested] 33+ messages in thread

* [PATCH v4 16/18] of: unittest: allow base devicetree to have symbol metadata
  2018-10-16  2:37 [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes frowand.list
                   ` (14 preceding siblings ...)
  2018-10-16  2:37 ` [PATCH v4 15/18] of: overlay: set node fields from properties when add new overlay node frowand.list
@ 2018-10-16  2:37 ` frowand.list
  2018-10-16  2:37 ` [PATCH v4 17/18] of: unittest: find overlays[] entry by name instead of index frowand.list
                   ` (2 subsequent siblings)
  18 siblings, 0 replies; 33+ messages in thread
From: frowand.list @ 2018-10-16  2:37 UTC (permalink / raw)
  To: Rob Herring, Pantelis Antoniou, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Alan Tull,
	Moritz Fischer
  Cc: devicetree, linux-fpga, linuxppc-dev, linux-kernel

From: Frank Rowand <frank.rowand@sony.com>

The overlay metadata nodes in the FDT created from testcases.dts
are not handled properly.

The __fixups__ and __local_fixups__ node were added to the live
devicetree, but should not be.

Only the first property in the /__symbols__ node was added to the
live devicetree if the live devicetree already contained a
/__symbols node.  All of the node's properties must be added.

Signed-off-by: Frank Rowand <frank.rowand@sony.com>
---
 drivers/of/unittest.c | 43 +++++++++++++++++++++++++++++++++++--------
 1 file changed, 35 insertions(+), 8 deletions(-)

diff --git a/drivers/of/unittest.c b/drivers/of/unittest.c
index 6d80f474c8f2..1c2bd8503095 100644
--- a/drivers/of/unittest.c
+++ b/drivers/of/unittest.c
@@ -1057,20 +1057,44 @@ static void __init of_unittest_platform_populate(void)
  *	of np into dup node (present in live tree) and
  *	updates parent of children of np to dup.
  *
- *	@np:	node already present in live tree
+ *	@np:	node whose properties are being added to the live tree
  *	@dup:	node present in live tree to be updated
  */
 static void update_node_properties(struct device_node *np,
 					struct device_node *dup)
 {
 	struct property *prop;
+	struct property *save_next;
 	struct device_node *child;
-
-	for_each_property_of_node(np, prop)
-		of_add_property(dup, prop);
+	int ret;
 
 	for_each_child_of_node(np, child)
 		child->parent = dup;
+
+	/*
+	 * "unittest internal error: unable to add testdata property"
+	 *
+	 *    If this message reports a property in node '/__symbols__' then
+	 *    the respective unittest overlay contains a label that has the
+	 *    same name as a label in the live devicetree.  The label will
+	 *    be in the live devicetree only if the devicetree source was
+	 *    compiled with the '-@' option.  If you encounter this error,
+	 *    please consider renaming __all__ of the labels in the unittest
+	 *    overlay dts files with an odd prefix that is unlikely to be
+	 *    used in a real devicetree.
+	 */
+
+	/*
+	 * open code for_each_property_of_node() because of_add_property()
+	 * sets prop->next to NULL
+	 */
+	for (prop = np->properties; prop != NULL; prop = save_next) {
+		save_next = prop->next;
+		ret = of_add_property(dup, prop);
+		if (ret)
+			pr_err("unittest internal error: unable to add testdata property %pOF/%s",
+			       np, prop->name);
+	}
 }
 
 /**
@@ -1079,18 +1103,23 @@ static void update_node_properties(struct device_node *np,
  *
  *	@np:	Node to attach to live tree
  */
-static int attach_node_and_children(struct device_node *np)
+static void attach_node_and_children(struct device_node *np)
 {
 	struct device_node *next, *dup, *child;
 	unsigned long flags;
 	const char *full_name;
 
 	full_name = kasprintf(GFP_KERNEL, "%pOF", np);
+
+	if (!strcmp(full_name, "/__local_fixups__") ||
+	    !strcmp(full_name, "/__fixups__"))
+		return;
+
 	dup = of_find_node_by_path(full_name);
 	kfree(full_name);
 	if (dup) {
 		update_node_properties(np, dup);
-		return 0;
+		return;
 	}
 
 	child = np->child;
@@ -1111,8 +1140,6 @@ static int attach_node_and_children(struct device_node *np)
 		attach_node_and_children(child);
 		child = next;
 	}
-
-	return 0;
 }
 
 /**
-- 
Frank Rowand <frank.rowand@sony.com>


^ permalink raw reply related	[flat|nested] 33+ messages in thread

* [PATCH v4 17/18] of: unittest: find overlays[] entry by name instead of index
  2018-10-16  2:37 [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes frowand.list
                   ` (15 preceding siblings ...)
  2018-10-16  2:37 ` [PATCH v4 16/18] of: unittest: allow base devicetree to have symbol metadata frowand.list
@ 2018-10-16  2:37 ` frowand.list
  2018-10-16  2:37 ` [PATCH v4 18/18] of: unittest: initialize args before calling of_*parse_*() frowand.list
  2018-10-16  9:47 ` [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes Michael Ellerman
  18 siblings, 0 replies; 33+ messages in thread
From: frowand.list @ 2018-10-16  2:37 UTC (permalink / raw)
  To: Rob Herring, Pantelis Antoniou, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Alan Tull,
	Moritz Fischer
  Cc: devicetree, linux-fpga, linuxppc-dev, linux-kernel

From: Frank Rowand <frank.rowand@sony.com>

One accessor of overlays[] was using a hard coded index value to
find the correct array entry instead of searching for the entry
containing the correct name.

Signed-off-by: Frank Rowand <frank.rowand@sony.com>
---
 drivers/of/unittest.c | 21 +++++++++++++++++----
 1 file changed, 17 insertions(+), 4 deletions(-)

diff --git a/drivers/of/unittest.c b/drivers/of/unittest.c
index 1c2bd8503095..785985bdbfa6 100644
--- a/drivers/of/unittest.c
+++ b/drivers/of/unittest.c
@@ -2178,7 +2178,7 @@ struct overlay_info {
 OVERLAY_INFO_EXTERN(overlay_bad_phandle);
 OVERLAY_INFO_EXTERN(overlay_bad_symbol);
 
-/* order of entries is hard-coded into users of overlays[] */
+/* entries found by name */
 static struct overlay_info overlays[] = {
 	OVERLAY_INFO(overlay_base, -9999),
 	OVERLAY_INFO(overlay, 0),
@@ -2201,7 +2201,8 @@ struct overlay_info {
 	OVERLAY_INFO(overlay_bad_add_dup_prop, -EINVAL),
 	OVERLAY_INFO(overlay_bad_phandle, -EINVAL),
 	OVERLAY_INFO(overlay_bad_symbol, -EINVAL),
-	{}
+	/* end marker */
+	{.dtb_begin = NULL, .dtb_end = NULL, .expected_result = 0, .name = NULL}
 };
 
 static struct device_node *overlay_base_root;
@@ -2231,6 +2232,19 @@ void __init unittest_unflatten_overlay_base(void)
 	u32 data_size;
 	void *new_fdt;
 	u32 size;
+	int found = 0;
+	const char *overlay_name = "overlay_base";
+
+	for (info = overlays; info && info->name; info++) {
+		if (!strcmp(overlay_name, info->name)) {
+			found = 1;
+			break;
+		}
+	}
+	if (!found) {
+		pr_err("no overlay data for %s\n", overlay_name);
+		return;
+	}
 
 	info = &overlays[0];
 
@@ -2278,11 +2292,10 @@ static int __init overlay_data_apply(const char *overlay_name, int *overlay_id)
 {
 	struct overlay_info *info;
 	int found = 0;
-	int k;
 	int ret;
 	u32 size;
 
-	for (k = 0, info = overlays; info && info->name; info++, k++) {
+	for (info = overlays; info && info->name; info++) {
 		if (!strcmp(overlay_name, info->name)) {
 			found = 1;
 			break;
-- 
Frank Rowand <frank.rowand@sony.com>


^ permalink raw reply related	[flat|nested] 33+ messages in thread

* [PATCH v4 18/18] of: unittest: initialize args before calling of_*parse_*()
  2018-10-16  2:37 [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes frowand.list
                   ` (16 preceding siblings ...)
  2018-10-16  2:37 ` [PATCH v4 17/18] of: unittest: find overlays[] entry by name instead of index frowand.list
@ 2018-10-16  2:37 ` frowand.list
  2018-10-16  9:47 ` [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes Michael Ellerman
  18 siblings, 0 replies; 33+ messages in thread
From: frowand.list @ 2018-10-16  2:37 UTC (permalink / raw)
  To: Rob Herring, Pantelis Antoniou, Michael Ellerman,
	Benjamin Herrenschmidt, Paul Mackerras, Alan Tull,
	Moritz Fischer
  Cc: devicetree, linux-fpga, linuxppc-dev, linux-kernel

From: Frank Rowand <frank.rowand@sony.com>

Callers of of_irq_parse_one() blindly use the pointer args.np
without checking whether of_irq_parse_one() had an error and
thus did not set the value of args.np.  Initialize args to
zero so that using the format "%pOF" to show the value of
args.np will show "(null)" when of_irq_parse_one() has an
error.  This prevents the dereference of a random value.

Make the same fix for callers of of_parse_phandle_with_args()
and of_parse_phandle_with_args_map().

Reported-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Frank Rowand <frank.rowand@sony.com>
---
 drivers/of/unittest.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/drivers/of/unittest.c b/drivers/of/unittest.c
index 785985bdbfa6..5f4db23e4752 100644
--- a/drivers/of/unittest.c
+++ b/drivers/of/unittest.c
@@ -375,6 +375,7 @@ static void __init of_unittest_parse_phandle_with_args(void)
 	for (i = 0; i < 8; i++) {
 		bool passed = true;
 
+		memset(&args, 0, sizeof(args));
 		rc = of_parse_phandle_with_args(np, "phandle-list",
 						"#phandle-cells", i, &args);
 
@@ -428,6 +429,7 @@ static void __init of_unittest_parse_phandle_with_args(void)
 	}
 
 	/* Check for missing list property */
+	memset(&args, 0, sizeof(args));
 	rc = of_parse_phandle_with_args(np, "phandle-list-missing",
 					"#phandle-cells", 0, &args);
 	unittest(rc == -ENOENT, "expected:%i got:%i\n", -ENOENT, rc);
@@ -436,6 +438,7 @@ static void __init of_unittest_parse_phandle_with_args(void)
 	unittest(rc == -ENOENT, "expected:%i got:%i\n", -ENOENT, rc);
 
 	/* Check for missing cells property */
+	memset(&args, 0, sizeof(args));
 	rc = of_parse_phandle_with_args(np, "phandle-list",
 					"#phandle-cells-missing", 0, &args);
 	unittest(rc == -EINVAL, "expected:%i got:%i\n", -EINVAL, rc);
@@ -444,6 +447,7 @@ static void __init of_unittest_parse_phandle_with_args(void)
 	unittest(rc == -EINVAL, "expected:%i got:%i\n", -EINVAL, rc);
 
 	/* Check for bad phandle in list */
+	memset(&args, 0, sizeof(args));
 	rc = of_parse_phandle_with_args(np, "phandle-list-bad-phandle",
 					"#phandle-cells", 0, &args);
 	unittest(rc == -EINVAL, "expected:%i got:%i\n", -EINVAL, rc);
@@ -452,6 +456,7 @@ static void __init of_unittest_parse_phandle_with_args(void)
 	unittest(rc == -EINVAL, "expected:%i got:%i\n", -EINVAL, rc);
 
 	/* Check for incorrectly formed argument list */
+	memset(&args, 0, sizeof(args));
 	rc = of_parse_phandle_with_args(np, "phandle-list-bad-args",
 					"#phandle-cells", 1, &args);
 	unittest(rc == -EINVAL, "expected:%i got:%i\n", -EINVAL, rc);
@@ -502,6 +507,7 @@ static void __init of_unittest_parse_phandle_with_args_map(void)
 	for (i = 0; i < 8; i++) {
 		bool passed = true;
 
+		memset(&args, 0, sizeof(args));
 		rc = of_parse_phandle_with_args_map(np, "phandle-list",
 						    "phandle", i, &args);
 
@@ -559,21 +565,25 @@ static void __init of_unittest_parse_phandle_with_args_map(void)
 	}
 
 	/* Check for missing list property */
+	memset(&args, 0, sizeof(args));
 	rc = of_parse_phandle_with_args_map(np, "phandle-list-missing",
 					    "phandle", 0, &args);
 	unittest(rc == -ENOENT, "expected:%i got:%i\n", -ENOENT, rc);
 
 	/* Check for missing cells,map,mask property */
+	memset(&args, 0, sizeof(args));
 	rc = of_parse_phandle_with_args_map(np, "phandle-list",
 					    "phandle-missing", 0, &args);
 	unittest(rc == -EINVAL, "expected:%i got:%i\n", -EINVAL, rc);
 
 	/* Check for bad phandle in list */
+	memset(&args, 0, sizeof(args));
 	rc = of_parse_phandle_with_args_map(np, "phandle-list-bad-phandle",
 					    "phandle", 0, &args);
 	unittest(rc == -EINVAL, "expected:%i got:%i\n", -EINVAL, rc);
 
 	/* Check for incorrectly formed argument list */
+	memset(&args, 0, sizeof(args));
 	rc = of_parse_phandle_with_args_map(np, "phandle-list-bad-args",
 					    "phandle", 1, &args);
 	unittest(rc == -EINVAL, "expected:%i got:%i\n", -EINVAL, rc);
@@ -780,7 +790,7 @@ static void __init of_unittest_parse_interrupts(void)
 	for (i = 0; i < 4; i++) {
 		bool passed = true;
 
-		args.args_count = 0;
+		memset(&args, 0, sizeof(args));
 		rc = of_irq_parse_one(np, i, &args);
 
 		passed &= !rc;
@@ -801,7 +811,7 @@ static void __init of_unittest_parse_interrupts(void)
 	for (i = 0; i < 4; i++) {
 		bool passed = true;
 
-		args.args_count = 0;
+		memset(&args, 0, sizeof(args));
 		rc = of_irq_parse_one(np, i, &args);
 
 		/* Test the values from tests-phandle.dtsi */
@@ -854,6 +864,7 @@ static void __init of_unittest_parse_interrupts_extended(void)
 	for (i = 0; i < 7; i++) {
 		bool passed = true;
 
+		memset(&args, 0, sizeof(args));
 		rc = of_irq_parse_one(np, i, &args);
 
 		/* Test the values from tests-phandle.dtsi */
-- 
Frank Rowand <frank.rowand@sony.com>


^ permalink raw reply related	[flat|nested] 33+ messages in thread

* Re: [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes
  2018-10-16  2:37 [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes frowand.list
                   ` (17 preceding siblings ...)
  2018-10-16  2:37 ` [PATCH v4 18/18] of: unittest: initialize args before calling of_*parse_*() frowand.list
@ 2018-10-16  9:47 ` Michael Ellerman
  2018-10-17  3:08   ` Frank Rowand
  18 siblings, 1 reply; 33+ messages in thread
From: Michael Ellerman @ 2018-10-16  9:47 UTC (permalink / raw)
  To: frowand.list, Rob Herring, Pantelis Antoniou,
	Benjamin Herrenschmidt, Paul Mackerras, Alan Tull,
	Moritz Fischer
  Cc: devicetree, linux-fpga, linuxppc-dev, linux-kernel

frowand.list@gmail.com writes:

> From: Frank Rowand <frank.rowand@sony.com>
>
> Add checks to (1) overlay apply process and (2) memory freeing
> triggered by overlay release.  The checks are intended to detect
> possible memory leaks and invalid overlays.
>
> The checks revealed bugs in existing code.  Fixed the bugs.
>
> While fixing bugs, noted other issues, which are fixed in
> separate patches.
>
> *****  Powerpc folks: I was not able to test the patches that
> *****  directly impact Powerpc systems that use dynamic
> *****  devicetree.  Please review that code carefully and
> *****  test.  The specific patches are: 03/16, 04/16, 07/16

Hi Frank,

Do you have this series in a git tree somewhere?

I tried applying it on top of linux-next but hit some conflicts which I
couldn't easily resolve.

cheers

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes
  2018-10-16  9:47 ` [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes Michael Ellerman
@ 2018-10-17  3:08   ` Frank Rowand
  2018-10-17 21:16     ` Alan Tull
  0 siblings, 1 reply; 33+ messages in thread
From: Frank Rowand @ 2018-10-17  3:08 UTC (permalink / raw)
  To: Michael Ellerman, Rob Herring, Pantelis Antoniou,
	Benjamin Herrenschmidt, Paul Mackerras, Alan Tull,
	Moritz Fischer
  Cc: devicetree, linux-fpga, linuxppc-dev, linux-kernel

On 10/16/18 02:47, Michael Ellerman wrote:
> frowand.list@gmail.com writes:
> 
>> From: Frank Rowand <frank.rowand@sony.com>
>>
>> Add checks to (1) overlay apply process and (2) memory freeing
>> triggered by overlay release.  The checks are intended to detect
>> possible memory leaks and invalid overlays.
>>
>> The checks revealed bugs in existing code.  Fixed the bugs.
>>
>> While fixing bugs, noted other issues, which are fixed in
>> separate patches.
>>
>> *****  Powerpc folks: I was not able to test the patches that
>> *****  directly impact Powerpc systems that use dynamic
>> *****  devicetree.  Please review that code carefully and
>> *****  test.  The specific patches are: 03/16, 04/16, 07/16
> 
> Hi Frank,
> 
> Do you have this series in a git tree somewhere?
> 
> I tried applying it on top of linux-next but hit some conflicts which I
> couldn't easily resolve.
> 
> cheers
> 



git://git.kernel.org/pub/scm/linux/kernel/git/frowand/linux.git

$ git checkout v4.19-rc1--kfree_validate--v4

$ git log --oneline v4.19-rc1..
2ba1b7d353dd of: unittest: initialize args before calling of_*parse_*()
4f9108209f79 of: unittest: find overlays[] entry by name instead of index
353403c76ff8 of: unittest: allow base devicetree to have symbol metadata
8fc37e04a01b of: overlay: set node fields from properties when add new overlay n
05d5df0e5151 of: unittest: remove unused of_unittest_apply_overlay() argument
8c021cba757a of: overlay: check prevents multiple fragments touching same proper
797a6f66e039 of: overlay: check prevents multiple fragments add or delete same n
c385e25a040d of: overlay: test case of two fragments adding same node
c88fd240f0e0 of: overlay: make all pr_debug() and pr_err() messages unique
1028a215d32a of: overlay: validate overlay properties #address-cells and #size-c
f1a97ef74ce4 of: overlay: reorder fields in struct fragment
ffe78cf7a1fb of: dynamic: change type of of_{at,de}tach_node() to void
5f5ff8ec0c0c of: overlay: do not duplicate properties from overlay for new nodes
06e72dcb2bb0 of: overlay: use prop add changeset entry for property in new nodes
a02f8d326a08 powerpc/pseries: add of_node_put() in dlpar_detach_node()
e203be664330 of: overlay: add missing of_node_get() in __of_attach_node_sysfs
8eb46208e7c8 of: overlay: add missing of_node_put() after add new node to change
b22067db7cf9 of: overlay: add tests to validate kfrees from overlay removal

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes
  2018-10-17  3:08   ` Frank Rowand
@ 2018-10-17 21:16     ` Alan Tull
  0 siblings, 0 replies; 33+ messages in thread
From: Alan Tull @ 2018-10-17 21:16 UTC (permalink / raw)
  To: Frank Rowand
  Cc: open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
	linux-fpga, Pantelis Antoniou, linux-kernel, Rob Herring,
	Moritz Fischer, Paul Mackerras, linuxppc-dev

On Tue, Oct 16, 2018 at 10:08 PM Frank Rowand <frowand.list@gmail.com> wrote:
>
> On 10/16/18 02:47, Michael Ellerman wrote:
> > frowand.list@gmail.com writes:
> >
> >> From: Frank Rowand <frank.rowand@sony.com>
> >>
> >> Add checks to (1) overlay apply process and (2) memory freeing
> >> triggered by overlay release.  The checks are intended to detect
> >> possible memory leaks and invalid overlays.
> >>
> >> The checks revealed bugs in existing code.  Fixed the bugs.
> >>
> >> While fixing bugs, noted other issues, which are fixed in
> >> separate patches.
> >>
> >> *****  Powerpc folks: I was not able to test the patches that
> >> *****  directly impact Powerpc systems that use dynamic
> >> *****  devicetree.  Please review that code carefully and
> >> *****  test.  The specific patches are: 03/16, 04/16, 07/16
> >
> > Hi Frank,
> >
> > Do you have this series in a git tree somewhere?
> >
> > I tried applying it on top of linux-next but hit some conflicts which I
> > couldn't easily resolve.
> >
> > cheers
> >
>
>
>
> git://git.kernel.org/pub/scm/linux/kernel/git/frowand/linux.git
>
> $ git checkout v4.19-rc1--kfree_validate--v4
>
> $ git log --oneline v4.19-rc1..
> 2ba1b7d353dd of: unittest: initialize args before calling of_*parse_*()
> 4f9108209f79 of: unittest: find overlays[] entry by name instead of index
> 353403c76ff8 of: unittest: allow base devicetree to have symbol metadata
> 8fc37e04a01b of: overlay: set node fields from properties when add new overlay n
> 05d5df0e5151 of: unittest: remove unused of_unittest_apply_overlay() argument
> 8c021cba757a of: overlay: check prevents multiple fragments touching same proper
> 797a6f66e039 of: overlay: check prevents multiple fragments add or delete same n
> c385e25a040d of: overlay: test case of two fragments adding same node
> c88fd240f0e0 of: overlay: make all pr_debug() and pr_err() messages unique
> 1028a215d32a of: overlay: validate overlay properties #address-cells and #size-c
> f1a97ef74ce4 of: overlay: reorder fields in struct fragment
> ffe78cf7a1fb of: dynamic: change type of of_{at,de}tach_node() to void
> 5f5ff8ec0c0c of: overlay: do not duplicate properties from overlay for new nodes
> 06e72dcb2bb0 of: overlay: use prop add changeset entry for property in new nodes
> a02f8d326a08 powerpc/pseries: add of_node_put() in dlpar_detach_node()
> e203be664330 of: overlay: add missing of_node_get() in __of_attach_node_sysfs
> 8eb46208e7c8 of: overlay: add missing of_node_put() after add new node to change
> b22067db7cf9 of: overlay: add tests to validate kfrees from overlay removal

That branch is a real time saver, thanks!

ALan

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [PATCH v4 01/18] of: overlay: add tests to validate kfrees from overlay removal
  2018-10-16  2:37 ` [PATCH v4 01/18] of: overlay: add tests to validate kfrees from overlay removal frowand.list
@ 2018-10-17 21:30   ` Alan Tull
  2018-10-18 20:24     ` Alan Tull
  2018-10-18 17:03   ` Rob Herring
  1 sibling, 1 reply; 33+ messages in thread
From: Alan Tull @ 2018-10-17 21:30 UTC (permalink / raw)
  To: Frank Rowand
  Cc: open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
	linux-fpga, Pantelis Antoniou, linux-kernel, Rob Herring,
	Moritz Fischer, Paul Mackerras, linuxppc-dev

On Mon, Oct 15, 2018 at 9:39 PM <frowand.list@gmail.com> wrote:

Hi Frank,

>
> From: Frank Rowand <frank.rowand@sony.com>
>
> Add checks:
>   - attempted kfree due to refcount reaching zero before overlay
>     is removed
>   - properties linked to an overlay node when the node is removed
>   - node refcount > one during node removal in a changeset destroy,
>     if the node was created by the changeset
>
> After applying this patch, several validation warnings will be
> reported from the devicetree unittest during boot due to
> pre-existing devicetree bugs. The warnings will be similar to:
>
>   OF: ERROR: of_node_release() overlay node /testcase-data/overlay-node/test-bus/test-unittest11/test-unittest111 contains unexpected properties
>   OF: ERROR: memory leak - destroy cset entry: attach overlay node /testcase-data-2/substation@100/hvac-medium-2 expected refcount 1 instead of 2.  of_node_get() / of_node_put() are unbalanced for this node.
>
> Signed-off-by: Frank Rowand <frank.rowand@sony.com>
> ---
> Changes since v3:
>   - Add expected value of refcount for destroy cset entry error.  Also
>     explain the cause of the error.
>
>  drivers/of/dynamic.c | 29 +++++++++++++++++++++++++++++
>  drivers/of/overlay.c |  1 +
>  include/linux/of.h   | 15 ++++++++++-----
>  3 files changed, 40 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/of/dynamic.c b/drivers/of/dynamic.c
> index f4f8ed9b5454..24c97b7a050f 100644
> --- a/drivers/of/dynamic.c
> +++ b/drivers/of/dynamic.c
> @@ -330,6 +330,25 @@ void of_node_release(struct kobject *kobj)
>         if (!of_node_check_flag(node, OF_DYNAMIC))
>                 return;
>
> +       if (of_node_check_flag(node, OF_OVERLAY)) {
> +
> +               if (!of_node_check_flag(node, OF_OVERLAY_FREE_CSET)) {
> +                       /* premature refcount of zero, do not free memory */
> +                       pr_err("ERROR: memory leak %s() overlay node %pOF before free overlay changeset\n",
> +                              __func__, node);
> +                       return;
> +               }
> +
> +               /*
> +                * If node->properties non-empty then properties were added
> +                * to this node either by different overlay that has not
> +                * yet been removed, or by a non-overlay mechanism.
> +                */
> +               if (node->properties)
> +                       pr_err("ERROR: %s() overlay node %pOF contains unexpected properties\n",
> +                              __func__, node);
> +       }
> +
>         property_list_free(node->properties);
>         property_list_free(node->deadprops);
>
> @@ -434,6 +453,16 @@ struct device_node *__of_node_dup(const struct device_node *np,
>
>  static void __of_changeset_entry_destroy(struct of_changeset_entry *ce)
>  {
> +       if (ce->action == OF_RECONFIG_ATTACH_NODE &&
> +           of_node_check_flag(ce->np, OF_OVERLAY)) {
> +               if (kref_read(&ce->np->kobj.kref) > 1) {
> +                       pr_err("ERROR: memory leak - destroy cset entry: attach overlay node %pOF expected refcount 1 instead of %d.  of_node_get() / of_node_put() are unbalanced for this node.\n",
> +                              ce->np, kref_read(&ce->np->kobj.kref));

Still testing as much as I have time to do.

I'm hitting this error message once when removing an overlay that adds
several child nodes.  The only node I get the message for was a node
that added a fixed-clock (the other nodes didn't trigger the error).
Then even if I edited all the rest of the overlay DTS and removed all
other child nodes and all references to the clock from other nodes, I
still got the error.

Removing dtbo: 1-socfpga_arria10_socdk_sdmmc_ghrd_ovl_ext_cfg.dtb
[   72.032270] OF: ERROR: memory leak - destroy cset entry: attach
overlay node /soc/base_fpga_region/clk_0 expected refcount 1 instead
of 2.  of_node_get() / of_node_put() are unbalanced for this node.

Here's the very stripped down overlay:

/dts-v1/;
/plugin/;
/ {
        fragment@0 {
                target-path = "/soc/base_fpga_region";
                #address-cells = <1>;
                #size-cells = <1>;

                __overlay__ {
                        external-fpga-config;

                        #address-cells = <1>;
                        #size-cells = <1>;

                        clk_0: clk_0 {
                                compatible = "fixed-clock";
                                #clock-cells = <0>;
                                clock-frequency = <100000000>;  /* 100.00 MHz */
                                clock-output-names = "clk_0-clk";
                        };
                };
        };
};

I'll look at it some more tomorrow and try to figure out what's
special about this node.

Alan

> +               } else {
> +                       of_node_set_flag(ce->np, OF_OVERLAY_FREE_CSET);
> +               }
> +       }
> +
>         of_node_put(ce->np);
>         list_del(&ce->node);
>         kfree(ce);
> diff --git a/drivers/of/overlay.c b/drivers/of/overlay.c
> index eda57ef12fd0..1176cb4b6e4e 100644
> --- a/drivers/of/overlay.c
> +++ b/drivers/of/overlay.c
> @@ -373,6 +373,7 @@ static int add_changeset_node(struct overlay_changeset *ovcs,
>                         return -ENOMEM;
>
>                 tchild->parent = target_node;
> +               of_node_set_flag(tchild, OF_OVERLAY);
>
>                 ret = of_changeset_attach_node(&ovcs->cset, tchild);
>                 if (ret)
> diff --git a/include/linux/of.h b/include/linux/of.h
> index 4d25e4f952d9..aa1dafaec6ae 100644
> --- a/include/linux/of.h
> +++ b/include/linux/of.h
> @@ -138,11 +138,16 @@ static inline void of_node_put(struct device_node *node) { }
>  extern struct device_node *of_stdout;
>  extern raw_spinlock_t devtree_lock;
>
> -/* flag descriptions (need to be visible even when !CONFIG_OF) */
> -#define OF_DYNAMIC     1 /* node and properties were allocated via kmalloc */
> -#define OF_DETACHED    2 /* node has been detached from the device tree */
> -#define OF_POPULATED   3 /* device already created for the node */
> -#define OF_POPULATED_BUS       4 /* of_platform_populate recursed to children of this node */
> +/*
> + * struct device_node flag descriptions
> + * (need to be visible even when !CONFIG_OF)
> + */
> +#define OF_DYNAMIC             1 /* (and properties) allocated via kmalloc */
> +#define OF_DETACHED            2 /* detached from the device tree */
> +#define OF_POPULATED           3 /* device already created */
> +#define OF_POPULATED_BUS       4 /* platform bus created for children */
> +#define OF_OVERLAY             5 /* allocated for an overlay */
> +#define OF_OVERLAY_FREE_CSET   6 /* in overlay cset being freed */
>
>  #define OF_BAD_ADDR    ((u64)-1)
>
> --
> Frank Rowand <frank.rowand@sony.com>
>

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [PATCH v4 01/18] of: overlay: add tests to validate kfrees from overlay removal
  2018-10-16  2:37 ` [PATCH v4 01/18] of: overlay: add tests to validate kfrees from overlay removal frowand.list
  2018-10-17 21:30   ` Alan Tull
@ 2018-10-18 17:03   ` Rob Herring
  2018-10-18 19:01     ` Frank Rowand
  1 sibling, 1 reply; 33+ messages in thread
From: Rob Herring @ 2018-10-18 17:03 UTC (permalink / raw)
  To: frowand.list
  Cc: devicetree, Alan Tull, linux-fpga, Pantelis Antoniou,
	linux-kernel, Moritz Fischer, Paul Mackerras, linuxppc-dev

On Mon, Oct 15, 2018 at 07:37:21PM -0700, frowand.list@gmail.com wrote:
> From: Frank Rowand <frank.rowand@sony.com>
> 
> Add checks:
>   - attempted kfree due to refcount reaching zero before overlay
>     is removed
>   - properties linked to an overlay node when the node is removed
>   - node refcount > one during node removal in a changeset destroy,
>     if the node was created by the changeset
> 
> After applying this patch, several validation warnings will be
> reported from the devicetree unittest during boot due to
> pre-existing devicetree bugs. The warnings will be similar to:
> 
>   OF: ERROR: of_node_release() overlay node /testcase-data/overlay-node/test-bus/test-unittest11/test-unittest111 contains unexpected properties
>   OF: ERROR: memory leak - destroy cset entry: attach overlay node /testcase-data-2/substation@100/hvac-medium-2 expected refcount 1 instead of 2.  of_node_get() / of_node_put() are unbalanced for this node.

These messages could be formatted more consistently. Put the path either 
at the beginning (after any prefix) or end. Beginning is more like a 
compiler error. End puts what the problem is before it's off the edge of 
the screen. 

> Signed-off-by: Frank Rowand <frank.rowand@sony.com>
> ---
> Changes since v3:
>   - Add expected value of refcount for destroy cset entry error.  Also
>     explain the cause of the error.
> 
>  drivers/of/dynamic.c | 29 +++++++++++++++++++++++++++++
>  drivers/of/overlay.c |  1 +
>  include/linux/of.h   | 15 ++++++++++-----
>  3 files changed, 40 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/of/dynamic.c b/drivers/of/dynamic.c
> index f4f8ed9b5454..24c97b7a050f 100644
> --- a/drivers/of/dynamic.c
> +++ b/drivers/of/dynamic.c
> @@ -330,6 +330,25 @@ void of_node_release(struct kobject *kobj)
>  	if (!of_node_check_flag(node, OF_DYNAMIC))
>  		return;
>  
> +	if (of_node_check_flag(node, OF_OVERLAY)) {
> +
> +		if (!of_node_check_flag(node, OF_OVERLAY_FREE_CSET)) {

I worry the flags are getting unwieldy.

> +			/* premature refcount of zero, do not free memory */
> +			pr_err("ERROR: memory leak %s() overlay node %pOF before free overlay changeset\n",
> +			       __func__, node);
> +			return;
> +		}
> +
> +		/*
> +		 * If node->properties non-empty then properties were added
> +		 * to this node either by different overlay that has not
> +		 * yet been removed, or by a non-overlay mechanism.
> +		 */
> +		if (node->properties)
> +			pr_err("ERROR: %s() overlay node %pOF contains unexpected properties\n",
> +			       __func__, node);
> +	}
> +
>  	property_list_free(node->properties);
>  	property_list_free(node->deadprops);
>  
> @@ -434,6 +453,16 @@ struct device_node *__of_node_dup(const struct device_node *np,
>  
>  static void __of_changeset_entry_destroy(struct of_changeset_entry *ce)
>  {
> +	if (ce->action == OF_RECONFIG_ATTACH_NODE &&
> +	    of_node_check_flag(ce->np, OF_OVERLAY)) {
> +		if (kref_read(&ce->np->kobj.kref) > 1) {
> +			pr_err("ERROR: memory leak - destroy cset entry: attach overlay node %pOF expected refcount 1 instead of %d.  of_node_get() / of_node_put() are unbalanced for this node.\n",
> +			       ce->np, kref_read(&ce->np->kobj.kref));
> +		} else {
> +			of_node_set_flag(ce->np, OF_OVERLAY_FREE_CSET);
> +		}
> +	}
> +
>  	of_node_put(ce->np);
>  	list_del(&ce->node);
>  	kfree(ce);
> diff --git a/drivers/of/overlay.c b/drivers/of/overlay.c
> index eda57ef12fd0..1176cb4b6e4e 100644
> --- a/drivers/of/overlay.c
> +++ b/drivers/of/overlay.c
> @@ -373,6 +373,7 @@ static int add_changeset_node(struct overlay_changeset *ovcs,
>  			return -ENOMEM;
>  
>  		tchild->parent = target_node;
> +		of_node_set_flag(tchild, OF_OVERLAY);
>  
>  		ret = of_changeset_attach_node(&ovcs->cset, tchild);
>  		if (ret)
> diff --git a/include/linux/of.h b/include/linux/of.h
> index 4d25e4f952d9..aa1dafaec6ae 100644
> --- a/include/linux/of.h
> +++ b/include/linux/of.h
> @@ -138,11 +138,16 @@ static inline void of_node_put(struct device_node *node) { }
>  extern struct device_node *of_stdout;
>  extern raw_spinlock_t devtree_lock;
>  
> -/* flag descriptions (need to be visible even when !CONFIG_OF) */
> -#define OF_DYNAMIC	1 /* node and properties were allocated via kmalloc */
> -#define OF_DETACHED	2 /* node has been detached from the device tree */
> -#define OF_POPULATED	3 /* device already created for the node */
> -#define OF_POPULATED_BUS	4 /* of_platform_populate recursed to children of this node */
> +/*
> + * struct device_node flag descriptions
> + * (need to be visible even when !CONFIG_OF)
> + */
> +#define OF_DYNAMIC		1 /* (and properties) allocated via kmalloc */
> +#define OF_DETACHED		2 /* detached from the device tree */
> +#define OF_POPULATED		3 /* device already created */
> +#define OF_POPULATED_BUS	4 /* platform bus created for children */
> +#define OF_OVERLAY		5 /* allocated for an overlay */
> +#define OF_OVERLAY_FREE_CSET	6 /* in overlay cset being freed */
>  
>  #define OF_BAD_ADDR	((u64)-1)
>  
> -- 
> Frank Rowand <frank.rowand@sony.com>
> 

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [PATCH v4 02/18] of: overlay: add missing of_node_put() after add new node to changeset
  2018-10-16  2:37 ` [PATCH v4 02/18] of: overlay: add missing of_node_put() after add new node to changeset frowand.list
@ 2018-10-18 17:05   ` Rob Herring
  2018-10-18 19:02     ` Frank Rowand
  0 siblings, 1 reply; 33+ messages in thread
From: Rob Herring @ 2018-10-18 17:05 UTC (permalink / raw)
  To: frowand.list
  Cc: devicetree, Alan Tull, linux-fpga, Pantelis Antoniou,
	linux-kernel, Moritz Fischer, Paul Mackerras, linuxppc-dev

On Mon, Oct 15, 2018 at 07:37:22PM -0700, frowand.list@gmail.com wrote:
> From: Frank Rowand <frank.rowand@sony.com>
> 
> The refcount of a newly added overlay node decrements to one
> (instead of zero) when the overlay changeset is destroyed.  This
> change will cause the final decrement be to zero.
> 
> After applying this patch, new validation warnings will be
> reported from the devicetree unittest during boot due to
> a pre-existing devicetree bug.  The warnings will be similar to:
> 
>   OF: ERROR: memory leak of_node_release() overlay node /testcase-data/overlay-node/test-bus/test-unittest4 before free overlay changeset

Same comment on formatting.

> 
> This pre-existing devicetree bug will also trigger a WARN_ONCE() from
> refcount_sub_and_test_checked() when an overlay changeset is
> destroyed without having first been applied.  This scenario occurs
> when an error in the overlay is detected during the overlay changeset
> creation:
> 
>   WARNING: CPU: 0 PID: 1 at lib/refcount.c:187 refcount_sub_and_test_checked+0xa8/0xbc
>   refcount_t: underflow; use-after-free.
> 
>   (unwind_backtrace) from (show_stack+0x10/0x14)
>   (show_stack) from (dump_stack+0x6c/0x8c)
>   (dump_stack) from (__warn+0xdc/0x104)
>   (__warn) from (warn_slowpath_fmt+0x44/0x6c)
>   (warn_slowpath_fmt) from (refcount_sub_and_test_checked+0xa8/0xbc)
>   (refcount_sub_and_test_checked) from (kobject_put+0x24/0x208)
>   (kobject_put) from (of_changeset_destroy+0x2c/0xb4)
>   (of_changeset_destroy) from (free_overlay_changeset+0x1c/0x9c)
>   (free_overlay_changeset) from (of_overlay_remove+0x284/0x2cc)
>   (of_overlay_remove) from (of_unittest_apply_revert_overlay_check.constprop.4+0xf8/0x1e8)
>   (of_unittest_apply_revert_overlay_check.constprop.4) from (of_unittest_overlay+0x960/0xed8)
>   (of_unittest_overlay) from (of_unittest+0x1cc4/0x2138)
>   (of_unittest) from (do_one_initcall+0x4c/0x28c)
>   (do_one_initcall) from (kernel_init_freeable+0x29c/0x378)
>   (kernel_init_freeable) from (kernel_init+0x8/0x110)
>   (kernel_init) from (ret_from_fork+0x14/0x2c)
> 
> Signed-off-by: Frank Rowand <frank.rowand@sony.com>
> ---
>  drivers/of/overlay.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/of/overlay.c b/drivers/of/overlay.c
> index 1176cb4b6e4e..32cfee68f2e3 100644
> --- a/drivers/of/overlay.c
> +++ b/drivers/of/overlay.c
> @@ -379,7 +379,9 @@ static int add_changeset_node(struct overlay_changeset *ovcs,
>  		if (ret)
>  			return ret;
>  
> -		return build_changeset_next_level(ovcs, tchild, node);
> +		ret = build_changeset_next_level(ovcs, tchild, node);
> +		of_node_put(tchild);
> +		return ret;
>  	}
>  
>  	if (node->phandle && tchild->phandle)
> -- 
> Frank Rowand <frank.rowand@sony.com>
> 

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [PATCH v4 04/18] powerpc/pseries: add of_node_put() in dlpar_detach_node()
  2018-10-16  2:37 ` [PATCH v4 04/18] powerpc/pseries: add of_node_put() in dlpar_detach_node() frowand.list
@ 2018-10-18 17:09   ` Rob Herring
  2018-10-18 19:09     ` Frank Rowand
  0 siblings, 1 reply; 33+ messages in thread
From: Rob Herring @ 2018-10-18 17:09 UTC (permalink / raw)
  To: frowand.list
  Cc: devicetree, Alan Tull, linux-fpga, Pantelis Antoniou,
	linux-kernel, Moritz Fischer, Paul Mackerras, linuxppc-dev

On Mon, Oct 15, 2018 at 07:37:24PM -0700, frowand.list@gmail.com wrote:
> From: Frank Rowand <frank.rowand@sony.com>
> 
> "of: overlay: add missing of_node_get() in __of_attach_node_sysfs"
> added a missing of_node_get() to __of_attach_node_sysfs().  This
> results in a refcount imbalance for nodes attached with
> dlpar_attach_node().  The calling sequence from dlpar_attach_node()
> to __of_attach_node_sysfs() is:
> 
>    dlpar_attach_node()
>       of_attach_node()
>          __of_attach_node_sysfs()

IIRC, there's a long standing item in the todo (Grant's) to convert the 
open coded dlpar code. Maybe you want to do that first?

> 
> Signed-off-by: Frank Rowand <frank.rowand@sony.com>
> ---
> 
> ***** UNTESTED.  I need people with the affected PowerPC systems
> *****            (systems that dynamically allocate and deallocate
> *****            devicetree nodes) to test this patch.

Can't we write a test case that does the same thing?

> 
>  arch/powerpc/platforms/pseries/dlpar.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/arch/powerpc/platforms/pseries/dlpar.c b/arch/powerpc/platforms/pseries/dlpar.c
> index a0b20c03f078..e3010b14aea5 100644
> --- a/arch/powerpc/platforms/pseries/dlpar.c
> +++ b/arch/powerpc/platforms/pseries/dlpar.c
> @@ -272,6 +272,8 @@ int dlpar_detach_node(struct device_node *dn)
>  	if (rc)
>  		return rc;
>  
> +	of_node_put(dn);
> +
>  	return 0;
>  }
>  
> -- 
> Frank Rowand <frank.rowand@sony.com>
> 

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [PATCH v4 09/18] of: overlay: validate overlay properties #address-cells and #size-cells
  2018-10-16  2:37 ` [PATCH v4 09/18] of: overlay: validate overlay properties #address-cells and #size-cells frowand.list
@ 2018-10-18 18:13   ` Rob Herring
  2018-10-18 19:13     ` Frank Rowand
  0 siblings, 1 reply; 33+ messages in thread
From: Rob Herring @ 2018-10-18 18:13 UTC (permalink / raw)
  To: frowand.list
  Cc: devicetree, Alan Tull, linux-fpga, Pantelis Antoniou,
	linux-kernel, Moritz Fischer, Paul Mackerras, linuxppc-dev

On Mon, Oct 15, 2018 at 07:37:29PM -0700, frowand.list@gmail.com wrote:
> From: Frank Rowand <frank.rowand@sony.com>
> 
> If overlay properties #address-cells or #size-cells are already in
> the live devicetree for any given node, then the values in the
> overlay must match the values in the live tree.
> 
> If the properties are already in the live tree then there is no
> need to create a changeset entry to add them since they must
> have the same value.  This reduces the memory used by the
> changeset and eliminates a possible memory leak.  This is
> verified by 12 fewer warnings during the devicetree unittest,
> as the possible memory leak warnings about #address-cells and

Still missing the rest...

And what about my other comments too?

> 
> Signed-off-by: Frank Rowand <frank.rowand@sony.com>
> ---
> Changes since v3:
>   - for errors of an overlay changing the value of #size-cells or
>     #address-cells, return -EINVAL so that overlay apply will fail
>   - for errors of an overlay changing the value of #size-cells or
>     #address-cells, make the message more direct.
>     Old message:
>       OF: overlay: ERROR: overlay and/or live tree #size-cells invalid in node /soc/base_fpga_region
>     New message:
>       OF: overlay: ERROR: changing value of /soc/base_fpga_region/#size-cells not allowed
> 
>  drivers/of/overlay.c | 42 +++++++++++++++++++++++++++++++++++++++---
>  1 file changed, 39 insertions(+), 3 deletions(-)

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [PATCH v4 01/18] of: overlay: add tests to validate kfrees from overlay removal
  2018-10-18 17:03   ` Rob Herring
@ 2018-10-18 19:01     ` Frank Rowand
  0 siblings, 0 replies; 33+ messages in thread
From: Frank Rowand @ 2018-10-18 19:01 UTC (permalink / raw)
  To: Rob Herring
  Cc: devicetree, Alan Tull, linux-fpga, Pantelis Antoniou,
	linux-kernel, Moritz Fischer, Paul Mackerras, linuxppc-dev

On 10/18/18 10:03, Rob Herring wrote:
> On Mon, Oct 15, 2018 at 07:37:21PM -0700, frowand.list@gmail.com wrote:
>> From: Frank Rowand <frank.rowand@sony.com>
>>
>> Add checks:
>>   - attempted kfree due to refcount reaching zero before overlay
>>     is removed
>>   - properties linked to an overlay node when the node is removed
>>   - node refcount > one during node removal in a changeset destroy,
>>     if the node was created by the changeset
>>
>> After applying this patch, several validation warnings will be
>> reported from the devicetree unittest during boot due to
>> pre-existing devicetree bugs. The warnings will be similar to:
>>
>>   OF: ERROR: of_node_release() overlay node /testcase-data/overlay-node/test-bus/test-unittest11/test-unittest111 contains unexpected properties
>>   OF: ERROR: memory leak - destroy cset entry: attach overlay node /testcase-data-2/substation@100/hvac-medium-2 expected refcount 1 instead of 2.  of_node_get() / of_node_put() are unbalanced for this node.
> 
> These messages could be formatted more consistently. Put the path either 
> at the beginning (after any prefix) or end. Beginning is more like a 
> compiler error. End puts what the problem is before it's off the edge of 
> the screen. 

The inconsistency makes the word flow more natural, but I agree that
consistency is more important.  I think I can make all the messages
say the problem first, then provide the path at the end.


>> Signed-off-by: Frank Rowand <frank.rowand@sony.com>
>> ---
>> Changes since v3:
>>   - Add expected value of refcount for destroy cset entry error.  Also
>>     explain the cause of the error.
>>
>>  drivers/of/dynamic.c | 29 +++++++++++++++++++++++++++++
>>  drivers/of/overlay.c |  1 +
>>  include/linux/of.h   | 15 ++++++++++-----
>>  3 files changed, 40 insertions(+), 5 deletions(-)
>>
>> diff --git a/drivers/of/dynamic.c b/drivers/of/dynamic.c
>> index f4f8ed9b5454..24c97b7a050f 100644
>> --- a/drivers/of/dynamic.c
>> +++ b/drivers/of/dynamic.c
>> @@ -330,6 +330,25 @@ void of_node_release(struct kobject *kobj)
>>  	if (!of_node_check_flag(node, OF_DYNAMIC))
>>  		return;
>>  
>> +	if (of_node_check_flag(node, OF_OVERLAY)) {
>> +
>> +		if (!of_node_check_flag(node, OF_OVERLAY_FREE_CSET)) {
> 
> I worry the flags are getting unwieldy.

I considered that.  I think we are still ok, and I don't have a better
solution than adding flag values.  (I did have some Rube Goldberg
variations.)


> 
>> +			/* premature refcount of zero, do not free memory */
>> +			pr_err("ERROR: memory leak %s() overlay node %pOF before free overlay changeset\n",
>> +			       __func__, node);
>> +			return;
>> +		}
>> +
>> +		/*
>> +		 * If node->properties non-empty then properties were added
>> +		 * to this node either by different overlay that has not
>> +		 * yet been removed, or by a non-overlay mechanism.
>> +		 */
>> +		if (node->properties)
>> +			pr_err("ERROR: %s() overlay node %pOF contains unexpected properties\n",
>> +			       __func__, node);
>> +	}
>> +
>>  	property_list_free(node->properties);
>>  	property_list_free(node->deadprops);
>>  
>> @@ -434,6 +453,16 @@ struct device_node *__of_node_dup(const struct device_node *np,
>>  
>>  static void __of_changeset_entry_destroy(struct of_changeset_entry *ce)
>>  {
>> +	if (ce->action == OF_RECONFIG_ATTACH_NODE &&
>> +	    of_node_check_flag(ce->np, OF_OVERLAY)) {
>> +		if (kref_read(&ce->np->kobj.kref) > 1) {
>> +			pr_err("ERROR: memory leak - destroy cset entry: attach overlay node %pOF expected refcount 1 instead of %d.  of_node_get() / of_node_put() are unbalanced for this node.\n",
>> +			       ce->np, kref_read(&ce->np->kobj.kref));
>> +		} else {
>> +			of_node_set_flag(ce->np, OF_OVERLAY_FREE_CSET);
>> +		}
>> +	}
>> +
>>  	of_node_put(ce->np);
>>  	list_del(&ce->node);
>>  	kfree(ce);
>> diff --git a/drivers/of/overlay.c b/drivers/of/overlay.c
>> index eda57ef12fd0..1176cb4b6e4e 100644
>> --- a/drivers/of/overlay.c
>> +++ b/drivers/of/overlay.c
>> @@ -373,6 +373,7 @@ static int add_changeset_node(struct overlay_changeset *ovcs,
>>  			return -ENOMEM;
>>  
>>  		tchild->parent = target_node;
>> +		of_node_set_flag(tchild, OF_OVERLAY);
>>  
>>  		ret = of_changeset_attach_node(&ovcs->cset, tchild);
>>  		if (ret)
>> diff --git a/include/linux/of.h b/include/linux/of.h
>> index 4d25e4f952d9..aa1dafaec6ae 100644
>> --- a/include/linux/of.h
>> +++ b/include/linux/of.h
>> @@ -138,11 +138,16 @@ static inline void of_node_put(struct device_node *node) { }
>>  extern struct device_node *of_stdout;
>>  extern raw_spinlock_t devtree_lock;
>>  
>> -/* flag descriptions (need to be visible even when !CONFIG_OF) */
>> -#define OF_DYNAMIC	1 /* node and properties were allocated via kmalloc */
>> -#define OF_DETACHED	2 /* node has been detached from the device tree */
>> -#define OF_POPULATED	3 /* device already created for the node */
>> -#define OF_POPULATED_BUS	4 /* of_platform_populate recursed to children of this node */
>> +/*
>> + * struct device_node flag descriptions
>> + * (need to be visible even when !CONFIG_OF)
>> + */
>> +#define OF_DYNAMIC		1 /* (and properties) allocated via kmalloc */
>> +#define OF_DETACHED		2 /* detached from the device tree */
>> +#define OF_POPULATED		3 /* device already created */
>> +#define OF_POPULATED_BUS	4 /* platform bus created for children */
>> +#define OF_OVERLAY		5 /* allocated for an overlay */
>> +#define OF_OVERLAY_FREE_CSET	6 /* in overlay cset being freed */
>>  
>>  #define OF_BAD_ADDR	((u64)-1)
>>  
>> -- 
>> Frank Rowand <frank.rowand@sony.com>
>>
> 


^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [PATCH v4 02/18] of: overlay: add missing of_node_put() after add new node to changeset
  2018-10-18 17:05   ` Rob Herring
@ 2018-10-18 19:02     ` Frank Rowand
  0 siblings, 0 replies; 33+ messages in thread
From: Frank Rowand @ 2018-10-18 19:02 UTC (permalink / raw)
  To: Rob Herring
  Cc: devicetree, Alan Tull, linux-fpga, Pantelis Antoniou,
	linux-kernel, Moritz Fischer, Paul Mackerras, linuxppc-dev

On 10/18/18 10:05, Rob Herring wrote:
> On Mon, Oct 15, 2018 at 07:37:22PM -0700, frowand.list@gmail.com wrote:
>> From: Frank Rowand <frank.rowand@sony.com>
>>
>> The refcount of a newly added overlay node decrements to one
>> (instead of zero) when the overlay changeset is destroyed.  This
>> change will cause the final decrement be to zero.
>>
>> After applying this patch, new validation warnings will be
>> reported from the devicetree unittest during boot due to
>> a pre-existing devicetree bug.  The warnings will be similar to:
>>
>>   OF: ERROR: memory leak of_node_release() overlay node /testcase-data/overlay-node/test-bus/test-unittest4 before free overlay changeset
> 
> Same comment on formatting.

OK.


>>
>> This pre-existing devicetree bug will also trigger a WARN_ONCE() from
>> refcount_sub_and_test_checked() when an overlay changeset is
>> destroyed without having first been applied.  This scenario occurs
>> when an error in the overlay is detected during the overlay changeset
>> creation:
>>
>>   WARNING: CPU: 0 PID: 1 at lib/refcount.c:187 refcount_sub_and_test_checked+0xa8/0xbc
>>   refcount_t: underflow; use-after-free.
>>
>>   (unwind_backtrace) from (show_stack+0x10/0x14)
>>   (show_stack) from (dump_stack+0x6c/0x8c)
>>   (dump_stack) from (__warn+0xdc/0x104)
>>   (__warn) from (warn_slowpath_fmt+0x44/0x6c)
>>   (warn_slowpath_fmt) from (refcount_sub_and_test_checked+0xa8/0xbc)
>>   (refcount_sub_and_test_checked) from (kobject_put+0x24/0x208)
>>   (kobject_put) from (of_changeset_destroy+0x2c/0xb4)
>>   (of_changeset_destroy) from (free_overlay_changeset+0x1c/0x9c)
>>   (free_overlay_changeset) from (of_overlay_remove+0x284/0x2cc)
>>   (of_overlay_remove) from (of_unittest_apply_revert_overlay_check.constprop.4+0xf8/0x1e8)
>>   (of_unittest_apply_revert_overlay_check.constprop.4) from (of_unittest_overlay+0x960/0xed8)
>>   (of_unittest_overlay) from (of_unittest+0x1cc4/0x2138)
>>   (of_unittest) from (do_one_initcall+0x4c/0x28c)
>>   (do_one_initcall) from (kernel_init_freeable+0x29c/0x378)
>>   (kernel_init_freeable) from (kernel_init+0x8/0x110)
>>   (kernel_init) from (ret_from_fork+0x14/0x2c)
>>
>> Signed-off-by: Frank Rowand <frank.rowand@sony.com>
>> ---
>>  drivers/of/overlay.c | 4 +++-
>>  1 file changed, 3 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/of/overlay.c b/drivers/of/overlay.c
>> index 1176cb4b6e4e..32cfee68f2e3 100644
>> --- a/drivers/of/overlay.c
>> +++ b/drivers/of/overlay.c
>> @@ -379,7 +379,9 @@ static int add_changeset_node(struct overlay_changeset *ovcs,
>>  		if (ret)
>>  			return ret;
>>  
>> -		return build_changeset_next_level(ovcs, tchild, node);
>> +		ret = build_changeset_next_level(ovcs, tchild, node);
>> +		of_node_put(tchild);
>> +		return ret;
>>  	}
>>  
>>  	if (node->phandle && tchild->phandle)
>> -- 
>> Frank Rowand <frank.rowand@sony.com>
>>
> 


^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [PATCH v4 04/18] powerpc/pseries: add of_node_put() in dlpar_detach_node()
  2018-10-18 17:09   ` Rob Herring
@ 2018-10-18 19:09     ` Frank Rowand
  2018-10-19 16:10       ` Rob Herring
  0 siblings, 1 reply; 33+ messages in thread
From: Frank Rowand @ 2018-10-18 19:09 UTC (permalink / raw)
  To: Rob Herring
  Cc: devicetree, Alan Tull, linux-fpga, Pantelis Antoniou,
	linux-kernel, Moritz Fischer, Paul Mackerras, linuxppc-dev

On 10/18/18 10:09, Rob Herring wrote:
> On Mon, Oct 15, 2018 at 07:37:24PM -0700, frowand.list@gmail.com wrote:
>> From: Frank Rowand <frank.rowand@sony.com>
>>
>> "of: overlay: add missing of_node_get() in __of_attach_node_sysfs"
>> added a missing of_node_get() to __of_attach_node_sysfs().  This
>> results in a refcount imbalance for nodes attached with
>> dlpar_attach_node().  The calling sequence from dlpar_attach_node()
>> to __of_attach_node_sysfs() is:
>>
>>    dlpar_attach_node()
>>       of_attach_node()
>>          __of_attach_node_sysfs()
> 
> IIRC, there's a long standing item in the todo (Grant's) to convert the 
> open coded dlpar code. Maybe you want to do that first?

I'd like to avoid extra delays to getting the current (with necesary
fixes) series accepted because the series is rather intrusive and
could have conflicts with other patches.

I'm also worried that I don't have access to any of the systems that
use the dynamic overlay code, and I don't have any way to test the
changes.

Can we encourage the users of this code to convert the open coded
dlpar code?


>>
>> Signed-off-by: Frank Rowand <frank.rowand@sony.com>
>> ---
>>
>> ***** UNTESTED.  I need people with the affected PowerPC systems
>> *****            (systems that dynamically allocate and deallocate
>> *****            devicetree nodes) to test this patch.
> 
> Can't we write a test case that does the same thing?

I could write a simplistic test case, but I'm concerned that
simplistic is not anywhere near sufficient.  And my test case
would reflect the same assumptions I already have when I
wrote this patch.


>>
>>  arch/powerpc/platforms/pseries/dlpar.c | 2 ++
>>  1 file changed, 2 insertions(+)
>>
>> diff --git a/arch/powerpc/platforms/pseries/dlpar.c b/arch/powerpc/platforms/pseries/dlpar.c
>> index a0b20c03f078..e3010b14aea5 100644
>> --- a/arch/powerpc/platforms/pseries/dlpar.c
>> +++ b/arch/powerpc/platforms/pseries/dlpar.c
>> @@ -272,6 +272,8 @@ int dlpar_detach_node(struct device_node *dn)
>>  	if (rc)
>>  		return rc;
>>  
>> +	of_node_put(dn);
>> +
>>  	return 0;
>>  }
>>  
>> -- 
>> Frank Rowand <frank.rowand@sony.com>
>>
> 


^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [PATCH v4 09/18] of: overlay: validate overlay properties #address-cells and #size-cells
  2018-10-18 18:13   ` Rob Herring
@ 2018-10-18 19:13     ` Frank Rowand
  0 siblings, 0 replies; 33+ messages in thread
From: Frank Rowand @ 2018-10-18 19:13 UTC (permalink / raw)
  To: Rob Herring
  Cc: devicetree, Alan Tull, linux-fpga, Pantelis Antoniou,
	linux-kernel, Moritz Fischer, Paul Mackerras, linuxppc-dev

On 10/18/18 11:13, Rob Herring wrote:
> On Mon, Oct 15, 2018 at 07:37:29PM -0700, frowand.list@gmail.com wrote:
>> From: Frank Rowand <frank.rowand@sony.com>
>>
>> If overlay properties #address-cells or #size-cells are already in
>> the live devicetree for any given node, then the values in the
>> overlay must match the values in the live tree.
>>
>> If the properties are already in the live tree then there is no
>> need to create a changeset entry to add them since they must
>> have the same value.  This reduces the memory used by the
>> changeset and eliminates a possible memory leak.  This is
>> verified by 12 fewer warnings during the devicetree unittest,
>> as the possible memory leak warnings about #address-cells and
> 
> Still missing the rest...
> 
> And what about my other comments too?

Apologies, paper bag.  Will fix all.

-Frank

> 
>>
>> Signed-off-by: Frank Rowand <frank.rowand@sony.com>
>> ---
>> Changes since v3:
>>   - for errors of an overlay changing the value of #size-cells or
>>     #address-cells, return -EINVAL so that overlay apply will fail
>>   - for errors of an overlay changing the value of #size-cells or
>>     #address-cells, make the message more direct.
>>     Old message:
>>       OF: overlay: ERROR: overlay and/or live tree #size-cells invalid in node /soc/base_fpga_region
>>     New message:
>>       OF: overlay: ERROR: changing value of /soc/base_fpga_region/#size-cells not allowed
>>
>>  drivers/of/overlay.c | 42 +++++++++++++++++++++++++++++++++++++++---
>>  1 file changed, 39 insertions(+), 3 deletions(-)
> 


^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [PATCH v4 01/18] of: overlay: add tests to validate kfrees from overlay removal
  2018-10-17 21:30   ` Alan Tull
@ 2018-10-18 20:24     ` Alan Tull
  0 siblings, 0 replies; 33+ messages in thread
From: Alan Tull @ 2018-10-18 20:24 UTC (permalink / raw)
  To: Frank Rowand
  Cc: open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
	linux-fpga, Pantelis Antoniou, linux-kernel, Rob Herring,
	Moritz Fischer, Paul Mackerras, linuxppc-dev

On Wed, Oct 17, 2018 at 4:30 PM Alan Tull <atull@kernel.org> wrote:
>
> On Mon, Oct 15, 2018 at 9:39 PM <frowand.list@gmail.com> wrote:
>
> Hi Frank,
>
> >
> > From: Frank Rowand <frank.rowand@sony.com>
> >
> > Add checks:
> >   - attempted kfree due to refcount reaching zero before overlay
> >     is removed
> >   - properties linked to an overlay node when the node is removed
> >   - node refcount > one during node removal in a changeset destroy,
> >     if the node was created by the changeset
> >
> > After applying this patch, several validation warnings will be
> > reported from the devicetree unittest during boot due to
> > pre-existing devicetree bugs. The warnings will be similar to:
> >
> >   OF: ERROR: of_node_release() overlay node /testcase-data/overlay-node/test-bus/test-unittest11/test-unittest111 contains unexpected properties
> >   OF: ERROR: memory leak - destroy cset entry: attach overlay node /testcase-data-2/substation@100/hvac-medium-2 expected refcount 1 instead of 2.  of_node_get() / of_node_put() are unbalanced for this node.
> >
> > Signed-off-by: Frank Rowand <frank.rowand@sony.com>
> > ---
> > Changes since v3:
> >   - Add expected value of refcount for destroy cset entry error.  Also
> >     explain the cause of the error.
> >
> >  drivers/of/dynamic.c | 29 +++++++++++++++++++++++++++++
> >  drivers/of/overlay.c |  1 +
> >  include/linux/of.h   | 15 ++++++++++-----
> >  3 files changed, 40 insertions(+), 5 deletions(-)
> >
> > diff --git a/drivers/of/dynamic.c b/drivers/of/dynamic.c
> > index f4f8ed9b5454..24c97b7a050f 100644
> > --- a/drivers/of/dynamic.c
> > +++ b/drivers/of/dynamic.c
> > @@ -330,6 +330,25 @@ void of_node_release(struct kobject *kobj)
> >         if (!of_node_check_flag(node, OF_DYNAMIC))
> >                 return;
> >
> > +       if (of_node_check_flag(node, OF_OVERLAY)) {
> > +
> > +               if (!of_node_check_flag(node, OF_OVERLAY_FREE_CSET)) {
> > +                       /* premature refcount of zero, do not free memory */
> > +                       pr_err("ERROR: memory leak %s() overlay node %pOF before free overlay changeset\n",
> > +                              __func__, node);
> > +                       return;
> > +               }
> > +
> > +               /*
> > +                * If node->properties non-empty then properties were added
> > +                * to this node either by different overlay that has not
> > +                * yet been removed, or by a non-overlay mechanism.
> > +                */
> > +               if (node->properties)
> > +                       pr_err("ERROR: %s() overlay node %pOF contains unexpected properties\n",
> > +                              __func__, node);
> > +       }
> > +
> >         property_list_free(node->properties);
> >         property_list_free(node->deadprops);
> >
> > @@ -434,6 +453,16 @@ struct device_node *__of_node_dup(const struct device_node *np,
> >
> >  static void __of_changeset_entry_destroy(struct of_changeset_entry *ce)
> >  {
> > +       if (ce->action == OF_RECONFIG_ATTACH_NODE &&
> > +           of_node_check_flag(ce->np, OF_OVERLAY)) {
> > +               if (kref_read(&ce->np->kobj.kref) > 1) {
> > +                       pr_err("ERROR: memory leak - destroy cset entry: attach overlay node %pOF expected refcount 1 instead of %d.  of_node_get() / of_node_put() are unbalanced for this node.\n",
> > +                              ce->np, kref_read(&ce->np->kobj.kref));
>
> Still testing as much as I have time to do.
>
> I'm hitting this error message once when removing an overlay that adds
> several child nodes.  The only node I get the message for was a node
> that added a fixed-clock (the other nodes didn't trigger the error).
> Then even if I edited all the rest of the overlay DTS and removed all
> other child nodes and all references to the clock from other nodes, I
> still got the error.
>
> Removing dtbo: 1-socfpga_arria10_socdk_sdmmc_ghrd_ovl_ext_cfg.dtb
> [   72.032270] OF: ERROR: memory leak - destroy cset entry: attach
> overlay node /soc/base_fpga_region/clk_0 expected refcount 1 instead
> of 2.  of_node_get() / of_node_put() are unbalanced for this node.

Update: with some helpful offline debug patches from Frank, I was able
to find the source of the of_node_get/put unbalance.  The fixed-rate
clock driver calls of_clk_add_provider() when probed but never calls
of_clk_del_provider()

This patchset quite likely will uncover other of_node_get/put
unbalances around the kernel.

Alan

>
> Here's the very stripped down overlay:
>
> /dts-v1/;
> /plugin/;
> / {
>         fragment@0 {
>                 target-path = "/soc/base_fpga_region";
>                 #address-cells = <1>;
>                 #size-cells = <1>;
>
>                 __overlay__ {
>                         external-fpga-config;
>
>                         #address-cells = <1>;
>                         #size-cells = <1>;
>
>                         clk_0: clk_0 {
>                                 compatible = "fixed-clock";
>                                 #clock-cells = <0>;
>                                 clock-frequency = <100000000>;  /* 100.00 MHz */
>                                 clock-output-names = "clk_0-clk";
>                         };
>                 };
>         };
> };

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [PATCH v4 04/18] powerpc/pseries: add of_node_put() in dlpar_detach_node()
  2018-10-18 19:09     ` Frank Rowand
@ 2018-10-19 16:10       ` Rob Herring
  0 siblings, 0 replies; 33+ messages in thread
From: Rob Herring @ 2018-10-19 16:10 UTC (permalink / raw)
  To: Frank Rowand
  Cc: devicetree, Alan Tull, linux-fpga, Pantelis Antoniou,
	linux-kernel, Moritz Fischer, Paul Mackerras, linuxppc-dev

On Thu, Oct 18, 2018 at 2:09 PM Frank Rowand <frowand.list@gmail.com> wrote:
>
> On 10/18/18 10:09, Rob Herring wrote:
> > On Mon, Oct 15, 2018 at 07:37:24PM -0700, frowand.list@gmail.com wrote:
> >> From: Frank Rowand <frank.rowand@sony.com>
> >>
> >> "of: overlay: add missing of_node_get() in __of_attach_node_sysfs"
> >> added a missing of_node_get() to __of_attach_node_sysfs().  This
> >> results in a refcount imbalance for nodes attached with
> >> dlpar_attach_node().  The calling sequence from dlpar_attach_node()
> >> to __of_attach_node_sysfs() is:
> >>
> >>    dlpar_attach_node()
> >>       of_attach_node()
> >>          __of_attach_node_sysfs()
> >
> > IIRC, there's a long standing item in the todo (Grant's) to convert the
> > open coded dlpar code. Maybe you want to do that first?
>
> I'd like to avoid extra delays to getting the current (with necesary
> fixes) series accepted because the series is rather intrusive and
> could have conflicts with other patches.
>
> I'm also worried that I don't have access to any of the systems that
> use the dynamic overlay code, and I don't have any way to test the
> changes.

Mainly I was thinking you are asking them to test changes now, so I
was thinking better to do that once than twice.

Either way is fine though.

> Can we encourage the users of this code to convert the open coded
> dlpar code?

That would be ideal...

Rob

^ permalink raw reply	[flat|nested] 33+ messages in thread

end of thread, other threads:[~2018-10-19 16:13 UTC | newest]

Thread overview: 33+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-10-16  2:37 [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes frowand.list
2018-10-16  2:37 ` [PATCH v4 01/18] of: overlay: add tests to validate kfrees from overlay removal frowand.list
2018-10-17 21:30   ` Alan Tull
2018-10-18 20:24     ` Alan Tull
2018-10-18 17:03   ` Rob Herring
2018-10-18 19:01     ` Frank Rowand
2018-10-16  2:37 ` [PATCH v4 02/18] of: overlay: add missing of_node_put() after add new node to changeset frowand.list
2018-10-18 17:05   ` Rob Herring
2018-10-18 19:02     ` Frank Rowand
2018-10-16  2:37 ` [PATCH v4 03/18] of: overlay: add missing of_node_get() in __of_attach_node_sysfs frowand.list
2018-10-16  2:37 ` [PATCH v4 04/18] powerpc/pseries: add of_node_put() in dlpar_detach_node() frowand.list
2018-10-18 17:09   ` Rob Herring
2018-10-18 19:09     ` Frank Rowand
2018-10-19 16:10       ` Rob Herring
2018-10-16  2:37 ` [PATCH v4 05/18] of: overlay: use prop add changeset entry for property in new nodes frowand.list
2018-10-16  2:37 ` [PATCH v4 06/18] of: overlay: do not duplicate properties from overlay for " frowand.list
2018-10-16  2:37 ` [PATCH v4 07/18] of: dynamic: change type of of_{at, de}tach_node() to void frowand.list
2018-10-16  2:37 ` [PATCH v4 08/18] of: overlay: reorder fields in struct fragment frowand.list
2018-10-16  2:37 ` [PATCH v4 09/18] of: overlay: validate overlay properties #address-cells and #size-cells frowand.list
2018-10-18 18:13   ` Rob Herring
2018-10-18 19:13     ` Frank Rowand
2018-10-16  2:37 ` [PATCH v4 10/18] of: overlay: make all pr_debug() and pr_err() messages unique frowand.list
2018-10-16  2:37 ` [PATCH v4 11/18] of: overlay: test case of two fragments adding same node frowand.list
2018-10-16  2:37 ` [PATCH v4 12/18] of: overlay: check prevents multiple fragments add or delete " frowand.list
2018-10-16  2:37 ` [PATCH v4 13/18] of: overlay: check prevents multiple fragments touching same property frowand.list
2018-10-16  2:37 ` [PATCH v4 14/18] of: unittest: remove unused of_unittest_apply_overlay() argument frowand.list
2018-10-16  2:37 ` [PATCH v4 15/18] of: overlay: set node fields from properties when add new overlay node frowand.list
2018-10-16  2:37 ` [PATCH v4 16/18] of: unittest: allow base devicetree to have symbol metadata frowand.list
2018-10-16  2:37 ` [PATCH v4 17/18] of: unittest: find overlays[] entry by name instead of index frowand.list
2018-10-16  2:37 ` [PATCH v4 18/18] of: unittest: initialize args before calling of_*parse_*() frowand.list
2018-10-16  9:47 ` [PATCH v4 00/18] of: overlay: validation checks, subsequent fixes Michael Ellerman
2018-10-17  3:08   ` Frank Rowand
2018-10-17 21:16     ` Alan Tull

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).