* [PATCH V4 0/3] scripts: dtc: Build fdtoverlay @ 2021-01-12 8:28 Viresh Kumar 2021-01-12 8:29 ` [PATCH V4 1/3] scripts: dtc: Add fdtoverlay.c to DTC_SOURCE Viresh Kumar ` (3 more replies) 0 siblings, 4 replies; 15+ messages in thread From: Viresh Kumar @ 2021-01-12 8:28 UTC (permalink / raw) To: Pantelis Antoniou, Frank Rowand, Rob Herring Cc: Viresh Kumar, devicetree, linux-kernel, linux-kbuild, Vincent Guittot, Bill Mills, anmar.oueja, Masahiro Yamada We will start building overlays for platforms soon in the kernel and would need fdtoverlay tool going forward. Lets start fetching and building it. While at it, also remove fdtdump.c file, which isn't used by the kernel. V4: - Don't fetch and build fdtdump.c - Remove fdtdump.c Viresh Kumar (3): scripts: dtc: Add fdtoverlay.c to DTC_SOURCE scripts: dtc: Build fdtoverlay tool scripts: dtc: Remove the unused fdtdump.c file scripts/dtc/Makefile | 6 +- scripts/dtc/fdtdump.c | 163 ------------------------------- scripts/dtc/update-dtc-source.sh | 6 +- 3 files changed, 8 insertions(+), 167 deletions(-) delete mode 100644 scripts/dtc/fdtdump.c -- 2.25.0.rc1.19.g042ed3e048af ^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH V4 1/3] scripts: dtc: Add fdtoverlay.c to DTC_SOURCE 2021-01-12 8:28 [PATCH V4 0/3] scripts: dtc: Build fdtoverlay Viresh Kumar @ 2021-01-12 8:29 ` Viresh Kumar 2021-01-19 16:32 ` Frank Rowand 2021-01-12 8:29 ` [PATCH V4 2/3] scripts: dtc: Build fdtoverlay tool Viresh Kumar ` (2 subsequent siblings) 3 siblings, 1 reply; 15+ messages in thread From: Viresh Kumar @ 2021-01-12 8:29 UTC (permalink / raw) To: Pantelis Antoniou, Frank Rowand, Rob Herring Cc: Viresh Kumar, devicetree, linux-kernel, linux-kbuild, Vincent Guittot, Bill Mills, anmar.oueja, Masahiro Yamada We will start building overlays for platforms soon in the kernel and would need fdtoverlay tool going forward. Lets start fetching it. Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org> --- scripts/dtc/update-dtc-source.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/scripts/dtc/update-dtc-source.sh b/scripts/dtc/update-dtc-source.sh index bc704e2a6a4a..f1c802011e1e 100755 --- a/scripts/dtc/update-dtc-source.sh +++ b/scripts/dtc/update-dtc-source.sh @@ -31,9 +31,9 @@ set -ev DTC_UPSTREAM_PATH=`pwd`/../dtc DTC_LINUX_PATH=`pwd`/scripts/dtc -DTC_SOURCE="checks.c data.c dtc.c dtc.h flattree.c fstree.c livetree.c srcpos.c \ - srcpos.h treesource.c util.c util.h version_gen.h yamltree.c \ - dtc-lexer.l dtc-parser.y" +DTC_SOURCE="checks.c data.c dtc.c dtc.h fdtoverlay.c flattree.c fstree.c \ + livetree.c srcpos.c srcpos.h treesource.c util.c \ + util.h version_gen.h yamltree.c dtc-lexer.l dtc-parser.y" LIBFDT_SOURCE="fdt.c fdt.h fdt_addresses.c fdt_empty_tree.c \ fdt_overlay.c fdt_ro.c fdt_rw.c fdt_strerror.c fdt_sw.c \ fdt_wip.c libfdt.h libfdt_env.h libfdt_internal.h" -- 2.25.0.rc1.19.g042ed3e048af ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH V4 1/3] scripts: dtc: Add fdtoverlay.c to DTC_SOURCE 2021-01-12 8:29 ` [PATCH V4 1/3] scripts: dtc: Add fdtoverlay.c to DTC_SOURCE Viresh Kumar @ 2021-01-19 16:32 ` Frank Rowand 0 siblings, 0 replies; 15+ messages in thread From: Frank Rowand @ 2021-01-19 16:32 UTC (permalink / raw) To: Viresh Kumar, Pantelis Antoniou, Rob Herring Cc: devicetree, linux-kernel, linux-kbuild, Vincent Guittot, Bill Mills, anmar.oueja, Masahiro Yamada On 1/12/21 2:29 AM, Viresh Kumar wrote: > We will start building overlays for platforms soon in the kernel and > would need fdtoverlay tool going forward. Lets start fetching it. > > Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org> > --- > scripts/dtc/update-dtc-source.sh | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) > > diff --git a/scripts/dtc/update-dtc-source.sh b/scripts/dtc/update-dtc-source.sh > index bc704e2a6a4a..f1c802011e1e 100755 > --- a/scripts/dtc/update-dtc-source.sh > +++ b/scripts/dtc/update-dtc-source.sh > @@ -31,9 +31,9 @@ set -ev > DTC_UPSTREAM_PATH=`pwd`/../dtc > DTC_LINUX_PATH=`pwd`/scripts/dtc > > -DTC_SOURCE="checks.c data.c dtc.c dtc.h flattree.c fstree.c livetree.c srcpos.c \ > - srcpos.h treesource.c util.c util.h version_gen.h yamltree.c \ > - dtc-lexer.l dtc-parser.y" > +DTC_SOURCE="checks.c data.c dtc.c dtc.h fdtoverlay.c flattree.c fstree.c \ > + livetree.c srcpos.c srcpos.h treesource.c util.c \ > + util.h version_gen.h yamltree.c dtc-lexer.l dtc-parser.y" > LIBFDT_SOURCE="fdt.c fdt.h fdt_addresses.c fdt_empty_tree.c \ > fdt_overlay.c fdt_ro.c fdt_rw.c fdt_strerror.c fdt_sw.c \ > fdt_wip.c libfdt.h libfdt_env.h libfdt_internal.h" > I made this comment in the v2 email thread. Copying it here since v4 is the current version of the patch series. DTC_SOURCE is for the dtc program. Please add a FDTOVERLAY_SOURCE and related use for the fdtoverlay program. -Frank ^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH V4 2/3] scripts: dtc: Build fdtoverlay tool 2021-01-12 8:28 [PATCH V4 0/3] scripts: dtc: Build fdtoverlay Viresh Kumar 2021-01-12 8:29 ` [PATCH V4 1/3] scripts: dtc: Add fdtoverlay.c to DTC_SOURCE Viresh Kumar @ 2021-01-12 8:29 ` Viresh Kumar 2021-01-19 16:37 ` Frank Rowand 2021-01-12 8:29 ` [PATCH V4 3/3] scripts: dtc: Remove the unused fdtdump.c file Viresh Kumar 2021-01-19 17:12 ` [PATCH V4 0/3] scripts: dtc: Build fdtoverlay Frank Rowand 3 siblings, 1 reply; 15+ messages in thread From: Viresh Kumar @ 2021-01-12 8:29 UTC (permalink / raw) To: Pantelis Antoniou, Frank Rowand, Rob Herring Cc: Viresh Kumar, devicetree, linux-kernel, linux-kbuild, Vincent Guittot, Bill Mills, anmar.oueja, Masahiro Yamada We will start building overlays for platforms soon in the kernel and would need fdtoverlay going forward. Lets start building it. The fdtoverlay program applies (or merges) one ore more overlay dtb blobs to a base dtb blob. The kernel build system would later use fdtoverlay to generate the overlaid blobs based on platform specific configurations. Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org> --- scripts/dtc/Makefile | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/scripts/dtc/Makefile b/scripts/dtc/Makefile index 4852bf44e913..5f19386a49eb 100644 --- a/scripts/dtc/Makefile +++ b/scripts/dtc/Makefile @@ -1,13 +1,17 @@ # SPDX-License-Identifier: GPL-2.0 # scripts/dtc makefile -hostprogs-always-$(CONFIG_DTC) += dtc +hostprogs-always-$(CONFIG_DTC) += dtc fdtoverlay hostprogs-always-$(CHECK_DT_BINDING) += dtc dtc-objs := dtc.o flattree.o fstree.o data.o livetree.o treesource.o \ srcpos.o checks.o util.o dtc-objs += dtc-lexer.lex.o dtc-parser.tab.o +libfdt-objs := fdt.o fdt_ro.o fdt_wip.o fdt_sw.o fdt_rw.o fdt_strerror.o fdt_empty_tree.o fdt_addresses.o fdt_overlay.o +libfdt = $(addprefix libfdt/,$(libfdt-objs)) +fdtoverlay-objs := $(libfdt) fdtoverlay.o util.o + # Source files need to get at the userspace version of libfdt_env.h to compile HOST_EXTRACFLAGS += -I $(srctree)/$(src)/libfdt -- 2.25.0.rc1.19.g042ed3e048af ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH V4 2/3] scripts: dtc: Build fdtoverlay tool 2021-01-12 8:29 ` [PATCH V4 2/3] scripts: dtc: Build fdtoverlay tool Viresh Kumar @ 2021-01-19 16:37 ` Frank Rowand 2021-01-20 15:47 ` Rob Herring 0 siblings, 1 reply; 15+ messages in thread From: Frank Rowand @ 2021-01-19 16:37 UTC (permalink / raw) To: Viresh Kumar, Pantelis Antoniou, Rob Herring Cc: devicetree, linux-kernel, linux-kbuild, Vincent Guittot, Bill Mills, anmar.oueja, Masahiro Yamada Hi Viresh, I made these comments in the v2 patch series. I am copying them here since this is the current version. On 1/12/21 2:29 AM, Viresh Kumar wrote: > We will start building overlays for platforms soon in the kernel and > would need fdtoverlay going forward. Lets start building it. > > The fdtoverlay program applies (or merges) one ore more overlay dtb > blobs to a base dtb blob. The kernel build system would later use > fdtoverlay to generate the overlaid blobs based on platform specific > configurations. > > Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org> > --- > scripts/dtc/Makefile | 6 +++++- > 1 file changed, 5 insertions(+), 1 deletion(-) > > diff --git a/scripts/dtc/Makefile b/scripts/dtc/Makefile > index 4852bf44e913..5f19386a49eb 100644 > --- a/scripts/dtc/Makefile > +++ b/scripts/dtc/Makefile > @@ -1,13 +1,17 @@ > # SPDX-License-Identifier: GPL-2.0 > # scripts/dtc makefile > > -hostprogs-always-$(CONFIG_DTC) += dtc > +hostprogs-always-$(CONFIG_DTC) += dtc fdtoverlay > hostprogs-always-$(CHECK_DT_BINDING) += dtc > > dtc-objs := dtc.o flattree.o fstree.o data.o livetree.o treesource.o \ > srcpos.o checks.o util.o > dtc-objs += dtc-lexer.lex.o dtc-parser.tab.o > # The upstream project builds libfdt as a separate library. We are choosing to # instead directly link the libfdt object files into fdtoverly > +libfdt-objs := fdt.o fdt_ro.o fdt_wip.o fdt_sw.o fdt_rw.o fdt_strerror.o fdt_empty_tree.o fdt_addresses.o fdt_overlay.o > +libfdt = $(addprefix libfdt/,$(libfdt-objs)) > +fdtoverlay-objs := $(libfdt) fdtoverlay.o util.o > + > # Source files need to get at the userspace version of libfdt_env.h to compile > HOST_EXTRACFLAGS += -I $(srctree)/$(src)/libfdt > > In general, I am a proponent of using shared libraries (which the upstream project builds by default) because if a security bug in the library is fixed, it is fixed for all users of the library. In this specific case, I actually prefer the implementation that the patch provides (directly linking the library object files into fdtoverlay, which uses the library) because it is the only user of the library _and_ fdtoverlay will not inadvertently use the system wide libfdt if it happens to be installed (as it is on my system). Any thoughts on this Rob? -Frank ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH V4 2/3] scripts: dtc: Build fdtoverlay tool 2021-01-19 16:37 ` Frank Rowand @ 2021-01-20 15:47 ` Rob Herring 0 siblings, 0 replies; 15+ messages in thread From: Rob Herring @ 2021-01-20 15:47 UTC (permalink / raw) To: Frank Rowand Cc: Viresh Kumar, Pantelis Antoniou, devicetree, linux-kernel, Linux Kbuild mailing list, Vincent Guittot, Bill Mills, Anmar Oueja, Masahiro Yamada On Tue, Jan 19, 2021 at 12:28 PM Frank Rowand <frowand.list@gmail.com> wrote: > > Hi Viresh, > > I made these comments in the v2 patch series. I am copying them here since > this is the current version. > > On 1/12/21 2:29 AM, Viresh Kumar wrote: > > We will start building overlays for platforms soon in the kernel and > > would need fdtoverlay going forward. Lets start building it. > > > > The fdtoverlay program applies (or merges) one ore more overlay dtb > > blobs to a base dtb blob. The kernel build system would later use > > fdtoverlay to generate the overlaid blobs based on platform specific > > configurations. > > > > Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org> > > --- > > scripts/dtc/Makefile | 6 +++++- > > 1 file changed, 5 insertions(+), 1 deletion(-) > > > > diff --git a/scripts/dtc/Makefile b/scripts/dtc/Makefile > > index 4852bf44e913..5f19386a49eb 100644 > > --- a/scripts/dtc/Makefile > > +++ b/scripts/dtc/Makefile > > @@ -1,13 +1,17 @@ > > # SPDX-License-Identifier: GPL-2.0 > > # scripts/dtc makefile > > > > -hostprogs-always-$(CONFIG_DTC) += dtc > > +hostprogs-always-$(CONFIG_DTC) += dtc fdtoverlay > > hostprogs-always-$(CHECK_DT_BINDING) += dtc > > > > dtc-objs := dtc.o flattree.o fstree.o data.o livetree.o treesource.o \ > > srcpos.o checks.o util.o > > dtc-objs += dtc-lexer.lex.o dtc-parser.tab.o > > > > # The upstream project builds libfdt as a separate library. We are choosing to > # instead directly link the libfdt object files into fdtoverly > > > +libfdt-objs := fdt.o fdt_ro.o fdt_wip.o fdt_sw.o fdt_rw.o fdt_strerror.o fdt_empty_tree.o fdt_addresses.o fdt_overlay.o > > +libfdt = $(addprefix libfdt/,$(libfdt-objs)) > > +fdtoverlay-objs := $(libfdt) fdtoverlay.o util.o > > + > > # Source files need to get at the userspace version of libfdt_env.h to compile > > HOST_EXTRACFLAGS += -I $(srctree)/$(src)/libfdt > > > > > > In general, I am a proponent of using shared libraries (which the upstream project > builds by default) because if a security bug in the library is fixed, it is fixed > for all users of the library. > > In this specific case, I actually prefer the implementation that the patch provides > (directly linking the library object files into fdtoverlay, which uses the library) > because it is the only user of the library _and_ fdtoverlay will not inadvertently > use the system wide libfdt if it happens to be installed (as it is on my system). > > Any thoughts on this Rob? I agree. No point in complicating the build to do a library. Rob ^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH V4 3/3] scripts: dtc: Remove the unused fdtdump.c file 2021-01-12 8:28 [PATCH V4 0/3] scripts: dtc: Build fdtoverlay Viresh Kumar 2021-01-12 8:29 ` [PATCH V4 1/3] scripts: dtc: Add fdtoverlay.c to DTC_SOURCE Viresh Kumar 2021-01-12 8:29 ` [PATCH V4 2/3] scripts: dtc: Build fdtoverlay tool Viresh Kumar @ 2021-01-12 8:29 ` Viresh Kumar 2021-01-19 17:12 ` [PATCH V4 0/3] scripts: dtc: Build fdtoverlay Frank Rowand 3 siblings, 0 replies; 15+ messages in thread From: Viresh Kumar @ 2021-01-12 8:29 UTC (permalink / raw) To: Pantelis Antoniou, Frank Rowand, Rob Herring Cc: Viresh Kumar, devicetree, linux-kernel, linux-kbuild, Vincent Guittot, Bill Mills, anmar.oueja, Masahiro Yamada This was copied from external DTC repository long back and isn't used anymore. Over that the dtc tool can be used to generate the dts source back from the dtb. Remove the unused fdtdump.c file. Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org> --- scripts/dtc/fdtdump.c | 163 ------------------------------------------ 1 file changed, 163 deletions(-) delete mode 100644 scripts/dtc/fdtdump.c diff --git a/scripts/dtc/fdtdump.c b/scripts/dtc/fdtdump.c deleted file mode 100644 index 7d460a50b513..000000000000 --- a/scripts/dtc/fdtdump.c +++ /dev/null @@ -1,163 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * fdtdump.c - Contributed by Pantelis Antoniou <pantelis.antoniou AT gmail.com> - */ - -#include <stdint.h> -#include <stdio.h> -#include <stdlib.h> -#include <string.h> -#include <ctype.h> - -#include <fdt.h> -#include <libfdt_env.h> - -#include "util.h" - -#define ALIGN(x, a) (((x) + ((a) - 1)) & ~((a) - 1)) -#define PALIGN(p, a) ((void *)(ALIGN((unsigned long)(p), (a)))) -#define GET_CELL(p) (p += 4, *((const uint32_t *)(p-4))) - -static void print_data(const char *data, int len) -{ - int i; - const char *p = data; - - /* no data, don't print */ - if (len == 0) - return; - - if (util_is_printable_string(data, len)) { - printf(" = \"%s\"", (const char *)data); - } else if ((len % 4) == 0) { - printf(" = <"); - for (i = 0; i < len; i += 4) - printf("0x%08x%s", fdt32_to_cpu(GET_CELL(p)), - i < (len - 4) ? " " : ""); - printf(">"); - } else { - printf(" = ["); - for (i = 0; i < len; i++) - printf("%02x%s", *p++, i < len - 1 ? " " : ""); - printf("]"); - } -} - -static void dump_blob(void *blob) -{ - struct fdt_header *bph = blob; - uint32_t off_mem_rsvmap = fdt32_to_cpu(bph->off_mem_rsvmap); - uint32_t off_dt = fdt32_to_cpu(bph->off_dt_struct); - uint32_t off_str = fdt32_to_cpu(bph->off_dt_strings); - struct fdt_reserve_entry *p_rsvmap = - (struct fdt_reserve_entry *)((char *)blob + off_mem_rsvmap); - const char *p_struct = (const char *)blob + off_dt; - const char *p_strings = (const char *)blob + off_str; - uint32_t version = fdt32_to_cpu(bph->version); - uint32_t totalsize = fdt32_to_cpu(bph->totalsize); - uint32_t tag; - const char *p, *s, *t; - int depth, sz, shift; - int i; - uint64_t addr, size; - - depth = 0; - shift = 4; - - printf("/dts-v1/;\n"); - printf("// magic:\t\t0x%x\n", fdt32_to_cpu(bph->magic)); - printf("// totalsize:\t\t0x%x (%d)\n", totalsize, totalsize); - printf("// off_dt_struct:\t0x%x\n", off_dt); - printf("// off_dt_strings:\t0x%x\n", off_str); - printf("// off_mem_rsvmap:\t0x%x\n", off_mem_rsvmap); - printf("// version:\t\t%d\n", version); - printf("// last_comp_version:\t%d\n", - fdt32_to_cpu(bph->last_comp_version)); - if (version >= 2) - printf("// boot_cpuid_phys:\t0x%x\n", - fdt32_to_cpu(bph->boot_cpuid_phys)); - - if (version >= 3) - printf("// size_dt_strings:\t0x%x\n", - fdt32_to_cpu(bph->size_dt_strings)); - if (version >= 17) - printf("// size_dt_struct:\t0x%x\n", - fdt32_to_cpu(bph->size_dt_struct)); - printf("\n"); - - for (i = 0; ; i++) { - addr = fdt64_to_cpu(p_rsvmap[i].address); - size = fdt64_to_cpu(p_rsvmap[i].size); - if (addr == 0 && size == 0) - break; - - printf("/memreserve/ %llx %llx;\n", - (unsigned long long)addr, (unsigned long long)size); - } - - p = p_struct; - while ((tag = fdt32_to_cpu(GET_CELL(p))) != FDT_END) { - - /* printf("tag: 0x%08x (%d)\n", tag, p - p_struct); */ - - if (tag == FDT_BEGIN_NODE) { - s = p; - p = PALIGN(p + strlen(s) + 1, 4); - - if (*s == '\0') - s = "/"; - - printf("%*s%s {\n", depth * shift, "", s); - - depth++; - continue; - } - - if (tag == FDT_END_NODE) { - depth--; - - printf("%*s};\n", depth * shift, ""); - continue; - } - - if (tag == FDT_NOP) { - printf("%*s// [NOP]\n", depth * shift, ""); - continue; - } - - if (tag != FDT_PROP) { - fprintf(stderr, "%*s ** Unknown tag 0x%08x\n", depth * shift, "", tag); - break; - } - sz = fdt32_to_cpu(GET_CELL(p)); - s = p_strings + fdt32_to_cpu(GET_CELL(p)); - if (version < 16 && sz >= 8) - p = PALIGN(p, 8); - t = p; - - p = PALIGN(p + sz, 4); - - printf("%*s%s", depth * shift, "", s); - print_data(t, sz); - printf(";\n"); - } -} - - -int main(int argc, char *argv[]) -{ - char *buf; - - if (argc < 2) { - fprintf(stderr, "supply input filename\n"); - return 5; - } - - buf = utilfdt_read(argv[1]); - if (buf) - dump_blob(buf); - else - return 10; - - return 0; -} -- 2.25.0.rc1.19.g042ed3e048af ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH V4 0/3] scripts: dtc: Build fdtoverlay 2021-01-12 8:28 [PATCH V4 0/3] scripts: dtc: Build fdtoverlay Viresh Kumar ` (2 preceding siblings ...) 2021-01-12 8:29 ` [PATCH V4 3/3] scripts: dtc: Remove the unused fdtdump.c file Viresh Kumar @ 2021-01-19 17:12 ` Frank Rowand 2021-01-20 5:17 ` Viresh Kumar 3 siblings, 1 reply; 15+ messages in thread From: Frank Rowand @ 2021-01-19 17:12 UTC (permalink / raw) To: Viresh Kumar, Pantelis Antoniou, Rob Herring Cc: devicetree, linux-kernel, linux-kbuild, Vincent Guittot, Bill Mills, anmar.oueja, Masahiro Yamada On 1/12/21 2:28 AM, Viresh Kumar wrote: > We will start building overlays for platforms soon in the kernel and > would need fdtoverlay tool going forward. Lets start fetching and > building it. > > While at it, also remove fdtdump.c file, which isn't used by the kernel. > > V4: > - Don't fetch and build fdtdump.c > - Remove fdtdump.c > > Viresh Kumar (3): > scripts: dtc: Add fdtoverlay.c to DTC_SOURCE > scripts: dtc: Build fdtoverlay tool > scripts: dtc: Remove the unused fdtdump.c file > > scripts/dtc/Makefile | 6 +- > scripts/dtc/fdtdump.c | 163 ------------------------------- > scripts/dtc/update-dtc-source.sh | 6 +- > 3 files changed, 8 insertions(+), 167 deletions(-) > delete mode 100644 scripts/dtc/fdtdump.c > My first inclination was to accept fdtoverlay, as is, from the upstream project. But my experiences debugging use of fdtoverlay against the existing unittest overlay files has me very wary of accepting fdtoverlay in it's current form. As an exmple, adding an overlay that fails to reply results in the following build messages: linux--5.11-rc> make zImage make[1]: Entering directory '/local/frowand_nobackup/src/git_linus/build/dragon_linus_5.11-rc' GEN Makefile CALL /local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/checksyscalls.sh CALL /local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/atomic/check-atomics.sh CHK include/generated/compile.h FDTOVERLAY drivers/of/unittest-data/static_test.dtb Failed to apply 'drivers/of/unittest-data/overlay.dtb': FDT_ERR_NOTFOUND make[4]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/drivers/of/unittest-data/Makefile:96: drivers/of/unittest-data/static_test.dtb] Error 1 make[3]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/Makefile.build:496: drivers/of/unittest-data] Error 2 make[2]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/Makefile.build:496: drivers/of] Error 2 make[1]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/Makefile:1805: drivers] Error 2 make[1]: Leaving directory '/local/frowand_nobackup/src/git_linus/build/dragon_linus_5.11-rc' make: *** [Makefile:185: __sub-make] Error 2 The specific error message (copied from above) is: Failed to apply 'drivers/of/unittest-data/overlay.dtb': FDT_ERR_NOTFOUND which is cryptic and does not even point to the location in the overlay that is problematic. If you look at the source of fdtoverlay / libfdt, you will find that FDT_ERR_NOTFOUND may be generated in one of many places. I do _not_ want to do a full review of fdtoverlay, but I think that it is reasonable to request enhancing fdtoverlay in the parent project to generate usable error messages before enabling fdtoverlay in the Linux kernel tree. fdtoverlay in it's current form adds a potential maintenance burden to me (as the overlay maintainer). I now have the experience of how difficult it was to debug the use of fdtoverlay in the context of the proposed patch to use it with the devicetree unittest overlay .dtb files. -Frank ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH V4 0/3] scripts: dtc: Build fdtoverlay 2021-01-19 17:12 ` [PATCH V4 0/3] scripts: dtc: Build fdtoverlay Frank Rowand @ 2021-01-20 5:17 ` Viresh Kumar 2021-01-22 6:34 ` David Gibson 0 siblings, 1 reply; 15+ messages in thread From: Viresh Kumar @ 2021-01-20 5:17 UTC (permalink / raw) To: Frank Rowand, David Gibson Cc: Pantelis Antoniou, Rob Herring, devicetree, linux-kernel, linux-kbuild, Vincent Guittot, Bill Mills, anmar.oueja, Masahiro Yamada +David. On 19-01-21, 11:12, Frank Rowand wrote: > On 1/12/21 2:28 AM, Viresh Kumar wrote: > > We will start building overlays for platforms soon in the kernel and > > would need fdtoverlay tool going forward. Lets start fetching and > > building it. > > > > While at it, also remove fdtdump.c file, which isn't used by the kernel. > > > > V4: > > - Don't fetch and build fdtdump.c > > - Remove fdtdump.c > > > > Viresh Kumar (3): > > scripts: dtc: Add fdtoverlay.c to DTC_SOURCE > > scripts: dtc: Build fdtoverlay tool > > scripts: dtc: Remove the unused fdtdump.c file > > > > scripts/dtc/Makefile | 6 +- > > scripts/dtc/fdtdump.c | 163 ------------------------------- > > scripts/dtc/update-dtc-source.sh | 6 +- > > 3 files changed, 8 insertions(+), 167 deletions(-) > > delete mode 100644 scripts/dtc/fdtdump.c > > > > My first inclination was to accept fdtoverlay, as is, from the upstream > project. > > But my experiences debugging use of fdtoverlay against the existing > unittest overlay files has me very wary of accepting fdtoverlay in > it's current form. > > As an exmple, adding an overlay that fails to reply results in the > following build messages: > > linux--5.11-rc> make zImage > make[1]: Entering directory '/local/frowand_nobackup/src/git_linus/build/dragon_linus_5.11-rc' > GEN Makefile > CALL /local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/checksyscalls.sh > CALL /local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/atomic/check-atomics.sh > CHK include/generated/compile.h > FDTOVERLAY drivers/of/unittest-data/static_test.dtb > > Failed to apply 'drivers/of/unittest-data/overlay.dtb': FDT_ERR_NOTFOUND > make[4]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/drivers/of/unittest-data/Makefile:96: drivers/of/unittest-data/static_test.dtb] Error 1 > make[3]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/Makefile.build:496: drivers/of/unittest-data] Error 2 > make[2]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/Makefile.build:496: drivers/of] Error 2 > make[1]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/Makefile:1805: drivers] Error 2 > make[1]: Leaving directory '/local/frowand_nobackup/src/git_linus/build/dragon_linus_5.11-rc' > make: *** [Makefile:185: __sub-make] Error 2 > > > The specific error message (copied from above) is: > > Failed to apply 'drivers/of/unittest-data/overlay.dtb': FDT_ERR_NOTFOUND > > which is cryptic and does not even point to the location in the overlay that > is problematic. If you look at the source of fdtoverlay / libfdt, you will > find that FDT_ERR_NOTFOUND may be generated in one of many places. > > I do _not_ want to do a full review of fdtoverlay, but I think that it is > reasonable to request enhancing fdtoverlay in the parent project to generate > usable error messages before enabling fdtoverlay in the Linux kernel tree. > > fdtoverlay in it's current form adds a potential maintenance burden to me > (as the overlay maintainer). I now have the experience of how difficult it > was to debug the use of fdtoverlay in the context of the proposed patch to > use it with the devicetree unittest overlay .dtb files. > > -Frank -- viresh ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH V4 0/3] scripts: dtc: Build fdtoverlay 2021-01-20 5:17 ` Viresh Kumar @ 2021-01-22 6:34 ` David Gibson 2021-01-26 3:42 ` Frank Rowand 0 siblings, 1 reply; 15+ messages in thread From: David Gibson @ 2021-01-22 6:34 UTC (permalink / raw) To: Viresh Kumar Cc: Frank Rowand, Pantelis Antoniou, Rob Herring, devicetree, linux-kernel, linux-kbuild, Vincent Guittot, Bill Mills, anmar.oueja, Masahiro Yamada [-- Attachment #1: Type: text/plain, Size: 4493 bytes --] On Wed, Jan 20, 2021 at 10:47:40AM +0530, Viresh Kumar wrote: > +David. > > On 19-01-21, 11:12, Frank Rowand wrote: > > On 1/12/21 2:28 AM, Viresh Kumar wrote: > > > We will start building overlays for platforms soon in the kernel and > > > would need fdtoverlay tool going forward. Lets start fetching and > > > building it. > > > > > > While at it, also remove fdtdump.c file, which isn't used by the kernel. > > > > > > V4: > > > - Don't fetch and build fdtdump.c > > > - Remove fdtdump.c > > > > > > Viresh Kumar (3): > > > scripts: dtc: Add fdtoverlay.c to DTC_SOURCE > > > scripts: dtc: Build fdtoverlay tool > > > scripts: dtc: Remove the unused fdtdump.c file > > > > > > scripts/dtc/Makefile | 6 +- > > > scripts/dtc/fdtdump.c | 163 ------------------------------- > > > scripts/dtc/update-dtc-source.sh | 6 +- > > > 3 files changed, 8 insertions(+), 167 deletions(-) > > > delete mode 100644 scripts/dtc/fdtdump.c > > > > > > > My first inclination was to accept fdtoverlay, as is, from the upstream > > project. > > > > But my experiences debugging use of fdtoverlay against the existing > > unittest overlay files has me very wary of accepting fdtoverlay in > > it's current form. > > > > As an exmple, adding an overlay that fails to reply results in the > > following build messages: > > > > linux--5.11-rc> make zImage > > make[1]: Entering directory '/local/frowand_nobackup/src/git_linus/build/dragon_linus_5.11-rc' > > GEN Makefile > > CALL /local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/checksyscalls.sh > > CALL /local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/atomic/check-atomics.sh > > CHK include/generated/compile.h > > FDTOVERLAY drivers/of/unittest-data/static_test.dtb > > > > Failed to apply 'drivers/of/unittest-data/overlay.dtb': FDT_ERR_NOTFOUND > > make[4]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/drivers/of/unittest-data/Makefile:96: drivers/of/unittest-data/static_test.dtb] Error 1 > > make[3]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/Makefile.build:496: drivers/of/unittest-data] Error 2 > > make[2]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/Makefile.build:496: drivers/of] Error 2 > > make[1]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/Makefile:1805: drivers] Error 2 > > make[1]: Leaving directory '/local/frowand_nobackup/src/git_linus/build/dragon_linus_5.11-rc' > > make: *** [Makefile:185: __sub-make] Error 2 > > > > > > The specific error message (copied from above) is: > > > > Failed to apply 'drivers/of/unittest-data/overlay.dtb': FDT_ERR_NOTFOUND > > > > which is cryptic and does not even point to the location in the overlay that > > is problematic. If you look at the source of fdtoverlay / libfdt, you will > > find that FDT_ERR_NOTFOUND may be generated in one of many places. > > > > I do _not_ want to do a full review of fdtoverlay, but I think that it is > > reasonable to request enhancing fdtoverlay in the parent project to generate > > usable error messages before enabling fdtoverlay in the Linux kernel tree. That's... actually much harder than it sounds. fdtoverlay is basically a trivial wrapper around the fdt_overlay_apply() function in libfdt. Matching the conventions of the rest of the library, really it's only way to report errors is a single error code. Returning richer errors is not an easy problem in a C library, especially one designed to be usable in embedded systems, without an allocator or much else available. Of course it would be possible to write a friendly command line tool specifically for applying overlays, which could give better errors. fdtoverlay as it stands isn't really that - it was pretty much written just to invoke fdt_overlay_apply() in testcases. > > fdtoverlay in it's current form adds a potential maintenance burden to me > > (as the overlay maintainer). I now have the experience of how difficult it > > was to debug the use of fdtoverlay in the context of the proposed patch to > > use it with the devicetree unittest overlay .dtb files. > > > > -Frank > -- David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH V4 0/3] scripts: dtc: Build fdtoverlay 2021-01-22 6:34 ` David Gibson @ 2021-01-26 3:42 ` Frank Rowand 2021-02-01 4:07 ` David Gibson 0 siblings, 1 reply; 15+ messages in thread From: Frank Rowand @ 2021-01-26 3:42 UTC (permalink / raw) To: David Gibson, Viresh Kumar Cc: Pantelis Antoniou, Rob Herring, devicetree, linux-kernel, linux-kbuild, Vincent Guittot, Bill Mills, anmar.oueja, Masahiro Yamada Hi David, On 1/22/21 12:34 AM, David Gibson wrote: > On Wed, Jan 20, 2021 at 10:47:40AM +0530, Viresh Kumar wrote: >> +David. >> >> On 19-01-21, 11:12, Frank Rowand wrote: >>> On 1/12/21 2:28 AM, Viresh Kumar wrote: >>>> We will start building overlays for platforms soon in the kernel and >>>> would need fdtoverlay tool going forward. Lets start fetching and >>>> building it. >>>> >>>> While at it, also remove fdtdump.c file, which isn't used by the kernel. >>>> >>>> V4: >>>> - Don't fetch and build fdtdump.c >>>> - Remove fdtdump.c >>>> >>>> Viresh Kumar (3): >>>> scripts: dtc: Add fdtoverlay.c to DTC_SOURCE >>>> scripts: dtc: Build fdtoverlay tool >>>> scripts: dtc: Remove the unused fdtdump.c file >>>> >>>> scripts/dtc/Makefile | 6 +- >>>> scripts/dtc/fdtdump.c | 163 ------------------------------- >>>> scripts/dtc/update-dtc-source.sh | 6 +- >>>> 3 files changed, 8 insertions(+), 167 deletions(-) >>>> delete mode 100644 scripts/dtc/fdtdump.c >>>> >>> >>> My first inclination was to accept fdtoverlay, as is, from the upstream >>> project. >>> >>> But my experiences debugging use of fdtoverlay against the existing >>> unittest overlay files has me very wary of accepting fdtoverlay in >>> it's current form. >>> >>> As an exmple, adding an overlay that fails to reply results in the >>> following build messages: >>> >>> linux--5.11-rc> make zImage >>> make[1]: Entering directory '/local/frowand_nobackup/src/git_linus/build/dragon_linus_5.11-rc' >>> GEN Makefile >>> CALL /local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/checksyscalls.sh >>> CALL /local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/atomic/check-atomics.sh >>> CHK include/generated/compile.h >>> FDTOVERLAY drivers/of/unittest-data/static_test.dtb >>> >>> Failed to apply 'drivers/of/unittest-data/overlay.dtb': FDT_ERR_NOTFOUND >>> make[4]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/drivers/of/unittest-data/Makefile:96: drivers/of/unittest-data/static_test.dtb] Error 1 >>> make[3]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/Makefile.build:496: drivers/of/unittest-data] Error 2 >>> make[2]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/Makefile.build:496: drivers/of] Error 2 >>> make[1]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/Makefile:1805: drivers] Error 2 >>> make[1]: Leaving directory '/local/frowand_nobackup/src/git_linus/build/dragon_linus_5.11-rc' >>> make: *** [Makefile:185: __sub-make] Error 2 >>> >>> >>> The specific error message (copied from above) is: >>> >>> Failed to apply 'drivers/of/unittest-data/overlay.dtb': FDT_ERR_NOTFOUND >>> >>> which is cryptic and does not even point to the location in the overlay that >>> is problematic. If you look at the source of fdtoverlay / libfdt, you will >>> find that FDT_ERR_NOTFOUND may be generated in one of many places. >>> >>> I do _not_ want to do a full review of fdtoverlay, but I think that it is >>> reasonable to request enhancing fdtoverlay in the parent project to generate >>> usable error messages before enabling fdtoverlay in the Linux kernel tree. > > That's... actually much harder than it sounds. fdtoverlay is > basically a trivial wrapper around the fdt_overlay_apply() function in > libfdt. Matching the conventions of the rest of the library, really > it's only way to report errors is a single error code. > > Returning richer errors is not an easy problem in a C library, > especially one designed to be usable in embedded systems, without an > allocator or much else available. > > Of course it would be possible to write a friendly command line tool > specifically for applying overlays, which could give better errors. > fdtoverlay as it stands isn't really that - it was pretty much written > just to invoke fdt_overlay_apply() in testcases. Thank you for providing that context. I do not know if there is a way to enable the code that is currently in libfdt to both be useful as an embedded library (for example, U-boot seems to often have a need to keep memory usage very small) and also be part of a tool with effective warning and error messages. Before having looked at libfdt only at a cursory level while debugging the proposed use of fdtoverlay in Linux, my first thought was that maybe it would be possible to add warning and error messages within "#ifdef" blocks, or other ways that cause the error code to _not_ be compiled as part of library version of libfdt, but only be compiled as part of fdtoverlay _when built in the Linux kernel_ (noting that the proposed Linux patch builds the libfdt files as part of the fdtoverlay compile instead of as a discrete library). After looking at the libfdt source a tiny bit more carefully, I would probably shoot down this suggestion, as it makes the source code uglier and harder to understand and maintain for the primary purpose of being an embedded library. Do you have any thoughts on how warning and error messages could be added, or if it is even possible? Or maybe your suggestion of writing a "friendly command line tool specifically for applying overlays" is the path that Viresh should pursue? -Frank > >>> fdtoverlay in it's current form adds a potential maintenance burden to me >>> (as the overlay maintainer). I now have the experience of how difficult it >>> was to debug the use of fdtoverlay in the context of the proposed patch to >>> use it with the devicetree unittest overlay .dtb files. >>> >>> -Frank >> > ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH V4 0/3] scripts: dtc: Build fdtoverlay 2021-01-26 3:42 ` Frank Rowand @ 2021-02-01 4:07 ` David Gibson 2021-02-03 9:51 ` Viresh Kumar 2021-02-04 14:25 ` Rob Herring 0 siblings, 2 replies; 15+ messages in thread From: David Gibson @ 2021-02-01 4:07 UTC (permalink / raw) To: Frank Rowand Cc: Viresh Kumar, Pantelis Antoniou, Rob Herring, devicetree, linux-kernel, linux-kbuild, Vincent Guittot, Bill Mills, anmar.oueja, Masahiro Yamada [-- Attachment #1: Type: text/plain, Size: 6028 bytes --] On Mon, Jan 25, 2021 at 09:42:21PM -0600, Frank Rowand wrote: > Hi David, > > On 1/22/21 12:34 AM, David Gibson wrote: > > On Wed, Jan 20, 2021 at 10:47:40AM +0530, Viresh Kumar wrote: > >> +David. > >> > >> On 19-01-21, 11:12, Frank Rowand wrote: > >>> On 1/12/21 2:28 AM, Viresh Kumar wrote: > >>>> We will start building overlays for platforms soon in the kernel and > >>>> would need fdtoverlay tool going forward. Lets start fetching and > >>>> building it. > >>>> > >>>> While at it, also remove fdtdump.c file, which isn't used by the kernel. > >>>> > >>>> V4: > >>>> - Don't fetch and build fdtdump.c > >>>> - Remove fdtdump.c > >>>> > >>>> Viresh Kumar (3): > >>>> scripts: dtc: Add fdtoverlay.c to DTC_SOURCE > >>>> scripts: dtc: Build fdtoverlay tool > >>>> scripts: dtc: Remove the unused fdtdump.c file > >>>> > >>>> scripts/dtc/Makefile | 6 +- > >>>> scripts/dtc/fdtdump.c | 163 ------------------------------- > >>>> scripts/dtc/update-dtc-source.sh | 6 +- > >>>> 3 files changed, 8 insertions(+), 167 deletions(-) > >>>> delete mode 100644 scripts/dtc/fdtdump.c > >>>> > >>> > >>> My first inclination was to accept fdtoverlay, as is, from the upstream > >>> project. > >>> > >>> But my experiences debugging use of fdtoverlay against the existing > >>> unittest overlay files has me very wary of accepting fdtoverlay in > >>> it's current form. > >>> > >>> As an exmple, adding an overlay that fails to reply results in the > >>> following build messages: > >>> > >>> linux--5.11-rc> make zImage > >>> make[1]: Entering directory '/local/frowand_nobackup/src/git_linus/build/dragon_linus_5.11-rc' > >>> GEN Makefile > >>> CALL /local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/checksyscalls.sh > >>> CALL /local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/atomic/check-atomics.sh > >>> CHK include/generated/compile.h > >>> FDTOVERLAY drivers/of/unittest-data/static_test.dtb > >>> > >>> Failed to apply 'drivers/of/unittest-data/overlay.dtb': FDT_ERR_NOTFOUND > >>> make[4]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/drivers/of/unittest-data/Makefile:96: drivers/of/unittest-data/static_test.dtb] Error 1 > >>> make[3]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/Makefile.build:496: drivers/of/unittest-data] Error 2 > >>> make[2]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/Makefile.build:496: drivers/of] Error 2 > >>> make[1]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/Makefile:1805: drivers] Error 2 > >>> make[1]: Leaving directory '/local/frowand_nobackup/src/git_linus/build/dragon_linus_5.11-rc' > >>> make: *** [Makefile:185: __sub-make] Error 2 > >>> > >>> > >>> The specific error message (copied from above) is: > >>> > >>> Failed to apply 'drivers/of/unittest-data/overlay.dtb': FDT_ERR_NOTFOUND > >>> > >>> which is cryptic and does not even point to the location in the overlay that > >>> is problematic. If you look at the source of fdtoverlay / libfdt, you will > >>> find that FDT_ERR_NOTFOUND may be generated in one of many places. > >>> > >>> I do _not_ want to do a full review of fdtoverlay, but I think that it is > >>> reasonable to request enhancing fdtoverlay in the parent project to generate > >>> usable error messages before enabling fdtoverlay in the Linux kernel tree. > > > > > That's... actually much harder than it sounds. fdtoverlay is > > basically a trivial wrapper around the fdt_overlay_apply() function in > > libfdt. Matching the conventions of the rest of the library, really > > it's only way to report errors is a single error code. > > > > Returning richer errors is not an easy problem in a C library, > > especially one designed to be usable in embedded systems, without an > > allocator or much else available. > > > > Of course it would be possible to write a friendly command line tool > > specifically for applying overlays, which could give better errors. > > fdtoverlay as it stands isn't really that - it was pretty much written > > just to invoke fdt_overlay_apply() in testcases. > > Thank you for providing that context. > > I do not know if there is a way to enable the code that is currently in libfdt > to both be useful as an embedded library (for example, U-boot seems to often > have a need to keep memory usage very small) and also be part of a tool with > effective warning and error messages. Yeah, I don't know either. > Before having looked at libfdt only at a cursory level while debugging the proposed > use of fdtoverlay in Linux, my first thought was that maybe it would be possible > to add warning and error messages within "#ifdef" blocks, or other ways that > cause the error code to _not_ be compiled as part of library version of libfdt, > but only be compiled as part of fdtoverlay _when built in the Linux kernel_ > (noting that the proposed Linux patch builds the libfdt files as part of > the fdtoverlay compile instead of as a discrete library). After looking at > the libfdt source a tiny bit more carefully, I would probably shoot down this > suggestion, as it makes the source code uglier and harder to understand and > maintain for the primary purpose of being an embedded library. Oof. That sounds really ugly, but maybe it could be pulled off. > Do you have any thoughts on how warning and error messages could be added, > or if it is even possible? Or maybe your suggestion of writing a "friendly > command line tool specifically for applying overlays" is the path that > Viresh should pursue? I think at this stage it's a matter of trying a few approaches and seeing what works out. -- David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH V4 0/3] scripts: dtc: Build fdtoverlay 2021-02-01 4:07 ` David Gibson @ 2021-02-03 9:51 ` Viresh Kumar 2021-02-04 14:25 ` Rob Herring 1 sibling, 0 replies; 15+ messages in thread From: Viresh Kumar @ 2021-02-03 9:51 UTC (permalink / raw) To: David Gibson Cc: Frank Rowand, Pantelis Antoniou, Rob Herring, devicetree, linux-kernel, linux-kbuild, Vincent Guittot, Bill Mills, anmar.oueja, Masahiro Yamada On 01-02-21, 15:07, David Gibson wrote: > On Mon, Jan 25, 2021 at 09:42:21PM -0600, Frank Rowand wrote: > > Before having looked at libfdt only at a cursory level while debugging the proposed > > use of fdtoverlay in Linux, my first thought was that maybe it would be possible > > to add warning and error messages within "#ifdef" blocks, or other ways that > > cause the error code to _not_ be compiled as part of library version of libfdt, > > but only be compiled as part of fdtoverlay _when built in the Linux kernel_ > > (noting that the proposed Linux patch builds the libfdt files as part of > > the fdtoverlay compile instead of as a discrete library). After looking at > > the libfdt source a tiny bit more carefully, I would probably shoot down this > > suggestion, as it makes the source code uglier and harder to understand and > > maintain for the primary purpose of being an embedded library. > > Oof. That sounds really ugly, but maybe it could be pulled off. I started looking at this and I was able to get to a not so ugly solution. Do this in dtc: -------------------------8<------------------------- --- dtc.h | 6 ++++++ fdtoverlay.c | 2 ++ 2 files changed, 8 insertions(+) diff --git a/dtc.h b/dtc.h index d3e82fb8e3db..cc1e591b3f8c 100644 --- a/dtc.h +++ b/dtc.h @@ -29,6 +29,12 @@ #define debug(...) #endif +#ifdef VERBOSE +#define pr_err(...) fprintf(stderr, __VA_ARGS__) +#else +#define pr_err(...) +#endif + #define DEFAULT_FDT_VERSION 17 /* diff --git a/fdtoverlay.c b/fdtoverlay.c index 5350af65679f..28ceac0d8079 100644 --- a/fdtoverlay.c +++ b/fdtoverlay.c @@ -16,6 +16,7 @@ #include <libfdt.h> +#include "dtc.h" #include "util.h" #define BUF_INCREMENT 65536 @@ -76,6 +77,7 @@ static void *apply_one(char *base, const char *overlay, size_t *buf_len, if (ret) { fprintf(stderr, "\nFailed to apply '%s': %s\n", name, fdt_strerror(ret)); + pr_err("New error\n"); goto fail; } -------------------------8<------------------------- And do this in kernel: -------------------------8<------------------------- diff --git a/scripts/dtc/Makefile b/scripts/dtc/Makefile index c8c21e0f2531..9dafb9773f06 100644 --- a/scripts/dtc/Makefile +++ b/scripts/dtc/Makefile @@ -13,6 +13,7 @@ dtc-objs += dtc-lexer.lex.o dtc-parser.tab.o libfdt-objs := fdt.o fdt_ro.o fdt_wip.o fdt_sw.o fdt_rw.o fdt_strerror.o fdt_empty_tree.o fdt_addresses.o fdt_overlay.o libfdt = $(addprefix libfdt/,$(libfdt-objs)) fdtoverlay-objs := $(libfdt) fdtoverlay.o util.o +HOSTCFLAGS_fdtoverlay.o := -DVERBOSE # Source files need to get at the userspace version of libfdt_env.h to compile HOST_EXTRACFLAGS += -I $(srctree)/$(src)/libfdt -------------------------8<------------------------- Will that be acceptable ? With this we can add as many error messages to libfdt without affecting any other users of it other than Linux. -- viresh ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH V4 0/3] scripts: dtc: Build fdtoverlay 2021-02-01 4:07 ` David Gibson 2021-02-03 9:51 ` Viresh Kumar @ 2021-02-04 14:25 ` Rob Herring 2021-02-22 6:17 ` David Gibson 1 sibling, 1 reply; 15+ messages in thread From: Rob Herring @ 2021-02-04 14:25 UTC (permalink / raw) To: David Gibson Cc: Frank Rowand, Viresh Kumar, Pantelis Antoniou, devicetree, linux-kernel, Linux Kbuild mailing list, Vincent Guittot, Bill Mills, Anmar Oueja, Masahiro Yamada On Sun, Jan 31, 2021 at 10:39 PM David Gibson <david@gibson.dropbear.id.au> wrote: > > On Mon, Jan 25, 2021 at 09:42:21PM -0600, Frank Rowand wrote: > > Hi David, > > > > On 1/22/21 12:34 AM, David Gibson wrote: > > > On Wed, Jan 20, 2021 at 10:47:40AM +0530, Viresh Kumar wrote: > > >> +David. > > >> > > >> On 19-01-21, 11:12, Frank Rowand wrote: > > >>> On 1/12/21 2:28 AM, Viresh Kumar wrote: > > >>>> We will start building overlays for platforms soon in the kernel and > > >>>> would need fdtoverlay tool going forward. Lets start fetching and > > >>>> building it. > > >>>> > > >>>> While at it, also remove fdtdump.c file, which isn't used by the kernel. > > >>>> > > >>>> V4: > > >>>> - Don't fetch and build fdtdump.c > > >>>> - Remove fdtdump.c > > >>>> > > >>>> Viresh Kumar (3): > > >>>> scripts: dtc: Add fdtoverlay.c to DTC_SOURCE > > >>>> scripts: dtc: Build fdtoverlay tool > > >>>> scripts: dtc: Remove the unused fdtdump.c file > > >>>> > > >>>> scripts/dtc/Makefile | 6 +- > > >>>> scripts/dtc/fdtdump.c | 163 ------------------------------- > > >>>> scripts/dtc/update-dtc-source.sh | 6 +- > > >>>> 3 files changed, 8 insertions(+), 167 deletions(-) > > >>>> delete mode 100644 scripts/dtc/fdtdump.c > > >>>> > > >>> > > >>> My first inclination was to accept fdtoverlay, as is, from the upstream > > >>> project. > > >>> > > >>> But my experiences debugging use of fdtoverlay against the existing > > >>> unittest overlay files has me very wary of accepting fdtoverlay in > > >>> it's current form. > > >>> > > >>> As an exmple, adding an overlay that fails to reply results in the > > >>> following build messages: > > >>> > > >>> linux--5.11-rc> make zImage > > >>> make[1]: Entering directory '/local/frowand_nobackup/src/git_linus/build/dragon_linus_5.11-rc' > > >>> GEN Makefile > > >>> CALL /local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/checksyscalls.sh > > >>> CALL /local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/atomic/check-atomics.sh > > >>> CHK include/generated/compile.h > > >>> FDTOVERLAY drivers/of/unittest-data/static_test.dtb > > >>> > > >>> Failed to apply 'drivers/of/unittest-data/overlay.dtb': FDT_ERR_NOTFOUND > > >>> make[4]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/drivers/of/unittest-data/Makefile:96: drivers/of/unittest-data/static_test.dtb] Error 1 > > >>> make[3]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/Makefile.build:496: drivers/of/unittest-data] Error 2 > > >>> make[2]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/Makefile.build:496: drivers/of] Error 2 > > >>> make[1]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/Makefile:1805: drivers] Error 2 > > >>> make[1]: Leaving directory '/local/frowand_nobackup/src/git_linus/build/dragon_linus_5.11-rc' > > >>> make: *** [Makefile:185: __sub-make] Error 2 > > >>> > > >>> > > >>> The specific error message (copied from above) is: > > >>> > > >>> Failed to apply 'drivers/of/unittest-data/overlay.dtb': FDT_ERR_NOTFOUND > > >>> > > >>> which is cryptic and does not even point to the location in the overlay that > > >>> is problematic. If you look at the source of fdtoverlay / libfdt, you will > > >>> find that FDT_ERR_NOTFOUND may be generated in one of many places. > > >>> > > >>> I do _not_ want to do a full review of fdtoverlay, but I think that it is > > >>> reasonable to request enhancing fdtoverlay in the parent project to generate > > >>> usable error messages before enabling fdtoverlay in the Linux kernel tree. > > > > > > > > That's... actually much harder than it sounds. fdtoverlay is > > > basically a trivial wrapper around the fdt_overlay_apply() function in > > > libfdt. Matching the conventions of the rest of the library, really > > > it's only way to report errors is a single error code. > > > > > > Returning richer errors is not an easy problem in a C library, > > > especially one designed to be usable in embedded systems, without an > > > allocator or much else available. > > > > > > Of course it would be possible to write a friendly command line tool > > > specifically for applying overlays, which could give better errors. > > > fdtoverlay as it stands isn't really that - it was pretty much written > > > just to invoke fdt_overlay_apply() in testcases. > > > > Thank you for providing that context. > > > > I do not know if there is a way to enable the code that is currently in libfdt > > to both be useful as an embedded library (for example, U-boot seems to often > > have a need to keep memory usage very small) and also be part of a tool with > > effective warning and error messages. > > Yeah, I don't know either. > > > Before having looked at libfdt only at a cursory level while debugging the proposed > > use of fdtoverlay in Linux, my first thought was that maybe it would be possible > > to add warning and error messages within "#ifdef" blocks, or other ways that > > cause the error code to _not_ be compiled as part of library version of libfdt, > > but only be compiled as part of fdtoverlay _when built in the Linux kernel_ > > (noting that the proposed Linux patch builds the libfdt files as part of > > the fdtoverlay compile instead of as a discrete library). After looking at > > the libfdt source a tiny bit more carefully, I would probably shoot down this > > suggestion, as it makes the source code uglier and harder to understand and > > maintain for the primary purpose of being an embedded library. > > Oof. That sounds really ugly, but maybe it could be pulled off. > > > Do you have any thoughts on how warning and error messages could be added, > > or if it is even possible? Or maybe your suggestion of writing a "friendly > > command line tool specifically for applying overlays" is the path that > > Viresh should pursue? > > I think at this stage it's a matter of trying a few approaches and > seeing what works out. Another way would be applying overlays to dtc's live tree. This could apply overlays from dts in addition to dtb. It could be a plug-in if we ever get that finished up. The downside of this is not testing libfdt's code and possible differences between 2 implementations. Rob ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH V4 0/3] scripts: dtc: Build fdtoverlay 2021-02-04 14:25 ` Rob Herring @ 2021-02-22 6:17 ` David Gibson 0 siblings, 0 replies; 15+ messages in thread From: David Gibson @ 2021-02-22 6:17 UTC (permalink / raw) To: Rob Herring Cc: Frank Rowand, Viresh Kumar, Pantelis Antoniou, devicetree, linux-kernel, Linux Kbuild mailing list, Vincent Guittot, Bill Mills, Anmar Oueja, Masahiro Yamada [-- Attachment #1: Type: text/plain, Size: 7790 bytes --] On Thu, Feb 04, 2021 at 08:25:23AM -0600, Rob Herring wrote: > On Sun, Jan 31, 2021 at 10:39 PM David Gibson > <david@gibson.dropbear.id.au> wrote: > > > > On Mon, Jan 25, 2021 at 09:42:21PM -0600, Frank Rowand wrote: > > > Hi David, > > > > > > On 1/22/21 12:34 AM, David Gibson wrote: > > > > On Wed, Jan 20, 2021 at 10:47:40AM +0530, Viresh Kumar wrote: > > > >> +David. > > > >> > > > >> On 19-01-21, 11:12, Frank Rowand wrote: > > > >>> On 1/12/21 2:28 AM, Viresh Kumar wrote: > > > >>>> We will start building overlays for platforms soon in the kernel and > > > >>>> would need fdtoverlay tool going forward. Lets start fetching and > > > >>>> building it. > > > >>>> > > > >>>> While at it, also remove fdtdump.c file, which isn't used by the kernel. > > > >>>> > > > >>>> V4: > > > >>>> - Don't fetch and build fdtdump.c > > > >>>> - Remove fdtdump.c > > > >>>> > > > >>>> Viresh Kumar (3): > > > >>>> scripts: dtc: Add fdtoverlay.c to DTC_SOURCE > > > >>>> scripts: dtc: Build fdtoverlay tool > > > >>>> scripts: dtc: Remove the unused fdtdump.c file > > > >>>> > > > >>>> scripts/dtc/Makefile | 6 +- > > > >>>> scripts/dtc/fdtdump.c | 163 ------------------------------- > > > >>>> scripts/dtc/update-dtc-source.sh | 6 +- > > > >>>> 3 files changed, 8 insertions(+), 167 deletions(-) > > > >>>> delete mode 100644 scripts/dtc/fdtdump.c > > > >>>> > > > >>> > > > >>> My first inclination was to accept fdtoverlay, as is, from the upstream > > > >>> project. > > > >>> > > > >>> But my experiences debugging use of fdtoverlay against the existing > > > >>> unittest overlay files has me very wary of accepting fdtoverlay in > > > >>> it's current form. > > > >>> > > > >>> As an exmple, adding an overlay that fails to reply results in the > > > >>> following build messages: > > > >>> > > > >>> linux--5.11-rc> make zImage > > > >>> make[1]: Entering directory '/local/frowand_nobackup/src/git_linus/build/dragon_linus_5.11-rc' > > > >>> GEN Makefile > > > >>> CALL /local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/checksyscalls.sh > > > >>> CALL /local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/atomic/check-atomics.sh > > > >>> CHK include/generated/compile.h > > > >>> FDTOVERLAY drivers/of/unittest-data/static_test.dtb > > > >>> > > > >>> Failed to apply 'drivers/of/unittest-data/overlay.dtb': FDT_ERR_NOTFOUND > > > >>> make[4]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/drivers/of/unittest-data/Makefile:96: drivers/of/unittest-data/static_test.dtb] Error 1 > > > >>> make[3]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/Makefile.build:496: drivers/of/unittest-data] Error 2 > > > >>> make[2]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/scripts/Makefile.build:496: drivers/of] Error 2 > > > >>> make[1]: *** [/local/frowand_nobackup/src/git_linus/linux--5.11-rc/Makefile:1805: drivers] Error 2 > > > >>> make[1]: Leaving directory '/local/frowand_nobackup/src/git_linus/build/dragon_linus_5.11-rc' > > > >>> make: *** [Makefile:185: __sub-make] Error 2 > > > >>> > > > >>> > > > >>> The specific error message (copied from above) is: > > > >>> > > > >>> Failed to apply 'drivers/of/unittest-data/overlay.dtb': FDT_ERR_NOTFOUND > > > >>> > > > >>> which is cryptic and does not even point to the location in the overlay that > > > >>> is problematic. If you look at the source of fdtoverlay / libfdt, you will > > > >>> find that FDT_ERR_NOTFOUND may be generated in one of many places. > > > >>> > > > >>> I do _not_ want to do a full review of fdtoverlay, but I think that it is > > > >>> reasonable to request enhancing fdtoverlay in the parent project to generate > > > >>> usable error messages before enabling fdtoverlay in the Linux kernel tree. > > > > > > > > > > > That's... actually much harder than it sounds. fdtoverlay is > > > > basically a trivial wrapper around the fdt_overlay_apply() function in > > > > libfdt. Matching the conventions of the rest of the library, really > > > > it's only way to report errors is a single error code. > > > > > > > > Returning richer errors is not an easy problem in a C library, > > > > especially one designed to be usable in embedded systems, without an > > > > allocator or much else available. > > > > > > > > Of course it would be possible to write a friendly command line tool > > > > specifically for applying overlays, which could give better errors. > > > > fdtoverlay as it stands isn't really that - it was pretty much written > > > > just to invoke fdt_overlay_apply() in testcases. > > > > > > Thank you for providing that context. > > > > > > I do not know if there is a way to enable the code that is currently in libfdt > > > to both be useful as an embedded library (for example, U-boot seems to often > > > have a need to keep memory usage very small) and also be part of a tool with > > > effective warning and error messages. > > > > Yeah, I don't know either. > > > > > Before having looked at libfdt only at a cursory level while debugging the proposed > > > use of fdtoverlay in Linux, my first thought was that maybe it would be possible > > > to add warning and error messages within "#ifdef" blocks, or other ways that > > > cause the error code to _not_ be compiled as part of library version of libfdt, > > > but only be compiled as part of fdtoverlay _when built in the Linux kernel_ > > > (noting that the proposed Linux patch builds the libfdt files as part of > > > the fdtoverlay compile instead of as a discrete library). After looking at > > > the libfdt source a tiny bit more carefully, I would probably shoot down this > > > suggestion, as it makes the source code uglier and harder to understand and > > > maintain for the primary purpose of being an embedded library. > > > > Oof. That sounds really ugly, but maybe it could be pulled off. > > > > > Do you have any thoughts on how warning and error messages could be added, > > > or if it is even possible? Or maybe your suggestion of writing a "friendly > > > command line tool specifically for applying overlays" is the path that > > > Viresh should pursue? > > > > I think at this stage it's a matter of trying a few approaches and > > seeing what works out. > > Another way would be applying overlays to dtc's live tree. This could > apply overlays from dts in addition to dtb. It could be a plug-in if > we ever get that finished up. This is actually a really interesting idea, because in a sense dtc already *does* apply overlays. It's just that it effectively resolves as it is parsing, rather than realizing separate overlay objects then merging as a separate step. I would actually like to change that, so that it *does* explicitly produce a chain of overlays internally. That has advantages for the checking code, because some checks make sense to apply to individual overlay fragments, but some only make sense on a fully resolved tree. As a bonus, it could handle this use case. Unlike libfdt, dtc is a much more normal userspace program and adding extra verbose debugging is no realy problem. It probably is more work in the short term, though. > The downside of this is not testing libfdt's code and possible > differences between 2 implementations. That can be mitigated by having a bunch of examples in the testsuite where we cross compare fdtoverlay's output with dtc's. -- David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2021-02-22 6:18 UTC | newest] Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-01-12 8:28 [PATCH V4 0/3] scripts: dtc: Build fdtoverlay Viresh Kumar 2021-01-12 8:29 ` [PATCH V4 1/3] scripts: dtc: Add fdtoverlay.c to DTC_SOURCE Viresh Kumar 2021-01-19 16:32 ` Frank Rowand 2021-01-12 8:29 ` [PATCH V4 2/3] scripts: dtc: Build fdtoverlay tool Viresh Kumar 2021-01-19 16:37 ` Frank Rowand 2021-01-20 15:47 ` Rob Herring 2021-01-12 8:29 ` [PATCH V4 3/3] scripts: dtc: Remove the unused fdtdump.c file Viresh Kumar 2021-01-19 17:12 ` [PATCH V4 0/3] scripts: dtc: Build fdtoverlay Frank Rowand 2021-01-20 5:17 ` Viresh Kumar 2021-01-22 6:34 ` David Gibson 2021-01-26 3:42 ` Frank Rowand 2021-02-01 4:07 ` David Gibson 2021-02-03 9:51 ` Viresh Kumar 2021-02-04 14:25 ` Rob Herring 2021-02-22 6:17 ` David Gibson
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).