* [PATCH v2] dt-bindings: Fix command line length limit calling dt-mk-schema
@ 2020-04-22 18:57 Rob Herring
2020-04-23 15:08 ` Masahiro Yamada
2020-04-23 16:06 ` Laurent Pinchart
0 siblings, 2 replies; 3+ messages in thread
From: Rob Herring @ 2020-04-22 18:57 UTC (permalink / raw)
To: devicetree; +Cc: linux-kernel, Laurent Pinchart, Masahiro Yamada
As the number of schemas has increased, we're starting to hit the error
"execvp: /bin/sh: Argument list too long". This is due to passing all the
schema files on the command line to dt-mk-schema. It currently is only
with out of tree builds and is intermittent depending on the file path
lengths.
Commit 2ba06cd8565b ("kbuild: Always validate DT binding examples") made
hitting this proplem more likely since the example validation now always
gets the full list of schemas.
Fix this by passing the schema file list in a pipe and using xargs. We end
up doing the find twice, but the time is insignificant compared to the
dt-mk-schema time.
Reported-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Rob Herring <robh@kernel.org>
---
Documentation/devicetree/bindings/Makefile | 18 ++++++++++--------
1 file changed, 10 insertions(+), 8 deletions(-)
diff --git a/Documentation/devicetree/bindings/Makefile b/Documentation/devicetree/bindings/Makefile
index 87c76bdabfe6..7782d9985082 100644
--- a/Documentation/devicetree/bindings/Makefile
+++ b/Documentation/devicetree/bindings/Makefile
@@ -14,16 +14,18 @@ $(obj)/%.example.dts: $(src)/%.yaml FORCE
# Use full schemas when checking %.example.dts
DT_TMP_SCHEMA := $(obj)/processed-schema-examples.yaml
+find_cmd = find $(srctree)/$(src) \( -name '*.yaml' ! \
+ -name 'processed-schema*' ! \
+ -name '*.example.dt.yaml' \)
+
quiet_cmd_mk_schema = SCHEMA $@
- cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@ $(real-prereqs)
+ cmd_mk_schema = rm -f $@ ; \
+ $(if $(DT_MK_SCHEMA_FLAGS), \
+ echo $(real-prereqs), \
+ $(find_cmd)) | \
+ xargs $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) >> $@
-DT_DOCS = $(addprefix $(src)/, \
- $(shell \
- cd $(srctree)/$(src) && \
- find * \( -name '*.yaml' ! \
- -name 'processed-schema*' ! \
- -name '*.example.dt.yaml' \) \
- ))
+DT_DOCS = $(shell $(find_cmd) | sed -e 's|^$(srctree)/||')
DT_SCHEMA_FILES ?= $(DT_DOCS)
--
2.20.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH v2] dt-bindings: Fix command line length limit calling dt-mk-schema
2020-04-22 18:57 [PATCH v2] dt-bindings: Fix command line length limit calling dt-mk-schema Rob Herring
@ 2020-04-23 15:08 ` Masahiro Yamada
2020-04-23 16:06 ` Laurent Pinchart
1 sibling, 0 replies; 3+ messages in thread
From: Masahiro Yamada @ 2020-04-23 15:08 UTC (permalink / raw)
To: Rob Herring; +Cc: DTML, Linux Kernel Mailing List, Laurent Pinchart
On Thu, Apr 23, 2020 at 3:57 AM Rob Herring <robh@kernel.org> wrote:
>
> As the number of schemas has increased, we're starting to hit the error
> "execvp: /bin/sh: Argument list too long". This is due to passing all the
> schema files on the command line to dt-mk-schema. It currently is only
> with out of tree builds and is intermittent depending on the file path
> lengths.
>
> Commit 2ba06cd8565b ("kbuild: Always validate DT binding examples") made
> hitting this proplem more likely since the example validation now always
> gets the full list of schemas.
>
> Fix this by passing the schema file list in a pipe and using xargs. We end
> up doing the find twice, but the time is insignificant compared to the
> dt-mk-schema time.
>
> Reported-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Cc: Masahiro Yamada <masahiroy@kernel.org>
> Signed-off-by: Rob Herring <robh@kernel.org>
Reviewed-by: Masahiro Yamada <masahiroy@kernel.org>
> ---
> Documentation/devicetree/bindings/Makefile | 18 ++++++++++--------
> 1 file changed, 10 insertions(+), 8 deletions(-)
>
> diff --git a/Documentation/devicetree/bindings/Makefile b/Documentation/devicetree/bindings/Makefile
> index 87c76bdabfe6..7782d9985082 100644
> --- a/Documentation/devicetree/bindings/Makefile
> +++ b/Documentation/devicetree/bindings/Makefile
> @@ -14,16 +14,18 @@ $(obj)/%.example.dts: $(src)/%.yaml FORCE
> # Use full schemas when checking %.example.dts
> DT_TMP_SCHEMA := $(obj)/processed-schema-examples.yaml
>
> +find_cmd = find $(srctree)/$(src) \( -name '*.yaml' ! \
> + -name 'processed-schema*' ! \
> + -name '*.example.dt.yaml' \)
> +
> quiet_cmd_mk_schema = SCHEMA $@
> - cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@ $(real-prereqs)
> + cmd_mk_schema = rm -f $@ ; \
> + $(if $(DT_MK_SCHEMA_FLAGS), \
> + echo $(real-prereqs), \
> + $(find_cmd)) | \
> + xargs $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) >> $@
>
> -DT_DOCS = $(addprefix $(src)/, \
> - $(shell \
> - cd $(srctree)/$(src) && \
> - find * \( -name '*.yaml' ! \
> - -name 'processed-schema*' ! \
> - -name '*.example.dt.yaml' \) \
> - ))
> +DT_DOCS = $(shell $(find_cmd) | sed -e 's|^$(srctree)/||')
>
> DT_SCHEMA_FILES ?= $(DT_DOCS)
>
> --
> 2.20.1
>
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH v2] dt-bindings: Fix command line length limit calling dt-mk-schema
2020-04-22 18:57 [PATCH v2] dt-bindings: Fix command line length limit calling dt-mk-schema Rob Herring
2020-04-23 15:08 ` Masahiro Yamada
@ 2020-04-23 16:06 ` Laurent Pinchart
1 sibling, 0 replies; 3+ messages in thread
From: Laurent Pinchart @ 2020-04-23 16:06 UTC (permalink / raw)
To: Rob Herring; +Cc: devicetree, linux-kernel, Masahiro Yamada
Hi Rob,
Thank you for the patch.
On Wed, Apr 22, 2020 at 01:57:08PM -0500, Rob Herring wrote:
> As the number of schemas has increased, we're starting to hit the error
> "execvp: /bin/sh: Argument list too long". This is due to passing all the
> schema files on the command line to dt-mk-schema. It currently is only
> with out of tree builds and is intermittent depending on the file path
> lengths.
>
> Commit 2ba06cd8565b ("kbuild: Always validate DT binding examples") made
> hitting this proplem more likely since the example validation now always
> gets the full list of schemas.
>
> Fix this by passing the schema file list in a pipe and using xargs. We end
> up doing the find twice, but the time is insignificant compared to the
> dt-mk-schema time.
>
> Reported-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Cc: Masahiro Yamada <masahiroy@kernel.org>
> Signed-off-by: Rob Herring <robh@kernel.org>
Tested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
> Documentation/devicetree/bindings/Makefile | 18 ++++++++++--------
> 1 file changed, 10 insertions(+), 8 deletions(-)
>
> diff --git a/Documentation/devicetree/bindings/Makefile b/Documentation/devicetree/bindings/Makefile
> index 87c76bdabfe6..7782d9985082 100644
> --- a/Documentation/devicetree/bindings/Makefile
> +++ b/Documentation/devicetree/bindings/Makefile
> @@ -14,16 +14,18 @@ $(obj)/%.example.dts: $(src)/%.yaml FORCE
> # Use full schemas when checking %.example.dts
> DT_TMP_SCHEMA := $(obj)/processed-schema-examples.yaml
>
> +find_cmd = find $(srctree)/$(src) \( -name '*.yaml' ! \
> + -name 'processed-schema*' ! \
> + -name '*.example.dt.yaml' \)
> +
> quiet_cmd_mk_schema = SCHEMA $@
> - cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@ $(real-prereqs)
> + cmd_mk_schema = rm -f $@ ; \
> + $(if $(DT_MK_SCHEMA_FLAGS), \
> + echo $(real-prereqs), \
> + $(find_cmd)) | \
> + xargs $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) >> $@
>
> -DT_DOCS = $(addprefix $(src)/, \
> - $(shell \
> - cd $(srctree)/$(src) && \
> - find * \( -name '*.yaml' ! \
> - -name 'processed-schema*' ! \
> - -name '*.example.dt.yaml' \) \
> - ))
> +DT_DOCS = $(shell $(find_cmd) | sed -e 's|^$(srctree)/||')
>
> DT_SCHEMA_FILES ?= $(DT_DOCS)
>
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2020-04-23 16:06 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-04-22 18:57 [PATCH v2] dt-bindings: Fix command line length limit calling dt-mk-schema Rob Herring
2020-04-23 15:08 ` Masahiro Yamada
2020-04-23 16:06 ` Laurent Pinchart
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).