All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/4] dt-bindings: fix 'make clean' and improve dt-schema check rules
@ 2020-06-25 17:04 Masahiro Yamada
  2020-06-25 17:04 ` [PATCH 1/4] dt-bindings: fix error in 'make clean' after 'make dt_binding_check' Masahiro Yamada
                   ` (3 more replies)
  0 siblings, 4 replies; 13+ messages in thread
From: Masahiro Yamada @ 2020-06-25 17:04 UTC (permalink / raw)
  To: Rob Herring, devicetree
  Cc: Frank Rowand, linux-kbuild, Masahiro Yamada, Michal Marek, linux-kernel




Masahiro Yamada (4):
  dt-bindings: fix error in 'make clean' after 'make dt_binding_check'
  dt-bindings: do not build processed-schema.yaml for 'make
    dt_binding_check'
  dt-bindings: copy process-schema-examples.yaml to process-schema.yaml
  dt-bindings: split DT schema check rules

 .gitignore                                 |  1 +
 Documentation/devicetree/bindings/Makefile | 35 ++++++++++++++++------
 Makefile                                   |  2 +-
 scripts/Makefile.build                     |  2 ++
 scripts/Makefile.lib                       | 27 +++++++++--------
 5 files changed, 44 insertions(+), 23 deletions(-)

-- 
2.25.1


^ permalink raw reply	[flat|nested] 13+ messages in thread

* [PATCH 1/4] dt-bindings: fix error in 'make clean' after 'make dt_binding_check'
  2020-06-25 17:04 [PATCH 0/4] dt-bindings: fix 'make clean' and improve dt-schema check rules Masahiro Yamada
@ 2020-06-25 17:04 ` Masahiro Yamada
  2020-06-29 21:24   ` Rob Herring
  2023-06-16 19:45   ` Eugeniu Rosca
  2020-06-25 17:04 ` [PATCH 2/4] dt-bindings: do not build processed-schema.yaml for " Masahiro Yamada
                   ` (2 subsequent siblings)
  3 siblings, 2 replies; 13+ messages in thread
From: Masahiro Yamada @ 2020-06-25 17:04 UTC (permalink / raw)
  To: Rob Herring, devicetree
  Cc: Frank Rowand, linux-kbuild, Masahiro Yamada, linux-kernel

We are having more and more schema files.

Commit 8b6b80218b01 ("dt-bindings: Fix command line length limit
calling dt-mk-schema") fixed the 'Argument list too long' error of
the schema checks, but the same error happens while cleaning too.

'make clean' after 'make dt_binding_check' fails as follows:

  $ make dt_binding_check
    [ snip ]
  $ make clean
  make[2]: execvp: /bin/sh: Argument list too long
  make[2]: *** [scripts/Makefile.clean:52: __clean] Error 127
  make[1]: *** [scripts/Makefile.clean:66: Documentation/devicetree/bindings] Error 2
  make: *** [Makefile:1763: _clean_Documentation] Error 2

'make dt_binding_check' generates so many .example.dts, .dt.yaml files,
which are passed to the 'rm' command when you run 'make clean'.

I added a small hack to use the 'find' command to clean up most of the
build artifacts before they are processed by scripts/Makefile.clean

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

 Documentation/devicetree/bindings/Makefile | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/Documentation/devicetree/bindings/Makefile b/Documentation/devicetree/bindings/Makefile
index a63898954068..f20d234aec46 100644
--- a/Documentation/devicetree/bindings/Makefile
+++ b/Documentation/devicetree/bindings/Makefile
@@ -53,3 +53,8 @@ $(obj)/processed-schema.yaml: $(DT_SCHEMA_FILES) check_dtschema_version FORCE
 	$(call if_changed,mk_schema)
 
 extra-y += processed-schema.yaml
+
+# Hack: avoid 'Argument list too long' error for 'make clean'. Remove most of
+# build artifacts here before they are processed by scripts/Makefile.clean
+clean-files = $(shell find $(obj) \( -name '*.example.dts' -o \
+			-name '*.example.dt.yaml' \) -delete 2>/dev/null)
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH 2/4] dt-bindings: do not build processed-schema.yaml for 'make dt_binding_check'
  2020-06-25 17:04 [PATCH 0/4] dt-bindings: fix 'make clean' and improve dt-schema check rules Masahiro Yamada
  2020-06-25 17:04 ` [PATCH 1/4] dt-bindings: fix error in 'make clean' after 'make dt_binding_check' Masahiro Yamada
@ 2020-06-25 17:04 ` Masahiro Yamada
  2020-06-29 21:24   ` Rob Herring
  2020-06-25 17:04 ` [PATCH 3/4] dt-bindings: copy process-schema-examples.yaml to process-schema.yaml Masahiro Yamada
  2020-06-25 17:04 ` [PATCH 4/4] dt-bindings: split DT schema check rules Masahiro Yamada
  3 siblings, 1 reply; 13+ messages in thread
From: Masahiro Yamada @ 2020-06-25 17:04 UTC (permalink / raw)
  To: Rob Herring, devicetree
  Cc: Frank Rowand, linux-kbuild, Masahiro Yamada, linux-kernel

Currently, processed-schema.yaml is always built, but it is actually
used only for 'make dtbs_check'.

'make dt_binding_check' uses processed-schema-example.yaml instead.

Build processed-schema.yaml only for 'make dtbs_check'.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

 Documentation/devicetree/bindings/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Documentation/devicetree/bindings/Makefile b/Documentation/devicetree/bindings/Makefile
index f20d234aec46..aa243328ed43 100644
--- a/Documentation/devicetree/bindings/Makefile
+++ b/Documentation/devicetree/bindings/Makefile
@@ -52,7 +52,7 @@ $(obj)/processed-schema.yaml: DT_MK_SCHEMA_FLAGS := $(DT_MK_SCHEMA_USERONLY_FLAG
 $(obj)/processed-schema.yaml: $(DT_SCHEMA_FILES) check_dtschema_version FORCE
 	$(call if_changed,mk_schema)
 
-extra-y += processed-schema.yaml
+extra-$(CHECK_DTBS) += processed-schema.yaml
 
 # Hack: avoid 'Argument list too long' error for 'make clean'. Remove most of
 # build artifacts here before they are processed by scripts/Makefile.clean
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH 3/4] dt-bindings: copy process-schema-examples.yaml to process-schema.yaml
  2020-06-25 17:04 [PATCH 0/4] dt-bindings: fix 'make clean' and improve dt-schema check rules Masahiro Yamada
  2020-06-25 17:04 ` [PATCH 1/4] dt-bindings: fix error in 'make clean' after 'make dt_binding_check' Masahiro Yamada
  2020-06-25 17:04 ` [PATCH 2/4] dt-bindings: do not build processed-schema.yaml for " Masahiro Yamada
@ 2020-06-25 17:04 ` Masahiro Yamada
  2020-06-29 21:25   ` Rob Herring
  2020-06-25 17:04 ` [PATCH 4/4] dt-bindings: split DT schema check rules Masahiro Yamada
  3 siblings, 1 reply; 13+ messages in thread
From: Masahiro Yamada @ 2020-06-25 17:04 UTC (permalink / raw)
  To: Rob Herring, devicetree
  Cc: Frank Rowand, linux-kbuild, Masahiro Yamada, Michal Marek, linux-kernel

There are two processed schema files:

 - processed-schema-examples.yaml

    Used for 'make dt_binding_check'. This is always a full schema.

 - processed-schema.yaml

    Used for 'make dtbs_check'. This may be a full schema, or a smaller
    subset if DT_SCHEMA_FILES is given by a user.

If DT_SCHEMA_FILES is not specified, they are the same. You can copy
the former to the latter instead of running dt-mk-schema twice. This
saves the cpu time a lot when you do 'make dt_binding_check dtbs_check'
because building the full schema takes a couple of seconds.

If DT_SCHEMA_FILES is specified, processed-schema.yaml is generated
based on the specified yaml files.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

 Documentation/devicetree/bindings/Makefile | 28 +++++++++++++++-------
 scripts/Makefile.lib                       |  3 +++
 2 files changed, 23 insertions(+), 8 deletions(-)

diff --git a/Documentation/devicetree/bindings/Makefile b/Documentation/devicetree/bindings/Makefile
index aa243328ed43..6aac57588e66 100644
--- a/Documentation/devicetree/bindings/Makefile
+++ b/Documentation/devicetree/bindings/Makefile
@@ -2,7 +2,6 @@
 DT_DOC_CHECKER ?= dt-doc-validate
 DT_EXTRACT_EX ?= dt-extract-example
 DT_MK_SCHEMA ?= dt-mk-schema
-DT_MK_SCHEMA_USERONLY_FLAG := $(if $(DT_SCHEMA_FILES), -u)
 
 DT_SCHEMA_MIN_VERSION = 2020.5
 
@@ -35,12 +34,6 @@ quiet_cmd_mk_schema = SCHEMA  $@
 
 DT_DOCS = $(shell $(find_cmd) | sed -e 's|^$(srctree)/||')
 
-DT_SCHEMA_FILES ?= $(DT_DOCS)
-
-extra-$(CHECK_DT_BINDING) += $(patsubst $(src)/%.yaml,%.example.dts, $(DT_SCHEMA_FILES))
-extra-$(CHECK_DT_BINDING) += $(patsubst $(src)/%.yaml,%.example.dt.yaml, $(DT_SCHEMA_FILES))
-extra-$(CHECK_DT_BINDING) += processed-schema-examples.yaml
-
 override DTC_FLAGS := \
 	-Wno-avoid_unnecessary_addr_size \
 	-Wno-graph_child_address
@@ -48,10 +41,29 @@ override DTC_FLAGS := \
 $(obj)/processed-schema-examples.yaml: $(DT_DOCS) check_dtschema_version FORCE
 	$(call if_changed,mk_schema)
 
-$(obj)/processed-schema.yaml: DT_MK_SCHEMA_FLAGS := $(DT_MK_SCHEMA_USERONLY_FLAG)
+ifeq ($(DT_SCHEMA_FILES),)
+
+# Unless DT_SCHEMA_FILES is specified, use the full schema for dtbs_check too.
+# Just copy processed-schema-examples.yaml
+
+$(obj)/processed-schema.yaml: $(obj)/processed-schema-examples.yaml FORCE
+	$(call if_changed,copy)
+
+DT_SCHEMA_FILES = $(DT_DOCS)
+
+else
+
+# If DT_SCHEMA_FILES is specified, use it for processed-schema.yaml
+
+$(obj)/processed-schema.yaml: DT_MK_SCHEMA_FLAGS := -u
 $(obj)/processed-schema.yaml: $(DT_SCHEMA_FILES) check_dtschema_version FORCE
 	$(call if_changed,mk_schema)
 
+endif
+
+extra-$(CHECK_DT_BINDING) += $(patsubst $(src)/%.yaml,%.example.dts, $(DT_SCHEMA_FILES))
+extra-$(CHECK_DT_BINDING) += $(patsubst $(src)/%.yaml,%.example.dt.yaml, $(DT_SCHEMA_FILES))
+extra-$(CHECK_DT_BINDING) += processed-schema-examples.yaml
 extra-$(CHECK_DTBS) += processed-schema.yaml
 
 # Hack: avoid 'Argument list too long' error for 'make clean'. Remove most of
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 99ac59c59826..fa5022133a25 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -212,6 +212,9 @@ $(foreach m, $(notdir $1), \
 	$(addprefix $(obj)/, $(foreach s, $3, $($(m:%$(strip $2)=%$(s)))))))
 endef
 
+quiet_cmd_copy = COPY    $@
+      cmd_copy = cp $< $@
+
 # Shipped files
 # ===========================================================================
 
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH 4/4] dt-bindings: split DT schema check rules
  2020-06-25 17:04 [PATCH 0/4] dt-bindings: fix 'make clean' and improve dt-schema check rules Masahiro Yamada
                   ` (2 preceding siblings ...)
  2020-06-25 17:04 ` [PATCH 3/4] dt-bindings: copy process-schema-examples.yaml to process-schema.yaml Masahiro Yamada
@ 2020-06-25 17:04 ` Masahiro Yamada
  2020-06-29 20:49   ` Rob Herring
  3 siblings, 1 reply; 13+ messages in thread
From: Masahiro Yamada @ 2020-06-25 17:04 UTC (permalink / raw)
  To: Rob Herring, devicetree
  Cc: Frank Rowand, linux-kbuild, Masahiro Yamada, Michal Marek, linux-kernel

When building %.dt.yaml from %.dts, two things happen in a row:

 [1] Run DTC to convert %.dts into %.dt.yaml

 [2] Run dt-validate against %.dt.yaml

Currently, when any .yaml schema file is updated, processed-schema.yaml
is regenerated, then both [1] and [2] are rerun for all .dts files.

Actually, we do not need to rerun [1] since the original .dts is not
updated.

Split the rule to avoid unneeded .dts/.dt.yaml conversion.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

 .gitignore                                 |  1 +
 Documentation/devicetree/bindings/Makefile |  4 ++--
 Makefile                                   |  2 +-
 scripts/Makefile.build                     |  2 ++
 scripts/Makefile.lib                       | 24 ++++++++++------------
 5 files changed, 17 insertions(+), 16 deletions(-)

diff --git a/.gitignore b/.gitignore
index 87b9dd8a163b..a1941faf7233 100644
--- a/.gitignore
+++ b/.gitignore
@@ -16,6 +16,7 @@
 *.bin
 *.bz2
 *.c.[012]*.*
+*.dt.checked
 *.dt.yaml
 *.dtb
 *.dtb.S
diff --git a/Documentation/devicetree/bindings/Makefile b/Documentation/devicetree/bindings/Makefile
index 6aac57588e66..b46910eb7cf4 100644
--- a/Documentation/devicetree/bindings/Makefile
+++ b/Documentation/devicetree/bindings/Makefile
@@ -62,11 +62,11 @@ $(obj)/processed-schema.yaml: $(DT_SCHEMA_FILES) check_dtschema_version FORCE
 endif
 
 extra-$(CHECK_DT_BINDING) += $(patsubst $(src)/%.yaml,%.example.dts, $(DT_SCHEMA_FILES))
-extra-$(CHECK_DT_BINDING) += $(patsubst $(src)/%.yaml,%.example.dt.yaml, $(DT_SCHEMA_FILES))
+extra-$(CHECK_DT_BINDING) += $(patsubst $(src)/%.yaml,%.example.dt.checked, $(DT_SCHEMA_FILES))
 extra-$(CHECK_DT_BINDING) += processed-schema-examples.yaml
 extra-$(CHECK_DTBS) += processed-schema.yaml
 
 # Hack: avoid 'Argument list too long' error for 'make clean'. Remove most of
 # build artifacts here before they are processed by scripts/Makefile.clean
 clean-files = $(shell find $(obj) \( -name '*.example.dts' -o \
-			-name '*.example.dt.yaml' \) -delete 2>/dev/null)
+			-name '*.example.dt.checked' \) -delete 2>/dev/null)
diff --git a/Makefile b/Makefile
index ac2c61c37a73..0fc778b656d1 100644
--- a/Makefile
+++ b/Makefile
@@ -1774,7 +1774,7 @@ clean: $(clean-dirs)
 		-o -name '*.lex.c' -o -name '*.tab.[ch]' \
 		-o -name '*.asn1.[ch]' \
 		-o -name '*.symtypes' -o -name 'modules.order' \
-		-o -name '.tmp_*.o.*' \
+		-o -name '.tmp_*.o.*' -o -name '*.dt.checked' \
 		-o -name '*.c.[012]*.*' \
 		-o -name '*.ll' \
 		-o -name '*.gcno' \) -type f -print | xargs rm -f
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 2e8810b7e5ed..ca24c3077fef 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -442,10 +442,12 @@ intermediate_targets = $(foreach sfx, $(2), \
 					$(filter %$(strip $(1)), $(targets))))
 # %.asn1.o <- %.asn1.[ch] <- %.asn1
 # %.dtb.o <- %.dtb.S <- %.dtb <- %.dts
+# %.dt.checked <- %.dt.yaml <- %.dts
 # %.lex.o <- %.lex.c <- %.l
 # %.tab.o <- %.tab.[ch] <- %.y
 targets += $(call intermediate_targets, .asn1.o, .asn1.c .asn1.h) \
 	   $(call intermediate_targets, .dtb.o, .dtb.S .dtb) \
+	   $(call intermediate_targets, .dt.checked, .dt.yaml) \
 	   $(call intermediate_targets, .lex.o, .lex.c) \
 	   $(call intermediate_targets, .tab.o, .tab.c .tab.h)
 
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index fa5022133a25..d72a7cf79f63 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -74,8 +74,8 @@ extra-y				+= $(dtb-y)
 extra-$(CONFIG_OF_ALL_DTBS)	+= $(dtb-)
 
 ifneq ($(CHECK_DTBS),)
-extra-y += $(patsubst %.dtb,%.dt.yaml, $(dtb-y))
-extra-$(CONFIG_OF_ALL_DTBS) += $(patsubst %.dtb,%.dt.yaml, $(dtb-))
+extra-y += $(patsubst %.dtb,%.dt.checked, $(dtb-y))
+extra-$(CONFIG_OF_ALL_DTBS) += $(patsubst %.dtb,%.dt.checked, $(dtb-))
 endif
 
 # Add subdir path
@@ -308,26 +308,24 @@ cmd_dtc = mkdir -p $(dir ${dtc-tmp}) ; \
 		-d $(depfile).dtc.tmp $(dtc-tmp) ; \
 	cat $(depfile).pre.tmp $(depfile).dtc.tmp > $(depfile)
 
+dtc-tmp = $(subst $(comma),_,$(dot-target).dts.tmp)
+
 $(obj)/%.dtb: $(src)/%.dts $(DTC) FORCE
 	$(call if_changed_dep,dtc)
 
+$(obj)/%.dt.yaml: $(src)/%.dts $(DTC) FORCE
+	$(call if_changed_dep,dtc)
+
 DT_CHECKER ?= dt-validate
 DT_BINDING_DIR := Documentation/devicetree/bindings
 # DT_TMP_SCHEMA may be overridden from Documentation/devicetree/bindings/Makefile
 DT_TMP_SCHEMA ?= $(objtree)/$(DT_BINDING_DIR)/processed-schema.yaml
 
-quiet_cmd_dtb_check =	CHECK   $@
-      cmd_dtb_check =	$(DT_CHECKER) -u $(srctree)/$(DT_BINDING_DIR) -p $(DT_TMP_SCHEMA) $@
+quiet_cmd_dt_check = CHECK   $<
+      cmd_dt_check = $(DT_CHECKER) -u $(srctree)/$(DT_BINDING_DIR) -p $(DT_TMP_SCHEMA) $<; touch $@
 
-define rule_dtc
-	$(call cmd_and_fixdep,dtc)
-	$(call cmd,dtb_check)
-endef
-
-$(obj)/%.dt.yaml: $(src)/%.dts $(DTC) $(DT_TMP_SCHEMA) FORCE
-	$(call if_changed_rule,dtc,yaml)
-
-dtc-tmp = $(subst $(comma),_,$(dot-target).dts.tmp)
+$(obj)/%.dt.checked: $(obj)/%.dt.yaml $(DT_TMP_SCHEMA) FORCE
+	$(call if_changed,dt_check)
 
 # Bzip2
 # ---------------------------------------------------------------------------
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 13+ messages in thread

* Re: [PATCH 4/4] dt-bindings: split DT schema check rules
  2020-06-25 17:04 ` [PATCH 4/4] dt-bindings: split DT schema check rules Masahiro Yamada
@ 2020-06-29 20:49   ` Rob Herring
  2020-06-30  1:51     ` Masahiro Yamada
  0 siblings, 1 reply; 13+ messages in thread
From: Rob Herring @ 2020-06-29 20:49 UTC (permalink / raw)
  To: Masahiro Yamada
  Cc: devicetree, Frank Rowand, Linux Kbuild mailing list,
	Michal Marek, linux-kernel

On Thu, Jun 25, 2020 at 11:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> When building %.dt.yaml from %.dts, two things happen in a row:
>
>  [1] Run DTC to convert %.dts into %.dt.yaml
>
>  [2] Run dt-validate against %.dt.yaml
>
> Currently, when any .yaml schema file is updated, processed-schema.yaml
> is regenerated, then both [1] and [2] are rerun for all .dts files.
>
> Actually, we do not need to rerun [1] since the original .dts is not
> updated.

I have plans (and an intern working on it) to integrate the schema
checks into dtc. That's going to make turning on the schema checks
just a flag to dtc. I'm not sure if adding the complexity here is
worth it as I'd expect much of this patch to go away again.

Is there any negative impact on the absolute clean build time? I'm
more concerned about that than optimizing rerunning.

Rob

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH 1/4] dt-bindings: fix error in 'make clean' after 'make dt_binding_check'
  2020-06-25 17:04 ` [PATCH 1/4] dt-bindings: fix error in 'make clean' after 'make dt_binding_check' Masahiro Yamada
@ 2020-06-29 21:24   ` Rob Herring
  2023-06-16 19:45   ` Eugeniu Rosca
  1 sibling, 0 replies; 13+ messages in thread
From: Rob Herring @ 2020-06-29 21:24 UTC (permalink / raw)
  To: Masahiro Yamada
  Cc: linux-kernel, linux-kbuild, devicetree, Rob Herring, Frank Rowand

On Fri, 26 Jun 2020 02:04:31 +0900, Masahiro Yamada wrote:
> We are having more and more schema files.
> 
> Commit 8b6b80218b01 ("dt-bindings: Fix command line length limit
> calling dt-mk-schema") fixed the 'Argument list too long' error of
> the schema checks, but the same error happens while cleaning too.
> 
> 'make clean' after 'make dt_binding_check' fails as follows:
> 
>   $ make dt_binding_check
>     [ snip ]
>   $ make clean
>   make[2]: execvp: /bin/sh: Argument list too long
>   make[2]: *** [scripts/Makefile.clean:52: __clean] Error 127
>   make[1]: *** [scripts/Makefile.clean:66: Documentation/devicetree/bindings] Error 2
>   make: *** [Makefile:1763: _clean_Documentation] Error 2
> 
> 'make dt_binding_check' generates so many .example.dts, .dt.yaml files,
> which are passed to the 'rm' command when you run 'make clean'.
> 
> I added a small hack to use the 'find' command to clean up most of the
> build artifacts before they are processed by scripts/Makefile.clean
> 
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---
> 
>  Documentation/devicetree/bindings/Makefile | 5 +++++
>  1 file changed, 5 insertions(+)
> 

Applied, thanks!

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH 2/4] dt-bindings: do not build processed-schema.yaml for 'make dt_binding_check'
  2020-06-25 17:04 ` [PATCH 2/4] dt-bindings: do not build processed-schema.yaml for " Masahiro Yamada
@ 2020-06-29 21:24   ` Rob Herring
  0 siblings, 0 replies; 13+ messages in thread
From: Rob Herring @ 2020-06-29 21:24 UTC (permalink / raw)
  To: Masahiro Yamada
  Cc: Rob Herring, Frank Rowand, devicetree, linux-kernel, linux-kbuild

On Fri, 26 Jun 2020 02:04:32 +0900, Masahiro Yamada wrote:
> Currently, processed-schema.yaml is always built, but it is actually
> used only for 'make dtbs_check'.
> 
> 'make dt_binding_check' uses processed-schema-example.yaml instead.
> 
> Build processed-schema.yaml only for 'make dtbs_check'.
> 
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---
> 
>  Documentation/devicetree/bindings/Makefile | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 

Applied, thanks!

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH 3/4] dt-bindings: copy process-schema-examples.yaml to process-schema.yaml
  2020-06-25 17:04 ` [PATCH 3/4] dt-bindings: copy process-schema-examples.yaml to process-schema.yaml Masahiro Yamada
@ 2020-06-29 21:25   ` Rob Herring
  0 siblings, 0 replies; 13+ messages in thread
From: Rob Herring @ 2020-06-29 21:25 UTC (permalink / raw)
  To: Masahiro Yamada
  Cc: Frank Rowand, linux-kernel, linux-kbuild, Rob Herring,
	devicetree, Michal Marek

On Fri, 26 Jun 2020 02:04:33 +0900, Masahiro Yamada wrote:
> There are two processed schema files:
> 
>  - processed-schema-examples.yaml
> 
>     Used for 'make dt_binding_check'. This is always a full schema.
> 
>  - processed-schema.yaml
> 
>     Used for 'make dtbs_check'. This may be a full schema, or a smaller
>     subset if DT_SCHEMA_FILES is given by a user.
> 
> If DT_SCHEMA_FILES is not specified, they are the same. You can copy
> the former to the latter instead of running dt-mk-schema twice. This
> saves the cpu time a lot when you do 'make dt_binding_check dtbs_check'
> because building the full schema takes a couple of seconds.
> 
> If DT_SCHEMA_FILES is specified, processed-schema.yaml is generated
> based on the specified yaml files.
> 
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---
> 
>  Documentation/devicetree/bindings/Makefile | 28 +++++++++++++++-------
>  scripts/Makefile.lib                       |  3 +++
>  2 files changed, 23 insertions(+), 8 deletions(-)
> 

Applied, thanks!

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH 4/4] dt-bindings: split DT schema check rules
  2020-06-29 20:49   ` Rob Herring
@ 2020-06-30  1:51     ` Masahiro Yamada
  0 siblings, 0 replies; 13+ messages in thread
From: Masahiro Yamada @ 2020-06-30  1:51 UTC (permalink / raw)
  To: Rob Herring
  Cc: DTML, Frank Rowand, Linux Kbuild mailing list, Michal Marek,
	linux-kernel

On Tue, Jun 30, 2020 at 5:50 AM Rob Herring <robh+dt@kernel.org> wrote:
>
> On Thu, Jun 25, 2020 at 11:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > When building %.dt.yaml from %.dts, two things happen in a row:
> >
> >  [1] Run DTC to convert %.dts into %.dt.yaml
> >
> >  [2] Run dt-validate against %.dt.yaml
> >
> > Currently, when any .yaml schema file is updated, processed-schema.yaml
> > is regenerated, then both [1] and [2] are rerun for all .dts files.
> >
> > Actually, we do not need to rerun [1] since the original .dts is not
> > updated.
>
> I have plans (and an intern working on it) to integrate the schema
> checks into dtc. That's going to make turning on the schema checks
> just a flag to dtc. I'm not sure if adding the complexity here is
> worth it as I'd expect much of this patch to go away again.
>
> Is there any negative impact on the absolute clean build time? I'm
> more concerned about that than optimizing rerunning.

No benefit on the absolute clean build time.

OK, then please skip this patch.




-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH 1/4] dt-bindings: fix error in 'make clean' after 'make dt_binding_check'
  2020-06-25 17:04 ` [PATCH 1/4] dt-bindings: fix error in 'make clean' after 'make dt_binding_check' Masahiro Yamada
  2020-06-29 21:24   ` Rob Herring
@ 2023-06-16 19:45   ` Eugeniu Rosca
  2023-06-17 18:09     ` Masahiro Yamada
  1 sibling, 1 reply; 13+ messages in thread
From: Eugeniu Rosca @ 2023-06-16 19:45 UTC (permalink / raw)
  To: Masahiro Yamada, linux-kbuild
  Cc: Rob Herring, devicetree, Frank Rowand, Matthias.Thomae,
	yyankovskyi, Eugeniu Rosca, Eugeniu Rosca

Dear Yamada-san,
Dear Kbuild experts,

On Fri, Jun 26, 2020 at 02:04:31AM +0900, Masahiro Yamada wrote:
> We are having more and more schema files.
> 
> Commit 8b6b80218b01 ("dt-bindings: Fix command line length limit
> calling dt-mk-schema") fixed the 'Argument list too long' error of
> the schema checks, but the same error happens while cleaning too.
> 
> 'make clean' after 'make dt_binding_check' fails as follows:
> 
>   $ make dt_binding_check
>     [ snip ]
>   $ make clean
>   make[2]: execvp: /bin/sh: Argument list too long
>   make[2]: *** [scripts/Makefile.clean:52: __clean] Error 127
>   make[1]: *** [scripts/Makefile.clean:66: Documentation/devicetree/bindings] Error 2
>   make: *** [Makefile:1763: _clean_Documentation] Error 2
> 
> 'make dt_binding_check' generates so many .example.dts, .dt.yaml files,
> which are passed to the 'rm' command when you run 'make clean'.
> 
> I added a small hack to use the 'find' command to clean up most of the
> build artifacts before they are processed by scripts/Makefile.clean
> 
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---
> 
>  Documentation/devicetree/bindings/Makefile | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/Documentation/devicetree/bindings/Makefile b/Documentation/devicetree/bindings/Makefile
> index a63898954068..f20d234aec46 100644
> --- a/Documentation/devicetree/bindings/Makefile
> +++ b/Documentation/devicetree/bindings/Makefile
> @@ -53,3 +53,8 @@ $(obj)/processed-schema.yaml: $(DT_SCHEMA_FILES) check_dtschema_version FORCE
>  	$(call if_changed,mk_schema)
>  
>  extra-y += processed-schema.yaml
> +
> +# Hack: avoid 'Argument list too long' error for 'make clean'. Remove most of
> +# build artifacts here before they are processed by scripts/Makefile.clean
> +clean-files = $(shell find $(obj) \( -name '*.example.dts' -o \
> +			-name '*.example.dt.yaml' \) -delete 2>/dev/null)

JFYI, we are running into the "Argument list too long" simply by trying to
clean an out-of-tree module (with lots of artifacts though), as below.
Seems to be happening on vanilla (v6.4-rc6-199-gb73056e9f82ebd) too.
Just in case there are any known hacks (or similar reports), please let us know.

$ make -C /data/linux-src M=my_module clean

make: Entering directory '/data/linux-src'
make -f ./scripts/Makefile.clean obj=my_module
set -e;  echo '  CLEAN   my_module';  rm -rf [~1k filenames / 200k char]
make[1]: /bin/sh: Argument list too long
make[1]: *** [scripts/Makefile.clean:45: __clean] Error 127

-- 
Best regards,
Eugeniu Rosca

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH 1/4] dt-bindings: fix error in 'make clean' after 'make dt_binding_check'
  2023-06-16 19:45   ` Eugeniu Rosca
@ 2023-06-17 18:09     ` Masahiro Yamada
  2023-06-19  9:27       ` Eugeniu Rosca
  0 siblings, 1 reply; 13+ messages in thread
From: Masahiro Yamada @ 2023-06-17 18:09 UTC (permalink / raw)
  To: Eugeniu Rosca
  Cc: linux-kbuild, Rob Herring, devicetree, Frank Rowand,
	Matthias.Thomae, yyankovskyi, Eugeniu Rosca

On Sat, Jun 17, 2023 at 4:45 AM Eugeniu Rosca <erosca@de.adit-jv.com> wrote:
>
> Dear Yamada-san,
> Dear Kbuild experts,
>
> On Fri, Jun 26, 2020 at 02:04:31AM +0900, Masahiro Yamada wrote:
> > We are having more and more schema files.
> >
> > Commit 8b6b80218b01 ("dt-bindings: Fix command line length limit
> > calling dt-mk-schema") fixed the 'Argument list too long' error of
> > the schema checks, but the same error happens while cleaning too.
> >
> > 'make clean' after 'make dt_binding_check' fails as follows:
> >
> >   $ make dt_binding_check
> >     [ snip ]
> >   $ make clean
> >   make[2]: execvp: /bin/sh: Argument list too long
> >   make[2]: *** [scripts/Makefile.clean:52: __clean] Error 127
> >   make[1]: *** [scripts/Makefile.clean:66: Documentation/devicetree/bindings] Error 2
> >   make: *** [Makefile:1763: _clean_Documentation] Error 2
> >
> > 'make dt_binding_check' generates so many .example.dts, .dt.yaml files,
> > which are passed to the 'rm' command when you run 'make clean'.
> >
> > I added a small hack to use the 'find' command to clean up most of the
> > build artifacts before they are processed by scripts/Makefile.clean
> >
> > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> > ---
> >
> >  Documentation/devicetree/bindings/Makefile | 5 +++++
> >  1 file changed, 5 insertions(+)
> >
> > diff --git a/Documentation/devicetree/bindings/Makefile b/Documentation/devicetree/bindings/Makefile
> > index a63898954068..f20d234aec46 100644
> > --- a/Documentation/devicetree/bindings/Makefile
> > +++ b/Documentation/devicetree/bindings/Makefile
> > @@ -53,3 +53,8 @@ $(obj)/processed-schema.yaml: $(DT_SCHEMA_FILES) check_dtschema_version FORCE
> >       $(call if_changed,mk_schema)
> >
> >  extra-y += processed-schema.yaml
> > +
> > +# Hack: avoid 'Argument list too long' error for 'make clean'. Remove most of
> > +# build artifacts here before they are processed by scripts/Makefile.clean
> > +clean-files = $(shell find $(obj) \( -name '*.example.dts' -o \
> > +                     -name '*.example.dt.yaml' \) -delete 2>/dev/null)
>
> JFYI, we are running into the "Argument list too long" simply by trying to
> clean an out-of-tree module (with lots of artifacts though), as below.
> Seems to be happening on vanilla (v6.4-rc6-199-gb73056e9f82ebd) too.
> Just in case there are any known hacks (or similar reports), please let us know.
>
> $ make -C /data/linux-src M=my_module clean
>
> make: Entering directory '/data/linux-src'
> make -f ./scripts/Makefile.clean obj=my_module
> set -e;  echo '  CLEAN   my_module';  rm -rf [~1k filenames / 200k char]
> make[1]: /bin/sh: Argument list too long
> make[1]: *** [scripts/Makefile.clean:45: __clean] Error 127


I sumitted a patch
https://patchwork.kernel.org/project/linux-kbuild/patch/20230617153025.1653851-1-masahiroy@kernel.org/

I do not know whether it will solve your problem or not.

Your module Makefile has ~1k files to remove.

It may still exceed ARG_MAX limit.




> --
> Best regards,
> Eugeniu Rosca



-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH 1/4] dt-bindings: fix error in 'make clean' after 'make dt_binding_check'
  2023-06-17 18:09     ` Masahiro Yamada
@ 2023-06-19  9:27       ` Eugeniu Rosca
  0 siblings, 0 replies; 13+ messages in thread
From: Eugeniu Rosca @ 2023-06-19  9:27 UTC (permalink / raw)
  To: Masahiro Yamada
  Cc: Eugeniu Rosca, linux-kbuild, Rob Herring, devicetree,
	Frank Rowand, Matthias.Thomae, yyankovskyi, Eugeniu Rosca

Hello Yamada-san,

On Sun, Jun 18, 2023 at 03:09:56AM +0900, Masahiro Yamada wrote:

[snip]

> I sumitted a patch
> https://urldefense.proofpoint.com/v2/url?u=https-3A__patchwork.kernel.org_project_linux-2Dkbuild_patch_20230617153025.1653851-2D1-2Dmasahiroy-40kernel.org_&d=DwIFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=SAhjP5GOmrADp1v_EE5jWoSuMlYCIt9gKduw-DCBPLs&m=_gWZwnUVw05obX2g0v2G-rIQlozQoBMXSi-pktMqVpo&s=L4Kw5qaiY1cU27bKrjQce8uJhLXdeiQ96IyX7l6Ki3o&e=

Thank you very much. Provided my comments in the new thread.

-- 
Best regards,
Eugeniu Rosca

^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2023-06-19  9:28 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-06-25 17:04 [PATCH 0/4] dt-bindings: fix 'make clean' and improve dt-schema check rules Masahiro Yamada
2020-06-25 17:04 ` [PATCH 1/4] dt-bindings: fix error in 'make clean' after 'make dt_binding_check' Masahiro Yamada
2020-06-29 21:24   ` Rob Herring
2023-06-16 19:45   ` Eugeniu Rosca
2023-06-17 18:09     ` Masahiro Yamada
2023-06-19  9:27       ` Eugeniu Rosca
2020-06-25 17:04 ` [PATCH 2/4] dt-bindings: do not build processed-schema.yaml for " Masahiro Yamada
2020-06-29 21:24   ` Rob Herring
2020-06-25 17:04 ` [PATCH 3/4] dt-bindings: copy process-schema-examples.yaml to process-schema.yaml Masahiro Yamada
2020-06-29 21:25   ` Rob Herring
2020-06-25 17:04 ` [PATCH 4/4] dt-bindings: split DT schema check rules Masahiro Yamada
2020-06-29 20:49   ` Rob Herring
2020-06-30  1:51     ` Masahiro Yamada

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.