[PATCH v2] kbuild: Enable DT schema checks for %.dtb targets

[PATCH v2] kbuild: Enable DT schema checks for %.dtb targets

[Rob Herring]

It is possible to build a single dtb, but not with DT schema validation
enabled. Enable the schema validation to run for %.dtb and %.dtbo
targets. Anyone building a dtb for a specific platform *should* pay
attention to schema warnings.

This could be supported with a separate %.dt.yaml target instead.
However, the .dt.yaml format is considered an intermediate format and
could possibly go away at some point if schema checking is integrated
into dtc. Also, the plan is to enable the schema checks by default once
platforms are free of warnings, and this is a move in that direction.

Cc: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Cc: Tom Rini <trini@konsulko.com>
Cc: Masahiro Yamada <masahiroy@kernel.org>
Cc: linux-kbuild@vger.kernel.org
Signed-off-by: Rob Herring <robh@kernel.org>
---
v2:
 - Also enable schema checks on %.dtbo targets
---
 Makefile | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/Makefile b/Makefile
index 7cfe4ff36f44..c57a7657c8dd 100644
--- a/Makefile
+++ b/Makefile
@@ -1402,17 +1402,17 @@ endif
 
 ifneq ($(dtstree),)
 
-%.dtb: include/config/kernel.release scripts_dtc
-	$(Q)$(MAKE) $(build)=$(dtstree) $(dtstree)/$@
+%.dtb: dt_binding_check include/config/kernel.release scripts_dtc
+	$(Q)$(MAKE) $(build)=$(dtstree) $(dtstree)/$@ $(dtstree)/$*.dt.yaml
 
-%.dtbo: include/config/kernel.release scripts_dtc
-	$(Q)$(MAKE) $(build)=$(dtstree) $(dtstree)/$@
+%.dtbo: dt_binding_check include/config/kernel.release scripts_dtc
+	$(Q)$(MAKE) $(build)=$(dtstree) $(dtstree)/$@ $(dtstree)/$*.dt.yaml
 
 PHONY += dtbs dtbs_install dtbs_check
 dtbs: include/config/kernel.release scripts_dtc
 	$(Q)$(MAKE) $(build)=$(dtstree)
 
-ifneq ($(filter dtbs_check, $(MAKECMDGOALS)),)
+ifneq ($(filter dtbs_check %.dtb %.dtbo, $(MAKECMDGOALS)),)
 export CHECK_DTBS=y
 dtbs: dt_binding_check
 endif
-- 
2.30.2

Re: [PATCH v2] kbuild: Enable DT schema checks for %.dtb targets

[Masahiro Yamada]

On Mon, Sep 13, 2021 at 11:51 PM Rob Herring <robh@kernel.org> wrote:
>
> It is possible to build a single dtb, but not with DT schema validation
> enabled. Enable the schema validation to run for %.dtb and %.dtbo
> targets. Anyone building a dtb for a specific platform *should* pay
> attention to schema warnings.
>
> This could be supported with a separate %.dt.yaml target instead.
> However, the .dt.yaml format is considered an intermediate format and
> could possibly go away at some point if schema checking is integrated
> into dtc. Also, the plan is to enable the schema checks by default once
> platforms are free of warnings, and this is a move in that direction.
>
> Cc: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
> Cc: Tom Rini <trini@konsulko.com>
> Cc: Masahiro Yamada <masahiroy@kernel.org>
> Cc: linux-kbuild@vger.kernel.org
> Signed-off-by: Rob Herring <robh@kernel.org>

Acked-by: Masahiro Yamada <masahiroy@kernel.org>


> ---
> v2:
>  - Also enable schema checks on %.dtbo targets
> ---
>  Makefile | 10 +++++-----
>  1 file changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index 7cfe4ff36f44..c57a7657c8dd 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1402,17 +1402,17 @@ endif
>
>  ifneq ($(dtstree),)
>
> -%.dtb: include/config/kernel.release scripts_dtc
> -       $(Q)$(MAKE) $(build)=$(dtstree) $(dtstree)/$@
> +%.dtb: dt_binding_check include/config/kernel.release scripts_dtc
> +       $(Q)$(MAKE) $(build)=$(dtstree) $(dtstree)/$@ $(dtstree)/$*.dt.yaml
>
> -%.dtbo: include/config/kernel.release scripts_dtc
> -       $(Q)$(MAKE) $(build)=$(dtstree) $(dtstree)/$@
> +%.dtbo: dt_binding_check include/config/kernel.release scripts_dtc
> +       $(Q)$(MAKE) $(build)=$(dtstree) $(dtstree)/$@ $(dtstree)/$*.dt.yaml
>
>  PHONY += dtbs dtbs_install dtbs_check
>  dtbs: include/config/kernel.release scripts_dtc
>         $(Q)$(MAKE) $(build)=$(dtstree)
>
> -ifneq ($(filter dtbs_check, $(MAKECMDGOALS)),)
> +ifneq ($(filter dtbs_check %.dtb %.dtbo, $(MAKECMDGOALS)),)
>  export CHECK_DTBS=y
>  dtbs: dt_binding_check
>  endif
> --
> 2.30.2
>


-- 
Best Regards
Masahiro Yamada

Re: [PATCH v2] kbuild: Enable DT schema checks for %.dtb targets

[Chris Packham]

Hi Rob,

On 14/09/21 2:51 am, Rob Herring wrote:
> It is possible to build a single dtb, but not with DT schema validation
> enabled. Enable the schema validation to run for %.dtb and %.dtbo
> targets. Anyone building a dtb for a specific platform *should* pay
> attention to schema warnings.
>
> This could be supported with a separate %.dt.yaml target instead.
> However, the .dt.yaml format is considered an intermediate format and
> could possibly go away at some point if schema checking is integrated
> into dtc. Also, the plan is to enable the schema checks by default once
> platforms are free of warnings, and this is a move in that direction.

Just started building 5.16-rc4 and hit the following error

/usr/src/linux/scripts/dtc/Makefile:23: *** dtc needs libyaml for DT 
schema validation support. Install the necessary libyaml development 
package..  Stop.
make: *** [/usr/src/linux/Makefile:1405: scripts_dtc] Error 2
make: *** Waiting for unfinished jobs....

I tracked it back to this patch and I gather that the "error" is very 
much intended. Fixing it means I need to get a native libyaml into my 
cross toolchain, which is doable but a bit of a hassle. This probably 
affects other meta build systems like buildroot and yocto.

I think I understand what you're getting at but is it possible to have 
some kind of escape hatch to avoid having to add a build time tool 
dependency (or even bundling libyaml next to scripts/dtc)?

I also notice that when I do supply a toolchain with libyaml the build 
times are impacted by a noticable factor.

> Cc: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
> Cc: Tom Rini <trini@konsulko.com>
> Cc: Masahiro Yamada <masahiroy@kernel.org>
> Cc: linux-kbuild@vger.kernel.org
> Signed-off-by: Rob Herring <robh@kernel.org>
> ---
> v2:
>   - Also enable schema checks on %.dtbo targets
> ---
>   Makefile | 10 +++++-----
>   1 file changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index 7cfe4ff36f44..c57a7657c8dd 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1402,17 +1402,17 @@ endif
>   
>   ifneq ($(dtstree),)
>   
> -%.dtb: include/config/kernel.release scripts_dtc
> -	$(Q)$(MAKE) $(build)=$(dtstree) $(dtstree)/$@
> +%.dtb: dt_binding_check include/config/kernel.release scripts_dtc
> +	$(Q)$(MAKE) $(build)=$(dtstree) $(dtstree)/$@ $(dtstree)/$*.dt.yaml
>   
> -%.dtbo: include/config/kernel.release scripts_dtc
> -	$(Q)$(MAKE) $(build)=$(dtstree) $(dtstree)/$@
> +%.dtbo: dt_binding_check include/config/kernel.release scripts_dtc
> +	$(Q)$(MAKE) $(build)=$(dtstree) $(dtstree)/$@ $(dtstree)/$*.dt.yaml
>   
>   PHONY += dtbs dtbs_install dtbs_check
>   dtbs: include/config/kernel.release scripts_dtc
>   	$(Q)$(MAKE) $(build)=$(dtstree)
>   
> -ifneq ($(filter dtbs_check, $(MAKECMDGOALS)),)
> +ifneq ($(filter dtbs_check %.dtb %.dtbo, $(MAKECMDGOALS)),)
>   export CHECK_DTBS=y
>   dtbs: dt_binding_check
>   endif

Re: [PATCH v2] kbuild: Enable DT schema checks for %.dtb targets

[Rob Herring]

On Mon, Dec 6, 2021 at 1:34 AM Chris Packham
<Chris.Packham@alliedtelesis.co.nz> wrote:
>
> Hi Rob,
>
> On 14/09/21 2:51 am, Rob Herring wrote:
> > It is possible to build a single dtb, but not with DT schema validation
> > enabled. Enable the schema validation to run for %.dtb and %.dtbo
> > targets. Anyone building a dtb for a specific platform *should* pay
> > attention to schema warnings.
> >
> > This could be supported with a separate %.dt.yaml target instead.
> > However, the .dt.yaml format is considered an intermediate format and
> > could possibly go away at some point if schema checking is integrated
> > into dtc. Also, the plan is to enable the schema checks by default once
> > platforms are free of warnings, and this is a move in that direction.
>
> Just started building 5.16-rc4 and hit the following error
>
> /usr/src/linux/scripts/dtc/Makefile:23: *** dtc needs libyaml for DT
> schema validation support. Install the necessary libyaml development
> package..  Stop.
> make: *** [/usr/src/linux/Makefile:1405: scripts_dtc] Error 2
> make: *** Waiting for unfinished jobs....
>
> I tracked it back to this patch and I gather that the "error" is very
> much intended. Fixing it means I need to get a native libyaml into my
> cross toolchain, which is doable but a bit of a hassle. This probably
> affects other meta build systems like buildroot and yocto.

Yes. I'm going to revert this for now.

Are python dependencies any easier? The libyaml dtc dependency may
actually go away, but dtschema and it's dependencies will remain.

> I think I understand what you're getting at but is it possible to have
> some kind of escape hatch to avoid having to add a build time tool
> dependency (or even bundling libyaml next to scripts/dtc)?

My current thought is to make it a kconfig option. I assume that would
work for you?

> I also notice that when I do supply a toolchain with libyaml the build
> times are impacted by a noticable factor.

How many dtbs are you building?

Rob

Re: [PATCH v2] kbuild: Enable DT schema checks for %.dtb targets

[Chris Packham]

On 9/12/21 10:31 am, Rob Herring wrote:
> On Mon, Dec 6, 2021 at 1:34 AM Chris Packham
> <Chris.Packham@alliedtelesis.co.nz> wrote:
>> Hi Rob,
>>
>> On 14/09/21 2:51 am, Rob Herring wrote:
>>> It is possible to build a single dtb, but not with DT schema validation
>>> enabled. Enable the schema validation to run for %.dtb and %.dtbo
>>> targets. Anyone building a dtb for a specific platform *should* pay
>>> attention to schema warnings.
>>>
>>> This could be supported with a separate %.dt.yaml target instead.
>>> However, the .dt.yaml format is considered an intermediate format and
>>> could possibly go away at some point if schema checking is integrated
>>> into dtc. Also, the plan is to enable the schema checks by default once
>>> platforms are free of warnings, and this is a move in that direction.
>> Just started building 5.16-rc4 and hit the following error
>>
>> /usr/src/linux/scripts/dtc/Makefile:23: *** dtc needs libyaml for DT
>> schema validation support. Install the necessary libyaml development
>> package..  Stop.
>> make: *** [/usr/src/linux/Makefile:1405: scripts_dtc] Error 2
>> make: *** Waiting for unfinished jobs....
>>
>> I tracked it back to this patch and I gather that the "error" is very
>> much intended. Fixing it means I need to get a native libyaml into my
>> cross toolchain, which is doable but a bit of a hassle. This probably
>> affects other meta build systems like buildroot and yocto.
> Yes. I'm going to revert this for now.
>
> Are python dependencies any easier? The libyaml dtc dependency may
> actually go away, but dtschema and it's dependencies will remain.

Probably no worse that non python dependencies.

Our particular special in-house build system might be a bit trickier as 
we can't just spin up a vitrualenv but that's our problem not yours. We 
do manage to build other packages that have build-time python 
dependencies so I'm sure we could make that work for the kernel. I'm not 
sure how much push-back you'd get from others if python became a 
dependency for building the kernel.

>> I think I understand what you're getting at but is it possible to have
>> some kind of escape hatch to avoid having to add a build time tool
>> dependency (or even bundling libyaml next to scripts/dtc)?
> My current thought is to make it a kconfig option. I assume that would
> work for you?

An option would work for us. It could also be the kind of thing that 
kicks in with make C=1/2.

I presume that you would want people to run this kind of thing when 
submitting new bindings or board.dts. For us we can do those builds 
manually outside our full build system.

>> I also notice that when I do supply a toolchain with libyaml the build
>> times are impacted by a noticable factor.
> How many dtbs are you building?
Nothing excessive. Something in the order of 7 depending on the 
architecture. It's avoided on most re-builds so generally it's just a 
one-off cost but it would affect any type of CI system that builds from 
clean.
>
> Rob