[Buildroot] [PATCH 0/2] Fix qemu package build

[Buildroot] [PATCH 0/2] Fix qemu package build

[Alexey Neyman]

Patch ping; no response received to patches submitted 5 days ago.

First, there is a problem that is currently reported by buildroot's
autobuilder: the --disable-git-update option is now deprecated and is
interpreted as --with-git-submodules=validate, which can only work in a
git clone. The correct replacement is --with-git-submodules=ignore.

With that fixed, package/qemu build succeeds. However, on hosts using
make 3.82 (e.g CentOS 7), despite reporting build success, it fails to
actually build or install any target binaries. The problem is that
Qemu's makefile incorrectly interprets the --no-print-directory passed
by buildroot as the -n option to make and injects -n (--just-print) into
its ninja build.

Alexey Neyman (2):
  qemu: do not use deprecated option
  qemu: filter out long make options

 package/qemu/qemu.mk | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

-- 
2.27.0

_______________________________________________
buildroot mailing list
buildroot@busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

[Buildroot] [PATCH 1/2] package/qemu: do not use deprecated option

[Alexey Neyman]

Qemu build fails with:

--disable-git-update deprecated, use --with-git-submodules=validate
ERROR: cannot validate git submodules without .git

Use --with-git-submodules=ignore instead.

Signed-off-by: Alexey Neyman <stilor@att.net>
---
 package/qemu/qemu.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/package/qemu/qemu.mk b/package/qemu/qemu.mk
index e0f722cf42..470c285f16 100644
--- a/package/qemu/qemu.mk
+++ b/package/qemu/qemu.mk
@@ -198,7 +198,7 @@ define QEMU_CONFIGURE_CMDS
 			--disable-vhost-crypto \
 			--disable-libxml2 \
 			--disable-capstone \
-			--disable-git-update \
+			--with-git-submodules=ignore \
 			--disable-opengl \
 			--disable-vhost-user-blk-server \
 			--disable-virtiofsd \
-- 
2.27.0

_______________________________________________
buildroot mailing list
buildroot@busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

[Buildroot] [PATCH 2/2] package/qemu: filter out long make options

[Alexey Neyman]

Since qemu started using ninja, they have the following fragment in the
Makefile:

MAKE.n = $(findstring n,$(firstword $(MAKEFLAGS)))
NINJAFLAGS = ... $(if $(MAKE.n), -n) ...

Buildroot's generated makefile in the O= directory invokes make in the
base buildroot with --no-print-directory. However, make's placement of
the --no-print-directory in MAKEFLAGS varies between the versions of
the host make; make 4.3 places that at the end while make 3.82 places
it at the beginning. As a result, if building on a system with an older
host make, qemu's makefile invokes `ninja -n` which does not generate
any outputs.

To reproduce, on a CentOS 7 machine or docker image:
  mkdir /tmp/br-build && cd /tmp/br-build
  make -C ~/buildroot pc_x86_64_bios_defconfig O=`pwd`
  make menuconfig # Switch to glibc, enable "QEMU" and "QEMU tools"
  make all # Build succeeds
  find target -name qemu-img # No binary has been built

Signed-off-by: Alexey Neyman <stilor@att.net>
---
 package/qemu/qemu.mk | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/package/qemu/qemu.mk b/package/qemu/qemu.mk
index 470c285f16..ee3e7c3676 100644
--- a/package/qemu/qemu.mk
+++ b/package/qemu/qemu.mk
@@ -206,14 +206,15 @@ define QEMU_CONFIGURE_CMDS
 			$(QEMU_OPTS)
 endef
 
+# QEMU Makefile incorrectly interprets long options to make
 define QEMU_BUILD_CMDS
 	unset TARGET_DIR; \
-	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D)
+	MAKEFLAGS="$(filter-out --%,$(MAKEFLAGS))" $(TARGET_MAKE_ENV) $(MAKE) -C $(@D)
 endef
 
 define QEMU_INSTALL_TARGET_CMDS
 	unset TARGET_DIR; \
-	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D) $(QEMU_MAKE_ENV) DESTDIR=$(TARGET_DIR) install
+	MAKEFLAGS="$(filter-out --%,$(MAKEFLAGS))" $(TARGET_MAKE_ENV) $(MAKE) -C $(@D) $(QEMU_MAKE_ENV) DESTDIR=$(TARGET_DIR) install
 endef
 
 $(eval $(generic-package))
-- 
2.27.0

_______________________________________________
buildroot mailing list
buildroot@busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 2/2] package/qemu: filter out long make options

[Thomas Petazzoni]

Hello,

On Wed, 21 Jul 2021 12:46:05 -0700
Alexey Neyman <stilor@att.net> wrote:

> Since qemu started using ninja, they have the following fragment in the
> Makefile:
> 
> MAKE.n = $(findstring n,$(firstword $(MAKEFLAGS)))
> NINJAFLAGS = ... $(if $(MAKE.n), -n) ...
> 
> Buildroot's generated makefile in the O= directory invokes make in the
> base buildroot with --no-print-directory. However, make's placement of
> the --no-print-directory in MAKEFLAGS varies between the versions of
> the host make; make 4.3 places that at the end while make 3.82 places
> it at the beginning. As a result, if building on a system with an older
> host make, qemu's makefile invokes `ninja -n` which does not generate
> any outputs.
> 
> To reproduce, on a CentOS 7 machine or docker image:
>   mkdir /tmp/br-build && cd /tmp/br-build
>   make -C ~/buildroot pc_x86_64_bios_defconfig O=`pwd`
>   make menuconfig # Switch to glibc, enable "QEMU" and "QEMU tools"
>   make all # Build succeeds
>   find target -name qemu-img # No binary has been built

I'm not sure to grasp all the implications of this, but isn't this a
bug in Qemu's build machinery, that should be fixed in Qemu's Makefile
instead ?

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 2/2] package/qemu: filter out long make options

[Alexey Neyman]

Hi,

On 7/21/21 1:15 PM, Thomas Petazzoni wrote:
> Hello,
>
> On Wed, 21 Jul 2021 12:46:05 -0700
> Alexey Neyman <stilor@att.net> wrote:
>
>> Since qemu started using ninja, they have the following fragment in the
>> Makefile:
>>
>> MAKE.n = $(findstring n,$(firstword $(MAKEFLAGS)))
>> NINJAFLAGS = ... $(if $(MAKE.n), -n) ...
>>
>> Buildroot's generated makefile in the O= directory invokes make in the
>> base buildroot with --no-print-directory. However, make's placement of
>> the --no-print-directory in MAKEFLAGS varies between the versions of
>> the host make; make 4.3 places that at the end while make 3.82 places
>> it at the beginning. As a result, if building on a system with an older
>> host make, qemu's makefile invokes `ninja -n` which does not generate
>> any outputs.
>>
>> To reproduce, on a CentOS 7 machine or docker image:
>>    mkdir /tmp/br-build && cd /tmp/br-build
>>    make -C ~/buildroot pc_x86_64_bios_defconfig O=`pwd`
>>    make menuconfig # Switch to glibc, enable "QEMU" and "QEMU tools"
>>    make all # Build succeeds
>>    find target -name qemu-img # No binary has been built
> I'm not sure to grasp all the implications of this, but isn't this a
> bug in Qemu's build machinery, that should be fixed in Qemu's Makefile
> instead ?

I was on the fence about it myself. Okay, I'll try to send a fix for 
qemu/Makefile to Qemu developers and see if they agree to pick it up; 
they could argue they don't support being invoked with --no-print-directory.

Regards,
Alexey.

_______________________________________________
buildroot mailing list
buildroot@busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 2/2] package/qemu: filter out long make options

[Thomas Petazzoni]

On Wed, 21 Jul 2021 15:18:36 -0700
Alexey Neyman <stilor@att.net> wrote:

> > I'm not sure to grasp all the implications of this, but isn't this a
> > bug in Qemu's build machinery, that should be fixed in Qemu's Makefile
> > instead ?  
> 
> I was on the fence about it myself. Okay, I'll try to send a fix for 
> qemu/Makefile to Qemu developers and see if they agree to pick it up; 
> they could argue they don't support being invoked with --no-print-directory.

It would be odd for them to consider that as a normal behavior,
--no-print-directory is really a standard make option.

Let us know what they say, of course.

Thanks a lot!

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/2] package/qemu: do not use deprecated option

[Yann E. MORIN]

Alexey, All,

On 2021-07-21 12:46 -0700, Alexey Neyman spake thusly:
> Qemu build fails with:
> 
> --disable-git-update deprecated, use --with-git-submodules=validate
> ERROR: cannot validate git submodules without .git
> 
> Use --with-git-submodules=ignore instead.
> 
> Signed-off-by: Alexey Neyman <stilor@att.net>

An identical patch by Joseph was already pending in the backlog, so I
applied his patch.

Thanks!

Regards,
Yann E. MORIN.

> ---
>  package/qemu/qemu.mk | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/package/qemu/qemu.mk b/package/qemu/qemu.mk
> index e0f722cf42..470c285f16 100644
> --- a/package/qemu/qemu.mk
> +++ b/package/qemu/qemu.mk
> @@ -198,7 +198,7 @@ define QEMU_CONFIGURE_CMDS
>  			--disable-vhost-crypto \
>  			--disable-libxml2 \
>  			--disable-capstone \
> -			--disable-git-update \
> +			--with-git-submodules=ignore \
>  			--disable-opengl \
>  			--disable-vhost-user-blk-server \
>  			--disable-virtiofsd \
> -- 
> 2.27.0
> 
> _______________________________________________
> buildroot mailing list
> buildroot@busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

[Buildroot] [PATCH] package/qemu: filter out long make options

[Alexey Neyman]

Since qemu started using ninja, they have the following fragment in the
Makefile:

MAKE.n = $(findstring n,$(firstword $(MAKEFLAGS)))
NINJAFLAGS = ... $(if $(MAKE.n), -n) ...

Buildroot's generated makefile in the O= directory invokes make in the
base buildroot with --no-print-directory. However, make's placement of
the --no-print-directory in MAKEFLAGS varies between the versions of
the host make; make 4.3 places that at the end while make 3.82 places
it at the beginning. As a result, if building on a system with an older
host make, qemu's makefile invokes `ninja -n` which does not generate
any outputs.

To reproduce, on a CentOS 7 machine or docker image:
  mkdir /tmp/br-build && cd /tmp/br-build
  make -C ~/buildroot pc_x86_64_bios_defconfig O=`pwd`
  make menuconfig # Switch to glibc, enable "QEMU" and "QEMU tools"
  make all # Build succeeds
  find target -name qemu-img # No binary has been built

Pick up the fix commited in Qemu upstream.

Signed-off-by: Alexey Neyman <stilor@att.net>
---
 .../0004-Makefile-ignore-long-options.patch   | 41 +++++++++++++++++++
 1 file changed, 41 insertions(+)
 create mode 100644 package/qemu/0004-Makefile-ignore-long-options.patch

diff --git a/package/qemu/0004-Makefile-ignore-long-options.patch b/package/qemu/0004-Makefile-ignore-long-options.patch
new file mode 100644
index 0000000000..2d14b70c03
--- /dev/null
+++ b/package/qemu/0004-Makefile-ignore-long-options.patch
@@ -0,0 +1,41 @@
+From 14833e24dea49303ebc2464813601054b6cdfcac Mon Sep 17 00:00:00 2001
+From: Alexey Neyman <stilor@att.net>
+Date: Wed, 21 Jul 2021 19:08:46 -0700
+Subject: [PATCH] Makefile: ignore long options
+
+When searching for options like -n in MAKEFLAGS, current code may result
+in a false positive match when make is invoked with long options like
+--no-print-directory. This has been observed with certain versions of
+host make (e.g. 3.82) while building the Qemu package in buildroot.
+
+Filter out such long options before searching for one-character options.
+
+Signed-off-by: Alexey Neyman <stilor@att.net>
+Message-Id: <20210722020846.3678817-1-stilor@att.net>
+Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
+---
+ Makefile | 8 +++++---
+ 1 file changed, 5 insertions(+), 3 deletions(-)
+
+diff --git a/Makefile b/Makefile
+index 6c36330eef..401c623a65 100644
+--- a/Makefile
++++ b/Makefile
+@@ -129,9 +129,11 @@ endif
+ # 4. Rules to bridge to other makefiles
+ 
+ ifneq ($(NINJA),)
+-MAKE.n = $(findstring n,$(firstword $(MAKEFLAGS)))
+-MAKE.k = $(findstring k,$(firstword $(MAKEFLAGS)))
+-MAKE.q = $(findstring q,$(firstword $(MAKEFLAGS)))
++# Filter out long options to avoid flags like --no-print-directory which
++# may result in false positive match for MAKE.n
++MAKE.n = $(findstring n,$(firstword $(filter-out --%,$(MAKEFLAGS))))
++MAKE.k = $(findstring k,$(firstword $(filter-out --%,$(MAKEFLAGS))))
++MAKE.q = $(findstring q,$(firstword $(filter-out --%,$(MAKEFLAGS))))
+ MAKE.nq = $(if $(word 2, $(MAKE.n) $(MAKE.q)),nq)
+ NINJAFLAGS = $(if $V,-v) $(if $(MAKE.n), -n) $(if $(MAKE.k), -k0) \
+         $(filter-out -j, $(lastword -j1 $(filter -l% -j%, $(MAKEFLAGS)))) \
+-- 
+2.27.0
+
-- 
2.27.0

_______________________________________________
buildroot mailing list
buildroot@busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH] package/qemu: filter out long make options

[Yann E. MORIN]

Alexey, All,

On 2021-08-07 12:15 -0700, Alexey Neyman spake thusly:
> Since qemu started using ninja, they have the following fragment in the
> Makefile:
> 
> MAKE.n = $(findstring n,$(firstword $(MAKEFLAGS)))
> NINJAFLAGS = ... $(if $(MAKE.n), -n) ...
> 
> Buildroot's generated makefile in the O= directory invokes make in the
> base buildroot with --no-print-directory. However, make's placement of
> the --no-print-directory in MAKEFLAGS varies between the versions of
> the host make; make 4.3 places that at the end while make 3.82 places
> it at the beginning. As a result, if building on a system with an older
> host make, qemu's makefile invokes `ninja -n` which does not generate
> any outputs.
> 
> To reproduce, on a CentOS 7 machine or docker image:
>   mkdir /tmp/br-build && cd /tmp/br-build
>   make -C ~/buildroot pc_x86_64_bios_defconfig O=`pwd`
>   make menuconfig # Switch to glibc, enable "QEMU" and "QEMU tools"
>   make all # Build succeeds
>   find target -name qemu-img # No binary has been built
> 
> Pick up the fix commited in Qemu upstream.
> 
> Signed-off-by: Alexey Neyman <stilor@att.net>

Applied to master, thanks.

Regards,
Yann E. MORIN.

> ---
>  .../0004-Makefile-ignore-long-options.patch   | 41 +++++++++++++++++++
>  1 file changed, 41 insertions(+)
>  create mode 100644 package/qemu/0004-Makefile-ignore-long-options.patch
> 
> diff --git a/package/qemu/0004-Makefile-ignore-long-options.patch b/package/qemu/0004-Makefile-ignore-long-options.patch
> new file mode 100644
> index 0000000000..2d14b70c03
> --- /dev/null
> +++ b/package/qemu/0004-Makefile-ignore-long-options.patch
> @@ -0,0 +1,41 @@
> +From 14833e24dea49303ebc2464813601054b6cdfcac Mon Sep 17 00:00:00 2001
> +From: Alexey Neyman <stilor@att.net>
> +Date: Wed, 21 Jul 2021 19:08:46 -0700
> +Subject: [PATCH] Makefile: ignore long options
> +
> +When searching for options like -n in MAKEFLAGS, current code may result
> +in a false positive match when make is invoked with long options like
> +--no-print-directory. This has been observed with certain versions of
> +host make (e.g. 3.82) while building the Qemu package in buildroot.
> +
> +Filter out such long options before searching for one-character options.
> +
> +Signed-off-by: Alexey Neyman <stilor@att.net>
> +Message-Id: <20210722020846.3678817-1-stilor@att.net>
> +Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> +---
> + Makefile | 8 +++++---
> + 1 file changed, 5 insertions(+), 3 deletions(-)
> +
> +diff --git a/Makefile b/Makefile
> +index 6c36330eef..401c623a65 100644
> +--- a/Makefile
> ++++ b/Makefile
> +@@ -129,9 +129,11 @@ endif
> + # 4. Rules to bridge to other makefiles
> + 
> + ifneq ($(NINJA),)
> +-MAKE.n = $(findstring n,$(firstword $(MAKEFLAGS)))
> +-MAKE.k = $(findstring k,$(firstword $(MAKEFLAGS)))
> +-MAKE.q = $(findstring q,$(firstword $(MAKEFLAGS)))
> ++# Filter out long options to avoid flags like --no-print-directory which
> ++# may result in false positive match for MAKE.n
> ++MAKE.n = $(findstring n,$(firstword $(filter-out --%,$(MAKEFLAGS))))
> ++MAKE.k = $(findstring k,$(firstword $(filter-out --%,$(MAKEFLAGS))))
> ++MAKE.q = $(findstring q,$(firstword $(filter-out --%,$(MAKEFLAGS))))
> + MAKE.nq = $(if $(word 2, $(MAKE.n) $(MAKE.q)),nq)
> + NINJAFLAGS = $(if $V,-v) $(if $(MAKE.n), -n) $(if $(MAKE.k), -k0) \
> +         $(filter-out -j, $(lastword -j1 $(filter -l% -j%, $(MAKEFLAGS)))) \
> +-- 
> +2.27.0
> +
> -- 
> 2.27.0
> 
> _______________________________________________
> buildroot mailing list
> buildroot@busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot