[Buildroot] [PATCH 0/4] System emulation support in host-qemu

[Buildroot] [PATCH 0/4] System emulation support in host-qemu

[Thomas Petazzoni]

Hello,

This is a respin of Simon Maes series adding system emulation support
to the host-qemu package. Compared to the original series from Simon:

 - The initial patch adding the support has been reworked, taking into
   account numerous comments from Arnout, plus additional
   simplifications and improvements.

 - The patch adding version selection has been dropped. We can revisit
   that later, but adding version selection is complicated (think
   about hash verification for example)

 - The cleanup patch from Simon has been dropped, as it didn't clean
   up things according to Buildroot best practices.

 - The patch adding the "qemu-system-run" target has been
   dropped. That's too specific, and if people really want to automate
   starting qemu, they can write a script for it outside of Buildroot.

 - A patch to bump to Qemu 2.6.0 has been added.

Thanks,

Thomas

Simon Maes (3):
  qemu: add support for host-qemu-system
  vde2: enable building host package
  qemu: add support for vde2

Thomas Petazzoni (1):
  qemu: bump to version 2.6.0

 package/qemu/Config.in.host | 32 ++++++++++++++++++++++++++++----
 package/qemu/qemu.hash      |  2 +-
 package/qemu/qemu.mk        | 30 +++++++++++++++++++++++++-----
 package/vde2/vde2.mk        | 12 ++++++++++++
 4 files changed, 66 insertions(+), 10 deletions(-)

-- 
2.7.4

[Buildroot] [PATCH 1/4] qemu: add support for host-qemu-system

[Thomas Petazzoni]

From: Simon Maes <simonn.maes@gmail.com>

This commit adds support for building the system emulation mode in
host-qemu. To do so, it adds the BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE and
BR2_PACKAGE_HOST_QEMU_LINUX_USER_MODE sub-options, making sure that the
latter is selected if the former is not enabled. This ensures that at
least one is enabled *and* that existing configurations continue to
build the user-land emulation (which was the only one we supported until
now).

The list of architectures supported by the system emulation mode is the
same as the one for the user-space emulation mode (as far as the
existing list is concerned), so we simply drop the comment about this
dependency list being related to the user-space emulation only.

Signed-off-by: Simon Maes <simonn.maes@gmail.com>
[Thomas:
 - Make sure either BR2_PACKAGE_HOST_QEMU_LINUX_USER_MODE or
   BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE get selected.
 - Unconditionally enable FDT support in host-qemu when in system
   mode (so the option BR2_PACKAGE_HOST_QEMU_FDT has been removed)
 - Remove the unneeded BR2_PACKAGE_HOST_QEMU_HAS_EMULS option.
 - Remove the SDL related option, we really on the system to provide
   SDL.
 - Remove the BR2_PACKAGE_HOST_QEMU_DEBUG and
   BR2_PACKAGE_HOST_QEMU_STRIP_BINARY options, since they are not really
   useful.
 - Remove HOST_QEMU_SITE and HOST_QEMU_SOURCE definitions, since they
   are automatically derived from QEMU_SITE and QEMU_SOURCE anyway.
 - Group things more logically in the .mk file.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/qemu/Config.in.host | 25 +++++++++++++++++++++----
 package/qemu/qemu.mk        | 25 ++++++++++++++++++++-----
 2 files changed, 41 insertions(+), 9 deletions(-)

diff --git a/package/qemu/Config.in.host b/package/qemu/Config.in.host
index c5c3f05..227749a 100644
--- a/package/qemu/Config.in.host
+++ b/package/qemu/Config.in.host
@@ -1,8 +1,7 @@
 config BR2_PACKAGE_HOST_QEMU
 	bool "host qemu"
-	# So far, we only build the user mode emulation, so this list
-	# of architecture dependencies only takes into account this
-	# emulation mode.
+	select BR2_PACKAGE_HOST_QEMU_LINUX_USER_MODE \
+	       if !BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE
 	depends on BR2_arm       || BR2_armeb       || BR2_aarch64      || \
 		BR2_i386         || BR2_m68k        || BR2_microblazeel || \
 		BR2_microblazebe || BR2_mips        || BR2_mipsel       || \
@@ -12,6 +11,24 @@ config BR2_PACKAGE_HOST_QEMU
 	help
 	  QEMU is a generic and open source machine emulator and virtualizer.
 
-	  This option builds a user emulator for your selected architecture.
+	  This option builds an emulator for your selected architecture.
 
 	  http://www.qemu.org
+
+if BR2_PACKAGE_HOST_QEMU
+
+comment "Emulators selection"
+
+config BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE
+	bool "Enable system emulation"
+	help
+	  Enables the build of the system emulator, which allows to
+	  boot an entire system in Qemu.
+
+config BR2_PACKAGE_HOST_QEMU_LINUX_USER_MODE
+	bool "Enable Linux user-land emulation"
+	help
+	  Enables the build of the user-land emulator, which allows to
+	  run user-space applications.
+
+endif
diff --git a/package/qemu/qemu.mk b/package/qemu/qemu.mk
index 5fa95bb..5c3cfea 100644
--- a/package/qemu/qemu.mk
+++ b/package/qemu/qemu.mk
@@ -61,9 +61,19 @@ endif
 ifeq ($(HOST_QEMU_ARCH),sh4aeb)
 HOST_QEMU_ARCH = sh4eb
 endif
-HOST_QEMU_TARGETS = $(HOST_QEMU_ARCH)-linux-user
 
-ifeq ($(BR2_PACKAGE_HOST_QEMU),y)
+ifeq ($(BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE),y)
+HOST_QEMU_TARGETS += $(HOST_QEMU_ARCH)-softmmu
+HOST_QEMU_OPTS += --enable-system --enable-fdt
+HOST_QEMU_DEPENDENCIES += host-dtc
+else
+HOST_QEMU_OPTS += --disable-system
+endif
+
+ifeq ($(BR2_PACKAGE_HOST_QEMU_LINUX_USER_MODE),y)
+HOST_QEMU_TARGETS += $(HOST_QEMU_ARCH)-linux-user
+HOST_QEMU_OPTS += --enable-linux-user
+
 HOST_QEMU_HOST_SYSTEM_TYPE = $(shell uname -s)
 ifneq ($(HOST_QEMU_HOST_SYSTEM_TYPE),Linux)
 $(error "qemu-user can only be used on Linux hosts")
@@ -84,12 +94,16 @@ HOST_QEMU_COMPARE_VERSION = $(shell test $(HOST_QEMU_HOST_SYSTEM_VERSION) -ge $(
 # built with kernel headers that are older or the same as the kernel
 # version running on the host machine.
 #
+
 ifeq ($(BR_BUILDING),y)
 ifneq ($(HOST_QEMU_COMPARE_VERSION),OK)
 $(error "Refusing to build qemu-user: target Linux version newer than host's.")
 endif
-endif
-endif
+endif # BR_BUILDING
+
+else # BR2_PACKAGE_HOST_QEMU_LINUX_USER_MODE
+HOST_QEMU_OPTS += --disable-linux-user
+endif # BR2_PACKAGE_HOST_QEMU_LINUX_USER_MODE
 
 define HOST_QEMU_CONFIGURE_CMDS
 	cd $(@D); $(HOST_CONFIGURE_OPTS) ./configure    \
@@ -100,7 +114,8 @@ define HOST_QEMU_CONFIGURE_CMDS
 		--host-cc="$(HOSTCC)"                   \
 		--python=$(HOST_DIR)/usr/bin/python2    \
 		--extra-cflags="$(HOST_CFLAGS)"         \
-		--extra-ldflags="$(HOST_LDFLAGS)"
+		--extra-ldflags="$(HOST_LDFLAGS)"       \
+		$(HOST_QEMU_OPTS)
 endef
 
 define HOST_QEMU_BUILD_CMDS
-- 
2.7.4

[Buildroot] [PATCH 2/4] vde2: enable building host package

[Thomas Petazzoni]

From: Simon Maes <simonn.maes@gmail.com>

In preparation of enabling vde2 support in host-qemu package.

Signed-off-by: Simon Maes <simonn.maes@gmail.com>
[Thomas:
 - add HOST_VDE2_CONF_OPTS and HOST_VDE2_MAKE, like we have for the
   target variant.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/vde2/vde2.mk | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/package/vde2/vde2.mk b/package/vde2/vde2.mk
index bad758d..d9b11bb 100644
--- a/package/vde2/vde2.mk
+++ b/package/vde2/vde2.mk
@@ -37,4 +37,16 @@ VDE2_CONF_OPTS = \
 #?Package does not build in parallel due to improper make rules
 VDE2_MAKE = $(MAKE1)
 
+HOST_VDE2_CONF_OPTS = \
+	--disable-experimental  \
+	--disable-cryptcab      \
+	--disable-pcap          \
+	--disable-python        \
+	--disable-profile       \
+	--disable-kernel-switch \
+	--enable-tuntap         \
+
+HOST_VDE2_MAKE = $(MAKE1)
+
 $(eval $(autotools-package))
+$(eval $(host-autotools-package))
-- 
2.7.4

[Buildroot] [PATCH 3/4] qemu: add support for vde2

[Thomas Petazzoni]

From: Simon Maes <simonn.maes@gmail.com>

Additional configuration for qemu package:
- Enable VDE2 support for qemu

Signed-off-by: Simon Maes <simonn.maes@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/qemu/Config.in.host | 7 +++++++
 package/qemu/qemu.mk        | 5 +++++
 2 files changed, 12 insertions(+)

diff --git a/package/qemu/Config.in.host b/package/qemu/Config.in.host
index 227749a..b3eb590 100644
--- a/package/qemu/Config.in.host
+++ b/package/qemu/Config.in.host
@@ -31,4 +31,11 @@ config BR2_PACKAGE_HOST_QEMU_LINUX_USER_MODE
 	  Enables the build of the user-land emulator, which allows to
 	  run user-space applications.
 
+config BR2_PACKAGE_HOST_QEMU_VDE2
+	bool "VDE2 support"
+	help
+	  Enables VDE2 support. VDE2 stands for Virtual Distributed
+	  Ethernet and can be used to create virtual switches to
+	  "plug" both physical and virtual machines in them.
+
 endif
diff --git a/package/qemu/qemu.mk b/package/qemu/qemu.mk
index 5c3cfea..3ca0958 100644
--- a/package/qemu/qemu.mk
+++ b/package/qemu/qemu.mk
@@ -105,6 +105,11 @@ else # BR2_PACKAGE_HOST_QEMU_LINUX_USER_MODE
 HOST_QEMU_OPTS += --disable-linux-user
 endif # BR2_PACKAGE_HOST_QEMU_LINUX_USER_MODE
 
+ifeq ($(BR2_PACKAGE_HOST_QEMU_VDE2),y)
+HOST_QEMU_OPTS += --enable-vde
+HOST_QEMU_DEPENDENCIES += host-vde2
+endif
+
 define HOST_QEMU_CONFIGURE_CMDS
 	cd $(@D); $(HOST_CONFIGURE_OPTS) ./configure    \
 		--target-list="$(HOST_QEMU_TARGETS)"    \
-- 
2.7.4

[Buildroot] [PATCH 4/4] qemu: bump to version 2.6.0

[Thomas Petazzoni]

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/qemu/qemu.hash | 2 +-
 package/qemu/qemu.mk   | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/package/qemu/qemu.hash b/package/qemu/qemu.hash
index d35960a..2d6d96c 100644
--- a/package/qemu/qemu.hash
+++ b/package/qemu/qemu.hash
@@ -1,2 +1,2 @@
 # Locally computed, tarball verified with GPG signature
-sha256 28d9946e43765a44ccccca3cba5f4f9034f2759ec1f2ce16594ddb6776c8efe6  qemu-2.5.1.1.tar.bz2
+sha256 c9ac4a651b273233d21b8bec32e30507cb9cce7900841febc330956a1a8434ec  qemu-2.6.0.tar.bz2
diff --git a/package/qemu/qemu.mk b/package/qemu/qemu.mk
index 3ca0958..5ca0e77 100644
--- a/package/qemu/qemu.mk
+++ b/package/qemu/qemu.mk
@@ -4,7 +4,7 @@
 #
 ################################################################################
 
-QEMU_VERSION = 2.5.1.1
+QEMU_VERSION = 2.6.0
 QEMU_SOURCE = qemu-$(QEMU_VERSION).tar.bz2
 QEMU_SITE = http://wiki.qemu.org/download
 QEMU_LICENSE = GPLv2, LGPLv2.1, MIT, BSD-3c, BSD-2c, Others/BSD-1c
-- 
2.7.4

[Buildroot] [PATCH 1/4] qemu: add support for host-qemu-system

[Yann E. MORIN]

Simon, Thomas, All,

On 2016-07-04 00:24 +0200, Thomas Petazzoni spake thusly:
> From: Simon Maes <simonn.maes@gmail.com>
> 
> This commit adds support for building the system emulation mode in
> host-qemu. To do so, it adds the BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE and
> BR2_PACKAGE_HOST_QEMU_LINUX_USER_MODE sub-options, making sure that the
> latter is selected if the former is not enabled. This ensures that at
> least one is enabled *and* that existing configurations continue to
> build the user-land emulation (which was the only one we supported until
> now).
> 
> The list of architectures supported by the system emulation mode is the
> same as the one for the user-space emulation mode (as far as the
> existing list is concerned), so we simply drop the comment about this
> dependency list being related to the user-space emulation only.
> 
> Signed-off-by: Simon Maes <simonn.maes@gmail.com>
> [Thomas:
>  - Make sure either BR2_PACKAGE_HOST_QEMU_LINUX_USER_MODE or
>    BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE get selected.
>  - Unconditionally enable FDT support in host-qemu when in system
>    mode (so the option BR2_PACKAGE_HOST_QEMU_FDT has been removed)
>  - Remove the unneeded BR2_PACKAGE_HOST_QEMU_HAS_EMULS option.
>  - Remove the SDL related option, we really on the system to provide
>    SDL.
>  - Remove the BR2_PACKAGE_HOST_QEMU_DEBUG and
>    BR2_PACKAGE_HOST_QEMU_STRIP_BINARY options, since they are not really
>    useful.
>  - Remove HOST_QEMU_SITE and HOST_QEMU_SOURCE definitions, since they
>    are automatically derived from QEMU_SITE and QEMU_SOURCE anyway.
>  - Group things more logically in the .mk file.]
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
[--SNIP--]
> diff --git a/package/qemu/qemu.mk b/package/qemu/qemu.mk
> index 5fa95bb..5c3cfea 100644
> --- a/package/qemu/qemu.mk
> +++ b/package/qemu/qemu.mk
> @@ -61,9 +61,19 @@ endif
>  ifeq ($(HOST_QEMU_ARCH),sh4aeb)
>  HOST_QEMU_ARCH = sh4eb
>  endif
> -HOST_QEMU_TARGETS = $(HOST_QEMU_ARCH)-linux-user
>  
> -ifeq ($(BR2_PACKAGE_HOST_QEMU),y)
> +ifeq ($(BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE),y)
> +HOST_QEMU_TARGETS += $(HOST_QEMU_ARCH)-softmmu
> +HOST_QEMU_OPTS += --enable-system --enable-fdt
> +HOST_QEMU_DEPENDENCIES += host-dtc
> +else
> +HOST_QEMU_OPTS += --disable-system
> +endif
> +
> +ifeq ($(BR2_PACKAGE_HOST_QEMU_LINUX_USER_MODE),y)
> +HOST_QEMU_TARGETS += $(HOST_QEMU_ARCH)-linux-user
> +HOST_QEMU_OPTS += --enable-linux-user

It is unnecessary to pass --enable-system and/or --enable-linux-user
when an explicit --target-list is passed.

Othrewise:

    Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

Regards,
Yann E. MORIN.

>  HOST_QEMU_HOST_SYSTEM_TYPE = $(shell uname -s)
>  ifneq ($(HOST_QEMU_HOST_SYSTEM_TYPE),Linux)
>  $(error "qemu-user can only be used on Linux hosts")
> @@ -84,12 +94,16 @@ HOST_QEMU_COMPARE_VERSION = $(shell test $(HOST_QEMU_HOST_SYSTEM_VERSION) -ge $(
>  # built with kernel headers that are older or the same as the kernel
>  # version running on the host machine.
>  #
> +
>  ifeq ($(BR_BUILDING),y)
>  ifneq ($(HOST_QEMU_COMPARE_VERSION),OK)
>  $(error "Refusing to build qemu-user: target Linux version newer than host's.")
>  endif
> -endif
> -endif
> +endif # BR_BUILDING
> +
> +else # BR2_PACKAGE_HOST_QEMU_LINUX_USER_MODE
> +HOST_QEMU_OPTS += --disable-linux-user
> +endif # BR2_PACKAGE_HOST_QEMU_LINUX_USER_MODE
>  
>  define HOST_QEMU_CONFIGURE_CMDS
>  	cd $(@D); $(HOST_CONFIGURE_OPTS) ./configure    \
> @@ -100,7 +114,8 @@ define HOST_QEMU_CONFIGURE_CMDS
>  		--host-cc="$(HOSTCC)"                   \
>  		--python=$(HOST_DIR)/usr/bin/python2    \
>  		--extra-cflags="$(HOST_CFLAGS)"         \
> -		--extra-ldflags="$(HOST_LDFLAGS)"
> +		--extra-ldflags="$(HOST_LDFLAGS)"       \
> +		$(HOST_QEMU_OPTS)
>  endef
>  
>  define HOST_QEMU_BUILD_CMDS
> -- 
> 2.7.4
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 2/4] vde2: enable building host package

[Yann E. MORIN]

Simon, Thomas, All,

On 2016-07-04 00:24 +0200, Thomas Petazzoni spake thusly:
> From: Simon Maes <simonn.maes@gmail.com>
> 
> In preparation of enabling vde2 support in host-qemu package.

I'm not sure why we need to have a host-qemu with VDE2 support...

As far as I understand it, VDE2 is needed for complex and complicated
cloud-like networking infrastructure, whereas we provide a host-qemu
that will serve to quickly test a Buildroot build.

I think ew should not add such a dependency.

However, I'll still do a review...

> Signed-off-by: Simon Maes <simonn.maes@gmail.com>
> [Thomas:
>  - add HOST_VDE2_CONF_OPTS and HOST_VDE2_MAKE, like we have for the
>    target variant.]
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  package/vde2/vde2.mk | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/package/vde2/vde2.mk b/package/vde2/vde2.mk
> index bad758d..d9b11bb 100644
> --- a/package/vde2/vde2.mk
> +++ b/package/vde2/vde2.mk
> @@ -37,4 +37,16 @@ VDE2_CONF_OPTS = \
>  #?Package does not build in parallel due to improper make rules
    ^
Non-breakable space here...
Note: there's another one for the first item in the list, above...

Probably my deed, btw...

>  VDE2_MAKE = $(MAKE1)
>  
> +HOST_VDE2_CONF_OPTS = \
> +	--disable-experimental  \
> +	--disable-cryptcab      \
> +	--disable-pcap          \
> +	--disable-python        \
> +	--disable-profile       \
> +	--disable-kernel-switch \
> +	--enable-tuntap         \
> +
> +HOST_VDE2_MAKE = $(MAKE1)
> +
>  $(eval $(autotools-package))
> +$(eval $(host-autotools-package))

    Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

Regards,
Yann E. MORIN.

> -- 
> 2.7.4
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 3/4] qemu: add support for vde2

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 > From: Simon Maes <simonn.maes@gmail.com>
 > Additional configuration for qemu package:
 > - Enable VDE2 support for qemu

 > Signed-off-by: Simon Maes <simonn.maes@gmail.com>
 > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

I've tweaked the config text to make it clear this is about host-qemu
and committed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH 0/4] System emulation support in host-qemu

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 > Hello,
 > This is a respin of Simon Maes series adding system emulation support
 > to the host-qemu package. Compared to the original series from Simon:

 >  - The initial patch adding the support has been reworked, taking into
 >    account numerous comments from Arnout, plus additional
 >    simplifications and improvements.

 >  - The patch adding version selection has been dropped. We can revisit
 >    that later, but adding version selection is complicated (think
 >    about hash verification for example)

 >  - The cleanup patch from Simon has been dropped, as it didn't clean
 >    up things according to Buildroot best practices.

 >  - The patch adding the "qemu-system-run" target has been
 >    dropped. That's too specific, and if people really want to automate
 >    starting qemu, they can write a script for it outside of Buildroot.

 >  - A patch to bump to Qemu 2.6.0 has been added.

Committed series, thanks!

-- 
Bye, Peter Korsgaard