[Buildroot] [PATCH v6 0/5] Introduce bmap-tools as target + host package

[Buildroot] [PATCH v6 0/5] Introduce bmap-tools as target + host package

[Thomas Petazzoni]

Hello,

Here is a v6 of the patch series adding bmaptools as a target + host
package, as well as corresponding test cases.

Compared to the v5:

 - Used a filesystem image size of 64 MB so that it is compatible with
   the requirement of Qemu >= 5.1 that SD card images should have
   power of two sizes.

 - Extended the timeout to 60 seconds for the target bmaptools test as
   it can take a bit of time on slow machines to run the Qemu test.

Thanks!

Thomas

Nicolas Carrier (2):
  package/bmap-tools: new package
  support/testing/tests/package/test_bmap_tools: new test

Thomas Petazzoni (3):
  package/bmap-tools: enable host package
  support/testing/infra/emulator.py: update pre-built kernels
  support/testing/tests/package/test_bmap_tools: add test for host
    bmap-tools

 DEVELOPERS                                    |  6 ++
 package/Config.in                             |  1 +
 package/Config.in.host                        |  1 +
 package/bmap-tools/Config.in                  | 44 +++++++++
 package/bmap-tools/Config.in.host             | 12 +++
 package/bmap-tools/bmap-tools.hash            |  3 +
 package/bmap-tools/bmap-tools.mk              | 14 +++
 support/testing/infra/emulator.py             |  8 +-
 .../tests/package/sample_bmap_tools.sh        | 15 ++++
 .../testing/tests/package/test_bmap_tools.py  | 90 +++++++++++++++++++
 10 files changed, 190 insertions(+), 4 deletions(-)
 create mode 100644 package/bmap-tools/Config.in
 create mode 100644 package/bmap-tools/Config.in.host
 create mode 100644 package/bmap-tools/bmap-tools.hash
 create mode 100644 package/bmap-tools/bmap-tools.mk
 create mode 100755 support/testing/tests/package/sample_bmap_tools.sh
 create mode 100644 support/testing/tests/package/test_bmap_tools.py

-- 
2.31.1

[Buildroot] [PATCH v6 1/5] package/bmap-tools: new package

[Thomas Petazzoni]

From: Nicolas Carrier <nicolas.carrier@orolia.com>

Signed-off-by: Nicolas Carrier <nicolas.carrier@orolia.com>
[Thomas: numerous reworks, add myself in DEVELOPERS]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
 DEVELOPERS                         |  2 ++
 package/Config.in                  |  1 +
 package/bmap-tools/Config.in       | 44 ++++++++++++++++++++++++++++++
 package/bmap-tools/bmap-tools.hash |  3 ++
 package/bmap-tools/bmap-tools.mk   | 13 +++++++++
 5 files changed, 63 insertions(+)
 create mode 100644 package/bmap-tools/Config.in
 create mode 100644 package/bmap-tools/bmap-tools.hash
 create mode 100644 package/bmap-tools/bmap-tools.mk

diff --git a/DEVELOPERS b/DEVELOPERS
index f6621e9670..ad6ad79ade 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -1985,6 +1985,7 @@ F:	board/intel/galileo/
 F:	configs/galileo_defconfig
 
 N:	Nicolas Carrier <nicolas.carrier@orolia.com>
+F:	package/bmap-tools/
 F:	package/php-xdebug/
 
 N:	Nicolas Cavallari <nicolas.cavallari@green-communications.fr>
@@ -2635,6 +2636,7 @@ F:	configs/stm32mp157c_dk2_defconfig
 F:	package/android-tools/
 F:	package/b43-firmware/
 F:	package/b43-fwcutter/
+F:	package/bmap-tools/
 F:	package/c-periphery/
 F:	package/cdrkit/
 F:	package/cifs-utils/
diff --git a/package/Config.in b/package/Config.in
index cb8ece4919..f3d6503427 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -196,6 +196,7 @@ menu "Filesystem and flash utilities"
 	source "package/aufs/Config.in"
 	source "package/aufs-util/Config.in"
 	source "package/autofs/Config.in"
+	source "package/bmap-tools/Config.in"
 	source "package/btrfs-progs/Config.in"
 	source "package/cifs-utils/Config.in"
 	source "package/cpio/Config.in"
diff --git a/package/bmap-tools/Config.in b/package/bmap-tools/Config.in
new file mode 100644
index 0000000000..83b4d92786
--- /dev/null
+++ b/package/bmap-tools/Config.in
@@ -0,0 +1,44 @@
+config BR2_PACKAGE_BMAP_TOOLS
+	bool "bmap-tools"
+	depends on BR2_PACKAGE_PYTHON3 || BR2_PACKAGE_PYTHON
+	select BR2_PACKAGE_PYTHON3_PYEXPAT if BR2_PACKAGE_PYTHON3 # runtime
+	select BR2_PACKAGE_PYTHON_PYEXPAT if BR2_PACKAGE_PYTHON # runtime
+	select BR2_PACKAGE_PYTHON_SETUPTOOLS # runtime, pkg_resources
+	select BR2_PACKAGE_PYTHON_SIX # runtime
+	help
+	  Tool to flash image files to block devices using the block map
+	  bmaptool is a generic tool for creating the block map (bmap)
+	  for a file, and copying files using the block map. The idea is
+	  that large file containing unused blocks, like raw system
+	  image files, can be copied or flashed a lot faster with
+	  bmaptool than with traditional tools like "dd" or "cp".
+
+	  Relevant optional dependencies:
+
+	  - Busybox compiled with CONFIG_BZIP2, or BR2_PACKAGE_BZIP2,
+	    to support bz2 compressed images
+
+	  - BR2_PACKAGE_PIGZ for optimized gzip decompression
+
+	  - Busybox compiled with CONFIG_LZOP, or BR2_PACKAGE_LZOP, to
+	    support lzo compressed images.
+
+	  - BR2_PACKAGE_ZIP to support zip compressed images
+
+	  - BR2_PACKAGE_LZ4 to support lz4 compressed images
+
+	  - BR2_PACKAGE_ZSTD to support zstd compressed images
+
+	  - Busybox compiled CONFIG_FEATURE_SEAMLESS_* or
+	    BR2_PACKAGE_TAR to support tar compressed images
+
+	  - BR2_PACKAGE_OPENSSH to retrieve images over SSH.
+
+	  - BR2_PACKAGE_SSHPASS to retrieve images over SSH if
+	    password authentication is used.
+
+	  Image signature verification is not supported as
+	  python-gpgme is not available. Optimized bzip2 decompression
+	  is not supported as pbzip2 is not available.
+
+	  https://github.com/intel/bmap-tools
diff --git a/package/bmap-tools/bmap-tools.hash b/package/bmap-tools/bmap-tools.hash
new file mode 100644
index 0000000000..82a570ecac
--- /dev/null
+++ b/package/bmap-tools/bmap-tools.hash
@@ -0,0 +1,3 @@
+# Locally calculated
+sha256  0658afb972e7221aa16fece3f84e29e5102e901c929253d7fb6040bde28243f7  bmap-tools-3.6.tar.gz
+sha256  8177f97513213526df2cf6184d8ff986c675afb514d4e68a404010521b880643  COPYING
diff --git a/package/bmap-tools/bmap-tools.mk b/package/bmap-tools/bmap-tools.mk
new file mode 100644
index 0000000000..c8b41c225f
--- /dev/null
+++ b/package/bmap-tools/bmap-tools.mk
@@ -0,0 +1,13 @@
+################################################################################
+#
+# bmap-tools
+#
+################################################################################
+
+BMAP_TOOLS_VERSION = 3.6
+BMAP_TOOLS_SITE = $(call github,intel,bmap-tools,v$(BMAP_TOOLS_VERSION))
+BMAP_TOOLS_LICENSE = GPL-2.0
+BMAP_TOOLS_LICENSE_FILES = COPYING
+BMAP_TOOLS_SETUP_TYPE = setuptools
+
+$(eval $(python-package))
-- 
2.31.1

[Buildroot] [PATCH v6 2/5] package/bmap-tools: enable host package

[Thomas Petazzoni]

bmaptool allows to drastically reduce the amount of data to transfer
when writing to an SD card. Example with a 544 MiB sdcard.img:

$ bmaptool create sdcard.img > sdcard.bmap
$ gzip sdcard.img
$ bmaptool copy sdcard.img.gz /dev/sdc
bmaptool: info: discovered bmap file 'sdcard.bmap'
bmaptool: info: block map format version 2.0
bmaptool: info: 139265 blocks of size 4096 (544.0 MiB), mapped 23918 blocks (93.4 MiB or 17.2%)
bmaptool: info: copying image 'sdcard.img.gz' to block device '/dev/sdc' using bmap file 'sdcard.bmap'
bmaptool: info: 100% copied
bmaptool: info: synchronizing '/dev/sdc'
bmaptool: info: copying time: 7.7s, copying speed 12.1 MiB/sec

So it means that instead of writing 544 MiB, only 93.4 MiB had to be
written.

In terms of implementation details, compared to the target bmap-tools
package, there are fewer "selects" that are needed because:

 - The dependency on setuptools is not needed, because the package
   uses the setuptools SETUP_TYPE, so host-python-setuptools is
   already a build dependency.

 - host-python and host-python3 are always built with Expat XML
   support.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
 package/Config.in.host            |  1 +
 package/bmap-tools/Config.in.host | 12 ++++++++++++
 package/bmap-tools/bmap-tools.mk  |  1 +
 3 files changed, 14 insertions(+)
 create mode 100644 package/bmap-tools/Config.in.host

diff --git a/package/Config.in.host b/package/Config.in.host
index 064c98af3d..8d443ea255 100644
--- a/package/Config.in.host
+++ b/package/Config.in.host
@@ -5,6 +5,7 @@ menu "Host utilities"
 	source "package/android-tools/Config.in.host"
 	source "package/asn1c/Config.in.host"
 	source "package/babeltrace2/Config.in.host"
+	source "package/bmap-tools/Config.in.host"
 	source "package/btrfs-progs/Config.in.host"
 	source "package/cbootimage/Config.in.host"
 	source "package/checkpolicy/Config.in.host"
diff --git a/package/bmap-tools/Config.in.host b/package/bmap-tools/Config.in.host
new file mode 100644
index 0000000000..a20829030d
--- /dev/null
+++ b/package/bmap-tools/Config.in.host
@@ -0,0 +1,12 @@
+config BR2_PACKAGE_HOST_BMAP_TOOLS
+	bool "host bmap-tools"
+	select BR2_PACKAGE_HOST_PYTHON_SIX # runtime
+	help
+	  Tool to flash image files to block devices using the block map
+	  bmaptool is a generic tool for creating the block map (bmap)
+	  for a file, and copying files using the block map. The idea is
+	  that large file containing unused blocks, like raw system
+	  image files, can be copied or flashed a lot faster with
+	  bmaptool than with traditional tools like "dd" or "cp".
+
+	  https://github.com/intel/bmap-tools
diff --git a/package/bmap-tools/bmap-tools.mk b/package/bmap-tools/bmap-tools.mk
index c8b41c225f..32399ca151 100644
--- a/package/bmap-tools/bmap-tools.mk
+++ b/package/bmap-tools/bmap-tools.mk
@@ -11,3 +11,4 @@ BMAP_TOOLS_LICENSE_FILES = COPYING
 BMAP_TOOLS_SETUP_TYPE = setuptools
 
 $(eval $(python-package))
+$(eval $(host-python-package))
-- 
2.31.1

[Buildroot] [PATCH v6 3/5] support/testing/infra/emulator.py: update pre-built kernels

[Thomas Petazzoni]

The pre-built vexpress kernel used by the testing infrastructure is a
4.0.0 kernel, which is getting old to be used with reasonably recent
toolchains.

This commit updates the pre-built kernels for both the versatile and
vexpress machines to 5.10.7 (they have already been put online).

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
 support/testing/infra/emulator.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/support/testing/infra/emulator.py b/support/testing/infra/emulator.py
index 5611ec96e8..0a77eb80fc 100644
--- a/support/testing/infra/emulator.py
+++ b/support/testing/infra/emulator.py
@@ -54,16 +54,16 @@ class Emulator(object):
 
                 if arch == "armv7":
                     kernel = infra.download(self.downloaddir,
-                                            "kernel-vexpress")
+                                            "kernel-vexpress-5.10.7")
                     dtb = infra.download(self.downloaddir,
-                                         "vexpress-v2p-ca9.dtb")
+                                         "vexpress-v2p-ca9-5.10.7.dtb")
                     qemu_cmd += ["-dtb", dtb]
                     qemu_cmd += ["-M", "vexpress-a9"]
                 elif arch == "armv5":
                     kernel = infra.download(self.downloaddir,
-                                            "kernel-versatile-4.19")
+                                            "kernel-versatile-5.10.7")
                     dtb = infra.download(self.downloaddir,
-                                         "versatile-pb-4.19.dtb")
+                                         "versatile-pb-5.10.7.dtb")
                     qemu_cmd += ["-dtb", dtb]
                     qemu_cmd += ["-M", "versatilepb"]
                     qemu_cmd += ["-device", "virtio-rng-pci"]
-- 
2.31.1

[Buildroot] [PATCH v6 4/5] support/testing/tests/package/test_bmap_tools: new test

[Thomas Petazzoni]

From: Nicolas Carrier <nicolas.carrier@orolia.com>

This patch implements a simple test in which a dummy file system image
is created, then `bmaptool create` and `bmaptool copy` are used to copy
it to another file.

Signed-off-by: Nicolas Carrier <nicolas.carrier@orolia.com>
[Thomas: several reworks, add myself to DEVELOPERS]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
 DEVELOPERS                                    |  4 ++
 .../tests/package/sample_bmap_tools.sh        | 15 +++++
 .../testing/tests/package/test_bmap_tools.py  | 61 +++++++++++++++++++
 3 files changed, 80 insertions(+)
 create mode 100755 support/testing/tests/package/sample_bmap_tools.sh
 create mode 100644 support/testing/tests/package/test_bmap_tools.py

diff --git a/DEVELOPERS b/DEVELOPERS
index ad6ad79ade..b4518b0cc7 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -1987,6 +1987,8 @@ F:	configs/galileo_defconfig
 N:	Nicolas Carrier <nicolas.carrier@orolia.com>
 F:	package/bmap-tools/
 F:	package/php-xdebug/
+F:	support/testing/tests/package/sample_bmap_tools.sh
+F:	support/testing/tests/package/test_bmap_tools.py
 
 N:	Nicolas Cavallari <nicolas.cavallari@green-communications.fr>
 F:	package/libgit2/
@@ -2687,6 +2689,8 @@ F:	package/squashfs/
 F:	package/wayland/
 F:	package/weston/
 F:	support/testing/tests/boot/test_syslinux.py
+F:	support/testing/tests/package/sample_bmap_tools.sh
+F:	support/testing/tests/package/test_bmap_tools.py
 F:	toolchain/
 
 N:	Timo Ketola <timo.ketola@exertus.fi>
diff --git a/support/testing/tests/package/sample_bmap_tools.sh b/support/testing/tests/package/sample_bmap_tools.sh
new file mode 100755
index 0000000000..7c90368c17
--- /dev/null
+++ b/support/testing/tests/package/sample_bmap_tools.sh
@@ -0,0 +1,15 @@
+#!/bin/sh
+# simple test which creates a dummy file system image, then use bmaptool create
+# and bmaptool copy to copy it to another file
+
+set -xeu
+
+# create the necessary test files
+dd if=/dev/zero of=disk.img bs=2M count=1
+mkfs.ext4 disk.img
+fallocate -d disk.img
+dd if=/dev/zero of=copy.img bs=2M count=1
+
+# do a test copy of the file system image
+bmaptool create -o disk.img.bmap disk.img
+bmaptool copy disk.img copy.img
diff --git a/support/testing/tests/package/test_bmap_tools.py b/support/testing/tests/package/test_bmap_tools.py
new file mode 100644
index 0000000000..e8802fc057
--- /dev/null
+++ b/support/testing/tests/package/test_bmap_tools.py
@@ -0,0 +1,61 @@
+import os
+import infra
+
+from infra.basetest import BRTest
+
+
+class TestBmapTools(BRTest):
+    __test__ = False
+    sample_script = "tests/package/sample_bmap_tools.sh"
+    copy_script = 'tests/package/copy-sample-script-to-target.sh'
+    config = \
+        """
+        BR2_arm=y
+        BR2_cortex_a8=y
+        BR2_TOOLCHAIN_EXTERNAL=y
+        BR2_TOOLCHAIN_EXTERNAL_BOOTLIN=y
+        BR2_PACKAGE_BMAP_TOOLS=y
+        BR2_ROOTFS_POST_BUILD_SCRIPT="{}"
+        BR2_ROOTFS_POST_SCRIPT_ARGS="{}"
+        BR2_TARGET_ROOTFS_EXT2=y
+        BR2_TARGET_ROOTFS_EXT2_4=y
+        BR2_TARGET_ROOTFS_EXT2_SIZE="65536"
+        # BR2_TARGET_ROOTFS_TAR is not set
+        BR2_PACKAGE_UTIL_LINUX=y
+        BR2_PACKAGE_UTIL_LINUX_FALLOCATE=y
+        BR2_PACKAGE_E2FSPROGS=y
+        BR2_PACKAGE_UTIL_LINUX_LIBUUID=y
+        """.format(infra.filepath(copy_script),
+                   infra.filepath(sample_script))
+    timeout = 60
+
+    def login(self):
+        img = os.path.join(self.builddir, "images", "rootfs.ext4")
+        self.emulator.boot(arch="armv7",
+                           kernel="builtin",
+                           kernel_cmdline=["root=/dev/mmcblk0",
+                                           "rootfstype=ext4"],
+                           options=["-drive", "file={},if=sd,format=raw".format(img)])
+        self.emulator.login()
+
+    def test_run(self):
+        self.login()
+        cmd = "/root/{}".format(os.path.basename(self.sample_script))
+        _, exit_code = self.emulator.run(cmd, timeout=20)
+        self.assertEqual(exit_code, 0)
+
+
+class TestPy2BmapTools(TestBmapTools):
+    __test__ = True
+    config = TestBmapTools.config + \
+        """
+        BR2_PACKAGE_PYTHON=y
+        """
+
+
+class TestPy3BmapTools(TestBmapTools):
+    __test__ = True
+    config = TestBmapTools.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        """
-- 
2.31.1

[Buildroot] [PATCH v6 5/5] support/testing/tests/package/test_bmap_tools: add test for host bmap-tools

[Thomas Petazzoni]

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
 .../testing/tests/package/test_bmap_tools.py  | 29 +++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/support/testing/tests/package/test_bmap_tools.py b/support/testing/tests/package/test_bmap_tools.py
index e8802fc057..a0f8d5c2ad 100644
--- a/support/testing/tests/package/test_bmap_tools.py
+++ b/support/testing/tests/package/test_bmap_tools.py
@@ -1,5 +1,6 @@
 import os
 import infra
+import subprocess
 
 from infra.basetest import BRTest
 
@@ -59,3 +60,31 @@ class TestPy3BmapTools(TestBmapTools):
         """
         BR2_PACKAGE_PYTHON3=y
         """
+
+
+class TestHostBmapTools(BRTest):
+    config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
+        """
+        BR2_PACKAGE_HOST_BMAP_TOOLS=y
+        # BR2_TARGET_ROOTFS_TAR is not set
+        BR2_TARGET_ROOTFS_EXT2=y
+        """
+
+    def test_run(self):
+        cmd = [os.path.join(self.b.builddir, "host", "bin", "bmaptool"),
+               "create",
+               "-o", os.path.join(self.b.builddir, "images", "rootfs.ext2.bmap"),
+               os.path.join(self.b.builddir, "images", "rootfs.ext2")]
+        ret = subprocess.call(cmd,
+                              stdout=self.b.logfile,
+                              stderr=self.b.logfile)
+        self.assertEqual(ret, 0)
+
+        cmd = [os.path.join(self.b.builddir, "host", "bin", "bmaptool"),
+               "copy",
+               os.path.join(self.b.builddir, "images", "rootfs.ext2"),
+               os.path.join(self.b.builddir, "images", "rootfs.ext2.copy")]
+        ret = subprocess.call(cmd,
+                              stdout=self.b.logfile,
+                              stderr=self.b.logfile)
+        self.assertEqual(ret, 0)
-- 
2.31.1

[Buildroot] [PATCH v6 1/5] package/bmap-tools: new package

[Yann E. MORIN]

Thomas, All,

On 2021-06-21 23:01 +0200, Thomas Petazzoni spake thusly:
> From: Nicolas Carrier <nicolas.carrier@orolia.com>
> 
> Signed-off-by: Nicolas Carrier <nicolas.carrier@orolia.com>
> [Thomas: numerous reworks, add myself in DEVELOPERS]
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>

I am not too sure what bmap provides over 'dd conv=sparse'...

dd has a blocking size, which can be adjusted to the underlying device,
and does not need the input file to actually be sparse (i.e. any
block-aligned runs of zeroes will not be transfered)...

Also, we have seen in the past that not writing zero-blocks on the
destination device, can lead to filesystem corruption, because all-zero
filesystem metadata suddenly may no longer be all-zero...

But oh well...

Applied to master, thanks.

Regards,
Yann E. MORIN.

> ---
>  DEVELOPERS                         |  2 ++
>  package/Config.in                  |  1 +
>  package/bmap-tools/Config.in       | 44 ++++++++++++++++++++++++++++++
>  package/bmap-tools/bmap-tools.hash |  3 ++
>  package/bmap-tools/bmap-tools.mk   | 13 +++++++++
>  5 files changed, 63 insertions(+)
>  create mode 100644 package/bmap-tools/Config.in
>  create mode 100644 package/bmap-tools/bmap-tools.hash
>  create mode 100644 package/bmap-tools/bmap-tools.mk
> 
> diff --git a/DEVELOPERS b/DEVELOPERS
> index f6621e9670..ad6ad79ade 100644
> --- a/DEVELOPERS
> +++ b/DEVELOPERS
> @@ -1985,6 +1985,7 @@ F:	board/intel/galileo/
>  F:	configs/galileo_defconfig
>  
>  N:	Nicolas Carrier <nicolas.carrier@orolia.com>
> +F:	package/bmap-tools/
>  F:	package/php-xdebug/
>  
>  N:	Nicolas Cavallari <nicolas.cavallari@green-communications.fr>
> @@ -2635,6 +2636,7 @@ F:	configs/stm32mp157c_dk2_defconfig
>  F:	package/android-tools/
>  F:	package/b43-firmware/
>  F:	package/b43-fwcutter/
> +F:	package/bmap-tools/
>  F:	package/c-periphery/
>  F:	package/cdrkit/
>  F:	package/cifs-utils/
> diff --git a/package/Config.in b/package/Config.in
> index cb8ece4919..f3d6503427 100644
> --- a/package/Config.in
> +++ b/package/Config.in
> @@ -196,6 +196,7 @@ menu "Filesystem and flash utilities"
>  	source "package/aufs/Config.in"
>  	source "package/aufs-util/Config.in"
>  	source "package/autofs/Config.in"
> +	source "package/bmap-tools/Config.in"
>  	source "package/btrfs-progs/Config.in"
>  	source "package/cifs-utils/Config.in"
>  	source "package/cpio/Config.in"
> diff --git a/package/bmap-tools/Config.in b/package/bmap-tools/Config.in
> new file mode 100644
> index 0000000000..83b4d92786
> --- /dev/null
> +++ b/package/bmap-tools/Config.in
> @@ -0,0 +1,44 @@
> +config BR2_PACKAGE_BMAP_TOOLS
> +	bool "bmap-tools"
> +	depends on BR2_PACKAGE_PYTHON3 || BR2_PACKAGE_PYTHON
> +	select BR2_PACKAGE_PYTHON3_PYEXPAT if BR2_PACKAGE_PYTHON3 # runtime
> +	select BR2_PACKAGE_PYTHON_PYEXPAT if BR2_PACKAGE_PYTHON # runtime
> +	select BR2_PACKAGE_PYTHON_SETUPTOOLS # runtime, pkg_resources
> +	select BR2_PACKAGE_PYTHON_SIX # runtime
> +	help
> +	  Tool to flash image files to block devices using the block map
> +	  bmaptool is a generic tool for creating the block map (bmap)
> +	  for a file, and copying files using the block map. The idea is
> +	  that large file containing unused blocks, like raw system
> +	  image files, can be copied or flashed a lot faster with
> +	  bmaptool than with traditional tools like "dd" or "cp".
> +
> +	  Relevant optional dependencies:
> +
> +	  - Busybox compiled with CONFIG_BZIP2, or BR2_PACKAGE_BZIP2,
> +	    to support bz2 compressed images
> +
> +	  - BR2_PACKAGE_PIGZ for optimized gzip decompression
> +
> +	  - Busybox compiled with CONFIG_LZOP, or BR2_PACKAGE_LZOP, to
> +	    support lzo compressed images.
> +
> +	  - BR2_PACKAGE_ZIP to support zip compressed images
> +
> +	  - BR2_PACKAGE_LZ4 to support lz4 compressed images
> +
> +	  - BR2_PACKAGE_ZSTD to support zstd compressed images
> +
> +	  - Busybox compiled CONFIG_FEATURE_SEAMLESS_* or
> +	    BR2_PACKAGE_TAR to support tar compressed images
> +
> +	  - BR2_PACKAGE_OPENSSH to retrieve images over SSH.
> +
> +	  - BR2_PACKAGE_SSHPASS to retrieve images over SSH if
> +	    password authentication is used.
> +
> +	  Image signature verification is not supported as
> +	  python-gpgme is not available. Optimized bzip2 decompression
> +	  is not supported as pbzip2 is not available.
> +
> +	  https://github.com/intel/bmap-tools
> diff --git a/package/bmap-tools/bmap-tools.hash b/package/bmap-tools/bmap-tools.hash
> new file mode 100644
> index 0000000000..82a570ecac
> --- /dev/null
> +++ b/package/bmap-tools/bmap-tools.hash
> @@ -0,0 +1,3 @@
> +# Locally calculated
> +sha256  0658afb972e7221aa16fece3f84e29e5102e901c929253d7fb6040bde28243f7  bmap-tools-3.6.tar.gz
> +sha256  8177f97513213526df2cf6184d8ff986c675afb514d4e68a404010521b880643  COPYING
> diff --git a/package/bmap-tools/bmap-tools.mk b/package/bmap-tools/bmap-tools.mk
> new file mode 100644
> index 0000000000..c8b41c225f
> --- /dev/null
> +++ b/package/bmap-tools/bmap-tools.mk
> @@ -0,0 +1,13 @@
> +################################################################################
> +#
> +# bmap-tools
> +#
> +################################################################################
> +
> +BMAP_TOOLS_VERSION = 3.6
> +BMAP_TOOLS_SITE = $(call github,intel,bmap-tools,v$(BMAP_TOOLS_VERSION))
> +BMAP_TOOLS_LICENSE = GPL-2.0
> +BMAP_TOOLS_LICENSE_FILES = COPYING
> +BMAP_TOOLS_SETUP_TYPE = setuptools
> +
> +$(eval $(python-package))
> -- 
> 2.31.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH v6 2/5] package/bmap-tools: enable host package

[Yann E. MORIN]

Thomas, All,

On 2021-06-21 23:01 +0200, Thomas Petazzoni spake thusly:
> bmaptool allows to drastically reduce the amount of data to transfer
> when writing to an SD card. Example with a 544 MiB sdcard.img:
> 
> $ bmaptool create sdcard.img > sdcard.bmap
> $ gzip sdcard.img
> $ bmaptool copy sdcard.img.gz /dev/sdc
> bmaptool: info: discovered bmap file 'sdcard.bmap'
> bmaptool: info: block map format version 2.0
> bmaptool: info: 139265 blocks of size 4096 (544.0 MiB), mapped 23918 blocks (93.4 MiB or 17.2%)
> bmaptool: info: copying image 'sdcard.img.gz' to block device '/dev/sdc' using bmap file 'sdcard.bmap'
> bmaptool: info: 100% copied
> bmaptool: info: synchronizing '/dev/sdc'
> bmaptool: info: copying time: 7.7s, copying speed 12.1 MiB/sec
> 
> So it means that instead of writing 544 MiB, only 93.4 MiB had to be
> written.
> 
> In terms of implementation details, compared to the target bmap-tools
> package, there are fewer "selects" that are needed because:
> 
>  - The dependency on setuptools is not needed, because the package
>    uses the setuptools SETUP_TYPE, so host-python-setuptools is
>    already a build dependency.
> 
>  - host-python and host-python3 are always built with Expat XML
>    support.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>

Applied to master, thanks.

Regards,
Yann E. MORIN.

> ---
>  package/Config.in.host            |  1 +
>  package/bmap-tools/Config.in.host | 12 ++++++++++++
>  package/bmap-tools/bmap-tools.mk  |  1 +
>  3 files changed, 14 insertions(+)
>  create mode 100644 package/bmap-tools/Config.in.host
> 
> diff --git a/package/Config.in.host b/package/Config.in.host
> index 064c98af3d..8d443ea255 100644
> --- a/package/Config.in.host
> +++ b/package/Config.in.host
> @@ -5,6 +5,7 @@ menu "Host utilities"
>  	source "package/android-tools/Config.in.host"
>  	source "package/asn1c/Config.in.host"
>  	source "package/babeltrace2/Config.in.host"
> +	source "package/bmap-tools/Config.in.host"
>  	source "package/btrfs-progs/Config.in.host"
>  	source "package/cbootimage/Config.in.host"
>  	source "package/checkpolicy/Config.in.host"
> diff --git a/package/bmap-tools/Config.in.host b/package/bmap-tools/Config.in.host
> new file mode 100644
> index 0000000000..a20829030d
> --- /dev/null
> +++ b/package/bmap-tools/Config.in.host
> @@ -0,0 +1,12 @@
> +config BR2_PACKAGE_HOST_BMAP_TOOLS
> +	bool "host bmap-tools"
> +	select BR2_PACKAGE_HOST_PYTHON_SIX # runtime
> +	help
> +	  Tool to flash image files to block devices using the block map
> +	  bmaptool is a generic tool for creating the block map (bmap)
> +	  for a file, and copying files using the block map. The idea is
> +	  that large file containing unused blocks, like raw system
> +	  image files, can be copied or flashed a lot faster with
> +	  bmaptool than with traditional tools like "dd" or "cp".
> +
> +	  https://github.com/intel/bmap-tools
> diff --git a/package/bmap-tools/bmap-tools.mk b/package/bmap-tools/bmap-tools.mk
> index c8b41c225f..32399ca151 100644
> --- a/package/bmap-tools/bmap-tools.mk
> +++ b/package/bmap-tools/bmap-tools.mk
> @@ -11,3 +11,4 @@ BMAP_TOOLS_LICENSE_FILES = COPYING
>  BMAP_TOOLS_SETUP_TYPE = setuptools
>  
>  $(eval $(python-package))
> +$(eval $(host-python-package))
> -- 
> 2.31.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH v6 3/5] support/testing/infra/emulator.py: update pre-built kernels

[Yann E. MORIN]

Thomas, All,

On 2021-06-21 23:01 +0200, Thomas Petazzoni spake thusly:
> The pre-built vexpress kernel used by the testing infrastructure is a
> 4.0.0 kernel, which is getting old to be used with reasonably recent
> toolchains.
> 
> This commit updates the pre-built kernels for both the versatile and
> vexpress machines to 5.10.7 (they have already been put online).
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>

Applied to master, thanks.

Regards,
Yann E. MORIN.

> ---
>  support/testing/infra/emulator.py | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/support/testing/infra/emulator.py b/support/testing/infra/emulator.py
> index 5611ec96e8..0a77eb80fc 100644
> --- a/support/testing/infra/emulator.py
> +++ b/support/testing/infra/emulator.py
> @@ -54,16 +54,16 @@ class Emulator(object):
>  
>                  if arch == "armv7":
>                      kernel = infra.download(self.downloaddir,
> -                                            "kernel-vexpress")
> +                                            "kernel-vexpress-5.10.7")
>                      dtb = infra.download(self.downloaddir,
> -                                         "vexpress-v2p-ca9.dtb")
> +                                         "vexpress-v2p-ca9-5.10.7.dtb")
>                      qemu_cmd += ["-dtb", dtb]
>                      qemu_cmd += ["-M", "vexpress-a9"]
>                  elif arch == "armv5":
>                      kernel = infra.download(self.downloaddir,
> -                                            "kernel-versatile-4.19")
> +                                            "kernel-versatile-5.10.7")
>                      dtb = infra.download(self.downloaddir,
> -                                         "versatile-pb-4.19.dtb")
> +                                         "versatile-pb-5.10.7.dtb")
>                      qemu_cmd += ["-dtb", dtb]
>                      qemu_cmd += ["-M", "versatilepb"]
>                      qemu_cmd += ["-device", "virtio-rng-pci"]
> -- 
> 2.31.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH v6 4/5] support/testing/tests/package/test_bmap_tools: new test

[Yann E. MORIN]

Thomas, All,

On 2021-06-21 23:01 +0200, Thomas Petazzoni spake thusly:
> From: Nicolas Carrier <nicolas.carrier@orolia.com>
> 
> This patch implements a simple test in which a dummy file system image
> is created, then `bmaptool create` and `bmaptool copy` are used to copy
> it to another file.
> 
> Signed-off-by: Nicolas Carrier <nicolas.carrier@orolia.com>
> [Thomas: several reworks, add myself to DEVELOPERS]
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> ---
[--SNIP--]
> diff --git a/support/testing/tests/package/sample_bmap_tools.sh b/support/testing/tests/package/sample_bmap_tools.sh
> new file mode 100755
> index 0000000000..7c90368c17
> --- /dev/null
> +++ b/support/testing/tests/package/sample_bmap_tools.sh
> @@ -0,0 +1,15 @@
> +#!/bin/sh
> +# simple test which creates a dummy file system image, then use bmaptool create
> +# and bmaptool copy to copy it to another file
> +
> +set -xeu
> +
> +# create the necessary test files
> +dd if=/dev/zero of=disk.img bs=2M count=1
> +mkfs.ext4 disk.img
> +fallocate -d disk.img
> +dd if=/dev/zero of=copy.img bs=2M count=1
> +
> +# do a test copy of the file system image
> +bmaptool create -o disk.img.bmap disk.img
> +bmaptool copy disk.img copy.img

It is sad that the copy was not actually tested to be identical to the
original, so I added:

    cmp disk.img copy.img

and applied to master, thanks.

Regards,
Yann E. MORIN.

> diff --git a/support/testing/tests/package/test_bmap_tools.py b/support/testing/tests/package/test_bmap_tools.py
> new file mode 100644
> index 0000000000..e8802fc057
> --- /dev/null
> +++ b/support/testing/tests/package/test_bmap_tools.py
> @@ -0,0 +1,61 @@
> +import os
> +import infra
> +
> +from infra.basetest import BRTest
> +
> +
> +class TestBmapTools(BRTest):
> +    __test__ = False
> +    sample_script = "tests/package/sample_bmap_tools.sh"
> +    copy_script = 'tests/package/copy-sample-script-to-target.sh'
> +    config = \
> +        """
> +        BR2_arm=y
> +        BR2_cortex_a8=y
> +        BR2_TOOLCHAIN_EXTERNAL=y
> +        BR2_TOOLCHAIN_EXTERNAL_BOOTLIN=y
> +        BR2_PACKAGE_BMAP_TOOLS=y
> +        BR2_ROOTFS_POST_BUILD_SCRIPT="{}"
> +        BR2_ROOTFS_POST_SCRIPT_ARGS="{}"
> +        BR2_TARGET_ROOTFS_EXT2=y
> +        BR2_TARGET_ROOTFS_EXT2_4=y
> +        BR2_TARGET_ROOTFS_EXT2_SIZE="65536"
> +        # BR2_TARGET_ROOTFS_TAR is not set
> +        BR2_PACKAGE_UTIL_LINUX=y
> +        BR2_PACKAGE_UTIL_LINUX_FALLOCATE=y
> +        BR2_PACKAGE_E2FSPROGS=y
> +        BR2_PACKAGE_UTIL_LINUX_LIBUUID=y
> +        """.format(infra.filepath(copy_script),
> +                   infra.filepath(sample_script))
> +    timeout = 60
> +
> +    def login(self):
> +        img = os.path.join(self.builddir, "images", "rootfs.ext4")
> +        self.emulator.boot(arch="armv7",
> +                           kernel="builtin",
> +                           kernel_cmdline=["root=/dev/mmcblk0",
> +                                           "rootfstype=ext4"],
> +                           options=["-drive", "file={},if=sd,format=raw".format(img)])
> +        self.emulator.login()
> +
> +    def test_run(self):
> +        self.login()
> +        cmd = "/root/{}".format(os.path.basename(self.sample_script))
> +        _, exit_code = self.emulator.run(cmd, timeout=20)
> +        self.assertEqual(exit_code, 0)
> +
> +
> +class TestPy2BmapTools(TestBmapTools):
> +    __test__ = True
> +    config = TestBmapTools.config + \
> +        """
> +        BR2_PACKAGE_PYTHON=y
> +        """
> +
> +
> +class TestPy3BmapTools(TestBmapTools):
> +    __test__ = True
> +    config = TestBmapTools.config + \
> +        """
> +        BR2_PACKAGE_PYTHON3=y
> +        """
> -- 
> 2.31.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH v6 5/5] support/testing/tests/package/test_bmap_tools: add test for host bmap-tools

[Yann E. MORIN]

Thomas, All,

On 2021-06-21 23:01 +0200, Thomas Petazzoni spake thusly:
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> ---
>  .../testing/tests/package/test_bmap_tools.py  | 29 +++++++++++++++++++
>  1 file changed, 29 insertions(+)
> 
> diff --git a/support/testing/tests/package/test_bmap_tools.py b/support/testing/tests/package/test_bmap_tools.py
> index e8802fc057..a0f8d5c2ad 100644
> --- a/support/testing/tests/package/test_bmap_tools.py
> +++ b/support/testing/tests/package/test_bmap_tools.py
> @@ -1,5 +1,6 @@
>  import os
>  import infra
> +import subprocess
>  
>  from infra.basetest import BRTest
>  
> @@ -59,3 +60,31 @@ class TestPy3BmapTools(TestBmapTools):
>          """
>          BR2_PACKAGE_PYTHON3=y
>          """
> +
> +
> +class TestHostBmapTools(BRTest):
> +    config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
> +        """
> +        BR2_PACKAGE_HOST_BMAP_TOOLS=y
> +        # BR2_TARGET_ROOTFS_TAR is not set
> +        BR2_TARGET_ROOTFS_EXT2=y
> +        """
> +
> +    def test_run(self):
> +        cmd = [os.path.join(self.b.builddir, "host", "bin", "bmaptool"),
> +               "create",
> +               "-o", os.path.join(self.b.builddir, "images", "rootfs.ext2.bmap"),
> +               os.path.join(self.b.builddir, "images", "rootfs.ext2")]
> +        ret = subprocess.call(cmd,
> +                              stdout=self.b.logfile,
> +                              stderr=self.b.logfile)
> +        self.assertEqual(ret, 0)
> +
> +        cmd = [os.path.join(self.b.builddir, "host", "bin", "bmaptool"),
> +               "copy",
> +               os.path.join(self.b.builddir, "images", "rootfs.ext2"),
> +               os.path.join(self.b.builddir, "images", "rootfs.ext2.copy")]
> +        ret = subprocess.call(cmd,
> +                              stdout=self.b.logfile,
> +                              stderr=self.b.logfile)
> +        self.assertEqual(ret, 0)

Again, it is sad that the copy is not cehcked to be identical to the
original, so I added a basic test that reads the two files, and compare
their content. Not opitimised, very crude, but we don;t care much for a
~64MiB image file

Applied to master, thanks.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH v6 1/5] package/bmap-tools: new package

[Thomas Petazzoni]

On Fri, 25 Jun 2021 14:35:14 +0200
"Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> I am not too sure what bmap provides over 'dd conv=sparse'...
> 
> dd has a blocking size, which can be adjusted to the underlying device,
> and does not need the input file to actually be sparse (i.e. any
> block-aligned runs of zeroes will not be transfered)...
> 
> Also, we have seen in the past that not writing zero-blocks on the
> destination device, can lead to filesystem corruption, because all-zero
> filesystem metadata suddenly may no longer be all-zero...

Well, this is precisely avoided by bmaptools I believe. bmaptools is
not about skipping zeroes, it's about skipping holes in sparse files.
So if a filesystem utility has explicitly written a huge amount of
zeroes, it's not a hole in the file, and therefore they will be written
as expected by bmaptools.

And with that, there are no risk: block filesystems do not make
assumptions about the state of the data they have not explicitly
written/initialized at the time of mkfs.

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH v6 4/5] support/testing/tests/package/test_bmap_tools: new test

[Thomas Petazzoni]

On Fri, 25 Jun 2021 14:36:57 +0200
"Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> > +# do a test copy of the file system image
> > +bmaptool create -o disk.img.bmap disk.img
> > +bmaptool copy disk.img copy.img  
> 
> It is sad that the copy was not actually tested to be identical to the
> original, so I added:
> 
>     cmp disk.img copy.img
> 
> and applied to master, thanks.

Good point, thanks for the change!

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH v6 1/5] package/bmap-tools: new package

[Yann E. MORIN]

On 2021-06-25 14:45 +0200, Thomas Petazzoni spake thusly:
> On Fri, 25 Jun 2021 14:35:14 +0200
> "Yann E. MORIN" <yann.morin.1998@free.fr> wrote:
> > I am not too sure what bmap provides over 'dd conv=sparse'...
[--SNIP--]
> Well, this is precisely avoided by bmaptools I believe. bmaptools is
> not about skipping zeroes, it's about skipping holes in sparse files.
> So if a filesystem utility has explicitly written a huge amount of
> zeroes, it's not a hole in the file, and therefore they will be written
> as expected by bmaptools.
> 
> And with that, there are no risk: block filesystems do not make
> assumptions about the state of the data they have not explicitly
> written/initialized at the time of mkfs.

Ah, bmap-tool is really just about skipping holes. And it is assumed
that the image has no hole where the data is relevant, even if only
zeroes.

Thanks.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH v6 1/5] package/bmap-tools: new package

[Thomas Petazzoni]

On Fri, 25 Jun 2021 14:54:47 +0200
"Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> Ah, bmap-tool is really just about skipping holes. And it is assumed
> that the image has no hole where the data is relevant, even if only
> zeroes.

Absolutely. For example, if I build an image on my build server, and
run bmaptool create there, it finds the holes and is able to generate a
correct bmap file. If however, I first scp that image file locally, and
then run bmaptool create, it finds no holes because scp-ing the file
has replaced the holes with actual data.

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH v6 3/5] support/testing/infra/emulator.py: update pre-built kernels

[Yann E. MORIN]

Thomas, All,

On 2021-06-21 23:01 +0200, Thomas Petazzoni spake thusly:
> The pre-built vexpress kernel used by the testing infrastructure is a
> 4.0.0 kernel, which is getting old to be used with reasonably recent
> toolchains.
> 
> This commit updates the pre-built kernels for both the versatile and
> vexpress machines to 5.10.7 (they have already been put online).

This breaks the squashfs tests:

    Filesystem uses "lz4" compression. This is not supported
    List of all partitions:
    1f00          131072 mtdblock0
     (driver?)
    1f01           32768 mtdblock1
     (driver?)
    b300            2048 mmcblk0
     driver: mmcblk
    No filesystem could mount root, tried:
     squashfs

    Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(179,0)

And indeed, if we boot this kernel with a gzip-compressed squashfs, ad
we look at the configuration (irrelevant options omitted for brevity):

    # zcat /proc/config.gz |grep SQUA
    CONFIG_SQUASHFS=y
    CONFIG_SQUASHFS_ZLIB=y
    # CONFIG_SQUASHFS_LZ4 is not set
    CONFIG_SQUASHFS_LZO=y
    # CONFIG_SQUASHFS_XZ is not set

Could you rebuild the kernel (but not nice for those like who already
have it cached locally), or build a newer-yet kernel?

Or should we just change our test(s) to use lzo instead of lz4?

Regards,
Yann E. MORIN.

> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> ---
>  support/testing/infra/emulator.py | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/support/testing/infra/emulator.py b/support/testing/infra/emulator.py
> index 5611ec96e8..0a77eb80fc 100644
> --- a/support/testing/infra/emulator.py
> +++ b/support/testing/infra/emulator.py
> @@ -54,16 +54,16 @@ class Emulator(object):
>  
>                  if arch == "armv7":
>                      kernel = infra.download(self.downloaddir,
> -                                            "kernel-vexpress")
> +                                            "kernel-vexpress-5.10.7")
>                      dtb = infra.download(self.downloaddir,
> -                                         "vexpress-v2p-ca9.dtb")
> +                                         "vexpress-v2p-ca9-5.10.7.dtb")
>                      qemu_cmd += ["-dtb", dtb]
>                      qemu_cmd += ["-M", "vexpress-a9"]
>                  elif arch == "armv5":
>                      kernel = infra.download(self.downloaddir,
> -                                            "kernel-versatile-4.19")
> +                                            "kernel-versatile-5.10.7")
>                      dtb = infra.download(self.downloaddir,
> -                                         "versatile-pb-4.19.dtb")
> +                                         "versatile-pb-5.10.7.dtb")
>                      qemu_cmd += ["-dtb", dtb]
>                      qemu_cmd += ["-M", "versatilepb"]
>                      qemu_cmd += ["-device", "virtio-rng-pci"]
> -- 
> 2.31.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'