[Buildroot] rpi: image generation

[Buildroot] rpi: image generation

[Benoît Thébaudeau]

Hi all,

I have local changes for the Raspberry Pi that can automatically generate an
image to be flashed on the SD card (e.g. with dd).

This mechanism uses genimage, so no sudo is needed. It generates a FAT32
partition of 64 MiB and an ext4 partition that just fits its contents with a
small free space. I also have scripts to auto-expand the ext4 partition and FS
to fill the SD card.

Are you interested in any of this, or do you prefer to keep the current manual
approach that makes it possible to do just anything custom?

Best regards,
Beno?t

[Buildroot] rpi: image generation

[Thomas Petazzoni]

Dear Beno?t Th?baudeau,

On Thu, 20 Aug 2015 00:23:50 +0200, Beno?t Th?baudeau wrote:

> I have local changes for the Raspberry Pi that can automatically generate an
> image to be flashed on the SD card (e.g. with dd).
> 
> This mechanism uses genimage, so no sudo is needed. It generates a FAT32
> partition of 64 MiB and an ext4 partition that just fits its contents with a
> small free space. I also have scripts to auto-expand the ext4 partition and FS
> to fill the SD card.
> 
> Are you interested in any of this, or do you prefer to keep the current manual
> approach that makes it possible to do just anything custom?

I am interested in seeing more examples of genimage usage. I believe
Vivien Didelot, who submitted the patch to use genimage for the
Wandboard configuration, was also planning on submitting more genimage
related patches.

Best regards,

Thomas Petazzoni
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] rpi: image generation

[Vivien Didelot]

Hi Beno?t, Thomas,

On 15-08-20 09:59:06, Thomas Petazzoni wrote:
> Dear Beno?t Th?baudeau,
> 
> On Thu, 20 Aug 2015 00:23:50 +0200, Beno?t Th?baudeau wrote:
> 
> > I have local changes for the Raspberry Pi that can automatically
> > generate an image to be flashed on the SD card (e.g. with dd).
> > 
> > This mechanism uses genimage, so no sudo is needed. It generates a
> > FAT32 partition of 64 MiB and an ext4 partition that just fits its
> > contents with a small free space. I also have scripts to auto-expand
> > the ext4 partition and FS to fill the SD card.
> > 
> > Are you interested in any of this, or do you prefer to keep the
> > current manual approach that makes it possible to do just anything
> > custom?
> 
> I am interested in seeing more examples of genimage usage. I believe
> Vivien Didelot, who submitted the patch to use genimage for the
> Wandboard configuration, was also planning on submitting more genimage
> related patches.

Indeed, I initial had genimage configs for Rpi(s), Wanboard and
BeagleBone. I didn't find the time yet to rebase all that.

Similar to the upstreamed Wandboard config (7fce297), the RPi(s) config
looks like this: https://gist.github.com/vivien/752f85b563b02afd772c

Note, this was based on 8d00fe4. With some new upstreamed patches for
RPi and DT, the patch 1/2 might not be relevant anymore.

Feel free to rebase mine or update yours. If you submit something,
please put me in Cc, I'd be happy to review and test any of your patch.

Best,
-v

[Buildroot] [PATCH 1/4] configs/raspberrypi: use EABIhf

[Benoît Thébaudeau]

The SoC of the Raspberry Pi 1 is an ARM1176JZF-S, which features a VFPv2
FPU, so use the EABIhf target ABI, which is more efficient than EABI.
This is also the default on Raspbian.

Signed-off-by: Beno?t Th?baudeau <benoit@wsystem.com>
---
 configs/raspberrypi_defconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/configs/raspberrypi_defconfig b/configs/raspberrypi_defconfig
index af702a3..2d47058 100644
--- a/configs/raspberrypi_defconfig
+++ b/configs/raspberrypi_defconfig
@@ -1,5 +1,6 @@
 BR2_arm=y
 BR2_arm1176jzf_s=y
+BR2_ARM_EABIHF=y
 
 BR2_TARGET_GENERIC_GETTY_PORT="tty1"
 
-- 
2.1.4

[Buildroot] [PATCH 2/4] board/raspberrypi: prepare the image files for the target

[Benoît Thébaudeau]

Automate the preparation of the generated image files for the Raspberry
Pi firmware:
 - mark the kernel image as supporting Device Tree,
 - place the marked kernel image and the DTBs into the rpi-firmware
   folder.

The instructions in readme.txt are updated accordingly.

Signed-off-by: Beno?t Th?baudeau <benoit@wsystem.com>
---
 board/raspberrypi/post-image.sh | 17 ++++++++
 board/raspberrypi/readme.txt    | 90 ++++++++++++++++++-----------------------
 configs/raspberrypi2_defconfig  |  2 +
 configs/raspberrypi_defconfig   |  2 +
 4 files changed, 60 insertions(+), 51 deletions(-)
 create mode 100755 board/raspberrypi/post-image.sh

diff --git a/board/raspberrypi/post-image.sh b/board/raspberrypi/post-image.sh
new file mode 100755
index 0000000..db9c53c
--- /dev/null
+++ b/board/raspberrypi/post-image.sh
@@ -0,0 +1,17 @@
+#!/bin/bash -e
+
+RPI_FW_BIN_DIR="${BINARIES_DIR}/rpi-firmware"
+KERNEL_IMG="${BINARIES_DIR}/zImage"
+KERNEL_RPI_FW_IMG="${RPI_FW_BIN_DIR}/zImage"
+
+# A special marker must be appended to the kernel image so that the firmware of
+# the Raspberry Pi knows that it supports Device Tree. Without this marker, the
+# firmware passes the ATAGS boot data instead of the appropriate DTB to the
+# kernel.
+# The final kernel image must be in the rpi-firmware folder.
+mkknlimg "${KERNEL_IMG}" "${KERNEL_RPI_FW_IMG}"
+
+# Move the DTBs where the firmware expects them to be, i.e. to the rpi-firmware
+# folder.
+find "${BINARIES_DIR}/" -maxdepth 1 -type f -name '*.dtb' -print0 | \
+	xargs -0r mv -ft "${RPI_FW_BIN_DIR}/"
diff --git a/board/raspberrypi/readme.txt b/board/raspberrypi/readme.txt
index cf95879..168fab5 100644
--- a/board/raspberrypi/readme.txt
+++ b/board/raspberrypi/readme.txt
@@ -12,7 +12,7 @@ To be able to use your Raspberry Pi board with the images generated by
 Buildroot, you have to choose whether you will use:
 
     * Volatile rootfs in RAM (the rootfs is an initramfs) or
-    * Persistent rootfs on the SDCard
+    * Persistent rootfs on the SD card
 
 Also, a good source of information is http://elinux.org/R-Pi_Hub
 
@@ -22,7 +22,7 @@ How to build it
 Configure Buildroot
 -------------------
 
-There are two RaspberryPi defconfig files in Buildroot, one for each
+There are two Raspberry Pi defconfig files in Buildroot, one for each
 major variant, which you should base your work on:
 
 For models A, B, A+ or B+:
@@ -61,77 +61,65 @@ Result of the build
 After building, you should obtain this tree:
 
     output/images/
-    +-- rootfs.tar                  [0]
+    +-- rootfs.cpio                 [0]
+    +-- rootfs.tar                  [1]
     +-- rpi-firmware/
+    |   +-- bcm2708-rpi-b.dtb       [2]
+    |   +-- bcm2708-rpi-b-plus.dtb  [3]
+    |   +-- bcm2709-rpi-2-b.dtb     [4]
     |   +-- bootcode.bin
+    |   +-- cmdline.txt             [5]
     |   +-- config.txt
     |   +-- fixup.dat
-    |   `-- start.elf
-    +-- bcm2708-rpi-b.dtb           [1]
-    +-- bcm2708-rpi-b-plus.dtb      [1]
-    +-- bcm2709-rpi-2-b.dtb         [1]
-    `-- zImage
+    |   +-- start.elf
+    |   `-- zImage                  [6]
+    `-- zImage                      [7]
 
-[0] Note for Volatile: rootfs.tar will only be there if you kept
+[0] rootfs.cpio will only be there if you selected a volatile rootfs.
+[1] Note for volatile: rootfs.tar will only be there if you kept
     "tar the root filesystem" option selected in "Filesystem images".
 
-[1] Not all of them will be present, depending on the RaspberryPi
-    model you are using.
+[2] Required for models A and B, generated by raspberrypi_defconfig.
+[3] Required for models A+ and B+, generated by raspberrypi_defconfig.
+[4] Required for model 2, generated by raspberrypi2_defconfig.
 
-Prepare you SDCard
-==================
+[5] Generated by Buildroot, but optional for the firmware.
+
+[6] Kernel image marked for the firmware as supporting Device Tree.
+    Note: The kernel image file name is defined in config.txt like this:
+    kernel=zImage
+[7] Original kernel image.
+
+Prepare your SD card
+====================
 
 For more information, visit
 http://elinux.org/RPi_Advanced_Setup#Advanced_SD_card_setup
 
-In summary, your SDCard must have first partition in fat32 and marked bootable.
+In summary, your SD card must have first partition in FAT and marked bootable.
 
 Create the required partitions:
-  - for a persistent rootfs, 10MiB is enough memory for the boot fat32
-    partition, and a second partition is required for the root filesystem
-  - for a volatile rootfs, 50MiB is required for the boot fat32 partition
+  - for a persistent rootfs, 10MiB is enough memory for the boot FAT
+    partition, and a second partition is required for the root filesystem,
+  - for a volatile rootfs, 50MiB is required for the boot FAT partition.
 
-Note: You can create any number of partitions you desire, the only requirement
-is that the first partition must be using fat32 and be bootable.
+Note: You can create any number of partitions you desire; the only requirement
+is that the first partition must be using FAT and be bootable.
 
-Mount the partitions (adjust 'sdX' to match your SDcard device):
+Mount the partitions (adjust 'sdX' to match your SD card device):
 
   $ sudo mount /dev/sdX1 /mnt/mountpointboot
   $ sudo mount /dev/sdX2 /mnt/mountpointroot    (only for persistent rootfs)
 
-Install the binaries to the SDCard
-----------------------------------
+Install the binaries to the SD card
+-----------------------------------
 
-At the root of the boot partition, the Raspberry Pi must find the following
-files:
-
-    * bcm2708-rpi-b.dtb         [2]
-    * bcm2708-rpi-b-plus.dtb    [3]
-    * bcm2709-rpi-2-b.dtb       [4]
-    * bootcode.bin
-    * config.txt
-    * fixup.dat
-    * start.elf
-    * zImage
-
-[2] For models A and B
-[3] For models A+ and B+
-[4] For model 2
-
-For example:
+At the root of the boot partition, the Raspberry Pi must find the files from
+output/images/rpi-firmware:
 
   $ cp output/images/rpi-firmware/* /mnt/mountpointboot
-  $ cp output/images/*.dtb /mnt/mountpointboot
 
-The kernel image must be marked with a special header so that the
-bootloader of the RaspberryPi knows it supports Device Tree:
-
-  $ ./output/host/usr/bin/mkknlimg output/images/zImage /mnt/mountpointboot/zImage
-
-Note: The kernel image file name is defined in config.txt like this:
-kernel=zImage
-
-If you use a volatile rootfs, Skip to "Finish", below. For a persistent
+If you use a volatile rootfs, skip to "Finish", below. For a persistent
 rootfs, there are further steps to do.
 
 Extract (as root!) the contents of the rootfs.tar archive into the second
@@ -147,7 +135,7 @@ Unmount all the partitions:
   $ sudo umount /mnt/mountpointboot
   $ sudo umount /mnt/mountpointroot     (only for persistent rootfs)
 
-And eject your SDcard from your computer SDcard reader.
+And eject your SD card from your computer SD card reader.
 
-Insert the SDcard into your Raspberry Pi, and power it up. Your new system
+Insert the SD card into your Raspberry Pi, and power it up. Your new system
 should come up, now.
diff --git a/configs/raspberrypi2_defconfig b/configs/raspberrypi2_defconfig
index 7e49f0c..bfbf243 100644
--- a/configs/raspberrypi2_defconfig
+++ b/configs/raspberrypi2_defconfig
@@ -26,3 +26,5 @@ BR2_LINUX_KERNEL_INTREE_DTS_NAME="bcm2709-rpi-2-b"
 
 BR2_PACKAGE_RPI_FIRMWARE=y
 # BR2_PACKAGE_RPI_FIRMWARE_INSTALL_DTB_OVERLAYS is not set
+
+BR2_ROOTFS_POST_IMAGE_SCRIPT="board/raspberrypi2/post-image.sh"
diff --git a/configs/raspberrypi_defconfig b/configs/raspberrypi_defconfig
index 2d47058..5ded72f 100644
--- a/configs/raspberrypi_defconfig
+++ b/configs/raspberrypi_defconfig
@@ -25,3 +25,5 @@ BR2_LINUX_KERNEL_INTREE_DTS_NAME="bcm2708-rpi-b bcm2708-rpi-b-plus"
 
 BR2_PACKAGE_RPI_FIRMWARE=y
 # BR2_PACKAGE_RPI_FIRMWARE_INSTALL_DTB_OVERLAYS is not set
+
+BR2_ROOTFS_POST_IMAGE_SCRIPT="board/raspberrypi/post-image.sh"
-- 
2.1.4

[Buildroot] [PATCH 3/4] board/raspberrypi: generate a medium image

[Benoît Thébaudeau]

Automate the generation of an sdcard.img medium image, ready to be
written to the SD card, while leaving the manual procedure available for
custom needs.

genimage is used, so no commands need to be run as root by Buildroot.

The layout of the generated image is adjusted automatically if an
initramfs is selected in the configuration.

The instructions in readme.txt are updated accordingly. The layout of
the generated image is described in this file.

Signed-off-by: Beno?t Th?baudeau <benoit@wsystem.com>
---
 board/raspberrypi/genimage-persistent-rootfs.cfg | 22 +++++++
 board/raspberrypi/genimage-volatile-rootfs.cfg   | 17 +++++
 board/raspberrypi/genimage.sh                    | 15 +++++
 board/raspberrypi/readme.txt                     | 79 +++++++++++++++++-------
 configs/raspberrypi2_defconfig                   |  6 +-
 configs/raspberrypi_defconfig                    |  6 +-
 6 files changed, 121 insertions(+), 24 deletions(-)
 create mode 100644 board/raspberrypi/genimage-persistent-rootfs.cfg
 create mode 100644 board/raspberrypi/genimage-volatile-rootfs.cfg
 create mode 100755 board/raspberrypi/genimage.sh

diff --git a/board/raspberrypi/genimage-persistent-rootfs.cfg b/board/raspberrypi/genimage-persistent-rootfs.cfg
new file mode 100644
index 0000000..e519ca6
--- /dev/null
+++ b/board/raspberrypi/genimage-persistent-rootfs.cfg
@@ -0,0 +1,22 @@
+image boot.vfat {
+	vfat {
+		extraargs = "-F 16 -n BOOT"
+	}
+	size = 10M
+	mountpoint = "/"
+}
+
+image sdcard.img {
+	hdimage {}
+
+	partition boot {
+		partition-type = 0x0e
+		bootable = true
+		image = "boot.vfat"
+	}
+
+	partition rootfs {
+		partition-type = 0x83
+		image = "rootfs.ext4"
+	}
+}
diff --git a/board/raspberrypi/genimage-volatile-rootfs.cfg b/board/raspberrypi/genimage-volatile-rootfs.cfg
new file mode 100644
index 0000000..488bd81
--- /dev/null
+++ b/board/raspberrypi/genimage-volatile-rootfs.cfg
@@ -0,0 +1,17 @@
+image boot.vfat {
+	vfat {
+		extraargs = "-F 32 -n BOOT"
+	}
+	size = 50M
+	mountpoint = "/"
+}
+
+image sdcard.img {
+	hdimage {}
+
+	partition boot {
+		partition-type = 0x0c
+		bootable = true
+		image = "boot.vfat"
+	}
+}
diff --git a/board/raspberrypi/genimage.sh b/board/raspberrypi/genimage.sh
new file mode 100755
index 0000000..663335c
--- /dev/null
+++ b/board/raspberrypi/genimage.sh
@@ -0,0 +1,15 @@
+#!/bin/bash -e
+
+BOARD_DIR="board/raspberrypi"
+RPI_FW_BIN_DIR="${BINARIES_DIR}/rpi-firmware"
+if grep "^BR2_TARGET_ROOTFS_INITRAMFS=y$" "${BR2_CONFIG}" &>/dev/null; then
+	GENIMAGE_CFG="${BOARD_DIR}/genimage-volatile-rootfs.cfg"
+else
+	GENIMAGE_CFG="${BOARD_DIR}/genimage-persistent-rootfs.cfg"
+fi
+GENIMAGE_TMP="${BUILD_DIR}/genimage.tmp"
+
+rm -Rf "${GENIMAGE_TMP}"
+genimage --rootpath "${RPI_FW_BIN_DIR}" --tmppath "${GENIMAGE_TMP}" \
+	--inputpath "${BINARIES_DIR}" --outputpath "${BINARIES_DIR}" \
+	--config "${GENIMAGE_CFG}"
diff --git a/board/raspberrypi/readme.txt b/board/raspberrypi/readme.txt
index 168fab5..d63e71e 100644
--- a/board/raspberrypi/readme.txt
+++ b/board/raspberrypi/readme.txt
@@ -33,7 +33,11 @@ And for model 2 B:
 
   $ make raspberrypi2_defconfig
 
-If you want to use a persistent rootfs, skip to "Build the rootfs", below.
+If you want to use a persistent rootfs, the generated ext4 rootfs image fits its
+contents with only a small (but sensible) margin of free space by default, so
+you might want to adjust "Filesystem images"/"ext2/3/4 root filesystem"/
+"size in blocks (leave at 0 for auto calculation)" depending on your needs, then
+skip to "Build the rootfs", below.
 
 For a volatile rootfs, you have to slightly adjust the configuration:
 
@@ -41,7 +45,7 @@ For a volatile rootfs, you have to slightly adjust the configuration:
     * Select "Filesystem images"
     * Select "initial RAM filesystem linked into linux kernel"
 
-You may also deselect "tar the root filesystem".
+You may also deselect "ext2/3/4 root filesystem" and "tar the root filesystem".
 
 Build the rootfs
 ----------------
@@ -61,38 +65,66 @@ Result of the build
 After building, you should obtain this tree:
 
     output/images/
-    +-- rootfs.cpio                 [0]
-    +-- rootfs.tar                  [1]
+    +-- boot.vfat                   [0]
+    +-- sdcard.img                  [1]
+    +-- rootfs.cpio                 [2]
+    +-- rootfs.ext2                 [3]
+    +-- rootfs.ext4 -> rootfs.ext2  [3]
+    +-- rootfs.tar                  [4]
     +-- rpi-firmware/
-    |   +-- bcm2708-rpi-b.dtb       [2]
-    |   +-- bcm2708-rpi-b-plus.dtb  [3]
-    |   +-- bcm2709-rpi-2-b.dtb     [4]
+    |   +-- bcm2708-rpi-b.dtb       [5]
+    |   +-- bcm2708-rpi-b-plus.dtb  [6]
+    |   +-- bcm2709-rpi-2-b.dtb     [7]
     |   +-- bootcode.bin
-    |   +-- cmdline.txt             [5]
+    |   +-- cmdline.txt             [8]
     |   +-- config.txt
     |   +-- fixup.dat
     |   +-- start.elf
-    |   `-- zImage                  [6]
-    `-- zImage                      [7]
+    |   `-- zImage                  [9]
+    `-- zImage                      [10]
 
-[0] rootfs.cpio will only be there if you selected a volatile rootfs.
-[1] Note for volatile: rootfs.tar will only be there if you kept
+[0] Image of the VFAT boot filesystem containing the files from
+    output/images/rpi-firmware.
+
+[1] Medium image ready to be written to your SD card.
+    For the volatile rootfs, it contains a single partition, filled with
+    boot.vfat.
+    For the persistent rootfs, it contains two partitions:
+    one filled with boot.vfat and one filled with rootfs.ext4.
+
+[2] rootfs.cpio will only be there if you selected a volatile rootfs.
+[3] Note for volatile: rootfs.ext* will only be there if you kept
+    "ext2/3/4 root filesystem" option selected in "Filesystem images".
+[4] Note for volatile: rootfs.tar will only be there if you kept
     "tar the root filesystem" option selected in "Filesystem images".
 
-[2] Required for models A and B, generated by raspberrypi_defconfig.
-[3] Required for models A+ and B+, generated by raspberrypi_defconfig.
-[4] Required for model 2, generated by raspberrypi2_defconfig.
+[5] Required for models A and B, generated by raspberrypi_defconfig.
+[6] Required for models A+ and B+, generated by raspberrypi_defconfig.
+[7] Required for model 2, generated by raspberrypi2_defconfig.
 
-[5] Generated by Buildroot, but optional for the firmware.
+[8] Generated by Buildroot, but optional for the firmware.
 
-[6] Kernel image marked for the firmware as supporting Device Tree.
+[9] Kernel image marked for the firmware as supporting Device Tree.
     Note: The kernel image file name is defined in config.txt like this:
     kernel=zImage
-[7] Original kernel image.
+[10] Original kernel image.
 
 Prepare your SD card
 ====================
 
+If you want to follow a manual procedure, then skip to "Manual procedure",
+below.
+
+Write (as root!) the generated medium image to your SD card (adjust 'sdX' to
+match your device):
+
+  $ sudo dd if=output/images/sdcard.img of=/dev/sdX bs=1M
+
+Skip to "Finish", below.
+
+Manual procedure
+----------------
+
 For more information, visit
 http://elinux.org/RPi_Advanced_Setup#Advanced_SD_card_setup
 
@@ -119,7 +151,7 @@ output/images/rpi-firmware:
 
   $ cp output/images/rpi-firmware/* /mnt/mountpointboot
 
-If you use a volatile rootfs, skip to "Finish", below. For a persistent
+If you use a volatile rootfs, skip to "Unmount", below. For a persistent
 rootfs, there are further steps to do.
 
 Extract (as root!) the contents of the rootfs.tar archive into the second
@@ -127,15 +159,18 @@ partition you created above:
 
   $ sudo tar xf ./output/images/rootfs.tar -C /mnt/mountpointroot
 
-Finish
-======
+Unmount
+-------
 
 Unmount all the partitions:
 
   $ sudo umount /mnt/mountpointboot
   $ sudo umount /mnt/mountpointroot     (only for persistent rootfs)
 
-And eject your SD card from your computer SD card reader.
+Finish
+======
+
+Eject your SD card from your computer SD card reader.
 
 Insert the SD card into your Raspberry Pi, and power it up. Your new system
 should come up, now.
diff --git a/configs/raspberrypi2_defconfig b/configs/raspberrypi2_defconfig
index bfbf243..73dc290 100644
--- a/configs/raspberrypi2_defconfig
+++ b/configs/raspberrypi2_defconfig
@@ -27,4 +27,8 @@ BR2_LINUX_KERNEL_INTREE_DTS_NAME="bcm2709-rpi-2-b"
 BR2_PACKAGE_RPI_FIRMWARE=y
 # BR2_PACKAGE_RPI_FIRMWARE_INSTALL_DTB_OVERLAYS is not set
 
-BR2_ROOTFS_POST_IMAGE_SCRIPT="board/raspberrypi2/post-image.sh"
+BR2_TARGET_ROOTFS_EXT2=y
+BR2_TARGET_ROOTFS_EXT2_4=y
+BR2_TARGET_ROOTFS_EXT2_LABEL="rootfs"
+BR2_PACKAGE_HOST_GENIMAGE=y
+BR2_ROOTFS_POST_IMAGE_SCRIPT="board/raspberrypi2/post-image.sh board/raspberrypi2/genimage.sh"
diff --git a/configs/raspberrypi_defconfig b/configs/raspberrypi_defconfig
index 5ded72f..58f4cab 100644
--- a/configs/raspberrypi_defconfig
+++ b/configs/raspberrypi_defconfig
@@ -26,4 +26,8 @@ BR2_LINUX_KERNEL_INTREE_DTS_NAME="bcm2708-rpi-b bcm2708-rpi-b-plus"
 BR2_PACKAGE_RPI_FIRMWARE=y
 # BR2_PACKAGE_RPI_FIRMWARE_INSTALL_DTB_OVERLAYS is not set
 
-BR2_ROOTFS_POST_IMAGE_SCRIPT="board/raspberrypi/post-image.sh"
+BR2_TARGET_ROOTFS_EXT2=y
+BR2_TARGET_ROOTFS_EXT2_4=y
+BR2_TARGET_ROOTFS_EXT2_LABEL="rootfs"
+BR2_PACKAGE_HOST_GENIMAGE=y
+BR2_ROOTFS_POST_IMAGE_SCRIPT="board/raspberrypi/post-image.sh board/raspberrypi/genimage.sh"
-- 
2.1.4

[Buildroot] [PATCH 4/4] board/raspberrypi: auto-expand rootfs on first boot

[Benoît Thébaudeau]

Add init scripts to auto-expand the persistent rootfs on the first boot
to fill the medium.

These scripts follow the same procedure as raspi-config. The root
partition is first expanded, then a reboot is required, then the rootfs
is expanded. Each script removes itself once run.

The volatile (initramfs) rootfs is not affected by this change.

The instructions in readme.txt are updated accordingly.

Signed-off-by: Beno?t Th?baudeau <benoit@wsystem.com>
---
 board/raspberrypi/post-build.sh                    |  7 +++++
 board/raspberrypi/readme.txt                       | 11 ++++---
 .../rootfs-overlay/etc/init.d/S22expand-rootpart   | 36 ++++++++++++++++++++++
 .../rootfs-overlay/etc/init.d/S23expand-rootfs     | 16 ++++++++++
 configs/raspberrypi2_defconfig                     |  8 +++++
 configs/raspberrypi_defconfig                      | 12 ++++++--
 6 files changed, 84 insertions(+), 6 deletions(-)
 create mode 100755 board/raspberrypi/post-build.sh
 create mode 100755 board/raspberrypi/rootfs-overlay/etc/init.d/S22expand-rootpart
 create mode 100755 board/raspberrypi/rootfs-overlay/etc/init.d/S23expand-rootfs

diff --git a/board/raspberrypi/post-build.sh b/board/raspberrypi/post-build.sh
new file mode 100755
index 0000000..4b92b33
--- /dev/null
+++ b/board/raspberrypi/post-build.sh
@@ -0,0 +1,7 @@
+#!/bin/bash -e
+
+# The init scripts used to auto-expand the persistent rootfs on the first boot
+# to fill the medium must not be used with the volatile (initramfs) rootfs.
+if grep "^BR2_TARGET_ROOTFS_INITRAMFS=y$" "${BR2_CONFIG}" &>/dev/null; then
+	rm -f "${TARGET_DIR}"/etc/init.d/{S22expand-rootpart,S23expand-rootfs}
+fi
diff --git a/board/raspberrypi/readme.txt b/board/raspberrypi/readme.txt
index d63e71e..7ee783b 100644
--- a/board/raspberrypi/readme.txt
+++ b/board/raspberrypi/readme.txt
@@ -34,10 +34,13 @@ And for model 2 B:
   $ make raspberrypi2_defconfig
 
 If you want to use a persistent rootfs, the generated ext4 rootfs image fits its
-contents with only a small (but sensible) margin of free space by default, so
-you might want to adjust "Filesystem images"/"ext2/3/4 root filesystem"/
-"size in blocks (leave at 0 for auto calculation)" depending on your needs, then
-skip to "Build the rootfs", below.
+contents with only a small (but sensible) margin of free space by default, but
+it is auto-expanded on the first boot to fill the medium. You can however adjust
+"Filesystem images"/"ext2/3/4 root filesystem"/
+"size in blocks (leave at 0 for auto calculation)" to fit your needs. The
+auto-expansion is controlled by
+board/raspberrypi/rootfs-overlay/etc/init.d/S*expand-root*. Skip to
+"Build the rootfs", below.
 
 For a volatile rootfs, you have to slightly adjust the configuration:
 
diff --git a/board/raspberrypi/rootfs-overlay/etc/init.d/S22expand-rootpart b/board/raspberrypi/rootfs-overlay/etc/init.d/S22expand-rootpart
new file mode 100755
index 0000000..7f19e7a
--- /dev/null
+++ b/board/raspberrypi/rootfs-overlay/etc/init.d/S22expand-rootpart
@@ -0,0 +1,36 @@
+#!/bin/sh
+
+case "$1" in
+	start)
+		echo -n "Expanding the root partition: "
+		BLOCK_DEV="/dev/mmcblk0"
+		PART_NUM="2"
+		PART_START="$(parted -ms "${BLOCK_DEV}" unit s p 2>/dev/null | \
+			grep "^${PART_NUM}:" | cut -d : -f 2 | sed 's/s$//')"
+		if [ -z "${PART_START}" ]; then
+			echo "failed"
+		else
+			fdisk "${BLOCK_DEV}" <<-EOF &>/dev/null
+				d
+				$PART_NUM
+				n
+				p
+				$PART_NUM
+				$PART_START
+
+				w
+			EOF
+			echo "done"
+		fi
+		rm -f /etc/init.d/S22expand-rootpart
+		if [ "${PART_START}" ]; then
+			reboot -f
+			while true; do :; done
+		fi
+		exit 2
+		;;
+	*)
+		echo "Usage: $0 start" >&2
+		exit 1
+		;;
+esac
diff --git a/board/raspberrypi/rootfs-overlay/etc/init.d/S23expand-rootfs b/board/raspberrypi/rootfs-overlay/etc/init.d/S23expand-rootfs
new file mode 100755
index 0000000..7b3f286
--- /dev/null
+++ b/board/raspberrypi/rootfs-overlay/etc/init.d/S23expand-rootfs
@@ -0,0 +1,16 @@
+#!/bin/sh
+
+case "$1" in
+	start)
+		echo -n "Expanding the root FS: "
+		resize2fs /dev/mmcblk0p2 &>/dev/null
+		ret=$?
+		[ ${ret} -eq 0 ] && echo "done" || echo "failed"
+		rm -f /etc/init.d/S23expand-rootfs
+		exit ${ret}
+		;;
+	*)
+		echo "Usage: $0 start" >&2
+		exit 1
+		;;
+esac
diff --git a/configs/raspberrypi2_defconfig b/configs/raspberrypi2_defconfig
index 73dc290..2e064e0 100644
--- a/configs/raspberrypi2_defconfig
+++ b/configs/raspberrypi2_defconfig
@@ -3,6 +3,7 @@ BR2_cortex_a7=y
 BR2_ARM_EABIHF=y
 BR2_ARM_FPU_NEON_VFPV4=y
 
+BR2_TOOLCHAIN_BUILDROOT_GLIBC=y
 BR2_TOOLCHAIN_BUILDROOT_CXX=y
 
 BR2_TARGET_GENERIC_GETTY_PORT="tty1"
@@ -27,6 +28,13 @@ BR2_LINUX_KERNEL_INTREE_DTS_NAME="bcm2709-rpi-2-b"
 BR2_PACKAGE_RPI_FIRMWARE=y
 # BR2_PACKAGE_RPI_FIRMWARE_INSTALL_DTB_OVERLAYS is not set
 
+BR2_PACKAGE_E2FSPROGS=y
+BR2_PACKAGE_E2FSPROGS_RESIZE2FS=y
+BR2_PACKAGE_PARTED=y
+BR2_PACKAGE_UTIL_LINUX_BINARIES=y
+
+BR2_ROOTFS_OVERLAY="board/raspberrypi2/rootfs-overlay"
+BR2_ROOTFS_POST_BUILD_SCRIPT="board/raspberrypi2/post-build.sh"
 BR2_TARGET_ROOTFS_EXT2=y
 BR2_TARGET_ROOTFS_EXT2_4=y
 BR2_TARGET_ROOTFS_EXT2_LABEL="rootfs"
diff --git a/configs/raspberrypi_defconfig b/configs/raspberrypi_defconfig
index 58f4cab..7e744f3 100644
--- a/configs/raspberrypi_defconfig
+++ b/configs/raspberrypi_defconfig
@@ -2,6 +2,9 @@ BR2_arm=y
 BR2_arm1176jzf_s=y
 BR2_ARM_EABIHF=y
 
+BR2_TOOLCHAIN_BUILDROOT_GLIBC=y
+BR2_TOOLCHAIN_BUILDROOT_CXX=y
+
 BR2_TARGET_GENERIC_GETTY_PORT="tty1"
 
 # Lock to 4.1 headers as the RPi kernel stable tree is
@@ -10,8 +13,6 @@ BR2_KERNEL_HEADERS_VERSION=y
 BR2_DEFAULT_KERNEL_VERSION="4.1.5"
 BR2_PACKAGE_HOST_LINUX_HEADERS_CUSTOM_4_1=y
 
-BR2_TOOLCHAIN_BUILDROOT_CXX=y
-
 BR2_LINUX_KERNEL=y
 BR2_LINUX_KERNEL_CUSTOM_GIT=y
 BR2_LINUX_KERNEL_CUSTOM_REPO_URL="https://github.com/raspberrypi/linux.git"
@@ -26,6 +27,13 @@ BR2_LINUX_KERNEL_INTREE_DTS_NAME="bcm2708-rpi-b bcm2708-rpi-b-plus"
 BR2_PACKAGE_RPI_FIRMWARE=y
 # BR2_PACKAGE_RPI_FIRMWARE_INSTALL_DTB_OVERLAYS is not set
 
+BR2_PACKAGE_E2FSPROGS=y
+BR2_PACKAGE_E2FSPROGS_RESIZE2FS=y
+BR2_PACKAGE_PARTED=y
+BR2_PACKAGE_UTIL_LINUX_BINARIES=y
+
+BR2_ROOTFS_OVERLAY="board/raspberrypi/rootfs-overlay"
+BR2_ROOTFS_POST_BUILD_SCRIPT="board/raspberrypi/post-build.sh"
 BR2_TARGET_ROOTFS_EXT2=y
 BR2_TARGET_ROOTFS_EXT2_4=y
 BR2_TARGET_ROOTFS_EXT2_LABEL="rootfs"
-- 
2.1.4

[Buildroot] [PATCH 4/4] board/raspberrypi: auto-expand rootfs on first boot

[Benoît Thébaudeau]

Hi Vivien, Yann, all,

On 22/08/2015 22:01, Beno?t Th?baudeau wrote:
> Add init scripts to auto-expand the persistent rootfs on the first boot
> to fill the medium.
> 
> These scripts follow the same procedure as raspi-config. The root
> partition is first expanded, then a reboot is required, then the rootfs
> is expanded. Each script removes itself once run.
> 
> The volatile (initramfs) rootfs is not affected by this change.
> 
> The instructions in readme.txt are updated accordingly.
> 
> Signed-off-by: Beno?t Th?baudeau <benoit@wsystem.com>

[...]

> diff --git a/board/raspberrypi/rootfs-overlay/etc/init.d/S23expand-rootfs b/board/raspberrypi/rootfs-overlay/etc/init.d/S23expand-rootfs
> new file mode 100755
> index 0000000..7b3f286
> --- /dev/null
> +++ b/board/raspberrypi/rootfs-overlay/etc/init.d/S23expand-rootfs
> @@ -0,0 +1,16 @@
> +#!/bin/sh
> +
> +case "$1" in
> +	start)
> +		echo -n "Expanding the root FS: "
> +		resize2fs /dev/mmcblk0p2 &>/dev/null

I had first tested this on a Raspberry Pi B using a 2-GiB SD card. It worked
fine and it was very quick.

I've just tested this on a Raspberry Pi B+ using a 8-GiB SD card. First, I got:

    [    6.831228] EXT4-fs (mmcblk0p2): resizing filesystem from 130322 to 7611388 blocks
    [    6.838822] EXT4-fs (mmcblk0p2): Converting file system to meta_bg
    [   16.888893] EXT4-fs (mmcblk0p2): resized to 3407537 blocks
    [   20.794071] ------------[ cut here ]------------
    [   20.798723] kernel BUG at fs/jbd2/journal.c:766!
    [   20.803338] Internal error: Oops - BUG: 0 [#1] PREEMPT ARM

    Entering kdb (current=0xcf339b80, pid 44) Oops: (null)
    due to oops @ 0xc02216a8
    CPU: 0 PID: 44 Comm: jbd2/mmcblk0p2- Not tainted 4.1.5 #1
    Hardware name: BCM2708
    task: cf339b80 ti: cf356000 task.ti: cf356000
    PC is at jbd2_journal_next_log_block+0xb0/0xb4
    LR is at jbd2_journal_commit_transaction+0x978/0x1f08
    pc : [<c02216a8>]    lr : [<c0219cd0>]    psr: 60000013
    sp : cf357e00  ip : 00000001  fp : cf357e1c
    r10: ce93851c  r9 : cef22140  r8 : c0218e0c
    r7 : 00000021  r6 : ceedfbc0  r5 : cf357e88  r4 : cedb41e0
    r3 : 00000001  r2 : cf357e00  r1 : cf357e88  r0 : cf336000
    Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment kernel
    Control: 00c5387d  Table: 0e6fc008  DAC: 00000015
    CPU: 0 PID: 44 Comm: jbd2/mmcblk0p2- Not tainted 4.1.5 #1
    Hardware name: BCM2708
    [<c0016d08>] (unwind_backtrace) from [<c0013ad0>] (show_stack+0x20/0x24)
    [<c0013ad0>] (show_stack) from [<c0566070>] (dump_stack+0x20/0x28)
    [<c0566070>] (dump_stack) from [<c0010964>] (show_regs+0x1c/0x20)
    [<c0010964>] (show_regs) from [<c00a4b38>] (kdb_main_loop+0x3c8/0x730)
    [<c00a4b38>] (kdb_main_loop) from [<c00a75dc>] (kdb_stub+0x19c/0x3cc)
    [<c00a75dc>] (kdb_stub) from [<c009da00>] (kgdb_handle_exception+0x284/0x8a8)
    more> 

I retried several times with the same result. Then, I fsck'ed and reformatted
the SD card, and I retried with the same sdcard.img as the first time. It worked
fine, but it was extremely slow:

    [    6.803146] EXT4-fs (mmcblk0p2): resizing filesystem from 130322 to 7611388 blocks
    [    6.826546] EXT4-fs (mmcblk0p2): Converting file system to meta_bg
    [   16.884732] EXT4-fs (mmcblk0p2): resized to 3423841 blocks
    [   26.912318] EXT4-fs (mmcblk0p2): resized to 4442841 blocks
[...]
    [  768.704191] EXT4-fs (mmcblk0p2): resized to 7581361 blocks
    [  775.571684] EXT4-fs (mmcblk0p2): resized filesystem to 7611388

So the initial contents of the unused space on the SD card seem to have an
influence, and there seems to be a bug somewhere (Linux, resize2fs, genext2fs,
tune2fs, or lack of call to e2fsck before calling resize2fs but this would not
be reliable with an online partition).

Then I retried with a 16-GiB SD card on a Raspberry Pi B, and it was very slow
too. Same if the resize is performed on a PC. However, it's very quick with
raspi-config and the latest Raspbian image. The main difference seemed to be the
block size (4 KiB on Raspbian vs. 1 KiB with mke2img), so I hacked mke2img to
test with genext2fs -B 4096 and tune2fs -J size=4, but it did not make things
faster, only the image bigger (about 512 MiB, probably the minimal size for
4-KiB blocks). So I don't know which ones, but it's probably some ext4
parameters that are having an influence on the resize2fs speed here. Any idea?

So what do you think would be the best thing to do here?
 1) Find the set of ext4 parameters making resize2fs faster and change mke2img
    and fs/ext2/* to make it possible to use these parameters.
 2) Just drop this patch from the series because the resize will probably always
    be slow for some larger SD cards.

If 2), would you keep the auto ext4 size as in the current 3/4, or would you
set a fixed size with some significant free space margin (e.g. 512 MiB)?

Best regards,
Beno?t

[Buildroot] [PATCH 4/4] board/raspberrypi: auto-expand rootfs on first boot

[Jérôme Pouiller]

Hello Beno?t,

On Thursday 27 August 2015 11:02:02 Beno?t Th?baudeau wrote:
> Hi Vivien, Yann, all,
> 
> On 22/08/2015 22:01, Beno?t Th?baudeau wrote:
[...]
> So the initial contents of the unused space on the SD card seem to have an
> influence, and there seems to be a bug somewhere (Linux, resize2fs,
> genext2fs, tune2fs, or lack of call to e2fsck before calling resize2fs but
> this would not be reliable with an online partition).
> 
> Then I retried with a 16-GiB SD card on a Raspberry Pi B, and it was very
> slow too. Same if the resize is performed on a PC. However, it's very quick
> with raspi-config and the latest Raspbian image. The main difference seemed
> to be the block size (4 KiB on Raspbian vs. 1 KiB with mke2img), so I
> hacked mke2img to test with genext2fs -B 4096 and tune2fs -J size=4, but it
> did not make things faster, only the image bigger (about 512 MiB, probably
> the minimal size for 4-KiB blocks). So I don't know which ones, but it's
> probably some ext4 parameters that are having an influence on the resize2fs
> speed here. Any idea?
Yes, I have an idea :-) Did you try sparse_super?

In fact a did nearly same task a few month ago and I found same problem. For 
information, I use following script to fill disk with last partition:

diff --git 1/sdflash.sh 2/sdflash.sh
new file mode 100755
index 0000000..5f9a9d5
--- /dev/null
+++ 2/sdflash.sh
@@ -0,0 +1,48 @@
+#!/bin/bash
+#
+# Copy a disk image to sdcard card and resize last partition in order to use
+# whole disk
+#
+
+device=$1
+image=$2
+
+if [ "$device" = "" ]; then
+    echo "Usage: $0 DEVICE IMAGE"
+    echo "Example: $0 /dev/sdd images/disk.img"
+    exit 1
+fi
+
+sfdisk -R $device       || exit 1
+echo "Copying $image to $device"
+dd if=$image of=$device conv=fsync,notrunc || exit 1
+sfdisk -R $device       || exit 1
+for part in 4 3 2 1; do
+    for file in $device$part ${device}p${part}; do
+        if [ -e $file ]; then
+            ID=$(sfdisk --print-id $device $part)
+            if [[ "0x$ID" -eq 0x83 ]]; then
+                e2fsck -f $file
+                [ $? -le 1 ] || exit 1
+                tune2fs -O sparse_super -c 0 -i 0 $file
+                [ $? -eq 0 ] || exit 1
+                e2fsck -yf $file > /dev/null
+                [ $? -le 1 ] || exit 1
+                sleep 1
+                echo ',+,' | sfdisk -L -D -uM -N $part $device
+                [ $? -eq 0 ] || exit 1
+                resize2fs $file
+                [ $? -eq 0 ] || exit 1
+                e2fsck -f $file
+                [ $? -le 1 ] || exit 1
+                exit 0
+            else
+                echo "Unknown partition type"
+                exit 1
+            fi
+        fi
+    done
+done
+echo "Cannot find partition on $device"
+exit 1
+


-- 
J?r?me Pouiller, Sysmic
Embedded Linux specialist
http://www.sysmic.fr

[Buildroot] [PATCH 4/4] board/raspberrypi: auto-expand rootfs on first boot

[Benoît Thébaudeau]

Hi J?r?me,

On 27/08/2015 23:22, J?r?me Pouiller wrote:
> On Thursday 27 August 2015 11:02:02 Beno?t Th?baudeau wrote:
>> Hi Vivien, Yann, all,
>>
>> On 22/08/2015 22:01, Beno?t Th?baudeau wrote:
> [...]
>> So the initial contents of the unused space on the SD card seem to have an
>> influence, and there seems to be a bug somewhere (Linux, resize2fs,
>> genext2fs, tune2fs, or lack of call to e2fsck before calling resize2fs but
>> this would not be reliable with an online partition).
>>
>> Then I retried with a 16-GiB SD card on a Raspberry Pi B, and it was very
>> slow too. Same if the resize is performed on a PC. However, it's very quick
>> with raspi-config and the latest Raspbian image. The main difference seemed
>> to be the block size (4 KiB on Raspbian vs. 1 KiB with mke2img), so I
>> hacked mke2img to test with genext2fs -B 4096 and tune2fs -J size=4, but it
>> did not make things faster, only the image bigger (about 512 MiB, probably
>> the minimal size for 4-KiB blocks). So I don't know which ones, but it's
>> probably some ext4 parameters that are having an influence on the resize2fs
>> speed here. Any idea?
> Yes, I have an idea :-) Did you try sparse_super?

[...]

Awesome, it works, even if resizing the partition online. Thanks for the tip!

I'll send a patch changing mke2img to allow the use of this option.

Best regards,
Beno?t

[Buildroot] [PATCH 4/4] board/raspberrypi: auto-expand rootfs on first boot

[Benoît Thébaudeau]

On 28/08/2015 12:36, Beno?t Th?baudeau wrote:
> On 27/08/2015 23:22, J?r?me Pouiller wrote:
>> On Thursday 27 August 2015 11:02:02 Beno?t Th?baudeau wrote:
>>> Hi Vivien, Yann, all,
>>>
>>> On 22/08/2015 22:01, Beno?t Th?baudeau wrote:
>> [...]
>>> So the initial contents of the unused space on the SD card seem to have an
>>> influence, and there seems to be a bug somewhere (Linux, resize2fs,
>>> genext2fs, tune2fs, or lack of call to e2fsck before calling resize2fs but
>>> this would not be reliable with an online partition).
>>>
>>> Then I retried with a 16-GiB SD card on a Raspberry Pi B, and it was very
>>> slow too. Same if the resize is performed on a PC. However, it's very quick
>>> with raspi-config and the latest Raspbian image. The main difference seemed
>>> to be the block size (4 KiB on Raspbian vs. 1 KiB with mke2img), so I
>>> hacked mke2img to test with genext2fs -B 4096 and tune2fs -J size=4, but it
>>> did not make things faster, only the image bigger (about 512 MiB, probably
>>> the minimal size for 4-KiB blocks). So I don't know which ones, but it's
>>> probably some ext4 parameters that are having an influence on the resize2fs
>>> speed here. Any idea?
>> Yes, I have an idea :-) Did you try sparse_super?
> 
> [...]
> 
> Awesome, it works, even if resizing the partition online. Thanks for the tip!
> 
> I'll send a patch changing mke2img to allow the use of this option.

Please apply the patch below before this series in order to avoid the resize
speed issue described above:
http://patchwork.ozlabs.org/patch/511861/

Best regards,
Beno?t

[Buildroot] [PATCH 4/4] board/raspberrypi: auto-expand rootfs on first boot

[Benoît Thébaudeau]

Hello J?r?me,

On 31/08/2015 10:51, J?r?me Pouiller wrote:
> On Saturday 22 August 2015 22:01:28 Beno?t Th?baudeau wrote:
>> Add init scripts to auto-expand the persistent rootfs on the first boot
>> to fill the medium.
>>
>> These scripts follow the same procedure as raspi-config. The root
>> partition is first expanded, then a reboot is required, then the rootfs
>> is expanded. Each script removes itself once run.
> [..]
>> diff --git a/board/raspberrypi/readme.txt b/board/raspberrypi/readme.txt
>> index d63e71e..7ee783b 100644
>> --- a/board/raspberrypi/readme.txt
>> +++ b/board/raspberrypi/readme.txt
>> @@ -34,10 +34,13 @@ And for model 2 B:
>>    $ make raspberrypi2_defconfig
>>
>>  If you want to use a persistent rootfs, the generated ext4 rootfs image
>> fits its -contents with only a small (but sensible) margin of free space by
>> default, so -you might want to adjust "Filesystem images"/"ext2/3/4 root
>> filesystem"/ -"size in blocks (leave at 0 for auto calculation)" depending
>> on your needs, then -skip to "Build the rootfs", below.
>> +contents with only a small (but sensible) margin of free space by default,
>> but +it is auto-expanded on the first boot to fill the medium. You can
>> however adjust +"Filesystem images"/"ext2/3/4 root filesystem"/
>> +"size in blocks (leave at 0 for auto calculation)" to fit your needs. The
>> +auto-expansion is controlled by
>> +board/raspberrypi/rootfs-overlay/etc/init.d/S*expand-root*. Skip to
>> +"Build the rootfs", below.
> Isn't safer to auto-expand partition while flashing SD Card instead of doing 
> it online?
> 
> Maybe you can explain how to resize partition in README or provide a script 
> like the one I sent you a few days ago (named sdflash.sh), no?

It would be safer if users pull the power plug during the few seconds that the
expansion lasts, without looking at their console output, which is not a big
risk. Doing it while flashing the SD card would have a major limitation: it
prevents the publication of an autonomous image file that you can program by any
means to get the same result.

Resizing a partition on a PC is trivial if the image is programmed first. There
are plenty of tools like gparted and tutorials for that on the web, so I don't
think adding such information to the readme.txt would be worth it.

Regarding a script writing the SD card and auto-expanding the rootfs, it might
be useful. Since it would require superuser rights, it should not be added to
the post-image scripts, but launched manually. That would give users a choice.
This script would just write the generated image file, auto-expand the rootfs
and remove the auto-expansion scripts from the rootfs. I'll look into it.

Best regards,
Beno?t

[Buildroot] [PATCH 4/4] board/raspberrypi: auto-expand rootfs on first boot

[Floris Bos]

On 08/22/2015 10:01 PM, Beno?t Th?baudeau wrote:
> Add init scripts to auto-expand the persistent rootfs on the first boot
> to fill the medium.

Wonder if it wouldn't be better to have a package section in buildroot 
for commonly used init scripts like this?

I don't think resizing the rootfs is specific to only this board.
And by making it a package you could do "depends on 
!BR2_TARGET_ROOTFS_INITRAMFS" instead of grep magic in the post build 
script.


Yours sincerely,

Floris Bos

[Buildroot] [PATCH 4/4] board/raspberrypi: auto-expand rootfs on first boot

[Floris Bos]

On 08/22/2015 10:01 PM, Beno?t Th?baudeau wrote:
> Add init scripts to auto-expand the persistent rootfs on the first boot
> to fill the medium.

Had a look at your script, but think it has more dependencies than 
strictly necessary.

>   
> diff --git a/board/raspberrypi/rootfs-overlay/etc/init.d/S22expand-rootpart b/board/raspberrypi/rootfs-overlay/etc/init.d/S22expand-rootpart
> new file mode 100755
> index 0000000..7f19e7a
> --- /dev/null
> +++ b/board/raspberrypi/rootfs-overlay/etc/init.d/S22expand-rootpart
> @@ -0,0 +1,36 @@
> +#!/bin/sh
> +
> +case "$1" in
> +	start)
> +		echo -n "Expanding the root partition: "
> +		BLOCK_DEV="/dev/mmcblk0"
> +		PART_NUM="2"
> +		PART_START="$(parted -ms "${BLOCK_DEV}" unit s p 2>/dev/null | \
> +			grep "^${PART_NUM}:" | cut -d : -f 2 | sed 's/s$//')"
> +		if [ -z "${PART_START}" ]; then
> +			echo "failed"

Note that the existing start sector can also be found in 
/sys/class/block/mmcblk0p2/start
So can remove the parted dependency.

> +		else
> +			fdisk "${BLOCK_DEV}" <<-EOF &>/dev/null
> +				d
> +				$PART_NUM
> +				n
> +				p
> +				$PART_NUM
> +				$PART_START
> +
> +				w
> +			EOF
> +			echo "done"
> +		fi
> +		rm -f /etc/init.d/S22expand-rootpart
> +		if [ "${PART_START}" ]; then
> +			reboot -f
> +			while true; do :; done
> +		fi
> +		exit 2
> +		;;
> +	*)
> +		echo "Usage: $0 start" >&2
> +		exit 1
> +		;;
> +esac
> diff --git a/board/raspberrypi/rootfs-overlay/etc/init.d/S23expand-rootfs b/board/raspberrypi/rootfs-overlay/etc/init.d/S23expand-rootfs
> new file mode 100755
> index 0000000..7b3f286
> --- /dev/null
> +++ b/board/raspberrypi/rootfs-overlay/etc/init.d/S23expand-rootfs
> @@ -0,0 +1,16 @@
> +#!/bin/sh
> +
> +case "$1" in
> +	start)
> +		echo -n "Expanding the root FS: "
> +		resize2fs /dev/mmcblk0p2 &>/dev/null
> +		ret=$?
> +		[ ${ret} -eq 0 ] && echo "done" || echo "failed"
> +		rm -f /etc/init.d/S23expand-rootfs
> +		exit ${ret}
> +		;;
> +	*)
> +		echo "Usage: $0 start" >&2
> +		exit 1
> +		;;
> +esac
> diff --git a/configs/raspberrypi2_defconfig b/configs/raspberrypi2_defconfig
> index 73dc290..2e064e0 100644
> --- a/configs/raspberrypi2_defconfig
> +++ b/configs/raspberrypi2_defconfig
> @@ -3,6 +3,7 @@ BR2_cortex_a7=y
>   BR2_ARM_EABIHF=y
>   BR2_ARM_FPU_NEON_VFPV4=y
>   
> +BR2_TOOLCHAIN_BUILDROOT_GLIBC=y

Why is glibc required?
Isn't it possible to just set the right options (like wchar, largefile) 
for uclibc?

>   BR2_TOOLCHAIN_BUILDROOT_CXX=y
>   
>   BR2_TARGET_GENERIC_GETTY_PORT="tty1"
> @@ -27,6 +28,13 @@ BR2_LINUX_KERNEL_INTREE_DTS_NAME="bcm2709-rpi-2-b"
>   BR2_PACKAGE_RPI_FIRMWARE=y
>   # BR2_PACKAGE_RPI_FIRMWARE_INSTALL_DTB_OVERLAYS is not set
>   
> +BR2_PACKAGE_E2FSPROGS=y
> +BR2_PACKAGE_E2FSPROGS_RESIZE2FS=y
> +BR2_PACKAGE_PARTED=y
> +BR2_PACKAGE_UTIL_LINUX_BINARIES=y
> +
> +BR2_ROOTFS_OVERLAY="board/raspberrypi2/rootfs-overlay"
> +BR2_ROOTFS_POST_BUILD_SCRIPT="board/raspberrypi2/post-build.sh"

Why is UTIL_LINUX_BINARIES necessary?
If for fdisk, doesn't busybox provides this as well?



Yours sincerely,

Floris Bos

[Buildroot] [PATCH 4/4] board/raspberrypi: auto-expand rootfs on first boot

[Benoît Thébaudeau]

Dear Floris Bos,

On Mon, Aug 31, 2015 at 2:17 PM, Floris Bos <bos@je-eigen-domein.nl> wrote:
> On 08/22/2015 10:01 PM, Beno?t Th?baudeau wrote:
>> Add init scripts to auto-expand the persistent rootfs on the first boot
>> to fill the medium.
>
>
> Wonder if it wouldn't be better to have a package section in buildroot for
> commonly used init scripts like this?
>
> I don't think resizing the rootfs is specific to only this board.
> And by making it a package you could do "depends on
> !BR2_TARGET_ROOTFS_INITRAMFS" instead of grep magic in the post build
> script.

That makes sense. It would do "depends on BR2_TARGET_ROOTFS_EXT2" and
select the required target utils. It would also have to ask the user
for the block device to use (or it could autodetect at runtime the
block device on which '/' is mounted). Should this be a new package,
or just new options in the ext2 rootfs menu? I think that the latter
makes more sense for now.

But regarding the "!BR2_TARGET_ROOTFS_INITRAMFS", it would not be
needed at all. This is only on RPi to make user choices easier by not
having to unselect ext4fs and the rootfs overlay besides selecting
initramfs. On RPi, we could just force users to select only one of
them, and say so in readme.txt, and there would be no rootfs overlay
to manage with this feature as new config options.

Best regards,
Beno?t

[Buildroot] [PATCH 4/4] board/raspberrypi: auto-expand rootfs on first boot

[Benoît Thébaudeau]

Dear Floris Bos,

On Mon, Aug 31, 2015 at 3:25 PM, Floris Bos <bos@je-eigen-domein.nl> wrote:
> On 08/22/2015 10:01 PM, Beno?t Th?baudeau wrote:
>> Add init scripts to auto-expand the persistent rootfs on the first boot
>> to fill the medium.
>
>
> Had a look at your script, but think it has more dependencies than strictly
> necessary.

Certainly. I've not tried to optimize dependencies in this patch.

>>   diff --git
>> a/board/raspberrypi/rootfs-overlay/etc/init.d/S22expand-rootpart
>> b/board/raspberrypi/rootfs-overlay/etc/init.d/S22expand-rootpart
>> new file mode 100755
>> index 0000000..7f19e7a
>> --- /dev/null
>> +++ b/board/raspberrypi/rootfs-overlay/etc/init.d/S22expand-rootpart
>> @@ -0,0 +1,36 @@
>> +#!/bin/sh
>> +
>> +case "$1" in
>> +       start)
>> +               echo -n "Expanding the root partition: "
>> +               BLOCK_DEV="/dev/mmcblk0"
>> +               PART_NUM="2"
>> +               PART_START="$(parted -ms "${BLOCK_DEV}" unit s p
>> 2>/dev/null | \
>> +                       grep "^${PART_NUM}:" | cut -d : -f 2 | sed
>> 's/s$//')"
>> +               if [ -z "${PART_START}" ]; then
>> +                       echo "failed"
>
>
> Note that the existing start sector can also be found in
> /sys/class/block/mmcblk0p2/start
> So can remove the parted dependency.

Yes, but we would have to check how portable this is among all Linux
versions if we want to move that to fs/ext2/ or packages/.

[...]

>> diff --git a/configs/raspberrypi2_defconfig
>> b/configs/raspberrypi2_defconfig
>> index 73dc290..2e064e0 100644
>> --- a/configs/raspberrypi2_defconfig
>> +++ b/configs/raspberrypi2_defconfig
>> @@ -3,6 +3,7 @@ BR2_cortex_a7=y
>>   BR2_ARM_EABIHF=y
>>   BR2_ARM_FPU_NEON_VFPV4=y
>>   +BR2_TOOLCHAIN_BUILDROOT_GLIBC=y
>
>
> Why is glibc required?
> Isn't it possible to just set the right options (like wchar, largefile) for
> uclibc?

Maybe. It was for e2fsprogs or parted. I will check that.

>>   BR2_TOOLCHAIN_BUILDROOT_CXX=y
>>     BR2_TARGET_GENERIC_GETTY_PORT="tty1"
>> @@ -27,6 +28,13 @@ BR2_LINUX_KERNEL_INTREE_DTS_NAME="bcm2709-rpi-2-b"
>>   BR2_PACKAGE_RPI_FIRMWARE=y
>>   # BR2_PACKAGE_RPI_FIRMWARE_INSTALL_DTB_OVERLAYS is not set
>>   +BR2_PACKAGE_E2FSPROGS=y
>> +BR2_PACKAGE_E2FSPROGS_RESIZE2FS=y
>> +BR2_PACKAGE_PARTED=y
>> +BR2_PACKAGE_UTIL_LINUX_BINARIES=y
>> +
>> +BR2_ROOTFS_OVERLAY="board/raspberrypi2/rootfs-overlay"
>> +BR2_ROOTFS_POST_BUILD_SCRIPT="board/raspberrypi2/post-build.sh"
>
>
> Why is UTIL_LINUX_BINARIES necessary?
> If for fdisk,

Yes.

> doesn't busybox provides this as well?

Yes, it does, and it's selected in the default Busybox configuration
provided by Buildroot and used on RPi. But I've not checked whether it
is compatible with the one from util-linux regarding the command line
interactions. I will check this.

Best regards,
Beno?t

[Buildroot] [PATCH 1/4] configs/raspberrypi: use EABIhf

[Thomas Petazzoni]

Dear Beno?t Th?baudeau,

On Sat, 22 Aug 2015 22:01:25 +0200, Beno?t Th?baudeau wrote:
> The SoC of the Raspberry Pi 1 is an ARM1176JZF-S, which features a VFPv2
> FPU, so use the EABIhf target ABI, which is more efficient than EABI.
> This is also the default on Raspbian.
> 
> Signed-off-by: Beno?t Th?baudeau <benoit@wsystem.com>
> ---
>  configs/raspberrypi_defconfig | 1 +
>  1 file changed, 1 insertion(+)

Applied, thanks. I actually believe we should be doing the same for all
ARMv6/ARMv7 platforms, except if there is a specific issue with using
EABIhf (such as existing pre-built binaries that are EABI only).

Maybe we should even use EABIhf by default on ARMv6+.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 1/4] configs/raspberrypi: use EABIhf

[Benoît Thébaudeau]

Dear Thomas Petazzoni,

On Mon, Oct 12, 2015 at 11:59 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Applied, thanks. I actually believe we should be doing the same for all
> ARMv6/ARMv7 platforms, except if there is a specific issue with using
> EABIhf (such as existing pre-built binaries that are EABI only).

Yes, there is such an issue because some hardware vendors provide
closed libraries that use soft-FP. It was the case e.g. for i.MX6
(audio/video CODEC and GPU libraries), but they now provide hard-float
binaries IIRC. So such a change should be checked on a per-platform
basis.

> Maybe we should even use EABIhf by default on ARMv6+.

Yes, I think that it would be a sane default, and it would still be
possible to set EABI for a specific platform for which EABIhf would
not apply.

Best regards,
Beno?t

[Buildroot] [PATCH 1/4] configs/raspberrypi: use EABIhf

[Arnout Vandecappelle]

On 13-10-15 01:17, Beno?t Th?baudeau wrote:
> Dear Thomas Petazzoni,
> 
> On Mon, Oct 12, 2015 at 11:59 PM, Thomas Petazzoni
> <thomas.petazzoni@free-electrons.com> wrote:
>> Applied, thanks. I actually believe we should be doing the same for all
>> ARMv6/ARMv7 platforms, except if there is a specific issue with using
>> EABIhf (such as existing pre-built binaries that are EABI only).
> 
> Yes, there is such an issue because some hardware vendors provide
> closed libraries that use soft-FP. It was the case e.g. for i.MX6
> (audio/video CODEC and GPU libraries), but they now provide hard-float
> binaries IIRC. So such a change should be checked on a per-platform
> basis.

 Actually, I think at the BR developer meeting we decided that even if there are
binary-only packages that require softfloat, our defconfig should still use
hardfloat if possible (since the defconfig doesn't enable that package). Anyway
the point is moot since those binary packages will in my experience also depend
on glibc, so the defconfig will not work for them anyway. Also, I can't find
this back in the report :-(


 Regards,
 Arnout

> 
>> Maybe we should even use EABIhf by default on ARMv6+.
> 
> Yes, I think that it would be a sane default, and it would still be
> possible to set EABI for a specific platform for which EABIhf would
> not apply.
> 
> Best regards,
> Beno?t
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
> 


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH 1/4] configs/raspberrypi: use EABIhf

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

Hi,

> Applied, thanks. I actually believe we should be doing the same for all
 > ARMv6/ARMv7 platforms, except if there is a specific issue with using
 > EABIhf (such as existing pre-built binaries that are EABI only).

 > Maybe we should even use EABIhf by default on ARMv6+.

Agreed!

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH 1/4] configs/raspberrypi: use EABIhf

[Peter Korsgaard]

>>>>> "Peter" == Peter Korsgaard <peter@korsgaard.com> writes:

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:
 > Hi,

 >> Applied, thanks. I actually believe we should be doing the same for all
 >> ARMv6/ARMv7 platforms, except if there is a specific issue with using
 >> EABIhf (such as existing pre-built binaries that are EABI only).

 >> Maybe we should even use EABIhf by default on ARMv6+.

 > Agreed!

Only potential disadvantage is that we currently don't have any
preconfigured external ARMv6 EABIhf toolchains :/

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH 2/4] board/raspberrypi: prepare the image files for the target

[Yann E. MORIN]

Beno?t, All,

On 2015-08-22 22:01 +0200, Beno?t Th?baudeau spake thusly:
> Automate the preparation of the generated image files for the Raspberry
> Pi firmware:
>  - mark the kernel image as supporting Device Tree,
>  - place the marked kernel image and the DTBs into the rpi-firmware
>    folder.
> 
> The instructions in readme.txt are updated accordingly.
> 
> Signed-off-by: Beno?t Th?baudeau <benoit@wsystem.com>
> ---
>  board/raspberrypi/post-image.sh | 17 ++++++++
>  board/raspberrypi/readme.txt    | 90 ++++++++++++++++++-----------------------
>  configs/raspberrypi2_defconfig  |  2 +
>  configs/raspberrypi_defconfig   |  2 +
>  4 files changed, 60 insertions(+), 51 deletions(-)
>  create mode 100755 board/raspberrypi/post-image.sh
> 
> diff --git a/board/raspberrypi/post-image.sh b/board/raspberrypi/post-image.sh
> new file mode 100755
> index 0000000..db9c53c
> --- /dev/null
> +++ b/board/raspberrypi/post-image.sh
> @@ -0,0 +1,17 @@
> +#!/bin/bash -e
> +
> +RPI_FW_BIN_DIR="${BINARIES_DIR}/rpi-firmware"
> +KERNEL_IMG="${BINARIES_DIR}/zImage"
> +KERNEL_RPI_FW_IMG="${RPI_FW_BIN_DIR}/zImage"
> +
> +# A special marker must be appended to the kernel image so that the firmware of
> +# the Raspberry Pi knows that it supports Device Tree. Without this marker, the
> +# firmware passes the ATAGS boot data instead of the appropriate DTB to the
> +# kernel.
> +# The final kernel image must be in the rpi-firmware folder.
> +mkknlimg "${KERNEL_IMG}" "${KERNEL_RPI_FW_IMG}"
> +
> +# Move the DTBs where the firmware expects them to be, i.e. to the rpi-firmware
> +# folder.
> +find "${BINARIES_DIR}/" -maxdepth 1 -type f -name '*.dtb' -print0 | \
> +	xargs -0r mv -ft "${RPI_FW_BIN_DIR}/"

Why not simply:
    cp -l "${BINARIES_DIR}/"*.dtb "${RPI_FW_BIN_DIR}"

I'd prefer we keep the DTB files where they were installed, so the user
does not get confused.

> diff --git a/board/raspberrypi/readme.txt b/board/raspberrypi/readme.txt
> index cf95879..168fab5 100644
> --- a/board/raspberrypi/readme.txt
> +++ b/board/raspberrypi/readme.txt
[--SNIP--]
> @@ -61,77 +61,65 @@ Result of the build
>  After building, you should obtain this tree:
>  
>      output/images/
> -    +-- rootfs.tar                  [0]
> +    +-- rootfs.cpio                 [0]
> +    +-- rootfs.tar                  [1]
>      +-- rpi-firmware/
> +    |   +-- bcm2708-rpi-b.dtb       [2]
> +    |   +-- bcm2708-rpi-b-plus.dtb  [3]
> +    |   +-- bcm2709-rpi-2-b.dtb     [4]
>      |   +-- bootcode.bin
> +    |   +-- cmdline.txt             [5]
>      |   +-- config.txt
>      |   +-- fixup.dat
> -    |   `-- start.elf
> -    +-- bcm2708-rpi-b.dtb           [1]
> -    +-- bcm2708-rpi-b-plus.dtb      [1]
> -    +-- bcm2709-rpi-2-b.dtb         [1]

As I said above, I'd prefer the DTB files be left in their original
location, with the ones in rpi-firmware/ being only copies of, or
hard-links to, the originals.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 2/4] board/raspberrypi: prepare the image files for the target

[Benoît Thébaudeau]

Dear Yann E. MORIN,

On Sat, Oct 17, 2015 at 1:45 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
[...]
> Why not simply:
>     cp -l "${BINARIES_DIR}/"*.dtb "${RPI_FW_BIN_DIR}"
>
> I'd prefer we keep the DTB files where they were installed, so the user
> does not get confused.
[...]
> As I said above, I'd prefer the DTB files be left in their original
> location, with the ones in rpi-firmware/ being only copies of, or
> hard-links to, the originals.

OK, I'll do that.

Best regards,
Beno?t

[Buildroot] [PATCH] arch/arm: use EABIhf by default with VFP

[Benoît Thébaudeau]

Set EABIhf as the default target ABI for the ARM processors that have or
may have a VFP unit, since this ABI is the most efficient in that case.
Of course, EABI can still be selected manually if needed.

Signed-off-by: Beno?t Th?baudeau <benoit.thebaudeau.dev@gmail.com>
---
 arch/Config.in.arm | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/Config.in.arm b/arch/Config.in.arm
index 4d10f4c..f0110b1 100644
--- a/arch/Config.in.arm
+++ b/arch/Config.in.arm
@@ -188,6 +188,7 @@ endchoice
 choice
 	prompt "Target ABI"
 	depends on BR2_arm || BR2_armeb
+	default BR2_ARM_EABIHF if BR2_ARM_CPU_MAYBE_HAS_VFPV2 || BR2_ARM_CPU_HAS_VFPV2
 	default BR2_ARM_EABI
 	help
 	  Application Binary Interface to use. The Application Binary
-- 
2.1.4

[Buildroot] [PATCH] arch/arm: use EABIhf by default with VFP

[Peter Korsgaard]

>>>>> "Beno?t" == Beno?t Th?baudeau <benoit.thebaudeau.dev@gmail.com> writes:

 > Set EABIhf as the default target ABI for the ARM processors that have or
 > may have a VFP unit, since this ABI is the most efficient in that case.
 > Of course, EABI can still be selected manually if needed.

 > Signed-off-by: Beno?t Th?baudeau <benoit.thebaudeau.dev@gmail.com>
 > ---
 >  arch/Config.in.arm | 1 +
 >  1 file changed, 1 insertion(+)

 > diff --git a/arch/Config.in.arm b/arch/Config.in.arm
 > index 4d10f4c..f0110b1 100644
 > --- a/arch/Config.in.arm
 > +++ b/arch/Config.in.arm
 > @@ -188,6 +188,7 @@ endchoice
 >  choice
 >  	prompt "Target ABI"
 >  	depends on BR2_arm || BR2_armeb
 > +	default BR2_ARM_EABIHF if BR2_ARM_CPU_MAYBE_HAS_VFPV2 || BR2_ARM_CPU_HAS_VFPV2

I think we should only do it when we are certain it will run,
E.G. arm926 (our default variant) selects MAYBE_HAS_VFPV2, but it is
afaik quite rare to see arm9s with a vfp.

Committed with that changed, thanks.

-- 
Venlig hilsen,
Peter Korsgaard 

[Buildroot] [PATCH] arch/arm: use EABIhf by default with VFP

[Peter Korsgaard]

>>>>> "Beno?t" == Beno?t Th?baudeau <benoit.thebaudeau.dev@gmail.com> writes:

 > Set EABIhf as the default target ABI for the ARM processors that have or
 > may have a VFP unit, since this ABI is the most efficient in that case.
 > Of course, EABI can still be selected manually if needed.

On a related note, should we default to thumb2 if the CPU supports it?

-- 
Venlig hilsen,
Peter Korsgaard 

[Buildroot] [PATCH] arch/arm: use EABIhf by default with VFP

[Benoît Thébaudeau]

On Sun, Oct 25, 2015 at 7:55 PM, Peter Korsgaard <peter@korsgaard.com> wrote:
>>>>>> "Beno?t" == Beno?t Th?baudeau <benoit.thebaudeau.dev@gmail.com> writes:
>
>  > Set EABIhf as the default target ABI for the ARM processors that have or
>  > may have a VFP unit, since this ABI is the most efficient in that case.
>  > Of course, EABI can still be selected manually if needed.
>
> On a related note, should we default to thumb2 if the CPU supports it?

I would say:

    choice
           prompt "ARM instruction set"
           default BR2_ARM_INSTRUCTIONS_ARM if BR2_ARM_CPU_HAS_ARM
           default BR2_ARM_INSTRUCTIONS_THUMBS2 if BR2_ARM_CPU_HAS_THUMBS2
           default BR2_ARM_INSTRUCTIONS_THUMBS if BR2_ARM_CPU_HAS_THUMBS

Best regards,
Beno?t

[Buildroot] [PATCH] arch/arm: use EABIhf by default with VFP

[Peter Korsgaard]

>>>>> "Beno?t" == Beno?t Th?baudeau <benoit.thebaudeau.dev@gmail.com> writes:

 > On Sun, Oct 25, 2015 at 7:55 PM, Peter Korsgaard <peter@korsgaard.com> wrote:
 >>>>>>> "Beno?t" == Beno?t Th?baudeau <benoit.thebaudeau.dev@gmail.com> writes:
 >> 
 >> > Set EABIhf as the default target ABI for the ARM processors that have or
 >> > may have a VFP unit, since this ABI is the most efficient in that case.
 >> > Of course, EABI can still be selected manually if needed.
 >> 
 >> On a related note, should we default to thumb2 if the CPU supports it?

 > I would say:

 >     choice
 >            prompt "ARM instruction set"
 >            default BR2_ARM_INSTRUCTIONS_ARM if BR2_ARM_CPU_HAS_ARM
 >            default BR2_ARM_INSTRUCTIONS_THUMBS2 if BR2_ARM_CPU_HAS_THUMBS2
 >            default BR2_ARM_INSTRUCTIONS_THUMBS if BR2_ARM_CPU_HAS_THUMBS

I'm not sure it makes sense to default to thumb1 even if the CPU
supports it because of various compatibility issues - So simply:

default BR2_ARM_INSTRUCTIONS_THUMBS2 if BR2_ARM_CPU_HAS_THUMBS2

(if no thumb2 is available, the first available option (ARM) is used).

-- 
Venlig hilsen,
Peter Korsgaard 

[Buildroot] [PATCH] arch/arm: use EABIhf by default with VFP

[Benoît Thébaudeau]

On Sun, Oct 25, 2015 at 9:40 PM, Peter Korsgaard <peter@korsgaard.com> wrote:
>>>>>> "Beno?t" == Beno?t Th?baudeau <benoit.thebaudeau.dev@gmail.com> writes:
>
>  > On Sun, Oct 25, 2015 at 7:55 PM, Peter Korsgaard <peter@korsgaard.com> wrote:
>  >>>>>>> "Beno?t" == Beno?t Th?baudeau <benoit.thebaudeau.dev@gmail.com> writes:
>  >>
>  >> > Set EABIhf as the default target ABI for the ARM processors that have or
>  >> > may have a VFP unit, since this ABI is the most efficient in that case.
>  >> > Of course, EABI can still be selected manually if needed.
>  >>
>  >> On a related note, should we default to thumb2 if the CPU supports it?
>
>  > I would say:
>
>  >     choice
>  >            prompt "ARM instruction set"
>  >            default BR2_ARM_INSTRUCTIONS_ARM if BR2_ARM_CPU_HAS_ARM
>  >            default BR2_ARM_INSTRUCTIONS_THUMBS2 if BR2_ARM_CPU_HAS_THUMBS2
>  >            default BR2_ARM_INSTRUCTIONS_THUMBS if BR2_ARM_CPU_HAS_THUMBS
>
> I'm not sure it makes sense to default to thumb1 even if the CPU
> supports it because of various compatibility issues

It's the first default that wins, not the last one, so above it's ARM
over Thumb1.

> - So simply:
>
> default BR2_ARM_INSTRUCTIONS_THUMBS2 if BR2_ARM_CPU_HAS_THUMBS2
>
> (if no thumb2 is available, the first available option (ARM) is used).

With ARM + Thumb2, I would rather use ARM to avoid compatibility
issues, except on platforms very constrained on memory. On Cortex-M3,
there's not even ARM. Or is it common to use Thumb2 on bigger
platforms like Cortex-A*?

Best regards,
Beno?t

[Buildroot] [PATCH] arch/arm: use EABIhf by default with VFP

[Yann E. MORIN]

Beno?t, Peter, All,

On 2015-10-25 21:52 +0100, Beno?t Th?baudeau spake thusly:
> On Sun, Oct 25, 2015 at 9:40 PM, Peter Korsgaard <peter@korsgaard.com> wrote:
> >>>>>> "Beno?t" == Beno?t Th?baudeau <benoit.thebaudeau.dev@gmail.com> writes:
> >
> >  > On Sun, Oct 25, 2015 at 7:55 PM, Peter Korsgaard <peter@korsgaard.com> wrote:
> >  >>>>>>> "Beno?t" == Beno?t Th?baudeau <benoit.thebaudeau.dev@gmail.com> writes:
> >  >>
> >  >> > Set EABIhf as the default target ABI for the ARM processors that have or
> >  >> > may have a VFP unit, since this ABI is the most efficient in that case.
> >  >> > Of course, EABI can still be selected manually if needed.
> >  >>
> >  >> On a related note, should we default to thumb2 if the CPU supports it?
> >
> >  > I would say:
> >
> >  >     choice
> >  >            prompt "ARM instruction set"
> >  >            default BR2_ARM_INSTRUCTIONS_ARM if BR2_ARM_CPU_HAS_ARM
> >  >            default BR2_ARM_INSTRUCTIONS_THUMBS2 if BR2_ARM_CPU_HAS_THUMBS2
> >  >            default BR2_ARM_INSTRUCTIONS_THUMBS if BR2_ARM_CPU_HAS_THUMBS
> >
> > I'm not sure it makes sense to default to thumb1 even if the CPU
> > supports it because of various compatibility issues
> 
> It's the first default that wins, not the last one,

To be more precise: it's the first default which dependencies are
fullfiled that wins.

> so above it's ARM over Thumb1.

> > - So simply:
> >
> > default BR2_ARM_INSTRUCTIONS_THUMBS2 if BR2_ARM_CPU_HAS_THUMBS2
> >
> > (if no thumb2 is available, the first available option (ARM) is used).
> 
> With ARM + Thumb2, I would rather use ARM to avoid compatibility
> issues, except on platforms very constrained on memory. On Cortex-M3,
> there's not even ARM. Or is it common to use Thumb2 on bigger
> platforms like Cortex-A*?

I agree that we should default to ARM if available. If not, then default
to Thumb2 if available. Finally, fallback to Thumb1.

We know that if neither ARM not Thumb2 is available, then we must have
Thumb1 available. otherwise it would not make sense to have an ARM CPU
that does not support any of those three.

So, slight ammendment to Beno?t's proposal:

    choice
        prompt "ARM instruction set"
        default BR2_ARM_INSTRUCTIONS_ARM    if BR2_ARM_CPU_HAS_ARM
        default BR2_ARM_INSTRUCTIONS_THUMB2 if BR2_ARM_CPU_HAS_THUMB2
        default BR2_ARM_INSTRUCTIONS_THUMB

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH] arch/arm: use EABIhf by default with VFP

[Thomas Petazzoni]

Dear Peter Korsgaard,

On Sun, 25 Oct 2015 19:53:06 +0100, Peter Korsgaard wrote:

>  > diff --git a/arch/Config.in.arm b/arch/Config.in.arm
>  > index 4d10f4c..f0110b1 100644
>  > --- a/arch/Config.in.arm
>  > +++ b/arch/Config.in.arm
>  > @@ -188,6 +188,7 @@ endchoice
>  >  choice
>  >  	prompt "Target ABI"
>  >  	depends on BR2_arm || BR2_armeb
>  > +	default BR2_ARM_EABIHF if BR2_ARM_CPU_MAYBE_HAS_VFPV2 || BR2_ARM_CPU_HAS_VFPV2
> 
> I think we should only do it when we are certain it will run,
> E.G. arm926 (our default variant) selects MAYBE_HAS_VFPV2, but it is
> afaik quite rare to see arm9s with a vfp.
> 
> Committed with that changed, thanks.

default BR2_ARM_EABIHF if BR2_ARM_CPU_MAYBE_HAS_VFPV2 || BR2_ARM_CPU_HAS_VFPV2

was indeed wrong, since having BR2_ARM_CPU_MAYBE_HAS_VFPV2 does not
indicate that the SoC has VFP.

However, unlike BR2_ARM_CPU_HAS_NEON which gets selected even on ARM
cores for which NEON is optional, but for which the user has explicitly
requested NEON support to be enabled (through BR2_ARM_ENABLE_NEON), it
is not the case for the VFP option. I think we should do that.
Otherwise, BR2_ARM_CPU_HAS_VFPV2 will not be true, even if the VFPv2
unit is actually available and used.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH] arch/arm: use EABIhf by default with VFP

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 > Dear Peter Korsgaard,
 > On Sun, 25 Oct 2015 19:53:06 +0100, Peter Korsgaard wrote:

 >> > diff --git a/arch/Config.in.arm b/arch/Config.in.arm
 >> > index 4d10f4c..f0110b1 100644
 >> > --- a/arch/Config.in.arm
 >> > +++ b/arch/Config.in.arm
 >> > @@ -188,6 +188,7 @@ endchoice
 >> >  choice
 >> >  	prompt "Target ABI"
 >> >  	depends on BR2_arm || BR2_armeb
 >> > +	default BR2_ARM_EABIHF if BR2_ARM_CPU_MAYBE_HAS_VFPV2 || BR2_ARM_CPU_HAS_VFPV2
 >> 
 >> I think we should only do it when we are certain it will run,
 >> E.G. arm926 (our default variant) selects MAYBE_HAS_VFPV2, but it is
 >> afaik quite rare to see arm9s with a vfp.
 >> 
 >> Committed with that changed, thanks.

 > default BR2_ARM_EABIHF if BR2_ARM_CPU_MAYBE_HAS_VFPV2 || BR2_ARM_CPU_HAS_VFPV2

 > was indeed wrong, since having BR2_ARM_CPU_MAYBE_HAS_VFPV2 does not
 > indicate that the SoC has VFP.

Indeed.

 > However, unlike BR2_ARM_CPU_HAS_NEON which gets selected even on ARM
 > cores for which NEON is optional, but for which the user has explicitly
 > requested NEON support to be enabled (through BR2_ARM_ENABLE_NEON), it
 > is not the case for the VFP option. I think we should do that.
 > Otherwise, BR2_ARM_CPU_HAS_VFPV2 will not be true, even if the VFPv2
 > unit is actually available and used.

But packages should use the BR2_ARM_FPU_VFPV2 (and 3/4) symbols to check
if they should use the VFP, and not BR2_ARM_CPU_MAYBE_HAS_VFPV2 || BR2_ARM_CPU_HAS_VFPV2

I don't think kconfig will let us both have these options depend on
BR2_ARM_CPU_MAYBE_HAS_VFPV2 || BR2_ARM_CPU_HAS_VFPV2 and select
BR2_ARM_CPU_HAS_VFPV2.

-- 
Venlig hilsen,
Peter Korsgaard 

[Buildroot] [PATCH] arch/arm: use EABIhf by default with VFP

[Thomas Petazzoni]

Peter,

On Mon, 26 Oct 2015 08:47:54 +0100, Peter Korsgaard wrote:

>  > However, unlike BR2_ARM_CPU_HAS_NEON which gets selected even on ARM
>  > cores for which NEON is optional, but for which the user has explicitly
>  > requested NEON support to be enabled (through BR2_ARM_ENABLE_NEON), it
>  > is not the case for the VFP option. I think we should do that.
>  > Otherwise, BR2_ARM_CPU_HAS_VFPV2 will not be true, even if the VFPv2
>  > unit is actually available and used.
> 
> But packages should use the BR2_ARM_FPU_VFPV2 (and 3/4) symbols to check
> if they should use the VFP, and not BR2_ARM_CPU_MAYBE_HAS_VFPV2 || BR2_ARM_CPU_HAS_VFPV2

But then they have to test BR2_ARM_FPU_VFPV2 || BR2_ARM_FPU_VFPV3 ||
BR2_ARM_FPU_VFPV3D16 || ...

If they just want to know if a VFP is available.

> I don't think kconfig will let us both have these options depend on
> BR2_ARM_CPU_MAYBE_HAS_VFPV2 || BR2_ARM_CPU_HAS_VFPV2 and select
> BR2_ARM_CPU_HAS_VFPV2.

Indeed, it doesn't. Some better solution is needed :)

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH] arch/arm: use EABIhf by default with VFP

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 > Peter,
 > On Mon, 26 Oct 2015 08:47:54 +0100, Peter Korsgaard wrote:

 >> > However, unlike BR2_ARM_CPU_HAS_NEON which gets selected even on ARM
 >> > cores for which NEON is optional, but for which the user has explicitly
 >> > requested NEON support to be enabled (through BR2_ARM_ENABLE_NEON), it
 >> > is not the case for the VFP option. I think we should do that.
 >> > Otherwise, BR2_ARM_CPU_HAS_VFPV2 will not be true, even if the VFPv2
 >> > unit is actually available and used.
 >> 
 >> But packages should use the BR2_ARM_FPU_VFPV2 (and 3/4) symbols to check
 >> if they should use the VFP, and not BR2_ARM_CPU_MAYBE_HAS_VFPV2 || BR2_ARM_CPU_HAS_VFPV2

 > But then they have to test BR2_ARM_FPU_VFPV2 || BR2_ARM_FPU_VFPV3 ||
 > BR2_ARM_FPU_VFPV3D16 || ...

 > If they just want to know if a VFP is available.

If that is the case we could make a hidden BR2_ARM_FPU_VFP that gets
selected by those, or simply check for !ARM_SOFT_FLOAT.

-- 
Venlig hilsen,
Peter Korsgaard