* [Buildroot] [PATCH 0/5 v4] SUBJECT
@ 2014-01-03 17:19 Yann E. MORIN
2014-01-03 17:19 ` [Buildroot] [PATCH 1/5] package/squashfs: add selection for the host variant Yann E. MORIN
` (5 more replies)
0 siblings, 6 replies; 22+ messages in thread
From: Yann E. MORIN @ 2014-01-03 17:19 UTC (permalink / raw)
To: buildroot
From: "Yann E. MORIN" <yann.morin.1998@free.fr>
Hello All!
During the last developpers' day in Edimburgh, we discussed my Buildroot.config
project, and especially the 'genimages' infrastructure I've done in there.
I've came up with 'genimages' after I came to the conclusion that the current
images generation in Buildroot is limited to generating a single filesystem
images, but most device do require a more complex setup.
For example. the Rapsberry Pi needs an SDcard (mmcblk0) formatted as thus:
mmcblk0
|- mmcblk0p1 (FAT32)
| |- bootcode.bin
| |- fixup.dat
| |- start.elf
| |- config.txt
| |- cmdline.txt
| `- kernel.bin
`- mmcblk0p2 (whatever, eg. ext3, btrfs...)
`- all of /
With the current output of Buildroot, that requires a lot of fiddling with
the partitioning of the SDcard (may require root), formatting the filesystems
(may require root), extract rootfs.tar (may require root to mount, requires
root to extract), copy select files from output/images/rpi-firmware/ to the
first partition (may require root to mount).
Each user would either have to manually perform those actions (error prone),
or write a small shell script to automate this (not trivial).
Also, some other devices require special, but different formatting as well.
Rather than have every users all suffer in silence (and sometimes not in
silence) by re-inventing such a script over and over again, for each device
that need such a setup, I came up with the 'genimages' idea: a description
of the layout of the storage devices, partitioning schemes, and partitions
content.
So, here is the 'genimages' from Buildroot.config, that I updated (in fact,
trimmed) to include in Buildroot itself.
The basis is that we bundle some trivial partition description for the
boards we have in board/ and reference it in the corresponding defconfigs.
How does 'genimages' work?
First, genimages extract the generated rootfs.tar. Hence, rootfs.tar is
always generated if a partition description is configured.
Second, genimages generates one filesystem image for each partition listed
in the description. Once the filesystem is generated, its mountpoint is
emptied, so it does not appear in the upper filesystem.
Third, genimages aggregates all the partition of a device into a single
image file
Finally, those device images can be flashed directly on the raw device of
the storage, without manually fiddling with fdisk, mount, tar and so
on...
- Patch 1/5 adds squahsfs selection in the host tools menu, and is not
a dependency to the rest of the series
- Patch 2/5 is the actual genimages implementation, with support for
some basic filesystems (ext and vfat), and support for MBR and GPT
partitioning schemes
- Patch 3/5 adds a partition table for the Raspberry Pi, as an example
of how to use genimages
- Patch 4/5 adds support to genimages to generate squashfs filesystems
- Patch 5/5 adds support for raw partitions and devices
Patch 4/5 and 5/5 are separate, just to show how easy it is to add a new
filesystem type. Raw is a little bit more involved, since it is a special
case that needs a very little bit of special-casing in the generic infra.
Otherwise, they could merged with patch 2/5.
How does 'genimages' work internally?
The partition layout is decribed in a .ini-like file.
genimages is a relatively simple shell script that parses that .ini file.
Based on each device/partition description, it calls to 'handlers' (or
hooks), for each kind of content we support.
For example, we have two types of partitioning scheme supported for
device: mbr and gpt. Each is implemented in its own shell-fragment.
Ditto for partitions: we support different filesystems, ext, squashfs,
and vfat; they are each implemented in their own shell-fragment.
Each handler has to define the 'do_image' shell function. That functions
should expect two arguments: the base dir of the filesystem, and the
filename of the image to generate, and it has access to the configuration
from the .ini file.
How to use 'genimages'?
It's all explained in the manual. A pre-rendered version is available for
reading, and easy review on-line there:
http://ymorin.is-a-geek.org/download/tmp/buildroot/manual/manual.html
The new section is "3.3.6 Customizing the generated filesystem images"
and there is a detailed description of the partition table layout in
appendix "12.3. Partition table layout description syntax".
Changes v3 -> v4:
- added support for raw partitions/devices
- fix MBR bootcode (J?r?me)
- a few fixes here and there
- less typoes
Changes v2 -> v3:
- misc typoes in the manual
Changes v1 -> v2:
- drop rpi-firmware install into /boot (Thomas, Ryan)
- drop move of rpi-firmware in bootloader sub-menu (Thomas)
- add possibility to generate an unmounted filesystem (Thomas, Ryan)
- add ability to specify files to add in a filesystem instead of a
sub-dir of target/ (Ryan, during a hacking session)
- properly depend on the host-tools if they are selected
- add GPT support
- add a third filesystem, squashfs (in its own patch, as an example
of how easy it is to add one)
Regards,
Yann E. MORIN.
----------------------------------------------------------------
Yann E. MORIN (5):
package/squashfs: add selection for the host variant
fs/custom: generate complete, partition-based device images
board/raspberrypi: provide partition description for the new genimages
fs/custom: add support for squashfs
fs/custom: add support for raw device/partition content
board/raspberrypi/partitions | 35 ++++
configs/raspberrypi_defconfig | 3 +
docs/manual/appendix.txt | 1 +
docs/manual/customize-filesystems.txt | 36 ++++
docs/manual/customize.txt | 2 +
docs/manual/partition-layout.txt | 315 ++++++++++++++++++++++++++++++
fs/Config.in | 1 +
fs/custom/Config.in | 16 ++
fs/custom/boot/gpt | 129 +++++++++++++
fs/custom/boot/mbr | 64 +++++++
fs/custom/boot/pre-post | 14 ++
fs/custom/custom.mk | 25 +++
fs/custom/fs/ext | 26 +++
fs/custom/fs/pre-post | 72 +++++++
fs/custom/fs/squashfs | 19 ++
fs/custom/fs/vfat | 21 ++
fs/custom/genimages | 347 ++++++++++++++++++++++++++++++++++
fs/custom/raw/pre-post | 14 ++
fs/custom/raw/raw | 12 ++
package/Config.in.host | 1 +
package/squashfs/Config.in.host | 6 +
21 files changed, 1159 insertions(+)
create mode 100644 board/raspberrypi/partitions
create mode 100644 docs/manual/customize-filesystems.txt
create mode 100644 docs/manual/partition-layout.txt
create mode 100644 fs/custom/Config.in
create mode 100644 fs/custom/boot/gpt
create mode 100644 fs/custom/boot/mbr
create mode 100644 fs/custom/boot/pre-post
create mode 100644 fs/custom/custom.mk
create mode 100644 fs/custom/fs/ext
create mode 100644 fs/custom/fs/pre-post
create mode 100644 fs/custom/fs/squashfs
create mode 100644 fs/custom/fs/vfat
create mode 100755 fs/custom/genimages
create mode 100644 fs/custom/raw/pre-post
create mode 100644 fs/custom/raw/raw
create mode 100644 package/squashfs/Config.in.host
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH 1/5] package/squashfs: add selection for the host variant
2014-01-03 17:19 [Buildroot] [PATCH 0/5 v4] SUBJECT Yann E. MORIN
@ 2014-01-03 17:19 ` Yann E. MORIN
2014-02-03 14:53 ` Thomas De Schampheleire
2014-02-08 22:20 ` Peter Korsgaard
2014-01-03 17:19 ` [Buildroot] [PATCH 2/5] fs/custom: generate complete, partition-based device images Yann E. MORIN
` (4 subsequent siblings)
5 siblings, 2 replies; 22+ messages in thread
From: Yann E. MORIN @ 2014-01-03 17:19 UTC (permalink / raw)
To: buildroot
From: "Yann E. MORIN" <yann.morin.1998@free.fr>
If any of the post-image scripts wants to handle squashfs filesystems,
we need to expose an option for squashfs-tools to be user-selectable.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ryan Barnett <rjbarnet@rockwellcollins.com>
---
package/Config.in.host | 1 +
package/squashfs/Config.in.host | 6 ++++++
2 files changed, 7 insertions(+)
create mode 100644 package/squashfs/Config.in.host
diff --git a/package/Config.in.host b/package/Config.in.host
index 34e84bf..71597bf 100644
--- a/package/Config.in.host
+++ b/package/Config.in.host
@@ -12,6 +12,7 @@ source "package/omap-u-boot-utils/Config.in.host"
source "package/openocd/Config.in.host"
source "package/parted/Config.in.host"
source "package/sam-ba/Config.in.host"
+source "package/squashfs/Config.in.host"
source "package/sunxi-tools/Config.in.host"
source "package/uboot-tools/Config.in.host"
diff --git a/package/squashfs/Config.in.host b/package/squashfs/Config.in.host
new file mode 100644
index 0000000..b353374
--- /dev/null
+++ b/package/squashfs/Config.in.host
@@ -0,0 +1,6 @@
+config BR2_PACKAGE_HOST_SQUASHFS
+ bool "host squashfs"
+ help
+ Tools to generate SquashFS filesystems.
+
+ http://squashfs.sourceforge.net/
--
1.8.1.2
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH 2/5] fs/custom: generate complete, partition-based device images
2014-01-03 17:19 [Buildroot] [PATCH 0/5 v4] SUBJECT Yann E. MORIN
2014-01-03 17:19 ` [Buildroot] [PATCH 1/5] package/squashfs: add selection for the host variant Yann E. MORIN
@ 2014-01-03 17:19 ` Yann E. MORIN
2014-01-04 16:38 ` Maxime Hadjinlian
2014-04-09 13:28 ` Andreas Naumann
2014-01-03 17:19 ` [Buildroot] [PATCH 3/5] board/raspberrypi: provide partition description for the new genimages Yann E. MORIN
` (3 subsequent siblings)
5 siblings, 2 replies; 22+ messages in thread
From: Yann E. MORIN @ 2014-01-03 17:19 UTC (permalink / raw)
To: buildroot
From: "Yann E. MORIN" <yann.morin.1998@free.fr>
Contrary to the existing fs/ schemes, which each generate only a single
filesystem image for the root filesystem, this new scheme allows the
user to generate more complex images.
The basis behind this is a .ini-like description of the layout of the
final target storage:
- the list of device(s)
- per-device, the list of partition(s)
- per-partition, the content
It is possible to create MBR- or GPT-based partitoining schemes. Adding
new ones should be relatively easy (but would need adequate host tools).
For now, the only content possible for partitions is a filesystem. It
should be pretty easy to add new types (eg. un-formated, or raw blob).
Also, only two filesystems are supported: ext{2,3,4} and vfat. Adding
more will be relatively easy, provided we have the necessary host
packages to deal with those filesystems.
The existing Buildroot filesystem generators are re-used as much as
possible when it makes sense; when it does not (eg. for vfat), a specific
generator is used.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Ryan Barnett <rjbarnet@rockwellcollins.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: "J?r?me Oufella" <jerome.oufella@savoirfairelinux.com>
---
docs/manual/appendix.txt | 1 +
docs/manual/customize-filesystems.txt | 36 ++++
docs/manual/customize.txt | 2 +
docs/manual/partition-layout.txt | 298 +++++++++++++++++++++++++++++
fs/Config.in | 1 +
fs/custom/Config.in | 16 ++
fs/custom/boot/gpt | 129 +++++++++++++
fs/custom/boot/mbr | 64 +++++++
fs/custom/boot/pre-post | 14 ++
fs/custom/custom.mk | 25 +++
fs/custom/fs/ext | 26 +++
fs/custom/fs/pre-post | 72 +++++++
fs/custom/fs/vfat | 21 +++
fs/custom/genimages | 342 ++++++++++++++++++++++++++++++++++
14 files changed, 1047 insertions(+)
create mode 100644 docs/manual/customize-filesystems.txt
create mode 100644 docs/manual/partition-layout.txt
create mode 100644 fs/custom/Config.in
create mode 100644 fs/custom/boot/gpt
create mode 100644 fs/custom/boot/mbr
create mode 100644 fs/custom/boot/pre-post
create mode 100644 fs/custom/custom.mk
create mode 100644 fs/custom/fs/ext
create mode 100644 fs/custom/fs/pre-post
create mode 100644 fs/custom/fs/vfat
create mode 100755 fs/custom/genimages
diff --git a/docs/manual/appendix.txt b/docs/manual/appendix.txt
index 74ee8fd..53f4205 100644
--- a/docs/manual/appendix.txt
+++ b/docs/manual/appendix.txt
@@ -6,6 +6,7 @@ Appendix
include::makedev-syntax.txt[]
include::makeusers-syntax.txt[]
+include::partition-layout.txt[]
// Automatically generated lists:
diff --git a/docs/manual/customize-filesystems.txt b/docs/manual/customize-filesystems.txt
new file mode 100644
index 0000000..fd65c97
--- /dev/null
+++ b/docs/manual/customize-filesystems.txt
@@ -0,0 +1,36 @@
+// -*- mode:doc; -*-
+// vim: set syntax=asciidoc:
+
+[[filesystem-custom]]
+Customizing the generated filesystem images
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
++Buildroot+ knows by default how to generate a few different kind of
+filesystems, such as +squashfs+, +ext2/3/4+, +cramfs+... But those
+filesystems are all generated to contain the complete target directory
+hierarchy in a single filesystem, mounted as the root filesystem +/+.
+That is, even if you select both an +ext2+ and a +squashfs+ filesystems,
+the content of the two generated images will be the exact same, only the
+types of the filesystems will be different.
+
+Most devices require a more complex setup, with different parts of the
+directory structure split across different filesystems, each stored on
+different partitions of one or more storage devices.
+
++Buildroot+ can generate such complex setups, using a +partition table layout
+description+. This is a simple text file, not unlike the +.ini+ style of
+configuration files, that describes how the target directory hierarchy has
+to be split across the target storage devices. It is a bit like a flattened
+tree of the storage layout.
+
+Set the variable +BR2_TARGET_ROOTFS_CUSTOM_PARTITION_TABLE+ to the path of
+the file containing your +partition table layout description+.
+
+See xref:part-layout-desc-syntax[] for the complete documentation of the
++partition table layout description+ syntax.
+
+[underline]*Note:* Although more versatile than the single filesystem image
+mechanism, the +partition table layout description+ might be unable to
+describe very complex setups. For example, it is not capable of handling
+NFS-mounted filesystems, and +initramfs+ setups is not trivial (although
+possible in most cases).
diff --git a/docs/manual/customize.txt b/docs/manual/customize.txt
index 7e46fd8..6d062ea 100644
--- a/docs/manual/customize.txt
+++ b/docs/manual/customize.txt
@@ -14,6 +14,8 @@ include::customize-kernel-config.txt[]
include::customize-toolchain.txt[]
+include::customize-filesystems.txt[]
+
include::customize-store.txt[]
include::customize-packages.txt[]
diff --git a/docs/manual/partition-layout.txt b/docs/manual/partition-layout.txt
new file mode 100644
index 0000000..092ae1b
--- /dev/null
+++ b/docs/manual/partition-layout.txt
@@ -0,0 +1,298 @@
+// -*- mode:doc; -*-
+// vim: set syntax=asciidoc:
+
+[[part-layout-desc-syntax]]
+
+Partition table layout description syntax
+-----------------------------------------
+
+The +partition table layout description+ syntax is not unlike the standard
+https://en.wikipedia.org/wiki/.ini[+.ini+] syntax. There are two types of
+entries: +sections+, that may each contain zero or more +properties+.
+
++Sections+ are specified between square brackets +[]+, _eg._: +[name]+.
+
++Properties+ are specified as key-value pairs, _eg._: +key=value+, and
+are documented as:
+
+* +key-name+ (optional or mandatory): description
+** +value1+: description
+** +value2+: description
+** ...
+
+[underline]*Note:* Unlike the standard +.ini+ syntax, the +partition table
+layout description+ _is_ case-sensitive.
+
+The order of +sections+ is irrelevant. However, for readability, we recomend
+the +partition table layout description+ starts with the +global+ section.
+
+The global section
+~~~~~~~~~~~~~~~~~~
+
+The +[global]+ section defines some global settings, and the list of devices.
+
+Properties for the global section
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* +extract+ (mandatory): the type of base image to extract
+** +tar+: extract the +rootfs.tar+ base image generated by +Buildroot+
+
+* +devices+ (mandatory): the comma-separated list of storage devices to
+ use on the device. Each device is the filename of the device node
+ present in +/dev+
+
+* +keep_partitions+ (optional): also copy the individual partition images
+ of all devices to +$(BINARIES_DIR)+. Settings from the device sections
+ or the partition sections take precedence over this one.
+** +yes+: copy the individual partition images
+** +no+ (the default): do not copy individual partition images
+
+The devices and partitions sections
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The devices and partitions sections define, for each device or partition,
+the content of that device or partition.
+
+For each device listed in the +[global]+ section, there must be a
+corresponding section named after that device.
+
+For each partition listed in a device section, there must be a corresponding
+section named after that partition.
+
+Properties for the device section
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* +type+ (mandatory): the type of content for that device or partition
+** +boot+: the device contains one or more partitions, and
+ _may_ serve as a boot device
+
+* +keep_partitions+ (optional): also copy the individual partition images
+ for this device to +$(BINARIES_DIR)+. Settings from the partition
+ sections take precedence over this one.
+** +yes+: copy the individual partition images
+** +no+ (the default): do not copy individual partition images
+
+Properties for the partition section
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* +type+ (mandatory): the type of content for that device or partition
+** +fs+: the partition contains a filesystem
+
+* +size+: the size of that partition, in bytes
+
+* +keep+ (optional): copy this partition image to +$(BINARIES_DIR)+
+** +yes+: copy this partition image
+** +no+ (the default): do not copy this partition image
+
+Properties for +type=boot+
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* +boot_type+ (mandatory): the partitioning scheme to use on this device
+** +mbr+: use an https://en.wikipedia.org/wiki/Master_boot_record[MBR]
+ partitioning scheme
+** +gpt+: use a https://en.wikipedia.org/wiki/GUID_Partition_Table[GPT]
+ partitioning scheme
+
+* +partitions+ (mandatory): the comma-separated list of partition(s) on
+ this device; no two partitions may have the same name, even if they
+ reside on different devices; partitions names shall match this regexp:
+ `^[[:alpha:]][[:alnum:]-_]*$` (_ie._ starts with a letter, followed by
+ zero or more alpha-numeric character or a dash or an underscore)
+
+* +partalign+ (optional): the alignment of partitions, in bytes; defaults
+ to an alignment of one, which means no alignment; depending on the
+ +boot_type+, some restrictions may apply, and are documented for each
+ +boot_type+
+
+Properties for +boot_type=mbr+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* +mbr_bootcode+ (optional): the bootcode to use, as a path to the file
+ containing the bootcode image, relative to the +$(BINARIES_DIR)+
+ directory; defaults to no bootcode (eg. filled with zeroes)
+
+* +partalign+: must be a multiple of 512
+
+Properties for +boot_type=gpt+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* +partalign+: must be a multiple of 512
+
+********
+Currently, only 512-byte sectors are supported. 4k sectors are not.
+********
+
+Properties for partitions whose containing device is +boot_type=mbr+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* +mbr_type+ (mandatory): the partition
+ https://en.wikipedia.org/wiki/Partition_type#List_of_partition_IDs[type]
+
+Properties for +type=fs+
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+* +fs_type+ (mandatory): the type of filesystem to generate
+** +ext+: generate an extended filesystem (ext2, ext3, ext4)
+** +vfat+: generate a VFAT filesystem (FAT16, FAT32)
+
+* +fs_label+ (optional): the label to assign to this filesystem, if that
+ filesystem supports a label
+
+* +fs_files_0+, +fs_files_1+, +fs_files_N+ (optional): the list of files,
+ relative to $(BINARIES_DIR), to store in the filesystem. These entries
+ must be indexed starting from 0, and must be sequential: the first
+ missing entry ends the list
+
+* +fs_root+ (optional): the mountpoint of the filesystem
+
+* +fs_vfstype+ (optional): the type of filesystem to use when calling
+ +mount+, if different from +fs_type+
+
+* +fs_mntopts+ (optional): the mount options; defaults to +defaults+
+
+_Note_: valid use-cases for +fs_root+ and +fs_files_N+:
+
+* if only +fs_root+ is specified (and no +fs_files_N+): the filesystem
+ content is made exclusively from +$(TARGET_DIR)/$(fs_root)+, and the
+ filesystem is mounted at runtime
+
+* if both +fs_root+ and at least one +fs_files_N+ are specified: the
+ filesystem content is made exclusively from the +fs_files_N+ entries,
+ and mounted at runtime. +$(TARGET_DIR)/$(fs_root)+ must be empty
+
+* if at least one +fs_files_N+ is specified, and +fs_root+ is not: the
+ filesystem content is made exclusively from the +fs_files_N+ entries,
+ and the filesystem is not mounted at runtime
+
+* if neither +fs_root+ nor +fs_files_N+ is specified: this is an error
+
+Properties for +fs_type=ext+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* +ext_gen+ (mandatory): the generation of extended filesystem to generate
+** +2+, +3+, +4+: for an ext2, ext3 or ext4 filesystem
+
+* +ext_rev+ (mandatory): the revision of the extended filesystem
+** +0+ (ext2 only): generate a revision 0 extended filesystem filesystem
+** +1+ (mandatory for ext3 or ext4): generate a revision 1 extended
+ filesystem
+
+Properties for +fs_type=vfat+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* +vfat_size+ (optional): the VFAT-size of the filesystem
+** +12+, +16+, +32+: generate a FAT12, FAT16, or FAT32
+
+Generation of the filesystems
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The filesystems are generated in an order such that a filesystem that is
+mounted as a sub-directory of another filesystem is generated first.
+
+A filesystem is filled with the content of the directory corresponding to
+its mountpoint, and then that directory is emptied before continuing to the
+next filesystem. That way, the mountpoints are empty before the filesystem
+that contains them are generated.
+
+Finally, an entry in added in +/etc/fstab+ for each generated filesystem, so
+they are mounted at boot time.
+
+Examples
+~~~~~~~~
+
+.Simplest partition table layout description
+====
+----
+[global]
+extract=tar
+devices=sda
+
+[sda]
+type=boot
+boot_type=mbr
+partitions=root
+partalign=1048576
+
+[root]
+type=fs
+fs_type=ext
+fs_vfstype=ext4
+fs_root=/
+ext_gen=4
+ext_rev=1
+----
+
+The +partition table layout description+ above defines a single device
++sda+. That device contains a single partition, +root+, with an ext4
+filesystem, which is filled with the whole content of the +rootfs.tar+,
+and is mounted on +/+.
+====
+
+.More complex table layout description
+====
+----
+[global]
+extract=tar
+devices=mmcblk0,sda
+
+[mmcblk0]
+type=boot
+boot_type=mbr
+partitions=boot,root
+partalign=$((1024*1024))
+
+[sda]
+type=boot
+boot_type=mbr
+partitions=data
+partalign=4096
+
+[boot]
+type=fs
+mbr_type=$((0xC))
+size=$((16*1048576))
+fs_type=vfat
+fs_mntopts=ro
+fs_label=BOOT
+fs_root=/boot
+vfat_size=32
+
+[root]
+type=fs
+mbr_type=$((0x83))
+size=268435456
+fs_type=ext
+fs_vfstype=ext4
+fs_mntopts=discard,delalloc
+fs_root=/
+fs_label=ROOT
+ext_gen=4
+ext_rev=1
+
+[data]
+type=fs
+mbr_type=$((0x83))
+size=$((4*1024*1048576))
+fs_type=ext
+fs_vfstype=ext2
+fs_root=/data
+fs_label=DATA
+ext_gen=2
+ext_rev=1
+----
+====
+
+The example above defines two devices, +mmcblk0+ and +sda+.
+
+The +mmcblk0+ device contains two partitions, +boot+ and +root+; partitions
+are aligned on a 1MiB boundary. The +sda+ device contains a single partition,
++data+, aligned on a 4KiB boundary.
+
+The +boot+ partition is a 16MiB FAT32 filesystem filled with the content
+of, and mounted on, +/boot+, and with label +BOOT+.
+
+The +data+ partition is a 4GiB ext2r1 filesystem filled with the content
+of, and mounted on, +/data+, and with label +DATA+.
+
+The +root+ partition is a 256MiB ext4 filesystem filled the the rest of,
+and mounted on, +/+, and with label +ROOT+.
diff --git a/fs/Config.in b/fs/Config.in
index da4c5ff..44e04f7 100644
--- a/fs/Config.in
+++ b/fs/Config.in
@@ -3,6 +3,7 @@ menu "Filesystem images"
source "fs/cloop/Config.in"
source "fs/cpio/Config.in"
source "fs/cramfs/Config.in"
+source "fs/custom/Config.in"
source "fs/ext2/Config.in"
source "fs/initramfs/Config.in"
source "fs/iso9660/Config.in"
diff --git a/fs/custom/Config.in b/fs/custom/Config.in
new file mode 100644
index 0000000..e5a8ee7
--- /dev/null
+++ b/fs/custom/Config.in
@@ -0,0 +1,16 @@
+config BR2_TARGET_ROOTFS_CUSTOM
+ bool "Custom partition table layout"
+ select BR2_TARGET_ROOTFS_TAR
+
+config BR2_TARGET_ROOTFS_CUSTOM_PARTITION_TABLE
+ string "path to the custom partition table layout description"
+ depends on BR2_TARGET_ROOTFS_CUSTOM
+ help
+ Enter the path to a partition-table for your device.
+
+ This will allow Buildroot to generate a more complex target
+ image, which may consist of more than one filesystem on more
+ than one partition.
+
+ See docs/manual/bla-bla on how to construct such a partition
+ table.
diff --git a/fs/custom/boot/gpt b/fs/custom/boot/gpt
new file mode 100644
index 0000000..f978524
--- /dev/null
+++ b/fs/custom/boot/gpt
@@ -0,0 +1,129 @@
+# Build a complete GPT-based image
+
+#-----------------------------------------------------------------------------
+DEPENDS+=( parted )
+
+#-----------------------------------------------------------------------------
+# For a GPT-based partitionning, we need to compute the complete
+# image size before we can attempt to generate the partition table.
+# Then, we need to add the size of the GPT itself, plus that of its
+# backup copy, plus the protective MBR.
+# The size of the GPT itself depends on the sector size, and the
+# number of partitions in the GPT. Sectors can be either 512-byte
+# or 4096-byte large; The numbers of partitions is unlimited, but
+# it is suggested there is space to store at least 128 of them; a
+# partition description is 128-byte large.
+#
+# https://en.wikipedia.org/wiki/GUID_Partition_Table
+#
+# So, here's what we do:
+# - consider 512-byte sectors (since GPT on 4k sectors is not well
+# documented)
+# - consider at least 128 partitions; if the layout defines more than
+# that, we need to round that number up to the smallest multiple of
+# 4 (since there are 4 partition descriptions in a 512-byte sector)
+# - generate an empty, sparse file that is big enough to store the MBR,
+# the two GPT copies, and the aligned partitions.
+# - dump each partition in turn in their final location in that file
+# - generate a parted script that creates the partition table in that
+# file
+#
+# Simg = 512 + 2*(Sgpt) + ?( aligned(Spart,512) )
+# Sgpt = 512 + Nent*128
+#
+# Where:
+# Simg : size of the image
+# Sgpt : size of one GPT
+# Spart : size of each partition
+# Nent : number of partition entries
+# aligned() : the alignment function
+#
+# Sicne 4k-large sectors are not really explained on Wikipedia, we can
+# add this later on.
+do_image() {
+ # ${1} is fs_root, irrelevant here
+ local img="${2}"
+
+ # How many partitions do we have?
+ nb_parts=0
+ for part in ${partitions[${dev}]//,/ }; do
+ nb_parts=$((nb_parts+1))
+ done
+
+ # How many partition entries do we need?
+ nb_entries=$((4*((nb_parts+3)/4)))
+ nb_entries=$((nb_entries<128?128:nb_entries))
+
+ # The size of a single GPT
+ gpt_size=$((512+(128*nb_entries)))
+
+ # Offset of the first partition
+ begin=$(align_val $((512+$(align_val ${gpt_size} 512))) ${partalign} )
+
+ # Initialise our image file
+ dd if=/dev/zero of="${img}" \
+ bs=1 seek=${begin} count=0 \
+ conv=sparse 2>/dev/null
+
+ # Compute the space required to store all partitions
+ # and store them in the image file
+ size_parts=0
+ _offset=${begin}
+ i=1
+ debug "adding partions descriptions\n"
+ for part in ${partitions[${dev}]//,/ }; do
+ debug " %s\n" "${part}"
+ _part_img="${tmp_dir}/${dev}.${part}.img"
+ _size=$( align_val $( stat -c '%s' "${_part_img}" ) 512 )
+ part_offset+=( ${_offset} )
+ _attr="${values["${part}:gpt_attr"]}"
+ _label="${values["${part}:gpt_label"]}"
+
+ # If the partition has no label, use the filesystem label
+ if [ -z "${_label}" ]; then
+ _label="${values["${part}:fs_label"]}"
+ fi
+ if [ -z "${_label}" ]; then
+ _label="data"
+ fi
+
+ debug " start=%s\n" "${_offset}"
+ debug " size =%s\n" "${_size}"
+ debug " end =%s\n" "$((_offset+_size-1))"
+
+ dd if="${_part_img}" of="${img}" \
+ bs=512 seek=$((_offset/512)) \
+ conv=notrunc,sparse 2>/dev/null
+
+ parted_script+=( mkpart "${_label}" \
+ ${_offset} \
+ $((_offset+_size-1)) \
+ )
+ if [ -n "${_attr}" ]; then
+ for attr in "${_attr//,/ }"; do
+ parted_script+=( set ${i} ${attr} on )
+ done
+ fi
+
+ size_parts=$((size_parts+_size))
+ _offset=$((_offset+_size))
+ i=$((i+1))
+ done
+
+ # Terminate our image file
+ img_size=$(align_val $(( begin + size_parts + gpt_size )) 512)
+ debug "begin =%s\n" ${begin}
+ debug "nb_entry=%s\n" ${nb_entries}
+ debug "gpt_size=%s\n" ${gpt_size}
+ debug "img_size=%s\n" ${img_size}
+ dd if=/dev/zero of="${img}" \
+ bs=1 seek=${img_size} \
+ count=0 conv=sparse 2>/dev/null
+
+ for i in parted -s "${img}" mklabel gpt unit B "${parted_script[@]}"; do
+ debug "--> '%s'\n" "${i}"
+ done
+ parted -s "${img}" mklabel gpt unit B "${parted_script[@]}"
+}
+
+# vim: ft=sh
diff --git a/fs/custom/boot/mbr b/fs/custom/boot/mbr
new file mode 100644
index 0000000..667feed
--- /dev/null
+++ b/fs/custom/boot/mbr
@@ -0,0 +1,64 @@
+# Build a complete MBR-based image
+
+#-----------------------------------------------------------------------------
+DEPENDS+=( genpart )
+
+#-----------------------------------------------------------------------------
+do_image() {
+ # ${1} is fs_root, irrelevant here
+ local img="${2}"
+ local i begin part part_img size type _begin _size
+ local -a part_offset part_file bootcode
+
+ # Fill-in the boot record
+ bootcode="${values["${dev}:mbr_bootcode"]}"
+ if [ -n "${bootcode}" ]; then
+ bootcode="${BINARIES_DIR}/${bootcode}"
+ else
+ bootcode="/dev/zero"
+ fi
+ debug "adding bootcode '%s'\n" "${bootcode}"
+ dd if="${bootcode}" of="${img}" bs=$((0x1be)) count=1 2>/dev/null
+
+ # Generate partition entries
+ i=0
+ begin=${partalign}
+ debug "adding partitions descriptors\n"
+ for part in ${partitions[${dev}]//,/ }; do
+ debug " %s\n" "${part}"
+ part_offset+=( ${begin} )
+ part_img="${tmp_dir}/${dev}.${part}.img"
+ part_file+=( "${part_img}" )
+ size=$( align_val $( stat -c '%s' "${part_img}" ) 512 )
+ type="${values["${part}:mbr_type"]}"
+ # LBA is exressed in a number of 512-byte blocks
+ # and genparts only deals with LBA
+ _begin=$((begin/512)) # begin is already 512-byte aligned
+ _size=$((size/512)) # size is already 512-byte aligned
+ debug " start=%s (LBA %s)\n" "${begin}" "${_begin}"
+ debug " size =%s (LBA %s)\n" "${size}" "${_size}"
+ debug " type =%s\n" "${type}"
+ genpart -b ${_begin} -s ${_size} -t ${type} >>"${img}"
+ begin=$( align_val $((begin+size)) ${partalign} )
+ i=$((i+1))
+ done
+ nb_parts=${i}
+ # Generate entries for empty partitions
+ for(( ; i<4; i++ )); do
+ debug " (empty)\n"
+ genpart -t 0 >>"${img}"
+ done
+ # Dump the boot signature
+ printf "\x55\xaa" >>"${img}"
+
+ for(( i=0; i<nb_parts; i++ )); do
+ part_img="${part_file[${i}]}"
+ offset=${part_offset[${i}]}
+ _offset=$(( offset/512 )) # offset is already 512-byte aligned
+ dd if="${part_img}" of="${img}" \
+ bs=512 seek=${_offset} \
+ conv=notrunc,sparse 2>/dev/null
+ done
+}
+
+# vim: ft=sh
diff --git a/fs/custom/boot/pre-post b/fs/custom/boot/pre-post
new file mode 100644
index 0000000..af4bcf5
--- /dev/null
+++ b/fs/custom/boot/pre-post
@@ -0,0 +1,14 @@
+#-----------------------------------------------------------------------------
+# No dependencies
+
+#-----------------------------------------------------------------------------
+do_image_pre() {
+ :
+}
+
+#-----------------------------------------------------------------------------
+do_image_post() {
+ :
+}
+
+#vim: set ft=sh
diff --git a/fs/custom/custom.mk b/fs/custom/custom.mk
new file mode 100644
index 0000000..ca53e0b
--- /dev/null
+++ b/fs/custom/custom.mk
@@ -0,0 +1,25 @@
+################################################################################
+#
+# custom partitioning
+#
+################################################################################
+
+# rootfs-custom uses rootfs.tar as the source to generate
+# the resulting image(s), so we need to build it first.
+ROOTFS_CUSTOM_DEPENDENCIES += rootfs-tar
+
+# If we are not selected, we won't have a partition table, so genimages
+# will complain (both on stdout and return code), so it will faill, so
+# we should not get our dependencies
+ifeq ($(BR2_TARGET_ROOTFS_CUSTOM),y)
+ROOTFS_CUSTOM_DEPENDENCIES += \
+ $(patsubst %,host-%,$(shell $(USER_HOOKS_EXTRA_ENV) fs/custom/genimages --show-depends \
+ '$(call qstrip,$(BR2_TARGET_ROOTFS_CUSTOM_PARTITION_TABLE))'))
+endif
+
+define ROOTFS_CUSTOM_CMD
+ $(USER_HOOKS_EXTRA_ENV) fs/custom/genimages \
+ '$(call qstrip,$(BR2_TARGET_ROOTFS_CUSTOM_PARTITION_TABLE))'
+endef
+
+$(eval $(call ROOTFS_TARGET,custom))
diff --git a/fs/custom/fs/ext b/fs/custom/fs/ext
new file mode 100644
index 0000000..a184969
--- /dev/null
+++ b/fs/custom/fs/ext
@@ -0,0 +1,26 @@
+# Create an extended file system
+
+#-----------------------------------------------------------------------------
+DEPENDS+=( e2fsprogs genext2fs )
+
+#-----------------------------------------------------------------------------
+do_image() {
+ local root_dir="${1}"
+ local img="${2}"
+ local -a fs_opts
+ local gen rev
+
+ fs_opts+=( -z )
+ fs_opts+=( -d "${root_dir}" )
+ [ -z "${size}" ] || fs_opts+=( -b $((size/1024)) )
+ [ -n "${ext_gen}" ] || ext_gen=2
+ [ -n "${ext_rev}" ] || ext_rev=1
+
+ # Remember, we're running from Buildroot's TOP_DIR
+ GEN=${ext_gen} REV=${ext_rev} \
+ ./fs/ext2/genext2fs.sh "${fs_opts[@]}" "${img}" >/dev/null
+
+ [ -z "${fs_label}" ] || tune2fs -L "${fs_label}" "${img}" >/dev/null
+}
+
+# vim: ft=sh
diff --git a/fs/custom/fs/pre-post b/fs/custom/fs/pre-post
new file mode 100644
index 0000000..d931437
--- /dev/null
+++ b/fs/custom/fs/pre-post
@@ -0,0 +1,72 @@
+#-----------------------------------------------------------------------------
+# No additional dependencies
+
+#-----------------------------------------------------------------------------
+do_image_pre() {
+ local i file
+
+ # if fs_root_dir is not specified, we have to create one
+ # It *does* override the caller's fs_root_dir value, but
+ # that's on purpose
+ # If fs_root_dir is specified, and we have at least fs_files_0,
+ # then fs_root_dir/ must be enpty
+ if [ -z "${fs_root_dir}" ]; then
+ if [ -n "${values["${part}:fs_files_0"]}" ]; then
+ error "%s: no fs_root specified, and no fs_files_0\n" "${part}"
+ fi
+ fs_root_dir="$( mktemp -d "${tmp_dir}/XXXXXX" )"
+ else
+ if [ -n "${values["${part}:fs_files_0"]}" \
+ -a $( ls -1A "${fs_root_dir}" 2>/dev/null \
+ |wc -l \
+ ) -ne 0 ]; then
+ error "%s: %s is not empty, but fs_files_0 is specified\n" \
+ "${part}" "${fs_root_dir#${fs_root}}"
+ fi
+ fi
+
+ i=0
+ while true; do
+ file="${values["${part}:fs_files_${i}"]}"
+ [ -n "${file}" ] || break
+ debug "%s: adding fs_files_%d %s\n" "${part}" ${i} "${file}"
+ install -D "${BINARIES_DIR}/${file}" "${fs_root_dir}/${file##*/}"
+ i=$((i+1))
+ done
+}
+
+#-----------------------------------------------------------------------------
+do_image_post() {
+ local rootfs_dir="${1}"
+ local fs_root="${2}"
+ local img_file="${3}"
+ local part="${4}"
+ local dev mntops vfstype fs_root_esc
+
+ subname+="[post-image]"
+
+ # Empty the partition's mountpoint
+ find "${fs_root_dir}" -maxdepth 1 \! -path "${fs_root_dir}" -exec rm -rf {} +
+
+ # Add entry in fstab, but not if this is '/'
+ # Don't add either if rootfs was not extracted
+ if [ "${fs_root}" = "/" -o -z "${fs_root}" \
+ -o -z "${values["global:extract"]}" ]; then
+ return 0
+ fi
+ fs_root_esc="$( sed -r -e 's:/:\\/:g;' <<<"${fs_root}" )"
+ sed -r -i -e "/[^[:space:]]+[[:space:]]+${fs_root_esc}[[:space:]]/d" \
+ "${rootfs_dir}/etc/fstab"
+ dev="$( get_part_dev_node "${part}" )"
+ vfstype="${fs_vfstype:-${fs_type}}"
+ mntops="${fs_mntops:-defaults}"
+ printf "/dev/%s %s %s %s 0 0\n" \
+ "${dev}" "${fs_root}" \
+ "${vfstype}" "${mntops}" \
+ >>"${rootfs_dir}/etc/fstab"
+
+ subname="${subname%\[post-image\]}"
+}
+
+#-----------------------------------------------------------------------------
+# vim: ft=sh
diff --git a/fs/custom/fs/vfat b/fs/custom/fs/vfat
new file mode 100644
index 0000000..5fc243d
--- /dev/null
+++ b/fs/custom/fs/vfat
@@ -0,0 +1,21 @@
+# Create a VFAT file system
+
+#-----------------------------------------------------------------------------
+DEPENDS+=( dosfstools mtools )
+
+#-----------------------------------------------------------------------------
+do_image() {
+ local root_dir="${1}"
+ local img="${2}"
+ local -a fs_opts
+
+ dd if=/dev/zero of="${img}" bs=${size} count=0 seek=1 2>/dev/null
+
+ [ -z "${vfat_size}" ] || fs_opts+=( -F ${vfat_size} )
+ [ -z "${fs_label}" ] || fs_opts+=( -n "${fs_label}" )
+ mkfs.vfat "${fs_opts[@]}" "${img}" >/dev/null
+
+ mcopy -i "${img}" "${root_dir}/"* '::'
+}
+
+# vim: ft=sh
diff --git a/fs/custom/genimages b/fs/custom/genimages
new file mode 100755
index 0000000..204f13b
--- /dev/null
+++ b/fs/custom/genimages
@@ -0,0 +1,342 @@
+#!/bin/bash
+
+#-----------------------------------------------------------------------------
+main() {
+ local part_table="${1}"
+ local tmp_dir
+ local rootfs_dir
+ local -a devices
+ local extract
+ local cur_section
+ local -a sections devices partitions
+ local -A variables values partdevs
+ local sec dev part var val
+ local secs devs parts vars vals
+ local has_global_section
+
+ # We need bash 4 or above for associative arrays
+ if [ ${BASH_VERSINFO[0]} -lt 4 ]; then
+ error "bash 4 or above is needed\n"
+ fi
+
+ if [ "${part_table}" = "--show-depends" ]; then
+ SHOW_DEPENDS=1
+ part_table="${2}"
+ trace() { :; } # Be silent, we just want the dependencies...
+ else
+ SHOW_DEPENDS=0
+ fi
+
+ if [ ! -f "${part_table}" ]; then
+ error "'%s': no such file\n" "${part_table}"
+ exit 1
+ fi
+
+ export PATH="${HOST_DIR}/usr/bin:${HOST_DIR}/usr/sbin:${PATH}"
+
+ # Parse all the sections in one go, we'll sort
+ # all the mess afterwards...
+ debug "parsing partitions descriptions file '%s'\n" \
+ "${part_table}"
+ parse_ini "${part_table}"
+
+ # The 'global' section is mandatory
+ has_global_section=0
+ for s in "${sections[@]}"; do
+ if [ "${s}" = "global" ]; then
+ has_global_section=1
+ break
+ fi
+ done
+ if [ ${has_global_section} -eq 0 ]; then
+ error "no global section defined\n"
+ fi
+
+ # Create lists of devices, partitions, and partition:device pairs.
+ debug "creating intermediate lists\n"
+ devices=( ${values["global:devices"]//,/ } )
+ for dev in "${devices[@]}"; do
+ # Sanity check first: all devices must have a corresponding
+ # section, which means they should have a type
+ if [ -z "${values["${dev}:type"]}" ]; then
+ error "device '%s' has no type (no section?)\n" "${dev}"
+ fi
+ partitions+=( ${values["${dev}:partitions"]//,/ } )
+ for part in ${values["${dev}:partitions"]//,/ }; do
+ # Sanity check first: all partitions must have a corresponding
+ # section, which means they should have a type
+ if [ -z "${values["${part}:type"]}" ]; then
+ error "partition '%s:%s' has no type (no section?)\n" \
+ "${dev}" "${part}"
+ fi
+ partdevs+=( ["${part}"]="${dev}" )
+ done
+ done
+
+ # Now, we must order the partitions so that their mountpoint
+ # is empty by the time we build the upper-level partition.
+ # For example, given this layout of mountpoints:
+ # /
+ # /usr
+ # /usr/var
+ # We must ensure /usr/var is empty at the time we create the /usr
+ # filesystem image; and similarly, we must ensure /usr is empty by
+ # the time we create the / filesystem image
+ # So, a simple reverse alphabetical sort will do the trick
+ debug "sorting partitions\n"
+ sorted_parts=( $(
+ for part in "${partitions[@]}"; do
+ # Partitions that are not mounted can be generated
+ # in any order
+ if [ -n "${values["${part}:fs_root"]}" ]; then
+ printf "%s:%s\n" "${part}" "${values["${part}:fs_root"]}"
+ else
+ printf "%s\n" "${part}"
+ fi
+ done \
+ |sort -t: -k2 -r \
+ |sed -r -e 's/:[^:]+$//;'
+ ) )
+
+ # We do not want to create anything if we only want the dependencies
+ if [ ${SHOW_DEPENDS} -eq 0 ]; then
+ tmp_dir="${BUILD_DIR}/genimages.tmp"
+ rootfs_dir="${tmp_dir}/rootfs"
+ # Since we don't remove it in case of error (to be able to inspect its
+ # content), we must remove it now (a previous run may have left it).
+ rm -rf "${tmp_dir}"
+ mkdir -p "${rootfs_dir}"
+
+ case "${values["global:extract"]}" in
+ tar)
+ # We must be root for the extract to work correctly
+ # (since it may have /dev nodes, or some files may
+ # belong to different users...)
+ if [ $(id -u) -ne 0 ]; then
+ error "error: not root\n"
+ fi
+ trace "extracting rootfs.tar\n"
+ tar xf "${BINARIES_DIR}/rootfs.tar" -C "${rootfs_dir}"
+ ;;
+ *) error "unknown extract method '%s'\n" "${extract:-(none)}"
+ ;;
+ esac
+ fi # ! SHOW_DEPENDS
+
+ # Render all partition images
+ for part in "${sorted_parts[@]}"; do
+ trace "preparing filesystem for partition '%s'\n" "${part}"
+ render_img "${rootfs_dir}" "${part}" \
+ "${tmp_dir}/${partdevs["${part}"]}.${part}.img"
+ done
+
+ # Aggregate all devices images
+ for dev in "${devices[@]}"; do
+ trace "assembling partitions in device '%s'\n" "${dev}"
+ render_img "${rootfs_dir}" "${dev}" "${tmp_dir}/${dev}.img"
+ done
+
+ # If we need the dependencies, we can stop right now
+ if [ ${SHOW_DEPENDS} -eq 1 ]; then
+ return 0
+ fi
+
+ # Copy all partitions and devices images to the image dir
+ if [ "${values["global:keep_partitions"]}" = "yes" ]; then
+ for part in "${sorted_parts[@]}"; do
+ debug "copying partition '%s' to image dir\n" "${part}"
+ dd if="${tmp_dir}/${partdevs["${part}"]}.${part}.img" \
+ of="${BINARIES_DIR}/$( get_part_dev_node "${part}" ).img" \
+ bs=4096 conv=sparse 2>/dev/null
+ done
+ fi
+ for dev in "${devices[@]}"; do
+ debug "copying device '%s' to image dir\n" "${dev}"
+ dd if="${tmp_dir}/${dev}.img" \
+ of="${BINARIES_DIR}/${dev}.img" \
+ bs=4096 conv=sparse 2>/dev/null
+ done
+
+ [ -n "${DEBUG}" ] || rm -rf "${tmp_dir}"
+}
+
+#-----------------------------------------------------------------------------
+render_img() {
+ local rootfs_dir="${1}"
+ local img="${2}"
+ local img_file="${3}"
+ local type sub_type fs_root_dir
+
+ type="${values["${img}:type"]}"
+ sub_type="${values["${img}:${type}_type"]}"
+
+ # Sanity checks
+ [ -n "${type}" ] || error "'%s': unspecified type\n" "${img}"
+ if [ ! -d "fs/custom/${type}" ]; then
+ error "'%s': unsupported type '%s'\n" "${img}" "${type}"
+ fi
+ [ -n "${sub_type}" ] || error "'%s': unspecified %s_type\n" "${img}" "${type}"
+ if [ ! -f "fs/custom/${type}/${sub_type}" ]; then
+ error "'%s': unknown %s_type '%s'\n" "${img}" "${type}" "${sub_type}"
+ fi
+
+ # Need to call the renderer in a subshell so that its definitions
+ # do not pollute our environment
+ subname="${sub_type}"
+ (
+ trap 'exit $?' ERR
+
+ declare -a DEPENDS
+
+ for var in ${variables["${img}"]//,/ }; do
+ eval "${var}=\"${values["${img}:${var}"]}\""
+ done
+ fs_root_dir="${rootfs_dir}${fs_root}"
+ . "fs/custom/${type}/pre-post"
+ . "fs/custom/${type}/${sub_type}"
+ if [ ${SHOW_DEPENDS} -eq 1 ]; then
+ for dep in "${DEPENDS[@]}"; do
+ printf "%s\n" "${dep}"
+ done
+ else
+ do_image_pre "${rootfs_dir}" "${fs_root}" "${img_file}" "${img}"
+ do_image "${fs_root_dir}" "${img_file}"
+ do_image_post "${rootfs_dir}" "${fs_root}" "${img_file}" "${img}"
+ fi
+ )
+ ret=${?}
+ [ ${ret} -eq 0 ] || exit ${ret}
+ subname=""
+}
+
+#------------------------------------------------------------------------------
+# Parse a .ini file
+# $1: .ini file to parse
+# The caller should define the following variables:
+# sections : array
+# variables : associative array
+# values : associative array
+# parse_ini() will fill those variables with:
+# sections : the list of sections, one section per array index
+# variables : the comma-separated list of varibles for a section,
+# indexed by the name of the section
+# values : the value of a variable in a section, indexed by the
+# 'section:variable' tuple
+# Eg.:
+# sections=( [0]='section-0' [1]='section-1' )
+# variables=( ['section-0']='var-0,var-1' ['section-1']='var-10,var-11' [...] )
+# values=( ['section-0:var-0']='value-0-0' ['section-1:var-10']='value-1-10' [...] )
+parse_ini() {
+ local ini_file="${1}"
+ local line var val
+ local cur_section
+ local var_sep
+
+ while read line; do
+ line="$( sed -r -e 's/[[:space:]]*#.*$//; //d;' <<<"${line}" )"
+
+ # Detect start of global section, skip anything else
+ case "${line}" in
+ "") continue;;
+ '['*']')
+ cur_section="$( sed -r -e 's/[][]//g;' <<<"${line}" )"
+ debug " entering section '%s'\n" "${cur_section}"
+ sections+=( "${cur_section}" )
+ continue
+ ;;
+ ?*=*) ;;
+ *) error "malformed entry '%s'\n" "${line}";;
+ esac
+
+ var="${line%%=*}"
+ eval val="${line#*=}"
+ debug " adding '%s'='%s'\n" "${var}" "${val}"
+ var_sep="${variables["${cur_section}"]+,}"
+ variables+=( ["${cur_section}"]="${var_sep}${var}" )
+ values+=( ["${cur_section}:${var}"]="${val}" )
+ done <"${ini_file}"
+}
+
+#-----------------------------------------------------------------------------
+get_part_dev_node() {
+ local part="${1}"
+ local dev
+ local i c p
+
+ dev="${partdevs["${part}"]}"
+ i="${values["${dev}:partstart"]:-1}"
+
+ # If device node ends with a number, partitions are denoted
+ # with a 'p' before the partition number, eg.:
+ # /dev/mmcblk0 --> /dev/mmcblk0p1
+ # /dev/sda --> /dev/sda1
+ case "${dev#${dev%?}}" in
+ [0-9]) c="p";;
+ *) c="";;
+ esac
+
+ for p in ${values["${dev}:partitions"]//,/ }; do
+ if [ "${p}" = "${part}" ]; then
+ printf "%s%s%d" "${dev}" "${c}" ${i}
+ return 0
+ fi
+ i=$((i+1))
+ done
+
+ error "'%s': partition not found. WTF?\n" "${part}"
+}
+
+#------------------------------------------------------------------------------
+align_val() {
+ local val="${1}"
+ local align="${2}"
+ local aligned
+
+ aligned=$(( ( (val+align-1) / align ) * align ))
+
+ printf "%d" ${aligned}
+}
+
+#------------------------------------------------------------------------------
+# Some trace functions
+_trace() {
+ local fmt="${1}"
+ shift
+
+ printf "%s" "${myname}"
+ if [ -n "${subname}" ]; then
+ printf "(%s)" "${subname}"
+ fi
+ printf ": ${fmt}" "${@}"
+}
+
+trace() {
+ _trace "${@}"
+}
+
+debug() { :; }
+if [ -n "${DEBUG}" ]; then
+ debug() {
+ _trace "${@}" >&2
+ }
+fi
+
+error() {
+ _trace "${@}" >&2
+ exit 1
+}
+
+on_error() {
+ local ret=${?}
+
+ error "unexpected error caught: %d\n" ${ret}
+}
+trap on_error ERR
+set -E -e
+
+#-----------------------------------------------------------------------------
+export myname="${0##*/}"
+
+main "${@}"
+
+# vim: ft=sh
--
1.8.1.2
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH 3/5] board/raspberrypi: provide partition description for the new genimages
2014-01-03 17:19 [Buildroot] [PATCH 0/5 v4] SUBJECT Yann E. MORIN
2014-01-03 17:19 ` [Buildroot] [PATCH 1/5] package/squashfs: add selection for the host variant Yann E. MORIN
2014-01-03 17:19 ` [Buildroot] [PATCH 2/5] fs/custom: generate complete, partition-based device images Yann E. MORIN
@ 2014-01-03 17:19 ` Yann E. MORIN
2014-01-04 16:40 ` Maxime Hadjinlian
2014-01-03 17:19 ` [Buildroot] [PATCH 4/5] fs/custom: add support for squashfs Yann E. MORIN
` (2 subsequent siblings)
5 siblings, 1 reply; 22+ messages in thread
From: Yann E. MORIN @ 2014-01-03 17:19 UTC (permalink / raw)
To: buildroot
From: "Yann E. MORIN" <yann.morin.1998@free.fr>
Now we can generate a complete target storage image with the genimages
infra, add a partition table layout description for the Raspberry Pi
as an example for how to use genimages.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ryan Barnett <rjbarnet@rockwellcollins.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
board/raspberrypi/partitions | 35 +++++++++++++++++++++++++++++++++++
configs/raspberrypi_defconfig | 3 +++
2 files changed, 38 insertions(+)
create mode 100644 board/raspberrypi/partitions
diff --git a/board/raspberrypi/partitions b/board/raspberrypi/partitions
new file mode 100644
index 0000000..dea9aff
--- /dev/null
+++ b/board/raspberrypi/partitions
@@ -0,0 +1,35 @@
+[global]
+extract=tar
+devices=mmcblk0
+
+[mmcblk0]
+type=boot
+boot_type=mbr
+partitions=boot,root
+partalign=$((1048576))
+
+[root]
+type=fs
+size=$((32*1048576))
+mbr_type=$((0x83))
+fs_type=ext
+fs_vfstype=ext4
+fs_mntops=discard
+fs_root=/
+fs_label=ROOT
+ext_gen=4
+ext_rev=1
+
+[boot]
+type=fs
+size=$((9*1048576))
+mbr_type=$((0xc))
+fs_type=vfat
+fs_files_0=rpi-firmware/bootcode.bin
+fs_files_1=rpi-firmware/start.elf
+fs_files_2=rpi-firmware/fixup.dat
+fs_files_3=rpi-firmware/config.txt
+fs_files_4=rpi-firmware/cmdline.txt
+fs_files_5=zImage
+fs_label=BOOT
+vfat_size=16
diff --git a/configs/raspberrypi_defconfig b/configs/raspberrypi_defconfig
index 6a17165..50caf5f 100644
--- a/configs/raspberrypi_defconfig
+++ b/configs/raspberrypi_defconfig
@@ -21,3 +21,6 @@ BR2_LINUX_KERNEL_CUSTOM_REPO_VERSION="75d03120bc36b1cc3166973b8edc11f033ab7c0d"
BR2_LINUX_KERNEL_USE_DEFCONFIG=y
BR2_LINUX_KERNEL_DEFCONFIG="bcmrpi_quick"
BR2_LINUX_KERNEL_ZIMAGE=y
+
+BR2_TARGET_ROOTFS_CUSTOM=y
+BR2_TARGET_ROOTFS_CUSTOM_PARTITION_TABLE="$(TOPDIR)/board/raspberrypi/partitions"
--
1.8.1.2
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH 4/5] fs/custom: add support for squashfs
2014-01-03 17:19 [Buildroot] [PATCH 0/5 v4] SUBJECT Yann E. MORIN
` (2 preceding siblings ...)
2014-01-03 17:19 ` [Buildroot] [PATCH 3/5] board/raspberrypi: provide partition description for the new genimages Yann E. MORIN
@ 2014-01-03 17:19 ` Yann E. MORIN
2014-01-04 16:43 ` Maxime Hadjinlian
2014-01-03 17:19 ` [Buildroot] [PATCH 5/5] fs/custom: add support for raw device/partition content Yann E. MORIN
2014-01-03 17:22 ` [Buildroot] [PATCH 0/5 v4] SUBJECT Yann E. MORIN
5 siblings, 1 reply; 22+ messages in thread
From: Yann E. MORIN @ 2014-01-03 17:19 UTC (permalink / raw)
To: buildroot
From: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Ryan Barnett <rjbarnet@rockwellcollins.com>
---
docs/manual/partition-layout.txt | 11 ++++++++++-
fs/custom/fs/squashfs | 19 +++++++++++++++++++
2 files changed, 29 insertions(+), 1 deletion(-)
create mode 100644 fs/custom/fs/squashfs
diff --git a/docs/manual/partition-layout.txt b/docs/manual/partition-layout.txt
index 092ae1b..e9027bf 100644
--- a/docs/manual/partition-layout.txt
+++ b/docs/manual/partition-layout.txt
@@ -133,7 +133,8 @@ Properties for +type=fs+
* +fs_type+ (mandatory): the type of filesystem to generate
** +ext+: generate an extended filesystem (ext2, ext3, ext4)
-** +vfat+: generate a VFAT filesystem (FAT16, FAT32)
+** +squashfs+: generate a squashfs (version 4) filesystem
+** +vfat+: generate a VFAT filesystem (FAT12, FAT16 or FAT32)
* +fs_label+ (optional): the label to assign to this filesystem, if that
filesystem supports a label
@@ -177,6 +178,14 @@ Properties for +fs_type=ext+
** +1+ (mandatory for ext3 or ext4): generate a revision 1 extended
filesystem
+Properties for +fs_type=squashfs+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* +squashfs_comp+ (mandatory): the compression type to use
+** +gzip+, +xz+, +lzo+: gzip, xz or lzo compression
+
+* +squashfs_block+ (optional): the size of blocks, in bytes
+
Properties for +fs_type=vfat+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
diff --git a/fs/custom/fs/squashfs b/fs/custom/fs/squashfs
new file mode 100644
index 0000000..132d3ef
--- /dev/null
+++ b/fs/custom/fs/squashfs
@@ -0,0 +1,19 @@
+# Create a squashfs filesystem
+
+#-----------------------------------------------------------------------------
+DEPENDS+=( squashfs )
+
+#-----------------------------------------------------------------------------
+do_image() {
+ local root_dir="${1}"
+ local img="${2}"
+ local -a fs_opts
+
+ fs_opts+=( -no-progress -noappend )
+ [ -z "${squashfs_comp}" ] || fs_opts+=( -comp "${squashfs_comp}" )
+ [ -z "${squashfs_block}" ] || fs_opts+=( -b ${squahsfs_block} )
+
+ mksquashfs "${root_dir}" "${img}" "${fs_opts[@]}" >/dev/null
+}
+
+# vim: ft=sh
--
1.8.1.2
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH 5/5] fs/custom: add support for raw device/partition content
2014-01-03 17:19 [Buildroot] [PATCH 0/5 v4] SUBJECT Yann E. MORIN
` (3 preceding siblings ...)
2014-01-03 17:19 ` [Buildroot] [PATCH 4/5] fs/custom: add support for squashfs Yann E. MORIN
@ 2014-01-03 17:19 ` Yann E. MORIN
2014-01-03 17:22 ` [Buildroot] [PATCH 0/5 v4] SUBJECT Yann E. MORIN
5 siblings, 0 replies; 22+ messages in thread
From: Yann E. MORIN @ 2014-01-03 17:19 UTC (permalink / raw)
To: buildroot
From: "Yann E. MORIN" <yann.morin.1998@free.fr>
Add the ability to fill a device or partition from a raw binary blob.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
---
docs/manual/partition-layout.txt | 8 ++++++++
fs/custom/genimages | 5 +++++
fs/custom/raw/pre-post | 14 ++++++++++++++
fs/custom/raw/raw | 12 ++++++++++++
4 files changed, 39 insertions(+)
create mode 100644 fs/custom/raw/pre-post
create mode 100644 fs/custom/raw/raw
diff --git a/docs/manual/partition-layout.txt b/docs/manual/partition-layout.txt
index e9027bf..c237893 100644
--- a/docs/manual/partition-layout.txt
+++ b/docs/manual/partition-layout.txt
@@ -65,6 +65,7 @@ Properties for the device section
* +type+ (mandatory): the type of content for that device or partition
** +boot+: the device contains one or more partitions, and
_may_ serve as a boot device
+** +raw+: the device contains a raw binary blob
* +keep_partitions+ (optional): also copy the individual partition images
for this device to +$(BINARIES_DIR)+. Settings from the partition
@@ -77,6 +78,7 @@ Properties for the partition section
* +type+ (mandatory): the type of content for that device or partition
** +fs+: the partition contains a filesystem
+** +raw+: the partition contains a raw binary blob
* +size+: the size of that partition, in bytes
@@ -104,6 +106,12 @@ Properties for +type=boot+
+boot_type+, some restrictions may apply, and are documented for each
+boot_type+
+Properties for +type=raw+
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* +raw_file+: the filename, relative to +$(BINARIES_DIR)+, that contains
+ the binary blob
+
Properties for +boot_type=mbr+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
diff --git a/fs/custom/genimages b/fs/custom/genimages
index 204f13b..e7d9065 100755
--- a/fs/custom/genimages
+++ b/fs/custom/genimages
@@ -175,6 +175,11 @@ render_img() {
if [ ! -d "fs/custom/${type}" ]; then
error "'%s': unsupported type '%s'\n" "${img}" "${type}"
fi
+
+ # Special case for 'raw': there is no sub-type, we just fake the
+ # sub-type to be 'raw' as well.
+ [ "${type}" != "raw" ] || sub_type="raw"
+
[ -n "${sub_type}" ] || error "'%s': unspecified %s_type\n" "${img}" "${type}"
if [ ! -f "fs/custom/${type}/${sub_type}" ]; then
error "'%s': unknown %s_type '%s'\n" "${img}" "${type}" "${sub_type}"
diff --git a/fs/custom/raw/pre-post b/fs/custom/raw/pre-post
new file mode 100644
index 0000000..af4bcf5
--- /dev/null
+++ b/fs/custom/raw/pre-post
@@ -0,0 +1,14 @@
+#-----------------------------------------------------------------------------
+# No dependencies
+
+#-----------------------------------------------------------------------------
+do_image_pre() {
+ :
+}
+
+#-----------------------------------------------------------------------------
+do_image_post() {
+ :
+}
+
+#vim: set ft=sh
diff --git a/fs/custom/raw/raw b/fs/custom/raw/raw
new file mode 100644
index 0000000..3cbee48
--- /dev/null
+++ b/fs/custom/raw/raw
@@ -0,0 +1,12 @@
+# Generate a partition from a plain file
+
+#-----------------------------------------------------------------------------
+# No extra dependencies
+
+#-----------------------------------------------------------------------------
+do_image() {
+ # ${1} is the root_dir, which is irrelevant here
+ local img="${2}"
+
+ dd if="${BINARIES_DIR}/${raw_file}" of="${img}" bs=4096 conv=sparse 2>/dev/null
+}
--
1.8.1.2
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH 0/5 v4] SUBJECT
2014-01-03 17:19 [Buildroot] [PATCH 0/5 v4] SUBJECT Yann E. MORIN
` (4 preceding siblings ...)
2014-01-03 17:19 ` [Buildroot] [PATCH 5/5] fs/custom: add support for raw device/partition content Yann E. MORIN
@ 2014-01-03 17:22 ` Yann E. MORIN
5 siblings, 0 replies; 22+ messages in thread
From: Yann E. MORIN @ 2014-01-03 17:22 UTC (permalink / raw)
To: buildroot
All,
s/SUBJECT/Introduce the 'genimages' infrastructure/
Doh... :-(
Regards,
Yann E. MORIN.
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH 2/5] fs/custom: generate complete, partition-based device images
2014-01-03 17:19 ` [Buildroot] [PATCH 2/5] fs/custom: generate complete, partition-based device images Yann E. MORIN
@ 2014-01-04 16:38 ` Maxime Hadjinlian
2014-01-04 17:43 ` Yann E. MORIN
2014-04-09 13:28 ` Andreas Naumann
1 sibling, 1 reply; 22+ messages in thread
From: Maxime Hadjinlian @ 2014-01-04 16:38 UTC (permalink / raw)
To: buildroot
Hi Yann,
When doing scripting in Buildroot, do we accept the uses of various
bash-ism or do we want to stick to POSIX shell ?
On Fri, Jan 3, 2014 at 6:19 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> From: "Yann E. MORIN" <yann.morin.1998@free.fr>
>
> Contrary to the existing fs/ schemes, which each generate only a single
> filesystem image for the root filesystem, this new scheme allows the
> user to generate more complex images.
>
> The basis behind this is a .ini-like description of the layout of the
> final target storage:
> - the list of device(s)
> - per-device, the list of partition(s)
> - per-partition, the content
>
> It is possible to create MBR- or GPT-based partitoining schemes. Adding
> new ones should be relatively easy (but would need adequate host tools).
>
> For now, the only content possible for partitions is a filesystem. It
> should be pretty easy to add new types (eg. un-formated, or raw blob).
>
> Also, only two filesystems are supported: ext{2,3,4} and vfat. Adding
> more will be relatively easy, provided we have the necessary host
> packages to deal with those filesystems.
>
> The existing Buildroot filesystem generators are re-used as much as
> possible when it makes sense; when it does not (eg. for vfat), a specific
> generator is used.
>
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> Cc: Ryan Barnett <rjbarnet@rockwellcollins.com>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: "J?r?me Oufella" <jerome.oufella@savoirfairelinux.com>
> ---
> docs/manual/appendix.txt | 1 +
> docs/manual/customize-filesystems.txt | 36 ++++
> docs/manual/customize.txt | 2 +
> docs/manual/partition-layout.txt | 298 +++++++++++++++++++++++++++++
> fs/Config.in | 1 +
> fs/custom/Config.in | 16 ++
> fs/custom/boot/gpt | 129 +++++++++++++
> fs/custom/boot/mbr | 64 +++++++
> fs/custom/boot/pre-post | 14 ++
> fs/custom/custom.mk | 25 +++
> fs/custom/fs/ext | 26 +++
> fs/custom/fs/pre-post | 72 +++++++
> fs/custom/fs/vfat | 21 +++
> fs/custom/genimages | 342 ++++++++++++++++++++++++++++++++++
> 14 files changed, 1047 insertions(+)
> create mode 100644 docs/manual/customize-filesystems.txt
> create mode 100644 docs/manual/partition-layout.txt
> create mode 100644 fs/custom/Config.in
> create mode 100644 fs/custom/boot/gpt
> create mode 100644 fs/custom/boot/mbr
> create mode 100644 fs/custom/boot/pre-post
> create mode 100644 fs/custom/custom.mk
> create mode 100644 fs/custom/fs/ext
> create mode 100644 fs/custom/fs/pre-post
> create mode 100644 fs/custom/fs/vfat
> create mode 100755 fs/custom/genimages
>
> diff --git a/docs/manual/appendix.txt b/docs/manual/appendix.txt
> index 74ee8fd..53f4205 100644
> --- a/docs/manual/appendix.txt
> +++ b/docs/manual/appendix.txt
> @@ -6,6 +6,7 @@ Appendix
>
> include::makedev-syntax.txt[]
> include::makeusers-syntax.txt[]
> +include::partition-layout.txt[]
>
>
> // Automatically generated lists:
> diff --git a/docs/manual/customize-filesystems.txt b/docs/manual/customize-filesystems.txt
> new file mode 100644
> index 0000000..fd65c97
> --- /dev/null
> +++ b/docs/manual/customize-filesystems.txt
> @@ -0,0 +1,36 @@
> +// -*- mode:doc; -*-
> +// vim: set syntax=asciidoc:
> +
> +[[filesystem-custom]]
> +Customizing the generated filesystem images
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> ++Buildroot+ knows by default how to generate a few different kind of
> +filesystems, such as +squashfs+, +ext2/3/4+, +cramfs+... But those
> +filesystems are all generated to contain the complete target directory
> +hierarchy in a single filesystem, mounted as the root filesystem +/+.
> +That is, even if you select both an +ext2+ and a +squashfs+ filesystems,
> +the content of the two generated images will be the exact same, only the
> +types of the filesystems will be different.
> +
> +Most devices require a more complex setup, with different parts of the
> +directory structure split across different filesystems, each stored on
> +different partitions of one or more storage devices.
> +
> ++Buildroot+ can generate such complex setups, using a +partition table layout
> +description+. This is a simple text file, not unlike the +.ini+ style of
> +configuration files, that describes how the target directory hierarchy has
> +to be split across the target storage devices. It is a bit like a flattened
> +tree of the storage layout.
> +
> +Set the variable +BR2_TARGET_ROOTFS_CUSTOM_PARTITION_TABLE+ to the path of
> +the file containing your +partition table layout description+.
> +
> +See xref:part-layout-desc-syntax[] for the complete documentation of the
> ++partition table layout description+ syntax.
> +
> +[underline]*Note:* Although more versatile than the single filesystem image
> +mechanism, the +partition table layout description+ might be unable to
> +describe very complex setups. For example, it is not capable of handling
> +NFS-mounted filesystems, and +initramfs+ setups is not trivial (although
> +possible in most cases).
> diff --git a/docs/manual/customize.txt b/docs/manual/customize.txt
> index 7e46fd8..6d062ea 100644
> --- a/docs/manual/customize.txt
> +++ b/docs/manual/customize.txt
> @@ -14,6 +14,8 @@ include::customize-kernel-config.txt[]
>
> include::customize-toolchain.txt[]
>
> +include::customize-filesystems.txt[]
> +
> include::customize-store.txt[]
>
> include::customize-packages.txt[]
> diff --git a/docs/manual/partition-layout.txt b/docs/manual/partition-layout.txt
> new file mode 100644
> index 0000000..092ae1b
> --- /dev/null
> +++ b/docs/manual/partition-layout.txt
> @@ -0,0 +1,298 @@
> +// -*- mode:doc; -*-
> +// vim: set syntax=asciidoc:
> +
> +[[part-layout-desc-syntax]]
> +
> +Partition table layout description syntax
> +-----------------------------------------
> +
> +The +partition table layout description+ syntax is not unlike the standard
> +https://en.wikipedia.org/wiki/.ini[+.ini+] syntax. There are two types of
> +entries: +sections+, that may each contain zero or more +properties+.
> +
> ++Sections+ are specified between square brackets +[]+, _eg._: +[name]+.
> +
> ++Properties+ are specified as key-value pairs, _eg._: +key=value+, and
> +are documented as:
> +
> +* +key-name+ (optional or mandatory): description
> +** +value1+: description
> +** +value2+: description
> +** ...
> +
> +[underline]*Note:* Unlike the standard +.ini+ syntax, the +partition table
> +layout description+ _is_ case-sensitive.
> +
> +The order of +sections+ is irrelevant. However, for readability, we recomend
> +the +partition table layout description+ starts with the +global+ section.
> +
> +The global section
> +~~~~~~~~~~~~~~~~~~
> +
> +The +[global]+ section defines some global settings, and the list of devices.
> +
> +Properties for the global section
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +* +extract+ (mandatory): the type of base image to extract
> +** +tar+: extract the +rootfs.tar+ base image generated by +Buildroot+
> +
> +* +devices+ (mandatory): the comma-separated list of storage devices to
> + use on the device. Each device is the filename of the device node
> + present in +/dev+
> +
> +* +keep_partitions+ (optional): also copy the individual partition images
> + of all devices to +$(BINARIES_DIR)+. Settings from the device sections
> + or the partition sections take precedence over this one.
> +** +yes+: copy the individual partition images
> +** +no+ (the default): do not copy individual partition images
> +
> +The devices and partitions sections
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +The devices and partitions sections define, for each device or partition,
> +the content of that device or partition.
> +
> +For each device listed in the +[global]+ section, there must be a
> +corresponding section named after that device.
> +
> +For each partition listed in a device section, there must be a corresponding
> +section named after that partition.
> +
> +Properties for the device section
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +* +type+ (mandatory): the type of content for that device or partition
> +** +boot+: the device contains one or more partitions, and
> + _may_ serve as a boot device
> +
> +* +keep_partitions+ (optional): also copy the individual partition images
> + for this device to +$(BINARIES_DIR)+. Settings from the partition
> + sections take precedence over this one.
> +** +yes+: copy the individual partition images
> +** +no+ (the default): do not copy individual partition images
> +
> +Properties for the partition section
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +* +type+ (mandatory): the type of content for that device or partition
> +** +fs+: the partition contains a filesystem
> +
> +* +size+: the size of that partition, in bytes
> +
> +* +keep+ (optional): copy this partition image to +$(BINARIES_DIR)+
> +** +yes+: copy this partition image
> +** +no+ (the default): do not copy this partition image
> +
> +Properties for +type=boot+
> +^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +* +boot_type+ (mandatory): the partitioning scheme to use on this device
> +** +mbr+: use an https://en.wikipedia.org/wiki/Master_boot_record[MBR]
> + partitioning scheme
> +** +gpt+: use a https://en.wikipedia.org/wiki/GUID_Partition_Table[GPT]
> + partitioning scheme
> +
> +* +partitions+ (mandatory): the comma-separated list of partition(s) on
> + this device; no two partitions may have the same name, even if they
> + reside on different devices; partitions names shall match this regexp:
> + `^[[:alpha:]][[:alnum:]-_]*$` (_ie._ starts with a letter, followed by
> + zero or more alpha-numeric character or a dash or an underscore)
> +
> +* +partalign+ (optional): the alignment of partitions, in bytes; defaults
> + to an alignment of one, which means no alignment; depending on the
> + +boot_type+, some restrictions may apply, and are documented for each
> + +boot_type+
> +
> +Properties for +boot_type=mbr+
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +* +mbr_bootcode+ (optional): the bootcode to use, as a path to the file
> + containing the bootcode image, relative to the +$(BINARIES_DIR)+
> + directory; defaults to no bootcode (eg. filled with zeroes)
> +
> +* +partalign+: must be a multiple of 512
> +
> +Properties for +boot_type=gpt+
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +* +partalign+: must be a multiple of 512
> +
> +********
> +Currently, only 512-byte sectors are supported. 4k sectors are not.
> +********
> +
> +Properties for partitions whose containing device is +boot_type=mbr+
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +* +mbr_type+ (mandatory): the partition
> + https://en.wikipedia.org/wiki/Partition_type#List_of_partition_IDs[type]
> +
> +Properties for +type=fs+
> +^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +* +fs_type+ (mandatory): the type of filesystem to generate
> +** +ext+: generate an extended filesystem (ext2, ext3, ext4)
> +** +vfat+: generate a VFAT filesystem (FAT16, FAT32)
> +
> +* +fs_label+ (optional): the label to assign to this filesystem, if that
> + filesystem supports a label
> +
> +* +fs_files_0+, +fs_files_1+, +fs_files_N+ (optional): the list of files,
> + relative to $(BINARIES_DIR), to store in the filesystem. These entries
> + must be indexed starting from 0, and must be sequential: the first
> + missing entry ends the list
> +
> +* +fs_root+ (optional): the mountpoint of the filesystem
> +
> +* +fs_vfstype+ (optional): the type of filesystem to use when calling
> + +mount+, if different from +fs_type+
> +
> +* +fs_mntopts+ (optional): the mount options; defaults to +defaults+
> +
> +_Note_: valid use-cases for +fs_root+ and +fs_files_N+:
> +
> +* if only +fs_root+ is specified (and no +fs_files_N+): the filesystem
> + content is made exclusively from +$(TARGET_DIR)/$(fs_root)+, and the
> + filesystem is mounted at runtime
> +
> +* if both +fs_root+ and at least one +fs_files_N+ are specified: the
> + filesystem content is made exclusively from the +fs_files_N+ entries,
> + and mounted at runtime. +$(TARGET_DIR)/$(fs_root)+ must be empty
> +
> +* if at least one +fs_files_N+ is specified, and +fs_root+ is not: the
> + filesystem content is made exclusively from the +fs_files_N+ entries,
> + and the filesystem is not mounted at runtime
> +
> +* if neither +fs_root+ nor +fs_files_N+ is specified: this is an error
> +
> +Properties for +fs_type=ext+
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +* +ext_gen+ (mandatory): the generation of extended filesystem to generate
> +** +2+, +3+, +4+: for an ext2, ext3 or ext4 filesystem
> +
> +* +ext_rev+ (mandatory): the revision of the extended filesystem
> +** +0+ (ext2 only): generate a revision 0 extended filesystem filesystem
> +** +1+ (mandatory for ext3 or ext4): generate a revision 1 extended
> + filesystem
> +
> +Properties for +fs_type=vfat+
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +* +vfat_size+ (optional): the VFAT-size of the filesystem
> +** +12+, +16+, +32+: generate a FAT12, FAT16, or FAT32
> +
> +Generation of the filesystems
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +The filesystems are generated in an order such that a filesystem that is
> +mounted as a sub-directory of another filesystem is generated first.
> +
> +A filesystem is filled with the content of the directory corresponding to
> +its mountpoint, and then that directory is emptied before continuing to the
> +next filesystem. That way, the mountpoints are empty before the filesystem
> +that contains them are generated.
> +
> +Finally, an entry in added in +/etc/fstab+ for each generated filesystem, so
> +they are mounted at boot time.
> +
> +Examples
> +~~~~~~~~
> +
> +.Simplest partition table layout description
> +====
> +----
> +[global]
> +extract=tar
> +devices=sda
> +
> +[sda]
> +type=boot
> +boot_type=mbr
> +partitions=root
> +partalign=1048576
> +
> +[root]
> +type=fs
> +fs_type=ext
> +fs_vfstype=ext4
> +fs_root=/
> +ext_gen=4
> +ext_rev=1
> +----
> +
> +The +partition table layout description+ above defines a single device
> ++sda+. That device contains a single partition, +root+, with an ext4
> +filesystem, which is filled with the whole content of the +rootfs.tar+,
> +and is mounted on +/+.
> +====
> +
> +.More complex table layout description
> +====
> +----
> +[global]
> +extract=tar
> +devices=mmcblk0,sda
> +
> +[mmcblk0]
> +type=boot
> +boot_type=mbr
> +partitions=boot,root
> +partalign=$((1024*1024))
> +
> +[sda]
> +type=boot
> +boot_type=mbr
> +partitions=data
> +partalign=4096
> +
> +[boot]
> +type=fs
> +mbr_type=$((0xC))
> +size=$((16*1048576))
> +fs_type=vfat
> +fs_mntopts=ro
> +fs_label=BOOT
> +fs_root=/boot
> +vfat_size=32
> +
> +[root]
> +type=fs
> +mbr_type=$((0x83))
> +size=268435456
> +fs_type=ext
> +fs_vfstype=ext4
> +fs_mntopts=discard,delalloc
> +fs_root=/
> +fs_label=ROOT
> +ext_gen=4
> +ext_rev=1
> +
> +[data]
> +type=fs
> +mbr_type=$((0x83))
> +size=$((4*1024*1048576))
> +fs_type=ext
> +fs_vfstype=ext2
> +fs_root=/data
> +fs_label=DATA
> +ext_gen=2
> +ext_rev=1
> +----
> +====
> +
> +The example above defines two devices, +mmcblk0+ and +sda+.
> +
> +The +mmcblk0+ device contains two partitions, +boot+ and +root+; partitions
> +are aligned on a 1MiB boundary. The +sda+ device contains a single partition,
> ++data+, aligned on a 4KiB boundary.
> +
> +The +boot+ partition is a 16MiB FAT32 filesystem filled with the content
> +of, and mounted on, +/boot+, and with label +BOOT+.
> +
> +The +data+ partition is a 4GiB ext2r1 filesystem filled with the content
> +of, and mounted on, +/data+, and with label +DATA+.
> +
> +The +root+ partition is a 256MiB ext4 filesystem filled the the rest of,
> +and mounted on, +/+, and with label +ROOT+.
How about comments ? Are they supported ?
In a next step, it would be really nice to have support for "human"
input for the different size, like 1G/M/K.
> diff --git a/fs/Config.in b/fs/Config.in
> index da4c5ff..44e04f7 100644
> --- a/fs/Config.in
> +++ b/fs/Config.in
> @@ -3,6 +3,7 @@ menu "Filesystem images"
> source "fs/cloop/Config.in"
> source "fs/cpio/Config.in"
> source "fs/cramfs/Config.in"
> +source "fs/custom/Config.in"
> source "fs/ext2/Config.in"
> source "fs/initramfs/Config.in"
> source "fs/iso9660/Config.in"
> diff --git a/fs/custom/Config.in b/fs/custom/Config.in
> new file mode 100644
> index 0000000..e5a8ee7
> --- /dev/null
> +++ b/fs/custom/Config.in
> @@ -0,0 +1,16 @@
> +config BR2_TARGET_ROOTFS_CUSTOM
> + bool "Custom partition table layout"
> + select BR2_TARGET_ROOTFS_TAR
> +
> +config BR2_TARGET_ROOTFS_CUSTOM_PARTITION_TABLE
> + string "path to the custom partition table layout description"
> + depends on BR2_TARGET_ROOTFS_CUSTOM
> + help
> + Enter the path to a partition-table for your device.
> +
> + This will allow Buildroot to generate a more complex target
> + image, which may consist of more than one filesystem on more
> + than one partition.
> +
> + See docs/manual/bla-bla on how to construct such a partition
> + table.
> diff --git a/fs/custom/boot/gpt b/fs/custom/boot/gpt
> new file mode 100644
> index 0000000..f978524
> --- /dev/null
> +++ b/fs/custom/boot/gpt
> @@ -0,0 +1,129 @@
> +# Build a complete GPT-based image
> +
> +#-----------------------------------------------------------------------------
> +DEPENDS+=( parted )
> +
> +#-----------------------------------------------------------------------------
> +# For a GPT-based partitionning, we need to compute the complete
> +# image size before we can attempt to generate the partition table.
> +# Then, we need to add the size of the GPT itself, plus that of its
> +# backup copy, plus the protective MBR.
> +# The size of the GPT itself depends on the sector size, and the
> +# number of partitions in the GPT. Sectors can be either 512-byte
> +# or 4096-byte large; The numbers of partitions is unlimited, but
> +# it is suggested there is space to store at least 128 of them; a
> +# partition description is 128-byte large.
> +#
> +# https://en.wikipedia.org/wiki/GUID_Partition_Table
> +#
> +# So, here's what we do:
> +# - consider 512-byte sectors (since GPT on 4k sectors is not well
> +# documented)
> +# - consider at least 128 partitions; if the layout defines more than
> +# that, we need to round that number up to the smallest multiple of
> +# 4 (since there are 4 partition descriptions in a 512-byte sector)
> +# - generate an empty, sparse file that is big enough to store the MBR,
> +# the two GPT copies, and the aligned partitions.
> +# - dump each partition in turn in their final location in that file
> +# - generate a parted script that creates the partition table in that
> +# file
> +#
> +# Simg = 512 + 2*(Sgpt) + ?( aligned(Spart,512) )
Don't we want only ASCII char there ?
> +# Sgpt = 512 + Nent*128
> +#
> +# Where:
> +# Simg : size of the image
> +# Sgpt : size of one GPT
> +# Spart : size of each partition
> +# Nent : number of partition entries
> +# aligned() : the alignment function
> +#
> +# Sicne 4k-large sectors are not really explained on Wikipedia, we can
Small typo here.
> +# add this later on.
> +do_image() {
> + # ${1} is fs_root, irrelevant here
> + local img="${2}"
> +
> + # How many partitions do we have?
> + nb_parts=0
> + for part in ${partitions[${dev}]//,/ }; do
> + nb_parts=$((nb_parts+1))
> + done
> +
> + # How many partition entries do we need?
> + nb_entries=$((4*((nb_parts+3)/4)))
> + nb_entries=$((nb_entries<128?128:nb_entries))
> +
> + # The size of a single GPT
> + gpt_size=$((512+(128*nb_entries)))
> +
> + # Offset of the first partition
> + begin=$(align_val $((512+$(align_val ${gpt_size} 512))) ${partalign} )
> +
> + # Initialise our image file
> + dd if=/dev/zero of="${img}" \
> + bs=1 seek=${begin} count=0 \
> + conv=sparse 2>/dev/null
Why the redirect to /dev/null ? Don't we want to see if there was an
error ? I know it's unlikely to happen but you never know.
I have noticed you did several times so maybe there is a reason I am
not seeing here.
> +
> + # Compute the space required to store all partitions
> + # and store them in the image file
> + size_parts=0
> + _offset=${begin}
> + i=1
> + debug "adding partions descriptions\n"
> + for part in ${partitions[${dev}]//,/ }; do
> + debug " %s\n" "${part}"
> + _part_img="${tmp_dir}/${dev}.${part}.img"
> + _size=$( align_val $( stat -c '%s' "${_part_img}" ) 512 )
> + part_offset+=( ${_offset} )
> + _attr="${values["${part}:gpt_attr"]}"
> + _label="${values["${part}:gpt_label"]}"
> +
> + # If the partition has no label, use the filesystem label
> + if [ -z "${_label}" ]; then
> + _label="${values["${part}:fs_label"]}"
> + fi
> + if [ -z "${_label}" ]; then
> + _label="data"
> + fi
> +
> + debug " start=%s\n" "${_offset}"
> + debug " size =%s\n" "${_size}"
> + debug " end =%s\n" "$((_offset+_size-1))"
> +
> + dd if="${_part_img}" of="${img}" \
> + bs=512 seek=$((_offset/512)) \
> + conv=notrunc,sparse 2>/dev/null
> +
> + parted_script+=( mkpart "${_label}" \
> + ${_offset} \
> + $((_offset+_size-1)) \
> + )
> + if [ -n "${_attr}" ]; then
> + for attr in "${_attr//,/ }"; do
> + parted_script+=( set ${i} ${attr} on )
> + done
> + fi
> +
> + size_parts=$((size_parts+_size))
> + _offset=$((_offset+_size))
> + i=$((i+1))
> + done
> +
> + # Terminate our image file
> + img_size=$(align_val $(( begin + size_parts + gpt_size )) 512)
> + debug "begin =%s\n" ${begin}
> + debug "nb_entry=%s\n" ${nb_entries}
> + debug "gpt_size=%s\n" ${gpt_size}
> + debug "img_size=%s\n" ${img_size}
> + dd if=/dev/zero of="${img}" \
> + bs=1 seek=${img_size} \
> + count=0 conv=sparse 2>/dev/null
> +
> + for i in parted -s "${img}" mklabel gpt unit B "${parted_script[@]}"; do
> + debug "--> '%s'\n" "${i}"
> + done
> + parted -s "${img}" mklabel gpt unit B "${parted_script[@]}"
> +}
> +
> +# vim: ft=sh
> diff --git a/fs/custom/boot/mbr b/fs/custom/boot/mbr
> new file mode 100644
> index 0000000..667feed
> --- /dev/null
> +++ b/fs/custom/boot/mbr
> @@ -0,0 +1,64 @@
> +# Build a complete MBR-based image
> +
> +#-----------------------------------------------------------------------------
> +DEPENDS+=( genpart )
> +
> +#-----------------------------------------------------------------------------
> +do_image() {
> + # ${1} is fs_root, irrelevant here
> + local img="${2}"
> + local i begin part part_img size type _begin _size
> + local -a part_offset part_file bootcode
> +
> + # Fill-in the boot record
> + bootcode="${values["${dev}:mbr_bootcode"]}"
> + if [ -n "${bootcode}" ]; then
> + bootcode="${BINARIES_DIR}/${bootcode}"
> + else
> + bootcode="/dev/zero"
> + fi
> + debug "adding bootcode '%s'\n" "${bootcode}"
> + dd if="${bootcode}" of="${img}" bs=$((0x1be)) count=1 2>/dev/null
> +
> + # Generate partition entries
> + i=0
> + begin=${partalign}
> + debug "adding partitions descriptors\n"
> + for part in ${partitions[${dev}]//,/ }; do
> + debug " %s\n" "${part}"
> + part_offset+=( ${begin} )
> + part_img="${tmp_dir}/${dev}.${part}.img"
> + part_file+=( "${part_img}" )
> + size=$( align_val $( stat -c '%s' "${part_img}" ) 512 )
> + type="${values["${part}:mbr_type"]}"
> + # LBA is exressed in a number of 512-byte blocks
> + # and genparts only deals with LBA
> + _begin=$((begin/512)) # begin is already 512-byte aligned
> + _size=$((size/512)) # size is already 512-byte aligned
> + debug " start=%s (LBA %s)\n" "${begin}" "${_begin}"
> + debug " size =%s (LBA %s)\n" "${size}" "${_size}"
> + debug " type =%s\n" "${type}"
> + genpart -b ${_begin} -s ${_size} -t ${type} >>"${img}"
> + begin=$( align_val $((begin+size)) ${partalign} )
> + i=$((i+1))
> + done
> + nb_parts=${i}
> + # Generate entries for empty partitions
> + for(( ; i<4; i++ )); do
> + debug " (empty)\n"
> + genpart -t 0 >>"${img}"
> + done
> + # Dump the boot signature
> + printf "\x55\xaa" >>"${img}"
> +
> + for(( i=0; i<nb_parts; i++ )); do
> + part_img="${part_file[${i}]}"
> + offset=${part_offset[${i}]}
> + _offset=$(( offset/512 )) # offset is already 512-byte aligned
> + dd if="${part_img}" of="${img}" \
> + bs=512 seek=${_offset} \
> + conv=notrunc,sparse 2>/dev/null
> + done
> +}
> +
> +# vim: ft=sh
> diff --git a/fs/custom/boot/pre-post b/fs/custom/boot/pre-post
> new file mode 100644
> index 0000000..af4bcf5
> --- /dev/null
> +++ b/fs/custom/boot/pre-post
> @@ -0,0 +1,14 @@
> +#-----------------------------------------------------------------------------
> +# No dependencies
> +
> +#-----------------------------------------------------------------------------
> +do_image_pre() {
> + :
> +}
> +
> +#-----------------------------------------------------------------------------
> +do_image_post() {
> + :
> +}
> +
> +#vim: set ft=sh
> diff --git a/fs/custom/custom.mk b/fs/custom/custom.mk
> new file mode 100644
> index 0000000..ca53e0b
> --- /dev/null
> +++ b/fs/custom/custom.mk
> @@ -0,0 +1,25 @@
> +################################################################################
> +#
> +# custom partitioning
> +#
> +################################################################################
> +
> +# rootfs-custom uses rootfs.tar as the source to generate
> +# the resulting image(s), so we need to build it first.
> +ROOTFS_CUSTOM_DEPENDENCIES += rootfs-tar
> +
> +# If we are not selected, we won't have a partition table, so genimages
> +# will complain (both on stdout and return code), so it will faill, so
> +# we should not get our dependencies
> +ifeq ($(BR2_TARGET_ROOTFS_CUSTOM),y)
> +ROOTFS_CUSTOM_DEPENDENCIES += \
> + $(patsubst %,host-%,$(shell $(USER_HOOKS_EXTRA_ENV) fs/custom/genimages --show-depends \
> + '$(call qstrip,$(BR2_TARGET_ROOTFS_CUSTOM_PARTITION_TABLE))'))
> +endif
> +
> +define ROOTFS_CUSTOM_CMD
> + $(USER_HOOKS_EXTRA_ENV) fs/custom/genimages \
> + '$(call qstrip,$(BR2_TARGET_ROOTFS_CUSTOM_PARTITION_TABLE))'
> +endef
> +
> +$(eval $(call ROOTFS_TARGET,custom))
> diff --git a/fs/custom/fs/ext b/fs/custom/fs/ext
> new file mode 100644
> index 0000000..a184969
> --- /dev/null
> +++ b/fs/custom/fs/ext
> @@ -0,0 +1,26 @@
> +# Create an extended file system
> +
> +#-----------------------------------------------------------------------------
> +DEPENDS+=( e2fsprogs genext2fs )
> +
> +#-----------------------------------------------------------------------------
> +do_image() {
> + local root_dir="${1}"
> + local img="${2}"
> + local -a fs_opts
> + local gen rev
> +
> + fs_opts+=( -z )
> + fs_opts+=( -d "${root_dir}" )
> + [ -z "${size}" ] || fs_opts+=( -b $((size/1024)) )
> + [ -n "${ext_gen}" ] || ext_gen=2
> + [ -n "${ext_rev}" ] || ext_rev=1
> +
> + # Remember, we're running from Buildroot's TOP_DIR
> + GEN=${ext_gen} REV=${ext_rev} \
> + ./fs/ext2/genext2fs.sh "${fs_opts[@]}" "${img}" >/dev/null
> +
> + [ -z "${fs_label}" ] || tune2fs -L "${fs_label}" "${img}" >/dev/null
> +}
> +
> +# vim: ft=sh
> diff --git a/fs/custom/fs/pre-post b/fs/custom/fs/pre-post
> new file mode 100644
> index 0000000..d931437
> --- /dev/null
> +++ b/fs/custom/fs/pre-post
> @@ -0,0 +1,72 @@
> +#-----------------------------------------------------------------------------
> +# No additional dependencies
> +
> +#-----------------------------------------------------------------------------
> +do_image_pre() {
> + local i file
> +
> + # if fs_root_dir is not specified, we have to create one
> + # It *does* override the caller's fs_root_dir value, but
> + # that's on purpose
> + # If fs_root_dir is specified, and we have at least fs_files_0,
> + # then fs_root_dir/ must be enpty
> + if [ -z "${fs_root_dir}" ]; then
> + if [ -n "${values["${part}:fs_files_0"]}" ]; then
> + error "%s: no fs_root specified, and no fs_files_0\n" "${part}"
> + fi
> + fs_root_dir="$( mktemp -d "${tmp_dir}/XXXXXX" )"
> + else
> + if [ -n "${values["${part}:fs_files_0"]}" \
> + -a $( ls -1A "${fs_root_dir}" 2>/dev/null \
> + |wc -l \
> + ) -ne 0 ]; then
> + error "%s: %s is not empty, but fs_files_0 is specified\n" \
> + "${part}" "${fs_root_dir#${fs_root}}"
> + fi
> + fi
> +
> + i=0
> + while true; do
> + file="${values["${part}:fs_files_${i}"]}"
> + [ -n "${file}" ] || break
> + debug "%s: adding fs_files_%d %s\n" "${part}" ${i} "${file}"
> + install -D "${BINARIES_DIR}/${file}" "${fs_root_dir}/${file##*/}"
> + i=$((i+1))
> + done
> +}
> +
> +#-----------------------------------------------------------------------------
> +do_image_post() {
> + local rootfs_dir="${1}"
> + local fs_root="${2}"
> + local img_file="${3}"
> + local part="${4}"
> + local dev mntops vfstype fs_root_esc
> +
> + subname+="[post-image]"
> +
> + # Empty the partition's mountpoint
> + find "${fs_root_dir}" -maxdepth 1 \! -path "${fs_root_dir}" -exec rm -rf {} +
> +
> + # Add entry in fstab, but not if this is '/'
> + # Don't add either if rootfs was not extracted
> + if [ "${fs_root}" = "/" -o -z "${fs_root}" \
> + -o -z "${values["global:extract"]}" ]; then
> + return 0
> + fi
> + fs_root_esc="$( sed -r -e 's:/:\\/:g;' <<<"${fs_root}" )"
> + sed -r -i -e "/[^[:space:]]+[[:space:]]+${fs_root_esc}[[:space:]]/d" \
> + "${rootfs_dir}/etc/fstab"
> + dev="$( get_part_dev_node "${part}" )"
> + vfstype="${fs_vfstype:-${fs_type}}"
> + mntops="${fs_mntops:-defaults}"
> + printf "/dev/%s %s %s %s 0 0\n" \
> + "${dev}" "${fs_root}" \
> + "${vfstype}" "${mntops}" \
> + >>"${rootfs_dir}/etc/fstab"
> +
> + subname="${subname%\[post-image\]}"
> +}
> +
> +#-----------------------------------------------------------------------------
> +# vim: ft=sh
> diff --git a/fs/custom/fs/vfat b/fs/custom/fs/vfat
> new file mode 100644
> index 0000000..5fc243d
> --- /dev/null
> +++ b/fs/custom/fs/vfat
> @@ -0,0 +1,21 @@
> +# Create a VFAT file system
> +
> +#-----------------------------------------------------------------------------
> +DEPENDS+=( dosfstools mtools )
> +
> +#-----------------------------------------------------------------------------
> +do_image() {
> + local root_dir="${1}"
> + local img="${2}"
> + local -a fs_opts
> +
> + dd if=/dev/zero of="${img}" bs=${size} count=0 seek=1 2>/dev/null
> +
> + [ -z "${vfat_size}" ] || fs_opts+=( -F ${vfat_size} )
> + [ -z "${fs_label}" ] || fs_opts+=( -n "${fs_label}" )
> + mkfs.vfat "${fs_opts[@]}" "${img}" >/dev/null
> +
> + mcopy -i "${img}" "${root_dir}/"* '::'
> +}
> +
> +# vim: ft=sh
> diff --git a/fs/custom/genimages b/fs/custom/genimages
> new file mode 100755
> index 0000000..204f13b
> --- /dev/null
> +++ b/fs/custom/genimages
> @@ -0,0 +1,342 @@
> +#!/bin/bash
> +
> +#-----------------------------------------------------------------------------
> +main() {
> + local part_table="${1}"
> + local tmp_dir
> + local rootfs_dir
> + local -a devices
> + local extract
> + local cur_section
> + local -a sections devices partitions
> + local -A variables values partdevs
> + local sec dev part var val
> + local secs devs parts vars vals
> + local has_global_section
> +
> + # We need bash 4 or above for associative arrays
> + if [ ${BASH_VERSINFO[0]} -lt 4 ]; then
> + error "bash 4 or above is needed\n"
> + fi
> +
> + if [ "${part_table}" = "--show-depends" ]; then
> + SHOW_DEPENDS=1
> + part_table="${2}"
> + trace() { :; } # Be silent, we just want the dependencies...
> + else
> + SHOW_DEPENDS=0
> + fi
> +
> + if [ ! -f "${part_table}" ]; then
> + error "'%s': no such file\n" "${part_table}"
> + exit 1
> + fi
> +
> + export PATH="${HOST_DIR}/usr/bin:${HOST_DIR}/usr/sbin:${PATH}"
> +
> + # Parse all the sections in one go, we'll sort
> + # all the mess afterwards...
> + debug "parsing partitions descriptions file '%s'\n" \
> + "${part_table}"
> + parse_ini "${part_table}"
> +
> + # The 'global' section is mandatory
> + has_global_section=0
> + for s in "${sections[@]}"; do
> + if [ "${s}" = "global" ]; then
> + has_global_section=1
> + break
> + fi
> + done
> + if [ ${has_global_section} -eq 0 ]; then
> + error "no global section defined\n"
> + fi
> +
> + # Create lists of devices, partitions, and partition:device pairs.
> + debug "creating intermediate lists\n"
> + devices=( ${values["global:devices"]//,/ } )
> + for dev in "${devices[@]}"; do
> + # Sanity check first: all devices must have a corresponding
> + # section, which means they should have a type
> + if [ -z "${values["${dev}:type"]}" ]; then
> + error "device '%s' has no type (no section?)\n" "${dev}"
> + fi
> + partitions+=( ${values["${dev}:partitions"]//,/ } )
> + for part in ${values["${dev}:partitions"]//,/ }; do
> + # Sanity check first: all partitions must have a corresponding
> + # section, which means they should have a type
> + if [ -z "${values["${part}:type"]}" ]; then
> + error "partition '%s:%s' has no type (no section?)\n" \
> + "${dev}" "${part}"
> + fi
> + partdevs+=( ["${part}"]="${dev}" )
> + done
> + done
> +
> + # Now, we must order the partitions so that their mountpoint
> + # is empty by the time we build the upper-level partition.
> + # For example, given this layout of mountpoints:
> + # /
> + # /usr
> + # /usr/var
> + # We must ensure /usr/var is empty at the time we create the /usr
> + # filesystem image; and similarly, we must ensure /usr is empty by
> + # the time we create the / filesystem image
> + # So, a simple reverse alphabetical sort will do the trick
> + debug "sorting partitions\n"
> + sorted_parts=( $(
> + for part in "${partitions[@]}"; do
> + # Partitions that are not mounted can be generated
> + # in any order
> + if [ -n "${values["${part}:fs_root"]}" ]; then
> + printf "%s:%s\n" "${part}" "${values["${part}:fs_root"]}"
> + else
> + printf "%s\n" "${part}"
> + fi
> + done \
> + |sort -t: -k2 -r \
> + |sed -r -e 's/:[^:]+$//;'
> + ) )
> +
> + # We do not want to create anything if we only want the dependencies
> + if [ ${SHOW_DEPENDS} -eq 0 ]; then
> + tmp_dir="${BUILD_DIR}/genimages.tmp"
> + rootfs_dir="${tmp_dir}/rootfs"
> + # Since we don't remove it in case of error (to be able to inspect its
> + # content), we must remove it now (a previous run may have left it).
> + rm -rf "${tmp_dir}"
> + mkdir -p "${rootfs_dir}"
> +
> + case "${values["global:extract"]}" in
> + tar)
> + # We must be root for the extract to work correctly
> + # (since it may have /dev nodes, or some files may
> + # belong to different users...)
> + if [ $(id -u) -ne 0 ]; then
> + error "error: not root\n"
> + fi
> + trace "extracting rootfs.tar\n"
> + tar xf "${BINARIES_DIR}/rootfs.tar" -C "${rootfs_dir}"
> + ;;
> + *) error "unknown extract method '%s'\n" "${extract:-(none)}"
> + ;;
> + esac
> + fi # ! SHOW_DEPENDS
> +
> + # Render all partition images
> + for part in "${sorted_parts[@]}"; do
> + trace "preparing filesystem for partition '%s'\n" "${part}"
> + render_img "${rootfs_dir}" "${part}" \
> + "${tmp_dir}/${partdevs["${part}"]}.${part}.img"
> + done
> +
> + # Aggregate all devices images
> + for dev in "${devices[@]}"; do
> + trace "assembling partitions in device '%s'\n" "${dev}"
> + render_img "${rootfs_dir}" "${dev}" "${tmp_dir}/${dev}.img"
> + done
> +
> + # If we need the dependencies, we can stop right now
> + if [ ${SHOW_DEPENDS} -eq 1 ]; then
> + return 0
> + fi
> +
> + # Copy all partitions and devices images to the image dir
> + if [ "${values["global:keep_partitions"]}" = "yes" ]; then
> + for part in "${sorted_parts[@]}"; do
> + debug "copying partition '%s' to image dir\n" "${part}"
> + dd if="${tmp_dir}/${partdevs["${part}"]}.${part}.img" \
> + of="${BINARIES_DIR}/$( get_part_dev_node "${part}" ).img" \
> + bs=4096 conv=sparse 2>/dev/null
> + done
> + fi
> + for dev in "${devices[@]}"; do
> + debug "copying device '%s' to image dir\n" "${dev}"
> + dd if="${tmp_dir}/${dev}.img" \
> + of="${BINARIES_DIR}/${dev}.img" \
> + bs=4096 conv=sparse 2>/dev/null
> + done
> +
> + [ -n "${DEBUG}" ] || rm -rf "${tmp_dir}"
> +}
> +
> +#-----------------------------------------------------------------------------
> +render_img() {
> + local rootfs_dir="${1}"
> + local img="${2}"
> + local img_file="${3}"
> + local type sub_type fs_root_dir
> +
> + type="${values["${img}:type"]}"
> + sub_type="${values["${img}:${type}_type"]}"
> +
> + # Sanity checks
> + [ -n "${type}" ] || error "'%s': unspecified type\n" "${img}"
> + if [ ! -d "fs/custom/${type}" ]; then
> + error "'%s': unsupported type '%s'\n" "${img}" "${type}"
> + fi
> + [ -n "${sub_type}" ] || error "'%s': unspecified %s_type\n" "${img}" "${type}"
> + if [ ! -f "fs/custom/${type}/${sub_type}" ]; then
> + error "'%s': unknown %s_type '%s'\n" "${img}" "${type}" "${sub_type}"
> + fi
> +
> + # Need to call the renderer in a subshell so that its definitions
> + # do not pollute our environment
> + subname="${sub_type}"
> + (
> + trap 'exit $?' ERR
> +
> + declare -a DEPENDS
> +
> + for var in ${variables["${img}"]//,/ }; do
> + eval "${var}=\"${values["${img}:${var}"]}\""
> + done
> + fs_root_dir="${rootfs_dir}${fs_root}"
> + . "fs/custom/${type}/pre-post"
> + . "fs/custom/${type}/${sub_type}"
> + if [ ${SHOW_DEPENDS} -eq 1 ]; then
> + for dep in "${DEPENDS[@]}"; do
> + printf "%s\n" "${dep}"
> + done
> + else
> + do_image_pre "${rootfs_dir}" "${fs_root}" "${img_file}" "${img}"
> + do_image "${fs_root_dir}" "${img_file}"
> + do_image_post "${rootfs_dir}" "${fs_root}" "${img_file}" "${img}"
> + fi
> + )
> + ret=${?}
> + [ ${ret} -eq 0 ] || exit ${ret}
Why not doing: [ ${ret} -ne 0 ] && exit ${ret} ?
> + subname=""
> +}
> +
> +#------------------------------------------------------------------------------
> +# Parse a .ini file
> +# $1: .ini file to parse
> +# The caller should define the following variables:
> +# sections : array
> +# variables : associative array
> +# values : associative array
> +# parse_ini() will fill those variables with:
> +# sections : the list of sections, one section per array index
> +# variables : the comma-separated list of varibles for a section,
> +# indexed by the name of the section
> +# values : the value of a variable in a section, indexed by the
> +# 'section:variable' tuple
> +# Eg.:
> +# sections=( [0]='section-0' [1]='section-1' )
> +# variables=( ['section-0']='var-0,var-1' ['section-1']='var-10,var-11' [...] )
> +# values=( ['section-0:var-0']='value-0-0' ['section-1:var-10']='value-1-10' [...] )
> +parse_ini() {
> + local ini_file="${1}"
> + local line var val
> + local cur_section
> + local var_sep
> +
> + while read line; do
> + line="$( sed -r -e 's/[[:space:]]*#.*$//; //d;' <<<"${line}" )"
> +
> + # Detect start of global section, skip anything else
> + case "${line}" in
> + "") continue;;
> + '['*']')
> + cur_section="$( sed -r -e 's/[][]//g;' <<<"${line}" )"
> + debug " entering section '%s'\n" "${cur_section}"
> + sections+=( "${cur_section}" )
> + continue
> + ;;
> + ?*=*) ;;
> + *) error "malformed entry '%s'\n" "${line}";;
> + esac
> +
> + var="${line%%=*}"
> + eval val="${line#*=}"
> + debug " adding '%s'='%s'\n" "${var}" "${val}"
> + var_sep="${variables["${cur_section}"]+,}"
> + variables+=( ["${cur_section}"]="${var_sep}${var}" )
> + values+=( ["${cur_section}:${var}"]="${val}" )
> + done <"${ini_file}"
> +}
> +
> +#-----------------------------------------------------------------------------
> +get_part_dev_node() {
> + local part="${1}"
> + local dev
> + local i c p
> +
> + dev="${partdevs["${part}"]}"
> + i="${values["${dev}:partstart"]:-1}"
> +
> + # If device node ends with a number, partitions are denoted
> + # with a 'p' before the partition number, eg.:
> + # /dev/mmcblk0 --> /dev/mmcblk0p1
> + # /dev/sda --> /dev/sda1
> + case "${dev#${dev%?}}" in
> + [0-9]) c="p";;
> + *) c="";;
> + esac
> +
> + for p in ${values["${dev}:partitions"]//,/ }; do
> + if [ "${p}" = "${part}" ]; then
> + printf "%s%s%d" "${dev}" "${c}" ${i}
> + return 0
> + fi
> + i=$((i+1))
> + done
> +
> + error "'%s': partition not found. WTF?\n" "${part}"
> +}
> +
> +#------------------------------------------------------------------------------
> +align_val() {
> + local val="${1}"
> + local align="${2}"
> + local aligned
> +
> + aligned=$(( ( (val+align-1) / align ) * align ))
> +
> + printf "%d" ${aligned}
> +}
> +
> +#------------------------------------------------------------------------------
> +# Some trace functions
> +_trace() {
> + local fmt="${1}"
> + shift
> +
> + printf "%s" "${myname}"
> + if [ -n "${subname}" ]; then
> + printf "(%s)" "${subname}"
> + fi
> + printf ": ${fmt}" "${@}"
> +}
> +
> +trace() {
> + _trace "${@}"
> +}
> +
> +debug() { :; }
> +if [ -n "${DEBUG}" ]; then
> + debug() {
> + _trace "${@}" >&2
> + }
> +fi
> +
> +error() {
> + _trace "${@}" >&2
> + exit 1
> +}
> +
> +on_error() {
> + local ret=${?}
> +
> + error "unexpected error caught: %d\n" ${ret}
> +}
> +trap on_error ERR
> +set -E -e
> +
> +#-----------------------------------------------------------------------------
> +export myname="${0##*/}"
> +
> +main "${@}"
> +
> +# vim: ft=sh
> --
> 1.8.1.2
>
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH 3/5] board/raspberrypi: provide partition description for the new genimages
2014-01-03 17:19 ` [Buildroot] [PATCH 3/5] board/raspberrypi: provide partition description for the new genimages Yann E. MORIN
@ 2014-01-04 16:40 ` Maxime Hadjinlian
2014-01-04 17:05 ` Yann E. MORIN
0 siblings, 1 reply; 22+ messages in thread
From: Maxime Hadjinlian @ 2014-01-04 16:40 UTC (permalink / raw)
To: buildroot
Hi Yann,
On Fri, Jan 3, 2014 at 6:19 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> From: "Yann E. MORIN" <yann.morin.1998@free.fr>
>
> Now we can generate a complete target storage image with the genimages
> infra, add a partition table layout description for the Raspberry Pi
> as an example for how to use genimages.
>
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Ryan Barnett <rjbarnet@rockwellcollins.com>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
> board/raspberrypi/partitions | 35 +++++++++++++++++++++++++++++++++++
> configs/raspberrypi_defconfig | 3 +++
> 2 files changed, 38 insertions(+)
> create mode 100644 board/raspberrypi/partitions
>
> diff --git a/board/raspberrypi/partitions b/board/raspberrypi/partitions
> new file mode 100644
> index 0000000..dea9aff
> --- /dev/null
> +++ b/board/raspberrypi/partitions
> @@ -0,0 +1,35 @@
> +[global]
> +extract=tar
> +devices=mmcblk0
> +
> +[mmcblk0]
> +type=boot
> +boot_type=mbr
> +partitions=boot,root
> +partalign=$((1048576))
> +
> +[root]
> +type=fs
> +size=$((32*1048576))
> +mbr_type=$((0x83))
> +fs_type=ext
> +fs_vfstype=ext4
> +fs_mntops=discard
> +fs_root=/
> +fs_label=ROOT
> +ext_gen=4
> +ext_rev=1
> +
> +[boot]
> +type=fs
> +size=$((9*1048576))
> +mbr_type=$((0xc))
> +fs_type=vfat
> +fs_files_0=rpi-firmware/bootcode.bin
> +fs_files_1=rpi-firmware/start.elf
> +fs_files_2=rpi-firmware/fixup.dat
> +fs_files_3=rpi-firmware/config.txt
> +fs_files_4=rpi-firmware/cmdline.txt
> +fs_files_5=zImage
> +fs_label=BOOT
> +vfat_size=16
> diff --git a/configs/raspberrypi_defconfig b/configs/raspberrypi_defconfig
> index 6a17165..50caf5f 100644
> --- a/configs/raspberrypi_defconfig
> +++ b/configs/raspberrypi_defconfig
> @@ -21,3 +21,6 @@ BR2_LINUX_KERNEL_CUSTOM_REPO_VERSION="75d03120bc36b1cc3166973b8edc11f033ab7c0d"
> BR2_LINUX_KERNEL_USE_DEFCONFIG=y
> BR2_LINUX_KERNEL_DEFCONFIG="bcmrpi_quick"
> BR2_LINUX_KERNEL_ZIMAGE=y
> +
> +BR2_TARGET_ROOTFS_CUSTOM=y
> +BR2_TARGET_ROOTFS_CUSTOM_PARTITION_TABLE="$(TOPDIR)/board/raspberrypi/partitions"
> --
> 1.8.1.2
Could you please add comments over the various size and also the
different type used, 0x83 is not really user friendly to read.
Thanks !
>
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH 4/5] fs/custom: add support for squashfs
2014-01-03 17:19 ` [Buildroot] [PATCH 4/5] fs/custom: add support for squashfs Yann E. MORIN
@ 2014-01-04 16:43 ` Maxime Hadjinlian
2014-01-04 17:01 ` Yann E. MORIN
0 siblings, 1 reply; 22+ messages in thread
From: Maxime Hadjinlian @ 2014-01-04 16:43 UTC (permalink / raw)
To: buildroot
Hi Yann,
On Fri, Jan 3, 2014 at 6:19 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> From: "Yann E. MORIN" <yann.morin.1998@free.fr>
>
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Ryan Barnett <rjbarnet@rockwellcollins.com>
> ---
> docs/manual/partition-layout.txt | 11 ++++++++++-
> fs/custom/fs/squashfs | 19 +++++++++++++++++++
> 2 files changed, 29 insertions(+), 1 deletion(-)
> create mode 100644 fs/custom/fs/squashfs
>
> diff --git a/docs/manual/partition-layout.txt b/docs/manual/partition-layout.txt
> index 092ae1b..e9027bf 100644
> --- a/docs/manual/partition-layout.txt
> +++ b/docs/manual/partition-layout.txt
> @@ -133,7 +133,8 @@ Properties for +type=fs+
>
> * +fs_type+ (mandatory): the type of filesystem to generate
> ** +ext+: generate an extended filesystem (ext2, ext3, ext4)
> -** +vfat+: generate a VFAT filesystem (FAT16, FAT32)
> +** +squashfs+: generate a squashfs (version 4) filesystem
> +** +vfat+: generate a VFAT filesystem (FAT12, FAT16 or FAT32)
>
> * +fs_label+ (optional): the label to assign to this filesystem, if that
> filesystem supports a label
> @@ -177,6 +178,14 @@ Properties for +fs_type=ext+
> ** +1+ (mandatory for ext3 or ext4): generate a revision 1 extended
> filesystem
>
> +Properties for +fs_type=squashfs+
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +* +squashfs_comp+ (mandatory): the compression type to use
> +** +gzip+, +xz+, +lzo+: gzip, xz or lzo compression
> +
> +* +squashfs_block+ (optional): the size of blocks, in bytes
> +
> Properties for +fs_type=vfat+
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> diff --git a/fs/custom/fs/squashfs b/fs/custom/fs/squashfs
> new file mode 100644
> index 0000000..132d3ef
> --- /dev/null
> +++ b/fs/custom/fs/squashfs
> @@ -0,0 +1,19 @@
> +# Create a squashfs filesystem
> +
> +#-----------------------------------------------------------------------------
> +DEPENDS+=( squashfs )
> +
> +#-----------------------------------------------------------------------------
> +do_image() {
> + local root_dir="${1}"
> + local img="${2}"
> + local -a fs_opts
> +
> + fs_opts+=( -no-progress -noappend )
> + [ -z "${squashfs_comp}" ] || fs_opts+=( -comp "${squashfs_comp}" )
It just a question of taste, but I really do prefer to read something like:
[ -n "${squashfs_comp}" ] && fs_opts+=( -comp "${squashfs_comp}" )
But that's just a personal taste.
> + [ -z "${squashfs_block}" ] || fs_opts+=( -b ${squahsfs_block} )
> +
> + mksquashfs "${root_dir}" "${img}" "${fs_opts[@]}" >/dev/null
> +}
> +
> +# vim: ft=sh
> --
> 1.8.1.2
>
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
Thanks !
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH 4/5] fs/custom: add support for squashfs
2014-01-04 16:43 ` Maxime Hadjinlian
@ 2014-01-04 17:01 ` Yann E. MORIN
0 siblings, 0 replies; 22+ messages in thread
From: Yann E. MORIN @ 2014-01-04 17:01 UTC (permalink / raw)
To: buildroot
Maxime, All,
Thanks for the review! :-)
On 2014-01-04 17:43 +0100, Maxime Hadjinlian spake thusly:
> On Fri, Jan 3, 2014 at 6:19 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> > From: "Yann E. MORIN" <yann.morin.1998@free.fr>
> >
> > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> > Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> > Cc: Ryan Barnett <rjbarnet@rockwellcollins.com>
> > ---
> > docs/manual/partition-layout.txt | 11 ++++++++++-
> > fs/custom/fs/squashfs | 19 +++++++++++++++++++
> > 2 files changed, 29 insertions(+), 1 deletion(-)
> > create mode 100644 fs/custom/fs/squashfs
> >
[--SNIP--]
> > diff --git a/fs/custom/fs/squashfs b/fs/custom/fs/squashfs
> > new file mode 100644
> > index 0000000..132d3ef
> > --- /dev/null
> > +++ b/fs/custom/fs/squashfs
> > @@ -0,0 +1,19 @@
> > +# Create a squashfs filesystem
> > +
> > +#-----------------------------------------------------------------------------
> > +DEPENDS+=( squashfs )
> > +
> > +#-----------------------------------------------------------------------------
> > +do_image() {
> > + local root_dir="${1}"
> > + local img="${2}"
> > + local -a fs_opts
> > +
> > + fs_opts+=( -no-progress -noappend )
> > + [ -z "${squashfs_comp}" ] || fs_opts+=( -comp "${squashfs_comp}" )
> It just a question of taste, but I really do prefer to read something like:
> [ -n "${squashfs_comp}" ] && fs_opts+=( -comp "${squashfs_comp}" )
> But that's just a personal taste.
The genimages script runs with 'set -e -E', which means to exit as soon
as one command exits with a !0 exit code.
If we do as you suggested:
[ -n "${squashfs_comp}" ] && fs_opts+=( -comp "${squashfs_comp}" )
and "${squashfs_comp}" is empty, then the test is false, and the whole
list also is false, which amkes the shell error out.
If we do the way I did:
[ -z "${squashfs_comp}" ] || fs_opts+=( -comp "${squashfs_comp}" )
then it ensures that the list is never false, and thus there is no
error.
If this construct is not welcome, then we have to use if...fi blocks
instead:
if [ -n "${squashfs_comp}" ]; then
fs_opts+=( -comp "${squashfs_comp}" )
fi
This is a bit more verbose, but maybe more explicit. I prefer concise,
since I know why it is written this way, but I don't really care.
Regards,
Yann E. MORIN.
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH 3/5] board/raspberrypi: provide partition description for the new genimages
2014-01-04 16:40 ` Maxime Hadjinlian
@ 2014-01-04 17:05 ` Yann E. MORIN
2014-01-04 17:12 ` Maxime Hadjinlian
0 siblings, 1 reply; 22+ messages in thread
From: Yann E. MORIN @ 2014-01-04 17:05 UTC (permalink / raw)
To: buildroot
Maxime, All,
On 2014-01-04 17:40 +0100, Maxime Hadjinlian spake thusly:
> On Fri, Jan 3, 2014 at 6:19 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> > From: "Yann E. MORIN" <yann.morin.1998@free.fr>
> >
> > Now we can generate a complete target storage image with the genimages
> > infra, add a partition table layout description for the Raspberry Pi
> > as an example for how to use genimages.
> >
> > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> > Cc: Ryan Barnett <rjbarnet@rockwellcollins.com>
> > Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> > ---
> > board/raspberrypi/partitions | 35 +++++++++++++++++++++++++++++++++++
> > configs/raspberrypi_defconfig | 3 +++
> > 2 files changed, 38 insertions(+)
> > create mode 100644 board/raspberrypi/partitions
> >
> > diff --git a/board/raspberrypi/partitions b/board/raspberrypi/partitions
> > new file mode 100644
> > index 0000000..dea9aff
> > --- /dev/null
> > +++ b/board/raspberrypi/partitions
> > @@ -0,0 +1,35 @@
> > +[global]
> > +extract=tar
> > +devices=mmcblk0
> > +
> > +[mmcblk0]
> > +type=boot
> > +boot_type=mbr
> > +partitions=boot,root
> > +partalign=$((1048576))
> > +
> > +[root]
> > +type=fs
> > +size=$((32*1048576))
> > +mbr_type=$((0x83))
> > +fs_type=ext
> > +fs_vfstype=ext4
> > +fs_mntops=discard
> > +fs_root=/
> > +fs_label=ROOT
> > +ext_gen=4
> > +ext_rev=1
> > +
> > +[boot]
> > +type=fs
> > +size=$((9*1048576))
> > +mbr_type=$((0xc))
> > +fs_type=vfat
> > +fs_files_0=rpi-firmware/bootcode.bin
> > +fs_files_1=rpi-firmware/start.elf
> > +fs_files_2=rpi-firmware/fixup.dat
> > +fs_files_3=rpi-firmware/config.txt
> > +fs_files_4=rpi-firmware/cmdline.txt
> > +fs_files_5=zImage
> > +fs_label=BOOT
> > +vfat_size=16
> > diff --git a/configs/raspberrypi_defconfig b/configs/raspberrypi_defconfig
> > index 6a17165..50caf5f 100644
> > --- a/configs/raspberrypi_defconfig
> > +++ b/configs/raspberrypi_defconfig
> > @@ -21,3 +21,6 @@ BR2_LINUX_KERNEL_CUSTOM_REPO_VERSION="75d03120bc36b1cc3166973b8edc11f033ab7c0d"
> > BR2_LINUX_KERNEL_USE_DEFCONFIG=y
> > BR2_LINUX_KERNEL_DEFCONFIG="bcmrpi_quick"
> > BR2_LINUX_KERNEL_ZIMAGE=y
> > +
> > +BR2_TARGET_ROOTFS_CUSTOM=y
> > +BR2_TARGET_ROOTFS_CUSTOM_PARTITION_TABLE="$(TOPDIR)/board/raspberrypi/partitions"
> > --
> > 1.8.1.2
> Could you please add comments over the various size and also the
> different type used, 0x83 is not really user friendly to read.
0x83 and 0xc are partition types, and there is a pointer in the manual
for where to find their meanings and the different values:
* mbr_type (mandatory): the partition [type]
where [type] is a pointer to:
https://en.wikipedia.org/wiki/Partition_type#List_of_partition_IDs
But I can add such comments as this is an example.
Regards,
Yann E. MORIN.
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH 3/5] board/raspberrypi: provide partition description for the new genimages
2014-01-04 17:05 ` Yann E. MORIN
@ 2014-01-04 17:12 ` Maxime Hadjinlian
2014-01-04 17:50 ` Yann E. MORIN
0 siblings, 1 reply; 22+ messages in thread
From: Maxime Hadjinlian @ 2014-01-04 17:12 UTC (permalink / raw)
To: buildroot
On Sat, Jan 4, 2014 at 6:05 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> Maxime, All,
>
> On 2014-01-04 17:40 +0100, Maxime Hadjinlian spake thusly:
>> On Fri, Jan 3, 2014 at 6:19 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
>> > From: "Yann E. MORIN" <yann.morin.1998@free.fr>
>> >
>> > Now we can generate a complete target storage image with the genimages
>> > infra, add a partition table layout description for the Raspberry Pi
>> > as an example for how to use genimages.
>> >
>> > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
>> > Cc: Ryan Barnett <rjbarnet@rockwellcollins.com>
>> > Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
>> > ---
>> > board/raspberrypi/partitions | 35 +++++++++++++++++++++++++++++++++++
>> > configs/raspberrypi_defconfig | 3 +++
>> > 2 files changed, 38 insertions(+)
>> > create mode 100644 board/raspberrypi/partitions
>> >
>> > diff --git a/board/raspberrypi/partitions b/board/raspberrypi/partitions
>> > new file mode 100644
>> > index 0000000..dea9aff
>> > --- /dev/null
>> > +++ b/board/raspberrypi/partitions
>> > @@ -0,0 +1,35 @@
>> > +[global]
>> > +extract=tar
>> > +devices=mmcblk0
>> > +
>> > +[mmcblk0]
>> > +type=boot
>> > +boot_type=mbr
>> > +partitions=boot,root
>> > +partalign=$((1048576))
>> > +
>> > +[root]
>> > +type=fs
>> > +size=$((32*1048576))
>> > +mbr_type=$((0x83))
>> > +fs_type=ext
>> > +fs_vfstype=ext4
>> > +fs_mntops=discard
>> > +fs_root=/
>> > +fs_label=ROOT
>> > +ext_gen=4
>> > +ext_rev=1
>> > +
>> > +[boot]
>> > +type=fs
>> > +size=$((9*1048576))
>> > +mbr_type=$((0xc))
>> > +fs_type=vfat
>> > +fs_files_0=rpi-firmware/bootcode.bin
>> > +fs_files_1=rpi-firmware/start.elf
>> > +fs_files_2=rpi-firmware/fixup.dat
>> > +fs_files_3=rpi-firmware/config.txt
>> > +fs_files_4=rpi-firmware/cmdline.txt
>> > +fs_files_5=zImage
>> > +fs_label=BOOT
>> > +vfat_size=16
>> > diff --git a/configs/raspberrypi_defconfig b/configs/raspberrypi_defconfig
>> > index 6a17165..50caf5f 100644
>> > --- a/configs/raspberrypi_defconfig
>> > +++ b/configs/raspberrypi_defconfig
>> > @@ -21,3 +21,6 @@ BR2_LINUX_KERNEL_CUSTOM_REPO_VERSION="75d03120bc36b1cc3166973b8edc11f033ab7c0d"
>> > BR2_LINUX_KERNEL_USE_DEFCONFIG=y
>> > BR2_LINUX_KERNEL_DEFCONFIG="bcmrpi_quick"
>> > BR2_LINUX_KERNEL_ZIMAGE=y
>> > +
>> > +BR2_TARGET_ROOTFS_CUSTOM=y
>> > +BR2_TARGET_ROOTFS_CUSTOM_PARTITION_TABLE="$(TOPDIR)/board/raspberrypi/partitions"
>> > --
>> > 1.8.1.2
>> Could you please add comments over the various size and also the
>> different type used, 0x83 is not really user friendly to read.
>
> 0x83 and 0xc are partition types, and there is a pointer in the manual
> for where to find their meanings and the different values:
>
> * mbr_type (mandatory): the partition [type]
>
> where [type] is a pointer to:
> https://en.wikipedia.org/wiki/Partition_type#List_of_partition_IDs
>
> But I can add such comments as this is an example.
Even if it's not an example.
This kind of file are meant to be read by humans, and as such, every
information such as partition type or size, should be along side a
comments for the human reader.
>
> Regards,
> Yann E. MORIN.
>
> --
> .-----------------.--------------------.------------------.--------------------.
> | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
> | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
> | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no |
> | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
> '------------------------------^-------^------------------^--------------------'
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH 2/5] fs/custom: generate complete, partition-based device images
2014-01-04 16:38 ` Maxime Hadjinlian
@ 2014-01-04 17:43 ` Yann E. MORIN
2014-01-04 17:52 ` Maxime Hadjinlian
0 siblings, 1 reply; 22+ messages in thread
From: Yann E. MORIN @ 2014-01-04 17:43 UTC (permalink / raw)
To: buildroot
Maxime, All,
Thanks for the review! :-)
On 2014-01-04 17:38 +0100, Maxime Hadjinlian spake thusly:
> When doing scripting in Buildroot, do we accept the uses of various
> bash-ism or do we want to stick to POSIX shell ?
bash is already a hard-dependency of Buildroot.
> On Fri, Jan 3, 2014 at 6:19 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
[--SNIP--]
> > diff --git a/docs/manual/customize-filesystems.txt b/docs/manual/customize-filesystems.txt
> > new file mode 100644
> > index 0000000..fd65c97
> > --- /dev/null
> > +++ b/docs/manual/customize-filesystems.txt
> > @@ -0,0 +1,36 @@
[--SNIP--]
> How about comments ? Are they supported ?
> In a next step, it would be really nice to have support for "human"
> input for the different size, like 1G/M/K.
Yes, comments are supported; I've now added that to the manual.
I've also added that values can be expresed as shell arith,metic
expansion, such as $((16*1048576)) (aka 16MiB).
Not that G/M/k are SI-prefixes that respectively mean 10^9, 10^6 and
10^3. The binary prefixes are Gi/Mi/ki, respectively 2^30, 2^20 and
2^10.
If/when we add support for this kind of values, we'll need to add both
decimal and binary prefixes.
> > diff --git a/fs/custom/boot/gpt b/fs/custom/boot/gpt
> > new file mode 100644
> > index 0000000..f978524
> > --- /dev/null
> > +++ b/fs/custom/boot/gpt
> > @@ -0,0 +1,129 @@
[--SNIP--]
> > +# Simg = 512 + 2*(Sgpt) + ?( aligned(Spart,512) )
> Don't we want only ASCII char there ?
I don't know.
I find it convenient not to limit ourselves to ASCII.
[--SNIP--]
> > +# Sicne 4k-large sectors are not really explained on Wikipedia, we can
> Small typo here.
Thanks, fixed.
[--SNIP--]
> > + dd if=/dev/zero of="${img}" \
> > + bs=1 seek=${begin} count=0 \
> > + conv=sparse 2>/dev/null
> Why the redirect to /dev/null ? Don't we want to see if there was an
> error ? I know it's unlikely to happen but you never know.
> I have noticed you did several times so maybe there is a reason I am
> not seeing here.
dd will always print something to stderr, even if successful. We do not
want it to pollute the output, since we're using nice traces.
[--SNIP--]
> > + ret=${?}
> > + [ ${ret} -eq 0 ] || exit ${ret}
> Why not doing: [ ${ret} -ne 0 ] && exit ${ret} ?
Same answer as to your comment in squashfs, since we're using set -e.
But here, it's a bit complex. POSIX does not specify whether a sub-shell
that exits with a return code !0 should be caught by set -e. And bash-4.0
does not catch it, while bash-4.2 does catch it. Both are POSIX complianti
in this respect, since POSIX does not state what should be done in this
case.
So, we have to explicitly store the return code, and test it.
But if ret==0, your test would fail, and the complete shell compound
command " [ ... ] && ... " would also be false, and we would create an
error that the shell would catch because of set -e, when there was no
error to begin with.
Thank you for the comments! :-)
Regards,
Yann E. MORIN.
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH 3/5] board/raspberrypi: provide partition description for the new genimages
2014-01-04 17:12 ` Maxime Hadjinlian
@ 2014-01-04 17:50 ` Yann E. MORIN
2014-01-04 17:57 ` Maxime Hadjinlian
0 siblings, 1 reply; 22+ messages in thread
From: Yann E. MORIN @ 2014-01-04 17:50 UTC (permalink / raw)
To: buildroot
Maxime, All,
On 2014-01-04 18:12 +0100, Maxime Hadjinlian spake thusly:
> On Sat, Jan 4, 2014 at 6:05 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> > On 2014-01-04 17:40 +0100, Maxime Hadjinlian spake thusly:
> >> On Fri, Jan 3, 2014 at 6:19 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> >> > +partalign=$((1048576))
> >> > +size=$((32*1048576))
> >> > +mbr_type=$((0x83))
> >> > +size=$((9*1048576))
> >> > +mbr_type=$((0xc))
> >> Could you please add comments over the various size and also the
> >> different type used, 0x83 is not really user friendly to read.
> >
> > 0x83 and 0xc are partition types, and there is a pointer in the manual
> > for where to find their meanings and the different values:
> >
> > * mbr_type (mandatory): the partition [type]
> >
> > where [type] is a pointer to:
> > https://en.wikipedia.org/wiki/Partition_type#List_of_partition_IDs
> >
> > But I can add such comments as this is an example.
> Even if it's not an example.
> This kind of file are meant to be read by humans, and as such, every
> information such as partition type or size, should be along side a
> comments for the human reader.
<humour>
Sorry, but I can read 9*1048576 very easily. But I'm no good reference,
since I know by heart those powers of 2 up to about 16777216... :-p
As for 0x83 and 0xc, I have run fdisk so many a time that I know what
they mean, too! :-p
</humour>
Regards,
Yann E. MORIN.
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH 2/5] fs/custom: generate complete, partition-based device images
2014-01-04 17:43 ` Yann E. MORIN
@ 2014-01-04 17:52 ` Maxime Hadjinlian
0 siblings, 0 replies; 22+ messages in thread
From: Maxime Hadjinlian @ 2014-01-04 17:52 UTC (permalink / raw)
To: buildroot
On Sat, Jan 4, 2014 at 6:43 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> Maxime, All,
>
> Thanks for the review! :-)
>
> On 2014-01-04 17:38 +0100, Maxime Hadjinlian spake thusly:
>> When doing scripting in Buildroot, do we accept the uses of various
>> bash-ism or do we want to stick to POSIX shell ?
>
> bash is already a hard-dependency of Buildroot.
>
>> On Fri, Jan 3, 2014 at 6:19 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> [--SNIP--]
>> > diff --git a/docs/manual/customize-filesystems.txt b/docs/manual/customize-filesystems.txt
>> > new file mode 100644
>> > index 0000000..fd65c97
>> > --- /dev/null
>> > +++ b/docs/manual/customize-filesystems.txt
>> > @@ -0,0 +1,36 @@
> [--SNIP--]
>> How about comments ? Are they supported ?
>> In a next step, it would be really nice to have support for "human"
>> input for the different size, like 1G/M/K.
>
> Yes, comments are supported; I've now added that to the manual.
>
> I've also added that values can be expresed as shell arith,metic
> expansion, such as $((16*1048576)) (aka 16MiB).
>
> Not that G/M/k are SI-prefixes that respectively mean 10^9, 10^6 and
> 10^3. The binary prefixes are Gi/Mi/ki, respectively 2^30, 2^20 and
> 2^10.
>
> If/when we add support for this kind of values, we'll need to add both
> decimal and binary prefixes.
You are absolutely right. Also, it is sad, almost no one uses the
right units :(.
>
>> > diff --git a/fs/custom/boot/gpt b/fs/custom/boot/gpt
>> > new file mode 100644
>> > index 0000000..f978524
>> > --- /dev/null
>> > +++ b/fs/custom/boot/gpt
>> > @@ -0,0 +1,129 @@
> [--SNIP--]
>> > +# Simg = 512 + 2*(Sgpt) + ?( aligned(Spart,512) )
>> Don't we want only ASCII char there ?
>
> I don't know.
>
> I find it convenient not to limit ourselves to ASCII.
I remember someone on this mailing saying something about it, don't
remember exactly what was the context.
>
> [--SNIP--]
>> > +# Sicne 4k-large sectors are not really explained on Wikipedia, we can
>> Small typo here.
>
> Thanks, fixed.
>
> [--SNIP--]
>> > + dd if=/dev/zero of="${img}" \
>> > + bs=1 seek=${begin} count=0 \
>> > + conv=sparse 2>/dev/null
>> Why the redirect to /dev/null ? Don't we want to see if there was an
>> error ? I know it's unlikely to happen but you never know.
>> I have noticed you did several times so maybe there is a reason I am
>> not seeing here.
>
> dd will always print something to stderr, even if successful. We do not
> want it to pollute the output, since we're using nice traces.
>
> [--SNIP--]
>> > + ret=${?}
>> > + [ ${ret} -eq 0 ] || exit ${ret}
>> Why not doing: [ ${ret} -ne 0 ] && exit ${ret} ?
>
> Same answer as to your comment in squashfs, since we're using set -e.
>
> But here, it's a bit complex. POSIX does not specify whether a sub-shell
> that exits with a return code !0 should be caught by set -e. And bash-4.0
> does not catch it, while bash-4.2 does catch it. Both are POSIX complianti
> in this respect, since POSIX does not state what should be done in this
> case.
>
> So, we have to explicitly store the return code, and test it.
>
> But if ret==0, your test would fail, and the complete shell compound
> command " [ ... ] && ... " would also be false, and we would create an
> error that the shell would catch because of set -e, when there was no
> error to begin with.
>
> Thank you for the comments! :-)
Thanks for the explanations !
>
> Regards,
> Yann E. MORIN.
>
> --
> .-----------------.--------------------.------------------.--------------------.
> | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
> | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
> | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no |
> | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
> '------------------------------^-------^------------------^--------------------'
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH 3/5] board/raspberrypi: provide partition description for the new genimages
2014-01-04 17:50 ` Yann E. MORIN
@ 2014-01-04 17:57 ` Maxime Hadjinlian
0 siblings, 0 replies; 22+ messages in thread
From: Maxime Hadjinlian @ 2014-01-04 17:57 UTC (permalink / raw)
To: buildroot
On Sat, Jan 4, 2014 at 6:50 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> Maxime, All,
>
> On 2014-01-04 18:12 +0100, Maxime Hadjinlian spake thusly:
>> On Sat, Jan 4, 2014 at 6:05 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
>> > On 2014-01-04 17:40 +0100, Maxime Hadjinlian spake thusly:
>> >> On Fri, Jan 3, 2014 at 6:19 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
>> >> > +partalign=$((1048576))
>> >> > +size=$((32*1048576))
>> >> > +mbr_type=$((0x83))
>> >> > +size=$((9*1048576))
>> >> > +mbr_type=$((0xc))
>> >> Could you please add comments over the various size and also the
>> >> different type used, 0x83 is not really user friendly to read.
>> >
>> > 0x83 and 0xc are partition types, and there is a pointer in the manual
>> > for where to find their meanings and the different values:
>> >
>> > * mbr_type (mandatory): the partition [type]
>> >
>> > where [type] is a pointer to:
>> > https://en.wikipedia.org/wiki/Partition_type#List_of_partition_IDs
>> >
>> > But I can add such comments as this is an example.
>> Even if it's not an example.
>> This kind of file are meant to be read by humans, and as such, every
>> information such as partition type or size, should be along side a
>> comments for the human reader.
>
> <humour>
> Sorry, but I can read 9*1048576 very easily. But I'm no good reference,
> since I know by heart those powers of 2 up to about 16777216... :-p
>
> As for 0x83 and 0xc, I have run fdisk so many a time that I know what
> they mean, too! :-p
> </humour>
"I don?t even see the code. All I see is blonde, brunette, red-head" ;)
>
> Regards,
> Yann E. MORIN.
>
> --
> .-----------------.--------------------.------------------.--------------------.
> | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
> | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
> | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no |
> | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
> '------------------------------^-------^------------------^--------------------'
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH 1/5] package/squashfs: add selection for the host variant
2014-01-03 17:19 ` [Buildroot] [PATCH 1/5] package/squashfs: add selection for the host variant Yann E. MORIN
@ 2014-02-03 14:53 ` Thomas De Schampheleire
2014-02-08 22:20 ` Peter Korsgaard
1 sibling, 0 replies; 22+ messages in thread
From: Thomas De Schampheleire @ 2014-02-03 14:53 UTC (permalink / raw)
To: buildroot
On Fri, Jan 3, 2014 at 6:19 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> From: "Yann E. MORIN" <yann.morin.1998@free.fr>
>
> If any of the post-image scripts wants to handle squashfs filesystems,
> we need to expose an option for squashfs-tools to be user-selectable.
>
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Ryan Barnett <rjbarnet@rockwellcollins.com>
> ---
> package/Config.in.host | 1 +
> package/squashfs/Config.in.host | 6 ++++++
> 2 files changed, 7 insertions(+)
> create mode 100644 package/squashfs/Config.in.host
>
> diff --git a/package/Config.in.host b/package/Config.in.host
> index 34e84bf..71597bf 100644
> --- a/package/Config.in.host
> +++ b/package/Config.in.host
> @@ -12,6 +12,7 @@ source "package/omap-u-boot-utils/Config.in.host"
> source "package/openocd/Config.in.host"
> source "package/parted/Config.in.host"
> source "package/sam-ba/Config.in.host"
> +source "package/squashfs/Config.in.host"
> source "package/sunxi-tools/Config.in.host"
> source "package/uboot-tools/Config.in.host"
>
> diff --git a/package/squashfs/Config.in.host b/package/squashfs/Config.in.host
> new file mode 100644
> index 0000000..b353374
> --- /dev/null
> +++ b/package/squashfs/Config.in.host
> @@ -0,0 +1,6 @@
> +config BR2_PACKAGE_HOST_SQUASHFS
> + bool "host squashfs"
> + help
> + Tools to generate SquashFS filesystems.
> +
> + http://squashfs.sourceforge.net/
Acked-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH 1/5] package/squashfs: add selection for the host variant
2014-01-03 17:19 ` [Buildroot] [PATCH 1/5] package/squashfs: add selection for the host variant Yann E. MORIN
2014-02-03 14:53 ` Thomas De Schampheleire
@ 2014-02-08 22:20 ` Peter Korsgaard
1 sibling, 0 replies; 22+ messages in thread
From: Peter Korsgaard @ 2014-02-08 22:20 UTC (permalink / raw)
To: buildroot
>>>>> "Yann" == Yann E MORIN <yann.morin.1998@free.fr> writes:
> From: "Yann E. MORIN" <yann.morin.1998@free.fr>
> If any of the post-image scripts wants to handle squashfs filesystems,
> we need to expose an option for squashfs-tools to be user-selectable.
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Ryan Barnett <rjbarnet@rockwellcollins.com>
Committed, thanks.
--
Bye, Peter Korsgaard
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH 2/5] fs/custom: generate complete, partition-based device images
2014-01-03 17:19 ` [Buildroot] [PATCH 2/5] fs/custom: generate complete, partition-based device images Yann E. MORIN
2014-01-04 16:38 ` Maxime Hadjinlian
@ 2014-04-09 13:28 ` Andreas Naumann
2014-04-09 17:09 ` Yann E. MORIN
1 sibling, 1 reply; 22+ messages in thread
From: Andreas Naumann @ 2014-04-09 13:28 UTC (permalink / raw)
To: buildroot
Hi Yann,
> From: "Yann E. MORIN" <yann.morin.1998@free.fr>
>
> Contrary to the existing fs/ schemes, which each generate only a single
> filesystem image for the root filesystem, this new scheme allows the
> user to generate more complex images.
if found this an interesting idea, but apparently it has not found its
way into master yet. What is the current state?
>
> The basis behind this is a .ini-like description of the layout of the
> final target storage:
> - the list of device(s)
> - per-device, the list of partition(s)
> - per-partition, the content
>
> It is possible to create MBR- or GPT-based partitoining schemes. Adding
> new ones should be relatively easy (but would need adequate host tools).
>
> For now, the only content possible for partitions is a filesystem. It
> should be pretty easy to add new types (eg. un-formated, or raw blob).
>
> Also, only two filesystems are supported: ext{2,3,4} and vfat. Adding
> more will be relatively easy, provided we have the necessary host
> packages to deal with those filesystems.
However, I'not sure if it actually is what I am looking for. Which is
dividing the target content into multiple partitions, which are then
mounted into the rootfs (like /data, /etc/somespecialconfig, /var/log).?
Are there other ways to do that?
cheers,
Andi
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH 2/5] fs/custom: generate complete, partition-based device images
2014-04-09 13:28 ` Andreas Naumann
@ 2014-04-09 17:09 ` Yann E. MORIN
2014-04-10 13:53 ` Andreas Naumann
0 siblings, 1 reply; 22+ messages in thread
From: Yann E. MORIN @ 2014-04-09 17:09 UTC (permalink / raw)
To: buildroot
Andreas, All,
On 2014-04-09 15:28 +0200, Andreas Naumann spake thusly:
> Hi Yann,
>
> >From: "Yann E. MORIN" <yann.morin.1998@free.fr>
> >
> >Contrary to the existing fs/ schemes, which each generate only a single
> >filesystem image for the root filesystem, this new scheme allows the
> >user to generate more complex images.
>
> if found this an interesting idea, but apparently it has not found its way
> into master yet. What is the current state?
This has been somewhat refused: we do not want to add new tools in
Buildroot, and this 'genimages' stuff is seen as a new tool, rather than
as infrastructure.
However, it has been suggested that we:
- transform this genimages into a separate project which provides a
new filesystem generator, and which Buildroot could use like it is
currently using the other filesystem generators;
- look at Pengutronix' genimage (without a trailing 's') to see if it
would fit our needs.
I haven't had time to properly look at Pengutronix' genimage, but last I
looked, it seemd a bit different than my genimages. I am not too fond of
genimage, but that's mostly a personal feeling, and is not really
founded on technical merits. I prefer my genimages implementation, which
I know how to deal with.
As for my genimages: I am still working on it, and it so far fits all my
personal needs. I still have some feature that are missing for now, that
are really needed before it can be widely deployed:
- mixed filesystem hierarchies split between an initramfs and partitions
on a real device;
- partitions to be mounted that are not to be written (eg. when you
upgrade, you do not want to overwrite a config partition, but you
still want to have an fstab entry for it).
That's being hamerred out, but it takes a bit of time...
> >The basis behind this is a .ini-like description of the layout of the
> >final target storage:
> > - the list of device(s)
> > - per-device, the list of partition(s)
> > - per-partition, the content
> >
> >It is possible to create MBR- or GPT-based partitoining schemes. Adding
> >new ones should be relatively easy (but would need adequate host tools).
> >
> >For now, the only content possible for partitions is a filesystem. It
> >should be pretty easy to add new types (eg. un-formated, or raw blob).
> >
> >Also, only two filesystems are supported: ext{2,3,4} and vfat. Adding
> >more will be relatively easy, provided we have the necessary host
> >packages to deal with those filesystems.
>
> However, I'not sure if it actually is what I am looking for. Which is
> dividing the target content into multiple partitions, which are then mounted
> into the rootfs (like /data, /etc/somespecialconfig, /var/log).?
Yes, that's exactly what genimages is supposed to cover: a filesystem
hierarchy made of more than the / mountpoint.
> Are there other ways to do that?
Depends what you mean. I am using a 'wrapper' project arounf Buildroot,
which genimages originates from. In this project, I'm using genimages as
a post-image script that just extract the rootfs.tar and works from
there to generate the multiple filesystem images for the different
partitions.
But it is not really usable outside of my project, since it is tightly
coupled with the post-build script.
Basically, if all you have are filesystem on reald decives (flash,
HDD...), then genimages could world stand-alone.
But if your setup involves an initramfs for the rootfs, plus one or more
partitions on a real device, you have to fill entries for those
partitions in etc/fstab, but doing so in post-image is too late, because
the etc/fstab is in the initramfs, which is in the kernel image, which
is assembled during the image phase. So we can't fill-in etc/fstab from
a post-image script in this case. [*]
However, it is always possible to fill-in etc/fstab from a post-build
(aka pre-image) script, whether your rootfs is on a real device or an
initramfs. And that's what my post-build script does (among many other
things).
So my genimages is tightly coupled to my rootfs-fixup post-build script,
and I currently do not see a clean way to provide genimages as just
another filesystem generator. If we wanted to truly have genimages in
Buildroot, we'd need more than hooking after the rootfs.tar generator.
[*] Well, it could be possible to call back to Buildroot to regenerate
the kernel image with the initramfs once we tweaked etc/fstab, but
that's pretty ugly, and prone to errors.)
If you want to have a look, here's my "buildroot.config' project:
http://ymorin.is-a-geek.org/git/buildroot.config/
There's currently pretty much no documentation, besides the one I wrote
for inclusion in Buildroot (available in this thread). That's something
I should really write... The README is a bit out-dated, so not completely
accurate, but the idea is there.
I have a tutorial on how to use it to build an XBMC for the RPi:
http://ymorin.is-a-geek.org/projects/rpi-xbmc
Any suggestion/idea on how to progress on Buildroot is welcome, of
course! ;-)
Regards,
Yann E. MORIN.
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Buildroot] [PATCH 2/5] fs/custom: generate complete, partition-based device images
2014-04-09 17:09 ` Yann E. MORIN
@ 2014-04-10 13:53 ` Andreas Naumann
0 siblings, 0 replies; 22+ messages in thread
From: Andreas Naumann @ 2014-04-10 13:53 UTC (permalink / raw)
To: buildroot
Hi Yann,
thanks a lot for the explanations!
Am 09.04.2014 19:09, schrieb Yann E. MORIN:
>> into master yet. What is the current state?
>
> This has been somewhat refused: we do not want to add new tools in
> Buildroot, and this 'genimages' stuff is seen as a new tool, rather than
> as infrastructure.
>
> However, it has been suggested that we:
>
> - transform this genimages into a separate project which provides a
> new filesystem generator, and which Buildroot could use like it is
> currently using the other filesystem generators;
>
> - look at Pengutronix' genimage (without a trailing 's') to see if it
> would fit our needs.
I had a very quick look at your buildroot.config as well as Pengutronix
genimage. You seem to do much more than what genimage does, but actually
creating a bunch of ubi images is sufficient for me. Well I have to try
it out, needs usually appear when doing things. Right now I dont find
the time but hopefully in a couple of weeks I should be able to post my
experience.
regards
Andi
>
> I haven't had time to properly look at Pengutronix' genimage, but last I
> looked, it seemd a bit different than my genimages. I am not too fond of
> genimage, but that's mostly a personal feeling, and is not really
> founded on technical merits. I prefer my genimages implementation, which
> I know how to deal with.
>
> As for my genimages: I am still working on it, and it so far fits all my
> personal needs. I still have some feature that are missing for now, that
> are really needed before it can be widely deployed:
>
> - mixed filesystem hierarchies split between an initramfs and partitions
> on a real device;
>
> - partitions to be mounted that are not to be written (eg. when you
> upgrade, you do not want to overwrite a config partition, but you
> still want to have an fstab entry for it).
>
> That's being hamerred out, but it takes a bit of time...
>
>>> The basis behind this is a .ini-like description of the layout of the
>>> final target storage:
>>> - the list of device(s)
>>> - per-device, the list of partition(s)
>>> - per-partition, the content
>>>
>>> It is possible to create MBR- or GPT-based partitoining schemes. Adding
>>> new ones should be relatively easy (but would need adequate host tools).
>>>
>>> For now, the only content possible for partitions is a filesystem. It
>>> should be pretty easy to add new types (eg. un-formated, or raw blob).
>>>
>>> Also, only two filesystems are supported: ext{2,3,4} and vfat. Adding
>>> more will be relatively easy, provided we have the necessary host
>>> packages to deal with those filesystems.
>>
>> However, I'not sure if it actually is what I am looking for. Which is
>> dividing the target content into multiple partitions, which are then mounted
>> into the rootfs (like /data, /etc/somespecialconfig, /var/log).?
>
> Yes, that's exactly what genimages is supposed to cover: a filesystem
> hierarchy made of more than the / mountpoint.
>
>> Are there other ways to do that?
>
> Depends what you mean. I am using a 'wrapper' project arounf Buildroot,
> which genimages originates from. In this project, I'm using genimages as
> a post-image script that just extract the rootfs.tar and works from
> there to generate the multiple filesystem images for the different
> partitions.
>
> But it is not really usable outside of my project, since it is tightly
> coupled with the post-build script.
>
> Basically, if all you have are filesystem on reald decives (flash,
> HDD...), then genimages could world stand-alone.
>
> But if your setup involves an initramfs for the rootfs, plus one or more
> partitions on a real device, you have to fill entries for those
> partitions in etc/fstab, but doing so in post-image is too late, because
> the etc/fstab is in the initramfs, which is in the kernel image, which
> is assembled during the image phase. So we can't fill-in etc/fstab from
> a post-image script in this case. [*]
>
> However, it is always possible to fill-in etc/fstab from a post-build
> (aka pre-image) script, whether your rootfs is on a real device or an
> initramfs. And that's what my post-build script does (among many other
> things).
>
> So my genimages is tightly coupled to my rootfs-fixup post-build script,
> and I currently do not see a clean way to provide genimages as just
> another filesystem generator. If we wanted to truly have genimages in
> Buildroot, we'd need more than hooking after the rootfs.tar generator.
>
>
> [*] Well, it could be possible to call back to Buildroot to regenerate
> the kernel image with the initramfs once we tweaked etc/fstab, but
> that's pretty ugly, and prone to errors.)
>
>
> If you want to have a look, here's my "buildroot.config' project:
> http://ymorin.is-a-geek.org/git/buildroot.config/
>
> There's currently pretty much no documentation, besides the one I wrote
> for inclusion in Buildroot (available in this thread). That's something
> I should really write... The README is a bit out-dated, so not completely
> accurate, but the idea is there.
>
> I have a tutorial on how to use it to build an XBMC for the RPi:
> http://ymorin.is-a-geek.org/projects/rpi-xbmc
>
> Any suggestion/idea on how to progress on Buildroot is welcome, of
> course! ;-)
>
> Regards,
> Yann E. MORIN.
>
^ permalink raw reply [flat|nested] 22+ messages in thread
end of thread, other threads:[~2014-04-10 13:53 UTC | newest]
Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-01-03 17:19 [Buildroot] [PATCH 0/5 v4] SUBJECT Yann E. MORIN
2014-01-03 17:19 ` [Buildroot] [PATCH 1/5] package/squashfs: add selection for the host variant Yann E. MORIN
2014-02-03 14:53 ` Thomas De Schampheleire
2014-02-08 22:20 ` Peter Korsgaard
2014-01-03 17:19 ` [Buildroot] [PATCH 2/5] fs/custom: generate complete, partition-based device images Yann E. MORIN
2014-01-04 16:38 ` Maxime Hadjinlian
2014-01-04 17:43 ` Yann E. MORIN
2014-01-04 17:52 ` Maxime Hadjinlian
2014-04-09 13:28 ` Andreas Naumann
2014-04-09 17:09 ` Yann E. MORIN
2014-04-10 13:53 ` Andreas Naumann
2014-01-03 17:19 ` [Buildroot] [PATCH 3/5] board/raspberrypi: provide partition description for the new genimages Yann E. MORIN
2014-01-04 16:40 ` Maxime Hadjinlian
2014-01-04 17:05 ` Yann E. MORIN
2014-01-04 17:12 ` Maxime Hadjinlian
2014-01-04 17:50 ` Yann E. MORIN
2014-01-04 17:57 ` Maxime Hadjinlian
2014-01-03 17:19 ` [Buildroot] [PATCH 4/5] fs/custom: add support for squashfs Yann E. MORIN
2014-01-04 16:43 ` Maxime Hadjinlian
2014-01-04 17:01 ` Yann E. MORIN
2014-01-03 17:19 ` [Buildroot] [PATCH 5/5] fs/custom: add support for raw device/partition content Yann E. MORIN
2014-01-03 17:22 ` [Buildroot] [PATCH 0/5 v4] SUBJECT Yann E. MORIN
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.