[U-Boot] [PATCH 0/4] Let the distro boot command scan all partitions

[U-Boot] [PATCH 0/4] Let the distro boot command scan all partitions

[Sjoerd Simons]

Not all devices use the convention of the first partition holding the boot
files. E.g. on chromebooks partition 1 and 2 are usually of the "Chromeos
kernel data" type. So instead of hardcoding just the first partitions scan all
partition on a storage device.

First two patches add some supporting commands, which help in determining the
list of partitions to scan and detect whether they have a known filesystem (No
need to scan for a bunch of different fiels if the filesystem isn't supported).

Third patch has the actual changes, while the last one tries to make it a bit
easier for board files to include the distro boot commands even if they don't
use it as their default.

Sjoerd Simons (4):
  fs: Add command to retrieve the filesystem type
  part: let list put the list in an environment variable
  config_distro_bootcmd: Scan all partitions for boot files
  distro_distro_bootcmd: use CONFIG_BOOTCOMMAND instead of setting
    bootcmd=

 common/cmd_fs.c                 | 15 +++++++++++++++
 common/cmd_part.c               | 24 ++++++++++++++++++++++--
 fs/fs.c                         | 27 +++++++++++++++++++++++++++
 include/config_distro_bootcmd.h | 21 +++++++++++++++++----
 include/fs.h                    |  6 ++++++
 5 files changed, 87 insertions(+), 6 deletions(-)

-- 
2.1.4

[U-Boot] [PATCH 1/4] fs: Add command to retrieve the filesystem type

[Sjoerd Simons]

New command to determine the filesystem type of a given partition.
Optionally stores the filesystem type in a environment variable.

Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>
---
 common/cmd_fs.c | 15 +++++++++++++++
 fs/fs.c         | 27 +++++++++++++++++++++++++++
 include/fs.h    |  6 ++++++
 3 files changed, 48 insertions(+)

diff --git a/common/cmd_fs.c b/common/cmd_fs.c
index 0d9da11..e146254 100644
--- a/common/cmd_fs.c
+++ b/common/cmd_fs.c
@@ -81,3 +81,18 @@ U_BOOT_CMD(
 	"    - List files in directory 'directory' of partition 'part' on\n"
 	"      device type 'interface' instance 'dev'."
 )
+
+static int do_fstype_wrapper(cmd_tbl_t *cmdtp, int flag, int argc,
+				char * const argv[])
+{
+	return do_fs_type(cmdtp, flag, argc, argv);
+}
+
+U_BOOT_CMD(
+	fstype, 4, 1, do_fstype_wrapper,
+	"Look up a filesystem type",
+	"<interface> <dev>:<part>\n"
+	"- print filesystem type\n"
+	"fstype <interface> <dev>:<part> <varname>\n"
+	"- set environment variable to filesystem type\n"
+);
diff --git a/fs/fs.c b/fs/fs.c
index ddd751c..483273f 100644
--- a/fs/fs.c
+++ b/fs/fs.c
@@ -79,6 +79,7 @@ static inline int fs_uuid_unsupported(char *uuid_str)
 
 struct fstype_info {
 	int fstype;
+	char *name;
 	/*
 	 * Is it legal to pass NULL as .probe()'s  fs_dev_desc parameter? This
 	 * should be false in most cases. For "virtual" filesystems which
@@ -105,6 +106,7 @@ static struct fstype_info fstypes[] = {
 #ifdef CONFIG_FS_FAT
 	{
 		.fstype = FS_TYPE_FAT,
+		.name = "fat",
 		.null_dev_desc_ok = false,
 		.probe = fat_set_blk_dev,
 		.close = fat_close,
@@ -123,6 +125,7 @@ static struct fstype_info fstypes[] = {
 #ifdef CONFIG_FS_EXT4
 	{
 		.fstype = FS_TYPE_EXT,
+		.name = "ext4",
 		.null_dev_desc_ok = false,
 		.probe = ext4fs_probe,
 		.close = ext4fs_close,
@@ -141,6 +144,7 @@ static struct fstype_info fstypes[] = {
 #ifdef CONFIG_SANDBOX
 	{
 		.fstype = FS_TYPE_SANDBOX,
+		.name = "sandbox",
 		.null_dev_desc_ok = true,
 		.probe = sandbox_fs_set_blk_dev,
 		.close = sandbox_fs_close,
@@ -154,6 +158,7 @@ static struct fstype_info fstypes[] = {
 #endif
 	{
 		.fstype = FS_TYPE_ANY,
+		.name = "unsupported",
 		.null_dev_desc_ok = true,
 		.probe = fs_probe_unsupported,
 		.close = fs_close_unsupported,
@@ -190,6 +195,7 @@ int fs_set_blk_dev(const char *ifname, const char *dev_part_str, int fstype)
 	if (!relocated) {
 		for (i = 0, info = fstypes; i < ARRAY_SIZE(fstypes);
 				i++, info++) {
+			info->name += gd->reloc_off;
 			info->probe += gd->reloc_off;
 			info->close += gd->reloc_off;
 			info->ls += gd->reloc_off;
@@ -503,3 +509,24 @@ int do_fs_uuid(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[],
 
 	return CMD_RET_SUCCESS;
 }
+
+int do_fs_type(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])
+{
+	struct fstype_info *info;
+
+	if (argc < 3 || argc > 4)
+		return CMD_RET_USAGE;
+
+	if (fs_set_blk_dev(argv[1], argv[2], FS_TYPE_ANY))
+		return 1;
+
+	info = fs_get_info(fs_type);
+
+	if (argc == 4)
+		setenv(argv[3], info->name);
+	else
+		printf("%s\n", info->name);
+
+	return CMD_RET_SUCCESS;
+}
+
diff --git a/include/fs.h b/include/fs.h
index ffb6ce7..fd1e4ab 100644
--- a/include/fs.h
+++ b/include/fs.h
@@ -109,4 +109,10 @@ int do_save(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[],
 int do_fs_uuid(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[],
 		int fstype);
 
+/*
+ * Determine the type of the specified filesystem and print it. Optionally it is
+ * possible to store the type directly in env.
+ */
+int do_fs_type(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[]);
+
 #endif /* _FS_H */
-- 
2.1.4

[U-Boot] [PATCH 1/4] fs: Add command to retrieve the filesystem type

[Stephen Warren]

On 01/05/2015 10:13 AM, Sjoerd Simons wrote:
> New command to determine the filesystem type of a given partition.
> Optionally stores the filesystem type in a environment variable.

> diff --git a/common/cmd_fs.c b/common/cmd_fs.c

> +U_BOOT_CMD(
> +	fstype, 4, 1, do_fstype_wrapper,
> +	"Look up a filesystem type",
> +	"<interface> <dev>:<part>\n"

Should this line ...

> +	"- print filesystem type\n"
> +	"fstype <interface> <dev>:<part> <varname>\n"

... be consistent with this one - namely either both or neither include 
"fstype" at the start?

> diff --git a/fs/fs.c b/fs/fs.c

> +int do_fs_type(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])
> +{
> +	struct fstype_info *info;
> +
> +	if (argc < 3 || argc > 4)
> +		return CMD_RET_USAGE;
> +
> +	if (fs_set_blk_dev(argv[1], argv[2], FS_TYPE_ANY))
> +		return 1;
> +
> +	info = fs_get_info(fs_type);
> +
> +	if (argc == 4)
> +		setenv(argv[3], info->name);
> +	else
> +		printf("%s\n", info->name);
> +
> +	return CMD_RET_SUCCESS;
> +}

That function has both the cmdline interface and implementation logic in 
one place. Many of the other features (read, write, ls, ...) separate 
them out into two functions so that U-Boot code can call the 
implementation, and shell code can call the cmdline parsing wrapper. 
Should we do the same here? Admittedly, the implementation would be 
pretty simple, but perhaps it's still useful?

I don't feel that strongly though, and we can easily refactor that later 
if required.

[U-Boot] [PATCH 1/4] fs: Add command to retrieve the filesystem type

[Sjoerd Simons]

On Mon, 2015-01-05 at 13:18 -0700, Stephen Warren wrote:
> On 01/05/2015 10:13 AM, Sjoerd Simons wrote:
> > New command to determine the filesystem type of a given partition.
> > Optionally stores the filesystem type in a environment variable.
> 
> > diff --git a/common/cmd_fs.c b/common/cmd_fs.c
> 
> > +U_BOOT_CMD(
> > +	fstype, 4, 1, do_fstype_wrapper,
> > +	"Look up a filesystem type",
> > +	"<interface> <dev>:<part>\n"
> 
> Should this line ...
> 
> > +	"- print filesystem type\n"
> > +	"fstype <interface> <dev>:<part> <varname>\n"
> 
> ... be consistent with this one - namely either both or neither include 
> "fstype" at the start?

Nope, the cmd_usage implementation does (summarized):

printf("Usage:\n%s ", cmdtp->name);
puts(cmdtp->help);
putc('\n');

So the "fstype" at the start of the first line gets added by that code,
hence the declaration needs to be inconsistent to have a consistent
output for the user :)

> > diff --git a/fs/fs.c b/fs/fs.c
> 
> > +int do_fs_type(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])
> > +{
> > +	struct fstype_info *info;
> > +
> > +	if (argc < 3 || argc > 4)
> > +		return CMD_RET_USAGE;
> > +
> > +	if (fs_set_blk_dev(argv[1], argv[2], FS_TYPE_ANY))
> > +		return 1;
> > +
> > +	info = fs_get_info(fs_type);
> > +
> > +	if (argc == 4)
> > +		setenv(argv[3], info->name);
> > +	else
> > +		printf("%s\n", info->name);
> > +
> > +	return CMD_RET_SUCCESS;
> > +}
> 
> That function has both the cmdline interface and implementation logic in 
> one place. Many of the other features (read, write, ls, ...) separate 
> them out into two functions so that U-Boot code can call the 
> implementation, and shell code can call the cmdline parsing wrapper. 
> Should we do the same here? Admittedly, the implementation would be 
> pretty simple, but perhaps it's still useful?

Ah i did wonder why the splitup was there in some functions, that
explains it :).

I would expect that u-boot code which would like to use it would be more
interested in getting the fstype struct rather then necessarily the name
as a string (or printed out).. But i could well be wrong. 

> I don't feel that strongly though, and we can easily refactor that later 
> if required.

Same here, although i'd slightly prefer refactoring if-needed rather
then pre-emptively :)

-- 
Sjoerd Simons <sjoerd.simons@collabora.co.uk>
Collabora Ltd.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 6170 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150106/58c2687c/attachment.bin>

[U-Boot] [PATCH 1/4] fs: Add command to retrieve the filesystem type

[Stephen Warren]

On 01/06/2015 09:40 AM, Sjoerd Simons wrote:
> On Mon, 2015-01-05 at 13:18 -0700, Stephen Warren wrote:
>> On 01/05/2015 10:13 AM, Sjoerd Simons wrote:
>>> New command to determine the filesystem type of a given partition.
>>> Optionally stores the filesystem type in a environment variable.
>>
>>> diff --git a/common/cmd_fs.c b/common/cmd_fs.c
>>
>>> +U_BOOT_CMD(
>>> +	fstype, 4, 1, do_fstype_wrapper,
>>> +	"Look up a filesystem type",
>>> +	"<interface> <dev>:<part>\n"
>>
>> Should this line ...
>>
>>> +	"- print filesystem type\n"
>>> +	"fstype <interface> <dev>:<part> <varname>\n"
>>
>> ... be consistent with this one - namely either both or neither include
>> "fstype" at the start?
>
> Nope, the cmd_usage implementation does (summarized):
>
> printf("Usage:\n%s ", cmdtp->name);
> puts(cmdtp->help);
> putc('\n');
>
> So the "fstype" at the start of the first line gets added by that code,
> hence the declaration needs to be inconsistent to have a consistent
> output for the user :)

Ah right. In that case,
Reviewed-by: Stephen Warren <swarren@nvidia.com>

[U-Boot] [U-Boot, 1/4] fs: Add command to retrieve the filesystem type

[Tom Rini]

On Mon, Jan 05, 2015 at 06:13:36PM +0100, Sjoerd Simons wrote:

> New command to determine the filesystem type of a given partition.
> Optionally stores the filesystem type in a environment variable.
> 
> Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>
> Reviewed-by: Stephen Warren <swarren@nvidia.com>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150202/956f3d38/attachment.sig>

[U-Boot] [PATCH 2/4] part: let list put the list in an environment variable

[Sjoerd Simons]

Add an optional third argument to the "part list" command which puts a
space seperated list of valid partitions into the given environment
variable. This is useful for allowing boot scripts to iterate of all
partitions of a device.

Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>
---
 common/cmd_part.c | 24 ++++++++++++++++++++++--
 1 file changed, 22 insertions(+), 2 deletions(-)

diff --git a/common/cmd_part.c b/common/cmd_part.c
index 39e8666..c99f527 100644
--- a/common/cmd_part.c
+++ b/common/cmd_part.c
@@ -54,13 +54,31 @@ static int do_part_list(int argc, char * const argv[])
 	int ret;
 	block_dev_desc_t *desc;
 
-	if (argc != 2)
+	if (argc < 2 || argc > 3)
 		return CMD_RET_USAGE;
 
 	ret = get_device(argv[0], argv[1], &desc);
 	if (ret < 0)
 		return 1;
 
+	if (argc == 3) {
+		int p;
+		char str[512] = { 0, };
+	  disk_partition_t info;
+
+		for (p = 1; p < 128; p++) {
+			int r = get_partition_info(desc, p, &info);
+
+			if (r == 0) {
+				char t[5];
+				sprintf(t, "%s%d", str[0] ? " " : "", p);
+				strcat(str, t);
+			}
+		}
+		setenv(argv[2], str);
+		return 0;
+	}
+
 	print_part(desc);
 
 	return 0;
@@ -87,5 +105,7 @@ U_BOOT_CMD(
 	"part uuid <interface> <dev>:<part> <varname>\n"
 	"    - set environment variable to partition UUID\n"
 	"part list <interface> <dev>\n"
-	"    - print a device's partition table"
+	"    - print a device's partition table\n"
+	"part list <interface> <dev> <varname>\n"
+	"    - set environment variable to the list of partitions"
 );
-- 
2.1.4

[U-Boot] [PATCH 2/4] part: let list put the list in an environment variable

[Stephen Warren]

On 01/05/2015 10:13 AM, Sjoerd Simons wrote:
> Add an optional third argument to the "part list" command which puts a
> space seperated list of valid partitions into the given environment
> variable. This is useful for allowing boot scripts to iterate of all
> partitions of a device.

Reviewed-by: Stephen Warren <swarren@nvidia.com>

[U-Boot] [U-Boot, 2/4] part: let list put the list in an environment variable

[Tom Rini]

On Mon, Jan 05, 2015 at 06:13:37PM +0100, Sjoerd Simons wrote:

> Add an optional third argument to the "part list" command which puts a
> space seperated list of valid partitions into the given environment
> variable. This is useful for allowing boot scripts to iterate of all
> partitions of a device.
> 
> Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>
> Reviewed-by: Stephen Warren <swarren@nvidia.com>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150202/151758af/attachment.sig>

[U-Boot] [PATCH 3/4] config_distro_bootcmd: Scan all partitions for boot files

[Sjoerd Simons]

Not all devices use the convention that the boot scripts are on the
first partition. For example on chromebooks it seems common for the
first two partitions to be ChromeOS kernel partitions.

So instead of just the first partition scan all partitions on a device
with a filesystem u-boot can recognize.

Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>
---
 include/config_distro_bootcmd.h | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/include/config_distro_bootcmd.h b/include/config_distro_bootcmd.h
index be616e8..68f5fcb 100644
--- a/include/config_distro_bootcmd.h
+++ b/include/config_distro_bootcmd.h
@@ -13,7 +13,7 @@
 #define BOOTENV_SHARED_BLKDEV_BODY(devtypel) \
 		"if " #devtypel " dev ${devnum}; then " \
 			"setenv devtype " #devtypel "; " \
-			"run scan_dev_for_boot; " \
+			"run scan_dev_for_boot_part; " \
 		"fi\0"
 
 #define BOOTENV_SHARED_BLKDEV(devtypel) \
@@ -163,7 +163,6 @@
 	"boot_prefixes=/ /boot/\0" \
 	"boot_scripts=boot.scr.uimg boot.scr\0" \
 	BOOTENV_BOOT_TARGETS \
-	"bootpart=1\0" \
 	\
 	"boot_extlinux="                                                  \
 		"sysboot ${devtype} ${devnum}:${bootpart} any "           \
@@ -194,12 +193,21 @@
 		"done\0"                                                  \
 	\
 	"scan_dev_for_boot="                                              \
-		"echo Scanning ${devtype} ${devnum}...; "                 \
+		"echo Scanning ${devtype} ${devnum}:${bootpart}...; "     \
 		"for prefix in ${boot_prefixes}; do "                     \
 			"run scan_dev_for_extlinux; "                     \
 			"run scan_dev_for_scripts; "                      \
 		"done\0"                                                  \
 	\
+	"scan_dev_for_boot_part="                                         \
+		"part list ${devtype} ${devnum} devplist; "               \
+		"for bootpart in ${devplist}; do "                        \
+			"if fstype ${devtype} ${devnum}:${bootpart} "     \
+					"bootfstype; then "               \
+				"run scan_dev_for_boot; "                 \
+			"fi; "                                            \
+		"done\0"                                                  \
+	\
 	BOOT_TARGET_DEVICES(BOOTENV_DEV)                                  \
 	\
 	"bootcmd=" BOOTENV_SET_USB_NEED_INIT BOOTENV_SET_SCSI_NEED_INIT   \
-- 
2.1.4

[U-Boot] [PATCH 3/4] config_distro_bootcmd: Scan all partitions for boot files

[Stephen Warren]

On 01/05/2015 10:13 AM, Sjoerd Simons wrote:
> Not all devices use the convention that the boot scripts are on the
> first partition. For example on chromebooks it seems common for the
> first two partitions to be ChromeOS kernel partitions.
>
> So instead of just the first partition scan all partitions on a device
> with a filesystem u-boot can recognize.

I had planned (but obviously never got around to...) enhancing the 
scripts to look up the (set of?) bootable partition(s) on the disk and 
to attempt to load the boot files from there. Bootable would be defined 
as the MBR bootable flag, or GPT legacy bootable attribute.

That would allow the code to zero in on the one specific partition that 
it was supposed to look at, rather than searching all partitions.

Do you have any thoughts re: which option is better?

This patch looks fine assuming this option (rather than bootable flag) 
is selected.

[U-Boot] [PATCH 3/4] config_distro_bootcmd: Scan all partitions for boot files

[Sjoerd Simons]

On Mon, 2015-01-05 at 13:24 -0700, Stephen Warren wrote:
> On 01/05/2015 10:13 AM, Sjoerd Simons wrote:
> > Not all devices use the convention that the boot scripts are on the
> > first partition. For example on chromebooks it seems common for the
> > first two partitions to be ChromeOS kernel partitions.
> >
> > So instead of just the first partition scan all partitions on a device
> > with a filesystem u-boot can recognize.
> 
> I had planned (but obviously never got around to...) enhancing the 
> scripts to look up the (set of?) bootable partition(s) on the disk and 
> to attempt to load the boot files from there. Bootable would be defined 
> as the MBR bootable flag, or GPT legacy bootable attribute.
> 
> That would allow the code to zero in on the one specific partition that 
> it was supposed to look at, rather than searching all partitions.
> 
> Do you have any thoughts re: which option is better?

I did wonder about this as well. I do personally consider the bootable
flag as a rather obsolete/legacy thing (GPT even specifies it as a
legacy flag), so i was wary about using it.. Also i've been bitten a few
times on systems that did rely on the bootable flag (what, what, why
does it not boot, oooooohhhh), which was another reason for heading this
route.

This way does no extra work if the first partition is the partition with
the boot partition when compared to only checking partitions with the
bootable flag as both would need to list existing partitions. 

If the first few partitions have no filesystems, the extra work compared
to the bootable-flag approach would just be probing the filesystem type,
which tends to be relatively simple, so i don't see a big issue there
(it's more work to scan for a missing boot file).

If your first few partitions are ones without the bootfiles, some more
effort is wasted as it will be probing those for viable boot files..
However, in my experience, partition layouts with the bootfiles not on
the first filesystem partitions is rather uncommmon. So again, i didn't
feel that that was problematic. If you have an odd parition layout, your
boot time will be ever so slightly longer :)

The only "issue" in my mind is when multiple partitions, for whatever
reason, have bootfiles. In which case the first one will get picked with
this approach, while with the partition-boot-flag approach you'd have a
way to specify, no really just look at that one.. However, i suspect the
likelihood of forgetting to set the boot flag is higher (been there,
done that) then accidentally leaving boot files on partitions before the
intended boot partition (which also requires on uncommon layout), so
even then i suspect this approach is more friendly/less error-prone.

> This patch looks fine assuming this option (rather than bootable flag) 
> is selected.

Well my thoughts on the matter are above, If folks feel strongly about
this approach being the wrong way I'd love to hear their arguments :).

-- 
Sjoerd Simons <sjoerd.simons@collabora.co.uk>
Collabora Ltd.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 6170 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150106/36493b70/attachment.bin>

[U-Boot] [PATCH 3/4] config_distro_bootcmd: Scan all partitions for boot files

[Stephen Warren]

(CCing Dennis so he can comment from a distro perspective re: partition 
table bootable flags v.s. scanning all partitions)

On 01/06/2015 10:07 AM, Sjoerd Simons wrote:
> On Mon, 2015-01-05 at 13:24 -0700, Stephen Warren wrote:
>> On 01/05/2015 10:13 AM, Sjoerd Simons wrote:
>>> Not all devices use the convention that the boot scripts are on the
>>> first partition. For example on chromebooks it seems common for the
>>> first two partitions to be ChromeOS kernel partitions.
>>>
>>> So instead of just the first partition scan all partitions on a device
>>> with a filesystem u-boot can recognize.
>>
>> I had planned (but obviously never got around to...) enhancing the
>> scripts to look up the (set of?) bootable partition(s) on the disk and
>> to attempt to load the boot files from there. Bootable would be defined
>> as the MBR bootable flag, or GPT legacy bootable attribute.
>>
>> That would allow the code to zero in on the one specific partition that
>> it was supposed to look at, rather than searching all partitions.
>>
>> Do you have any thoughts re: which option is better?
>
> I did wonder about this as well. I do personally consider the bootable
> flag as a rather obsolete/legacy thing (GPT even specifies it as a
> legacy flag), so i was wary about using it.. Also i've been bitten a few
> times on systems that did rely on the bootable flag (what, what, why
> does it not boot, oooooohhhh), which was another reason for heading this
> route.
>
> This way does no extra work if the first partition is the partition with
> the boot partition when compared to only checking partitions with the
> bootable flag as both would need to list existing partitions.
>
> If the first few partitions have no filesystems, the extra work compared
> to the bootable-flag approach would just be probing the filesystem type,
> which tends to be relatively simple, so i don't see a big issue there
> (it's more work to scan for a missing boot file).
>
> If your first few partitions are ones without the bootfiles, some more
> effort is wasted as it will be probing those for viable boot files..
> However, in my experience, partition layouts with the bootfiles not on
> the first filesystem partitions is rather uncommmon. So again, i didn't
> feel that that was problematic. If you have an odd parition layout, your
> boot time will be ever so slightly longer :)
>
> The only "issue" in my mind is when multiple partitions, for whatever
> reason, have bootfiles. In which case the first one will get picked with
> this approach, while with the partition-boot-flag approach you'd have a
> way to specify, no really just look at that one.. However, i suspect the
> likelihood of forgetting to set the boot flag is higher (been there,
> done that) then accidentally leaving boot files on partitions before the
> intended boot partition (which also requires on uncommon layout), so
> even then i suspect this approach is more friendly/less error-prone.
>
>> This patch looks fine assuming this option (rather than bootable flag)
>> is selected.
>
> Well my thoughts on the matter are above, If folks feel strongly about
> this approach being the wrong way I'd love to hear their arguments :).

One issue with this approach is that there's no way for the user to 
short-circuit the scanning. If I put a ChromiumOS install on an SD card 
and leave it plugged into a system that's going to end up booting from 
eMMC since that's where the boot files are, there are lots of partitions 
to scan on that SD card, which will be a bit annoying.

[U-Boot] [PATCH 3/4] config_distro_bootcmd: Scan all partitions for boot files

[Sjoerd Simons]

On Tue, 2015-01-06 at 17:43 -0700, Stephen Warren wrote:
> (CCing Dennis so he can comment from a distro perspective re: partition 
> table bootable flags v.s. scanning all partitions)
> 
> On 01/06/2015 10:07 AM, Sjoerd Simons wrote:
> > On Mon, 2015-01-05 at 13:24 -0700, Stephen Warren wrote:
> >> On 01/05/2015 10:13 AM, Sjoerd Simons wrote:

> > Well my thoughts on the matter are above, If folks feel strongly about
> > this approach being the wrong way I'd love to hear their arguments :).
> 
> One issue with this approach is that there's no way for the user to 
> short-circuit the scanning. If I put a ChromiumOS install on an SD card 
> and leave it plugged into a system that's going to end up booting from 
> eMMC since that's where the boot files are, there are lots of partitions 
> to scan on that SD card, which will be a bit annoying.


I don't remember exactly how many partitions with fat/ext* filesystems a
ChromiumOS installation has (order of 3-5 iirc?), but indeed it means
your boot will be a bit slower due to it probing more partitions.
Wouldn't expect it to significantly slow down the total boot time
though.

I didn't think of this one my WIP is on an Odroid X2 which has a boot
selector jumper, so I have it always starting from mmc0 (which is either
SD or EMMC depending on the jumper setting). 

However, it raises an interesting question. The current convention for
Exynos is to first scans external storage (SD, mmc 1) and then internal
storage (eMMC, mmc 0), which opens up a whole different can of worms. As
that means that e.g. my chromebook will try to boot from whatever random
SD i've put into it first rather the OS installed on eMMC.  It would be
nice to have some general guidelines in this area so the behaviour of
various boards can be somewhat consistent in the default behaviour.
(Added Ian Cambell to the cc as he introduce the usage on exynos
devices)

-- 
Sjoerd Simons <sjoerd.simons@collabora.co.uk>
Collabora Ltd.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 6170 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150107/8b06646e/attachment.bin>

[U-Boot] [PATCH 3/4] config_distro_bootcmd: Scan all partitions for boot files

[Ian Campbell]

On Wed, 2015-01-07 at 11:10 +0100, Sjoerd Simons wrote:
> On Tue, 2015-01-06 at 17:43 -0700, Stephen Warren wrote:
> > (CCing Dennis so he can comment from a distro perspective re: partition 
> > table bootable flags v.s. scanning all partitions)
> > 
> > On 01/06/2015 10:07 AM, Sjoerd Simons wrote:
> > > On Mon, 2015-01-05 at 13:24 -0700, Stephen Warren wrote:
> > >> On 01/05/2015 10:13 AM, Sjoerd Simons wrote:
> 
> > > Well my thoughts on the matter are above, If folks feel strongly about
> > > this approach being the wrong way I'd love to hear their arguments :).
> > 
> > One issue with this approach is that there's no way for the user to 
> > short-circuit the scanning. If I put a ChromiumOS install on an SD card 
> > and leave it plugged into a system that's going to end up booting from 
> > eMMC since that's where the boot files are, there are lots of partitions 
> > to scan on that SD card, which will be a bit annoying.
> 
> 
> I don't remember exactly how many partitions with fat/ext* filesystems a
> ChromiumOS installation has (order of 3-5 iirc?), but indeed it means
> your boot will be a bit slower due to it probing more partitions.
> Wouldn't expect it to significantly slow down the total boot time
> though.

I thought u-boot would scan that partitions which were marked bootable
first, in which case you just need to set the bit correctly in the
partition table. I might be wrong about that though. (and re-reading
$subject, it seems like changing this is the subject of the thread...)

> I didn't think of this one my WIP is on an Odroid X2 which has a boot
> selector jumper, so I have it always starting from mmc0 (which is either
> SD or EMMC depending on the jumper setting). 
> 
> However, it raises an interesting question. The current convention for
> Exynos is to first scans external storage (SD, mmc 1) and then internal
> storage (eMMC, mmc 0), which opens up a whole different can of worms. As
> that means that e.g. my chromebook will try to boot from whatever random
> SD i've put into it first rather the OS installed on eMMC.  It would be
> nice to have some general guidelines in this area so the behaviour of
> various boards can be somewhat consistent in the default behaviour.

My understanding was that the various ${boot_*} envvars, including
boot_targets, are considered to be user serviceable parts. IOW if you
want to boot from mmc0 only then:
        setenv boot_targets mmc0
        saveenv

Maybe it makes sense for the default boot order to differ depending on
the specific exynos platform though?

Ian.

> (Added Ian Cambell to the cc as he introduce the usage on exynos
> devices)
> 

[U-Boot] [PATCH 3/4] config_distro_bootcmd: Scan all partitions for boot files

[Sjoerd Simons]

On Wed, 2015-01-07 at 10:22 +0000, Ian Campbell wrote:
> On Wed, 2015-01-07 at 11:10 +0100, Sjoerd Simons wrote:
> > On Tue, 2015-01-06 at 17:43 -0700, Stephen Warren wrote:
> > > (CCing Dennis so he can comment from a distro perspective re: partition 
> > > table bootable flags v.s. scanning all partitions)
> > > 
> > > On 01/06/2015 10:07 AM, Sjoerd Simons wrote:
> > > > On Mon, 2015-01-05 at 13:24 -0700, Stephen Warren wrote:
> > > >> On 01/05/2015 10:13 AM, Sjoerd Simons wrote:
> > 
> > > > Well my thoughts on the matter are above, If folks feel strongly about
> > > > this approach being the wrong way I'd love to hear their arguments :).
> > > 
> > > One issue with this approach is that there's no way for the user to 
> > > short-circuit the scanning. If I put a ChromiumOS install on an SD card 
> > > and leave it plugged into a system that's going to end up booting from 
> > > eMMC since that's where the boot files are, there are lots of partitions 
> > > to scan on that SD card, which will be a bit annoying.
> > 
> > 
> > I don't remember exactly how many partitions with fat/ext* filesystems a
> > ChromiumOS installation has (order of 3-5 iirc?), but indeed it means
> > your boot will be a bit slower due to it probing more partitions.
> > Wouldn't expect it to significantly slow down the total boot time
> > though.
> 
> I thought u-boot would scan that partitions which were marked bootable
> first, in which case you just need to set the bit correctly in the
> partition table. I might be wrong about that though. (and re-reading
> $subject, it seems like changing this is the subject of the thread...)

Nope current u-boot just always uses the first partition. My changes
make it scan all the partition on the device instead to find a viable
set of bootfiles. As you can see in the previous mails, i do have some
dislike for using the legacy bootable flag for this purpose..

> > I didn't think of this one my WIP is on an Odroid X2 which has a boot
> > selector jumper, so I have it always starting from mmc0 (which is either
> > SD or EMMC depending on the jumper setting). 
> > 
> > However, it raises an interesting question. The current convention for
> > Exynos is to first scans external storage (SD, mmc 1) and then internal
> > storage (eMMC, mmc 0), which opens up a whole different can of worms. As
> > that means that e.g. my chromebook will try to boot from whatever random
> > SD i've put into it first rather the OS installed on eMMC.  It would be
> > nice to have some general guidelines in this area so the behaviour of
> > various boards can be somewhat consistent in the default behaviour.
> 
> My understanding was that the various ${boot_*} envvars, including
> boot_targets, are considered to be user serviceable parts. IOW if you
> want to boot from mmc0 only then:
>         setenv boot_targets mmc0
>         saveenv
> 
> Maybe it makes sense for the default boot order to differ depending on
> the specific exynos platform though?

Sure, which is something that really should get some documentation (I've
had that on my mental today, together with documenting what boot scripts
can expect wrt. variables set).. 

This is about the default setup though, it would be really nice to get
consistent behaviour. I would be inclined to say that the defaults
should conservatively try the internal/main storage first (assuming
there will be an OS is installed there) and only fallback to other
options later. 

-- 
Sjoerd Simons <sjoerd.simons@collabora.co.uk>
Collabora Ltd.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 6170 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150107/7f6c804e/attachment.bin>

[U-Boot] [PATCH 3/4] config_distro_bootcmd: Scan all partitions for boot files

[Ian Campbell]

On Wed, 2015-01-07 at 12:01 +0100, Sjoerd Simons wrote:
> On Wed, 2015-01-07 at 10:22 +0000, Ian Campbell wrote:
> > On Wed, 2015-01-07 at 11:10 +0100, Sjoerd Simons wrote:
> > > On Tue, 2015-01-06 at 17:43 -0700, Stephen Warren wrote:
> > > > (CCing Dennis so he can comment from a distro perspective re: partition 
> > > > table bootable flags v.s. scanning all partitions)
> > > > 
> > > > On 01/06/2015 10:07 AM, Sjoerd Simons wrote:
> > > > > On Mon, 2015-01-05 at 13:24 -0700, Stephen Warren wrote:
> > > > >> On 01/05/2015 10:13 AM, Sjoerd Simons wrote:
> > > 
> > > > > Well my thoughts on the matter are above, If folks feel strongly about
> > > > > this approach being the wrong way I'd love to hear their arguments :).
> > > > 
> > > > One issue with this approach is that there's no way for the user to 
> > > > short-circuit the scanning. If I put a ChromiumOS install on an SD card 
> > > > and leave it plugged into a system that's going to end up booting from 
> > > > eMMC since that's where the boot files are, there are lots of partitions 
> > > > to scan on that SD card, which will be a bit annoying.
> > > 
> > > 
> > > I don't remember exactly how many partitions with fat/ext* filesystems a
> > > ChromiumOS installation has (order of 3-5 iirc?), but indeed it means
> > > your boot will be a bit slower due to it probing more partitions.
> > > Wouldn't expect it to significantly slow down the total boot time
> > > though.
> > 
> > I thought u-boot would scan that partitions which were marked bootable
> > first, in which case you just need to set the bit correctly in the
> > partition table. I might be wrong about that though. (and re-reading
> > $subject, it seems like changing this is the subject of the thread...)
> 
> Nope current u-boot just always uses the first partition. My changes
> make it scan all the partition on the device instead to find a viable
> set of bootfiles. As you can see in the previous mails, i do have some
> dislike for using the legacy bootable flag for this purpose..

I think when using a legacy partition table it is fine to pay attention
to that bit, it's not legacy in that context.

WRT GPT, I think "legacy" in that context can reasonably be inferred to
mean "non-EFI", and u-boot isn't EFI so I don't think it is so very
wrong for u-boot to pay some attention to it, or at least search those
first.

GPT defines bit 1, as "No Block IO Protocol", so if you disagree with
treating u-boot as "not EFI", perhaps you'd instead prefer to be "EFI
compatible" to the extent of honouring that bit.

> > > I didn't think of this one my WIP is on an Odroid X2 which has a boot
> > > selector jumper, so I have it always starting from mmc0 (which is either
> > > SD or EMMC depending on the jumper setting). 
> > > 
> > > However, it raises an interesting question. The current convention for
> > > Exynos is to first scans external storage (SD, mmc 1) and then internal
> > > storage (eMMC, mmc 0), which opens up a whole different can of worms. As
> > > that means that e.g. my chromebook will try to boot from whatever random
> > > SD i've put into it first rather the OS installed on eMMC.  It would be
> > > nice to have some general guidelines in this area so the behaviour of
> > > various boards can be somewhat consistent in the default behaviour.
> > 
> > My understanding was that the various ${boot_*} envvars, including
> > boot_targets, are considered to be user serviceable parts. IOW if you
> > want to boot from mmc0 only then:
> >         setenv boot_targets mmc0
> >         saveenv
> > 
> > Maybe it makes sense for the default boot order to differ depending on
> > the specific exynos platform though?
> 
> Sure, which is something that really should get some documentation (I've
> had that on my mental today, together with documenting what boot scripts
> can expect wrt. variables set).. 

http://patchwork.ozlabs.org/patch/425412/ covers all this, I think.

> This is about the default setup though, it would be really nice to get
> consistent behaviour. I would be inclined to say that the defaults
> should conservatively try the internal/main storage first (assuming
> there will be an OS is installed there) and only fallback to other
> options later. 

I'm inclined the other way, which is to boot of a removable media first
if someone has gone to the effort to plug one in. People building kiosks
etc who want to lock it down to internal only can still do so.

Ian.

[U-Boot] [PATCH 3/4] config_distro_bootcmd: Scan all partitions for boot files

[Sjoerd Simons]

On Wed, 2015-01-07 at 11:17 +0000, Ian Campbell wrote:
> On Wed, 2015-01-07 at 12:01 +0100, Sjoerd Simons wrote:
> > On Wed, 2015-01-07 at 10:22 +0000, Ian Campbell wrote:
> > > On Wed, 2015-01-07 at 11:10 +0100, Sjoerd Simons wrote:
> > > > On Tue, 2015-01-06 at 17:43 -0700, Stephen Warren wrote:
> > > > > (CCing Dennis so he can comment from a distro perspective re: partition 
> > > > > table bootable flags v.s. scanning all partitions)
> > > > > 
> > > > > On 01/06/2015 10:07 AM, Sjoerd Simons wrote:
> > > > > > On Mon, 2015-01-05 at 13:24 -0700, Stephen Warren wrote:
> > > > > >> On 01/05/2015 10:13 AM, Sjoerd Simons wrote:
> > > > 
> > > > > > Well my thoughts on the matter are above, If folks feel strongly about
> > > > > > this approach being the wrong way I'd love to hear their arguments :).
> > > > > 
> > > > > One issue with this approach is that there's no way for the user to 
> > > > > short-circuit the scanning. If I put a ChromiumOS install on an SD card 
> > > > > and leave it plugged into a system that's going to end up booting from 
> > > > > eMMC since that's where the boot files are, there are lots of partitions 
> > > > > to scan on that SD card, which will be a bit annoying.
> > > > 
> > > > 
> > > > I don't remember exactly how many partitions with fat/ext* filesystems a
> > > > ChromiumOS installation has (order of 3-5 iirc?), but indeed it means
> > > > your boot will be a bit slower due to it probing more partitions.
> > > > Wouldn't expect it to significantly slow down the total boot time
> > > > though.
> > > 
> > > I thought u-boot would scan that partitions which were marked bootable
> > > first, in which case you just need to set the bit correctly in the
> > > partition table. I might be wrong about that though. (and re-reading
> > > $subject, it seems like changing this is the subject of the thread...)
> > 
> > Nope current u-boot just always uses the first partition. My changes
> > make it scan all the partition on the device instead to find a viable
> > set of bootfiles. As you can see in the previous mails, i do have some
> > dislike for using the legacy bootable flag for this purpose..
> 
> I think when using a legacy partition table it is fine to pay attention
> to that bit, it's not legacy in that context.
> 
> WRT GPT, I think "legacy" in that context can reasonably be inferred to
> mean "non-EFI", and u-boot isn't EFI so I don't think it is so very
> wrong for u-boot to pay some attention to it, or at least search those
> first.
> 
> GPT defines bit 1, as "No Block IO Protocol", so if you disagree with
> treating u-boot as "not EFI", perhaps you'd instead prefer to be "EFI
> compatible" to the extent of honouring that bit.

I prefer to treat GPT as a partition table layout seperate from EFI. In
the GPT context, i tend to read "legacy" as "Intel BIOS". But that's
really opinions. Respecting bit 1 would be a nice improvement though. 

Wrt. the boot flag. I can be convinced that partitions with that flag
should be prioritized in their scanning above others, although i'm
really not sure how valuable that actually is (i don't think it is).
What i'm opposed to would be *only* searching partitions with that flag
as that just tends to be unnecessarily error-prone.


> > > > I didn't think of this one my WIP is on an Odroid X2 which has a boot
> > > > selector jumper, so I have it always starting from mmc0 (which is either
> > > > SD or EMMC depending on the jumper setting). 
> > > > 
> > > > However, it raises an interesting question. The current convention for
> > > > Exynos is to first scans external storage (SD, mmc 1) and then internal
> > > > storage (eMMC, mmc 0), which opens up a whole different can of worms. As
> > > > that means that e.g. my chromebook will try to boot from whatever random
> > > > SD i've put into it first rather the OS installed on eMMC.  It would be
> > > > nice to have some general guidelines in this area so the behaviour of
> > > > various boards can be somewhat consistent in the default behaviour.
> > > 
> > > My understanding was that the various ${boot_*} envvars, including
> > > boot_targets, are considered to be user serviceable parts. IOW if you
> > > want to boot from mmc0 only then:
> > >         setenv boot_targets mmc0
> > >         saveenv
> > > 
> > > Maybe it makes sense for the default boot order to differ depending on
> > > the specific exynos platform though?
> > 
> > Sure, which is something that really should get some documentation (I've
> > had that on my mental today, together with documenting what boot scripts
> > can expect wrt. variables set).. 
> 
> http://patchwork.ozlabs.org/patch/425412/ covers all this, I think.

Oh awesome, didn't see that patch yet. Thanks for the pointer.

> > This is about the default setup though, it would be really nice to get
> > consistent behaviour. I would be inclined to say that the defaults
> > should conservatively try the internal/main storage first (assuming
> > there will be an OS is installed there) and only fallback to other
> > options later. 
> 
> I'm inclined the other way, which is to boot of a removable media first
> if someone has gone to the effort to plug one in. People building kiosks
> etc who want to lock it down to internal only can still do so.

This is the general problem of the user doing an action but the system
really being unable to devine the intention behind that action.. An
external storage device can have been plugged in for a lot of reasons,
not just to boot from (you might have wnated to copy/inspect/change some
data on the card). I would argue that the main reason for folks to plug
in external storage devices into computers is not because they'd like to
boot from it and furthermore that a lot of people would be confused if
leaving in a storage card/stick makes the system unbootable.

I suspect this argument comes down to how you expect users to normally
use the system.. Iotw as a general computing device, just like your
laptop or desktop machine or as a development toy where you regularly
boot from external media (I would expect the former, but that's just
me).


-- 
Sjoerd Simons <sjoerd.simons@collabora.co.uk>
Collabora Ltd.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 6170 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150107/62449504/attachment.bin>

[U-Boot] [PATCH 3/4] config_distro_bootcmd: Scan all partitions for boot files

[Ian Campbell]

On Wed, 2015-01-07 at 12:46 +0100, Sjoerd Simons wrote:
> On Wed, 2015-01-07 at 11:17 +0000, Ian Campbell wrote:
> > On Wed, 2015-01-07 at 12:01 +0100, Sjoerd Simons wrote:
> > > On Wed, 2015-01-07 at 10:22 +0000, Ian Campbell wrote:
> > > > On Wed, 2015-01-07 at 11:10 +0100, Sjoerd Simons wrote:
> > > > > On Tue, 2015-01-06 at 17:43 -0700, Stephen Warren wrote:
> > > > > > (CCing Dennis so he can comment from a distro perspective re: partition 
> > > > > > table bootable flags v.s. scanning all partitions)
> > > > > > 
> > > > > > On 01/06/2015 10:07 AM, Sjoerd Simons wrote:
> > > > > > > On Mon, 2015-01-05 at 13:24 -0700, Stephen Warren wrote:
> > > > > > >> On 01/05/2015 10:13 AM, Sjoerd Simons wrote:
> > > > > 
> > > > > > > Well my thoughts on the matter are above, If folks feel strongly about
> > > > > > > this approach being the wrong way I'd love to hear their arguments :).
> > > > > > 
> > > > > > One issue with this approach is that there's no way for the user to 
> > > > > > short-circuit the scanning. If I put a ChromiumOS install on an SD card 
> > > > > > and leave it plugged into a system that's going to end up booting from 
> > > > > > eMMC since that's where the boot files are, there are lots of partitions 
> > > > > > to scan on that SD card, which will be a bit annoying.
> > > > > 
> > > > > 
> > > > > I don't remember exactly how many partitions with fat/ext* filesystems a
> > > > > ChromiumOS installation has (order of 3-5 iirc?), but indeed it means
> > > > > your boot will be a bit slower due to it probing more partitions.
> > > > > Wouldn't expect it to significantly slow down the total boot time
> > > > > though.
> > > > 
> > > > I thought u-boot would scan that partitions which were marked bootable
> > > > first, in which case you just need to set the bit correctly in the
> > > > partition table. I might be wrong about that though. (and re-reading
> > > > $subject, it seems like changing this is the subject of the thread...)
> > > 
> > > Nope current u-boot just always uses the first partition. My changes
> > > make it scan all the partition on the device instead to find a viable
> > > set of bootfiles. As you can see in the previous mails, i do have some
> > > dislike for using the legacy bootable flag for this purpose..
> > 
> > I think when using a legacy partition table it is fine to pay attention
> > to that bit, it's not legacy in that context.
> > 
> > WRT GPT, I think "legacy" in that context can reasonably be inferred to
> > mean "non-EFI", and u-boot isn't EFI so I don't think it is so very
> > wrong for u-boot to pay some attention to it, or at least search those
> > first.
> > 
> > GPT defines bit 1, as "No Block IO Protocol", so if you disagree with
> > treating u-boot as "not EFI", perhaps you'd instead prefer to be "EFI
> > compatible" to the extent of honouring that bit.
> 
> I prefer to treat GPT as a partition table layout seperate from EFI. In
> the GPT context, i tend to read "legacy" as "Intel BIOS". But that's
> really opinions. Respecting bit 1 would be a nice improvement though. 
> 
> Wrt. the boot flag. I can be convinced that partitions with that flag
> should be prioritized in their scanning above others, although i'm
> really not sure how valuable that actually is (i don't think it is).
> What i'm opposed to would be *only* searching partitions with that flag
> as that just tends to be unnecessarily error-prone.

I agree it'd be better to eventually scan everything, but I do think
it'd be useful to follow any hints the partition table might offer.

> > > This is about the default setup though, it would be really nice to get
> > > consistent behaviour. I would be inclined to say that the defaults
> > > should conservatively try the internal/main storage first (assuming
> > > there will be an OS is installed there) and only fallback to other
> > > options later. 
> > 
> > I'm inclined the other way, which is to boot of a removable media first
> > if someone has gone to the effort to plug one in. People building kiosks
> > etc who want to lock it down to internal only can still do so.
> 
> This is the general problem of the user doing an action but the system
> really being unable to devine the intention behind that action.. An
> external storage device can have been plugged in for a lot of reasons,
> not just to boot from (you might have wnated to copy/inspect/change some
> data on the card). I would argue that the main reason for folks to plug
> in external storage devices into computers is not because they'd like to
> boot from it and furthermore that a lot of people would be confused if
> leaving in a storage card/stick makes the system unbootable.

If the external media is present but unbootable then I'd expect it to
(by default) try the internal media next, not fail to boot.

> I suspect this argument comes down to how you expect users to normally
> use the system.. Iotw as a general computing device, just like your
> laptop or desktop machine or as a development toy where you regularly
> boot from external media (I would expect the former, but that's just
> me).

A laptop or desktop machine is typically configured (from the factory at
least) to boot from the optical device first if present (and bootable),
and it's not uncommon for the e.g. a USB stick to be ahead of the hdd in
the default boot order.

But that's not necessarily relevant here, in the context of e.g. arndale
the "external mmc" is an sdcard slot, which it is perfectly reasonable
to boot from and use as a rootfs, and in many cases more convenient
because you can install it from some other host by e.g. dd-ing a distro
provided image to it, or using debootstrap etc. It a bit different from
what one might call "removable media", even if it does happen to be
removable.

Ian.

[U-Boot] [PATCH 3/4] config_distro_bootcmd: Scan all partitions for boot files

[Dennis Gilmore]

On Wed, 07 Jan 2015 12:46:20 +0100
Sjoerd Simons <sjoerd.simons@collabora.co.uk> wrote:

> 
> > > This is about the default setup though, it would be really nice
> > > to get consistent behaviour. I would be inclined to say that the
> > > defaults should conservatively try the internal/main storage
> > > first (assuming there will be an OS is installed there) and only
> > > fallback to other options later. 
> > 
> > I'm inclined the other way, which is to boot of a removable media
> > first if someone has gone to the effort to plug one in. People
> > building kiosks etc who want to lock it down to internal only can
> > still do so.
> 
> This is the general problem of the user doing an action but the system
> really being unable to devine the intention behind that action.. An
> external storage device can have been plugged in for a lot of reasons,
> not just to boot from (you might have wnated to copy/inspect/change
> some data on the card). I would argue that the main reason for folks
> to plug in external storage devices into computers is not because
> they'd like to boot from it and furthermore that a lot of people
> would be confused if leaving in a storage card/stick makes the system
> unbootable.
> 
> I suspect this argument comes down to how you expect users to normally
> use the system.. Iotw as a general computing device, just like your
> laptop or desktop machine or as a development toy where you regularly
> boot from external media (I would expect the former, but that's just
> me).

Well the intention here is for general computing devices. Where when
you get a machine you would pxe install or boot from a install image
provided by the os vendor. I think we should follow how x86 works and
default to external media first, then internal media followed by
network. All with the user being able to override the settings either
permanently or in a one off fashion.

Dennis

[U-Boot] [PATCH 3/4] config_distro_bootcmd: Scan all partitions for boot files

[Stephen Warren]

On 01/07/2015 04:17 AM, Ian Campbell wrote:
> On Wed, 2015-01-07 at 12:01 +0100, Sjoerd Simons wrote:
>> On Wed, 2015-01-07 at 10:22 +0000, Ian Campbell wrote:
>>> On Wed, 2015-01-07 at 11:10 +0100, Sjoerd Simons wrote:
>>>> On Tue, 2015-01-06 at 17:43 -0700, Stephen Warren wrote:
>>>>> (CCing Dennis so he can comment from a distro perspective re: partition
>>>>> table bootable flags v.s. scanning all partitions)
>>>>>
>>>>> On 01/06/2015 10:07 AM, Sjoerd Simons wrote:
>>>>>> On Mon, 2015-01-05 at 13:24 -0700, Stephen Warren wrote:
>>>>>>> On 01/05/2015 10:13 AM, Sjoerd Simons wrote:
>>>>
>>>>>> Well my thoughts on the matter are above, If folks feel strongly about
>>>>>> this approach being the wrong way I'd love to hear their arguments :).
>>>>>
>>>>> One issue with this approach is that there's no way for the user to
>>>>> short-circuit the scanning. If I put a ChromiumOS install on an SD card
>>>>> and leave it plugged into a system that's going to end up booting from
>>>>> eMMC since that's where the boot files are, there are lots of partitions
>>>>> to scan on that SD card, which will be a bit annoying.
>>>>
>>>>
>>>> I don't remember exactly how many partitions with fat/ext* filesystems a
>>>> ChromiumOS installation has (order of 3-5 iirc?), but indeed it means
>>>> your boot will be a bit slower due to it probing more partitions.
>>>> Wouldn't expect it to significantly slow down the total boot time
>>>> though.
>>>
>>> I thought u-boot would scan that partitions which were marked bootable
>>> first, in which case you just need to set the bit correctly in the
>>> partition table. I might be wrong about that though. (and re-reading
>>> $subject, it seems like changing this is the subject of the thread...)
>>
>> Nope current u-boot just always uses the first partition. My changes
>> make it scan all the partition on the device instead to find a viable
>> set of bootfiles. As you can see in the previous mails, i do have some
>> dislike for using the legacy bootable flag for this purpose..
>
> I think when using a legacy partition table it is fine to pay attention
> to that bit, it's not legacy in that context.
>
> WRT GPT, I think "legacy" in that context can reasonably be inferred to
> mean "non-EFI", and u-boot isn't EFI so I don't think it is so very
> wrong for u-boot to pay some attention to it, or at least search those
> first.
>
> GPT defines bit 1, as "No Block IO Protocol", so if you disagree with
> treating u-boot as "not EFI", perhaps you'd instead prefer to be "EFI
> compatible" to the extent of honouring that bit.
>
>>>> I didn't think of this one my WIP is on an Odroid X2 which has a boot
>>>> selector jumper, so I have it always starting from mmc0 (which is either
>>>> SD or EMMC depending on the jumper setting).
>>>>
>>>> However, it raises an interesting question. The current convention for
>>>> Exynos is to first scans external storage (SD, mmc 1) and then internal
>>>> storage (eMMC, mmc 0), which opens up a whole different can of worms. As
>>>> that means that e.g. my chromebook will try to boot from whatever random
>>>> SD i've put into it first rather the OS installed on eMMC.  It would be
>>>> nice to have some general guidelines in this area so the behaviour of
>>>> various boards can be somewhat consistent in the default behaviour.
>>>
>>> My understanding was that the various ${boot_*} envvars, including
>>> boot_targets, are considered to be user serviceable parts. IOW if you
>>> want to boot from mmc0 only then:
>>>          setenv boot_targets mmc0
>>>          saveenv
>>>
>>> Maybe it makes sense for the default boot order to differ depending on
>>> the specific exynos platform though?
>>
>> Sure, which is something that really should get some documentation (I've
>> had that on my mental today, together with documenting what boot scripts
>> can expect wrt. variables set)..
>
> http://patchwork.ozlabs.org/patch/425412/ covers all this, I think.
>
>> This is about the default setup though, it would be really nice to get
>> consistent behaviour. I would be inclined to say that the defaults
>> should conservatively try the internal/main storage first (assuming
>> there will be an OS is installed there) and only fallback to other
>> options later.
>
> I'm inclined the other way, which is to boot of a removable media first
> if someone has gone to the effort to plug one in. People building kiosks
> etc who want to lock it down to internal only can still do so.

Yes, I agree. This means that if you already have a (perhaps broken or 
old) distro installed on the internal media, you can place boot media 
into the external slot and boot that without having to fiddle with 
modifying the boot configuration variables. If that behaviour isn't what 
a particular user wants, they can just edit $boot_targets.

Each platform can make their own decision if they want though; the order 
of entries in BOOT_TARGET_DEVICES (part of the U-Boot config header 
file) determines the default order of entries in $boot_targets.

[U-Boot] [PATCH 3/4] config_distro_bootcmd: Scan all partitions for boot files

[Sjoerd Simons]

On Wed, 2015-01-07 at 13:22 -0700, Stephen Warren wrote:
> On 01/07/2015 04:17 AM, Ian Campbell wrote:
> > On Wed, 2015-01-07 at 12:01 +0100, Sjoerd Simons wrote:
> >> On Wed, 2015-01-07 at 10:22 +0000, Ian Campbell wrote:

> >> This is about the default setup though, it would be really nice to get
> >> consistent behaviour. I would be inclined to say that the defaults
> >> should conservatively try the internal/main storage first (assuming
> >> there will be an OS is installed there) and only fallback to other
> >> options later.
> >
> > I'm inclined the other way, which is to boot of a removable media first
> > if someone has gone to the effort to plug one in. People building kiosks
> > etc who want to lock it down to internal only can still do so.
> 
> Yes, I agree. This means that if you already have a (perhaps broken or 
> old) distro installed on the internal media, you can place boot media 
> into the external slot and boot that without having to fiddle with 
> modifying the boot configuration variables. If that behaviour isn't what 
> a particular user wants, they can just edit $boot_targets.
> 
> Each platform can make their own decision if they want though; the order 
> of entries in BOOT_TARGET_DEVICES (part of the U-Boot config header 
> file) determines the default order of entries in $boot_targets.

Fair enough. One thing i would like to mention though that respective of
the preference here, It would be nice to have some guidelines here.

One thing which always irks me is that for every new ARM i get on my
desk, I need to re-discover how this particular one happens to boot..
The distro bootcmd stuff goes a very long way here, but would be even
nicer if the sequence was predictable as well (e.g. always external,
internal, network boots unless the board has a boot selector switch).
(One can wish right :p)

-- 
Sjoerd Simons <sjoerd.simons@collabora.co.uk>
Collabora Ltd.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 6170 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150108/e2f4c038/attachment.bin>

[U-Boot] [PATCH 3/4] config_distro_bootcmd: Scan all partitions for boot files

[Stephen Warren]

On 01/07/2015 03:10 AM, Sjoerd Simons wrote:
> On Tue, 2015-01-06 at 17:43 -0700, Stephen Warren wrote:
>> (CCing Dennis so he can comment from a distro perspective re: partition
>> table bootable flags v.s. scanning all partitions)
>>
>> On 01/06/2015 10:07 AM, Sjoerd Simons wrote:
>>> On Mon, 2015-01-05 at 13:24 -0700, Stephen Warren wrote:
>>>> On 01/05/2015 10:13 AM, Sjoerd Simons wrote:
>
>>> Well my thoughts on the matter are above, If folks feel strongly about
>>> this approach being the wrong way I'd love to hear their arguments :).
>>
>> One issue with this approach is that there's no way for the user to
>> short-circuit the scanning. If I put a ChromiumOS install on an SD card
>> and leave it plugged into a system that's going to end up booting from
>> eMMC since that's where the boot files are, there are lots of partitions
>> to scan on that SD card, which will be a bit annoying.
>
>
> I don't remember exactly how many partitions with fat/ext* filesystems a
> ChromiumOS installation has (order of 3-5 iirc?), but indeed it means
> your boot will be a bit slower due to it probing more partitions.
> Wouldn't expect it to significantly slow down the total boot time
> though.

IIRC something more like 12-16, at least for the developer builds I have 
used in the past.

> I didn't think of this one my WIP is on an Odroid X2 which has a boot
> selector jumper, so I have it always starting from mmc0 (which is either
> SD or EMMC depending on the jumper setting).
>
> However, it raises an interesting question. The current convention for
> Exynos is to first scans external storage (SD, mmc 1) and then internal
> storage (eMMC, mmc 0), which opens up a whole different can of worms. As
> that means that e.g. my chromebook will try to boot from whatever random
> SD i've put into it first rather the OS installed on eMMC.  It would be
> nice to have some general guidelines in this area so the behaviour of
> various boards can be somewhat consistent in the default behaviour.
> (Added Ian Cambell to the cc as he introduce the usage on exynos
> devices)

The user can select which devices get looked at by setting $boot_targets 
to a list of devices names, e.g. "mmc1 mmc0 usb pxe".

[U-Boot] [PATCH 3/4] config_distro_bootcmd: Scan all partitions for boot files

[Dennis Gilmore]

On Tue, 06 Jan 2015 17:43:19 -0700
Stephen Warren <swarren@wwwdotorg.org> wrote:

> (CCing Dennis so he can comment from a distro perspective re:
> partition table bootable flags v.s. scanning all partitions)
> 
> On 01/06/2015 10:07 AM, Sjoerd Simons wrote:
> > On Mon, 2015-01-05 at 13:24 -0700, Stephen Warren wrote:
> >> On 01/05/2015 10:13 AM, Sjoerd Simons wrote:
> >>> Not all devices use the convention that the boot scripts are on
> >>> the first partition. For example on chromebooks it seems common
> >>> for the first two partitions to be ChromeOS kernel partitions.

ChromeOS seems to have adopted its own unique setup. it is not a
typical configuration.

> >>>
> >>> So instead of just the first partition scan all partitions on a
> >>> device with a filesystem u-boot can recognize.
> >>
> >> I had planned (but obviously never got around to...) enhancing the
> >> scripts to look up the (set of?) bootable partition(s) on the disk
> >> and to attempt to load the boot files from there. Bootable would
> >> be defined as the MBR bootable flag, or GPT legacy bootable
> >> attribute.
> >>
> >> That would allow the code to zero in on the one specific partition
> >> that it was supposed to look at, rather than searching all
> >> partitions.
> >>
> >> Do you have any thoughts re: which option is better?
> >
> > I did wonder about this as well. I do personally consider the
> > bootable flag as a rather obsolete/legacy thing (GPT even specifies
> > it as a legacy flag), so i was wary about using it.. Also i've been
> > bitten a few times on systems that did rely on the bootable flag
> > (what, what, why does it not boot, oooooohhhh), which was another
> > reason for heading this route.

I really like the idea of using the bootable flag and looking at it but
if its legacy in GPT will it go away in some future partition table
layout? UEFI Requires that a ESP exist. I think requiring that the
bootable flag exist is acceptable.

> > This way does no extra work if the first partition is the partition
> > with the boot partition when compared to only checking partitions
> > with the bootable flag as both would need to list existing
> > partitions.
> >
> > If the first few partitions have no filesystems, the extra work
> > compared to the bootable-flag approach would just be probing the
> > filesystem type, which tends to be relatively simple, so i don't
> > see a big issue there (it's more work to scan for a missing boot
> > file).
> >
> > If your first few partitions are ones without the bootfiles, some
> > more effort is wasted as it will be probing those for viable boot
> > files.. However, in my experience, partition layouts with the
> > bootfiles not on the first filesystem partitions is rather
> > uncommmon. So again, i didn't feel that that was problematic. If
> > you have an odd parition layout, your boot time will be ever so
> > slightly longer :)
> >
> > The only "issue" in my mind is when multiple partitions, for
> > whatever reason, have bootfiles. In which case the first one will
> > get picked with this approach, while with the partition-boot-flag
> > approach you'd have a way to specify, no really just look at that
> > one.. However, i suspect the likelihood of forgetting to set the
> > boot flag is higher (been there, done that) then accidentally
> > leaving boot files on partitions before the intended boot partition
> > (which also requires on uncommon layout), so even then i suspect
> > this approach is more friendly/less error-prone.
> >
> >> This patch looks fine assuming this option (rather than bootable
> >> flag) is selected.
> >
> > Well my thoughts on the matter are above, If folks feel strongly
> > about this approach being the wrong way I'd love to hear their
> > arguments :).
> 
> One issue with this approach is that there's no way for the user to 
> short-circuit the scanning. If I put a ChromiumOS install on an SD
> card and leave it plugged into a system that's going to end up
> booting from eMMC since that's where the boot files are, there are
> lots of partitions to scan on that SD card, which will be a bit
> annoying.

That is what happens on x86 today though. if you had a bootable
cdrom/dvdrom or usb stick it will boot from that before the local
install.

Dennis

[U-Boot] [PATCH 3/4] config_distro_bootcmd: Scan all partitions for boot files

[Stephen Warren]

On 01/10/2015 11:27 AM, Dennis Gilmore wrote:
> On Tue, 06 Jan 2015 17:43:19 -0700
> Stephen Warren <swarren@wwwdotorg.org> wrote:
>
>> (CCing Dennis so he can comment from a distro perspective re:
>> partition table bootable flags v.s. scanning all partitions)
>>
>> On 01/06/2015 10:07 AM, Sjoerd Simons wrote:
>>> On Mon, 2015-01-05 at 13:24 -0700, Stephen Warren wrote:
>>>> On 01/05/2015 10:13 AM, Sjoerd Simons wrote:
>>>>> Not all devices use the convention that the boot scripts are on
>>>>> the first partition. For example on chromebooks it seems common
>>>>> for the first two partitions to be ChromeOS kernel partitions.
>
> ChromeOS seems to have adopted its own unique setup. it is not a
> typical configuration.
>
>>>>>
>>>>> So instead of just the first partition scan all partitions on a
>>>>> device with a filesystem u-boot can recognize.
>>>>
>>>> I had planned (but obviously never got around to...) enhancing the
>>>> scripts to look up the (set of?) bootable partition(s) on the disk
>>>> and to attempt to load the boot files from there. Bootable would
>>>> be defined as the MBR bootable flag, or GPT legacy bootable
>>>> attribute.
>>>>
>>>> That would allow the code to zero in on the one specific partition
>>>> that it was supposed to look at, rather than searching all
>>>> partitions.
>>>>
>>>> Do you have any thoughts re: which option is better?
>>>
>>> I did wonder about this as well. I do personally consider the
>>> bootable flag as a rather obsolete/legacy thing (GPT even specifies
>>> it as a legacy flag), so i was wary about using it.. Also i've been
>>> bitten a few times on systems that did rely on the bootable flag
>>> (what, what, why does it not boot, oooooohhhh), which was another
>>> reason for heading this route.
>
> I really like the idea of using the bootable flag and looking at it but
> if its legacy in GPT will it go away in some future partition table
> layout? UEFI Requires that a ESP exist. I think requiring that the
> bootable flag exist is acceptable.

One other alternative for GPT is to invent a new partition type UUID for 
bootable partitions. This likely has more implications though, since any 
tool that looks at the partition type UUID would have to be updated. I 
have no idea how many such tools exist though.

>>> This way does no extra work if the first partition is the partition
>>> with the boot partition when compared to only checking partitions
>>> with the bootable flag as both would need to list existing
>>> partitions.
>>>
>>> If the first few partitions have no filesystems, the extra work
>>> compared to the bootable-flag approach would just be probing the
>>> filesystem type, which tends to be relatively simple, so i don't
>>> see a big issue there (it's more work to scan for a missing boot
>>> file).
>>>
>>> If your first few partitions are ones without the bootfiles, some
>>> more effort is wasted as it will be probing those for viable boot
>>> files.. However, in my experience, partition layouts with the
>>> bootfiles not on the first filesystem partitions is rather
>>> uncommmon. So again, i didn't feel that that was problematic. If
>>> you have an odd parition layout, your boot time will be ever so
>>> slightly longer :)
>>>
>>> The only "issue" in my mind is when multiple partitions, for
>>> whatever reason, have bootfiles. In which case the first one will
>>> get picked with this approach, while with the partition-boot-flag
>>> approach you'd have a way to specify, no really just look at that
>>> one.. However, i suspect the likelihood of forgetting to set the
>>> boot flag is higher (been there, done that) then accidentally
>>> leaving boot files on partitions before the intended boot partition
>>> (which also requires on uncommon layout), so even then i suspect
>>> this approach is more friendly/less error-prone.
>>>
>>>> This patch looks fine assuming this option (rather than bootable
>>>> flag) is selected.
>>>
>>> Well my thoughts on the matter are above, If folks feel strongly
>>> about this approach being the wrong way I'd love to hear their
>>> arguments :).
>>
>> One issue with this approach is that there's no way for the user to
>> short-circuit the scanning. If I put a ChromiumOS install on an SD
>> card and leave it plugged into a system that's going to end up
>> booting from eMMC since that's where the boot files are, there are
>> lots of partitions to scan on that SD card, which will be a bit
>> annoying.
>
> That is what happens on x86 today though. if you had a bootable
> cdrom/dvdrom or usb stick it will boot from that before the local
> install.

x86 doesn't search all the partitions though, only those marked with the 
bootable flag. That's why I'm trying to drive the standard distro boot 
process (as implemented by U-Boot) to honor the bootable flag and ignore 
other partitions.

[U-Boot] [PATCH 3/4] config_distro_bootcmd: Scan all partitions for boot files

[Dennis Gilmore]

On Mon, 12 Jan 2015 10:42:27 -0700
Stephen Warren <swarren@wwwdotorg.org> wrote:

> On 01/10/2015 11:27 AM, Dennis Gilmore wrote:
> > On Tue, 06 Jan 2015 17:43:19 -0700
> > Stephen Warren <swarren@wwwdotorg.org> wrote:
> >
> >> (CCing Dennis so he can comment from a distro perspective re:
> >> partition table bootable flags v.s. scanning all partitions)
> >>
> >> On 01/06/2015 10:07 AM, Sjoerd Simons wrote:
> >>> On Mon, 2015-01-05 at 13:24 -0700, Stephen Warren wrote:
> >>>> On 01/05/2015 10:13 AM, Sjoerd Simons wrote:
> >>>>> Not all devices use the convention that the boot scripts are on
> >>>>> the first partition. For example on chromebooks it seems common
> >>>>> for the first two partitions to be ChromeOS kernel partitions.
> >
> > ChromeOS seems to have adopted its own unique setup. it is not a
> > typical configuration.
> >
> >>>>>
> >>>>> So instead of just the first partition scan all partitions on a
> >>>>> device with a filesystem u-boot can recognize.
> >>>>
> >>>> I had planned (but obviously never got around to...) enhancing
> >>>> the scripts to look up the (set of?) bootable partition(s) on
> >>>> the disk and to attempt to load the boot files from there.
> >>>> Bootable would be defined as the MBR bootable flag, or GPT
> >>>> legacy bootable attribute.
> >>>>
> >>>> That would allow the code to zero in on the one specific
> >>>> partition that it was supposed to look at, rather than searching
> >>>> all partitions.
> >>>>
> >>>> Do you have any thoughts re: which option is better?
> >>>
> >>> I did wonder about this as well. I do personally consider the
> >>> bootable flag as a rather obsolete/legacy thing (GPT even
> >>> specifies it as a legacy flag), so i was wary about using it..
> >>> Also i've been bitten a few times on systems that did rely on the
> >>> bootable flag (what, what, why does it not boot, oooooohhhh),
> >>> which was another reason for heading this route.
> >
> > I really like the idea of using the bootable flag and looking at it
> > but if its legacy in GPT will it go away in some future partition
> > table layout? UEFI Requires that a ESP exist. I think requiring
> > that the bootable flag exist is acceptable.
> 
> One other alternative for GPT is to invent a new partition type UUID
> for bootable partitions. This likely has more implications though,
> since any tool that looks at the partition type UUID would have to be
> updated. I have no idea how many such tools exist though.

or perhaps use the ESP flag. though that might be totally confusing for
all.


> >>
> >> One issue with this approach is that there's no way for the user to
> >> short-circuit the scanning. If I put a ChromiumOS install on an SD
> >> card and leave it plugged into a system that's going to end up
> >> booting from eMMC since that's where the boot files are, there are
> >> lots of partitions to scan on that SD card, which will be a bit
> >> annoying.
> >
> > That is what happens on x86 today though. if you had a bootable
> > cdrom/dvdrom or usb stick it will boot from that before the local
> > install.
> 
> x86 doesn't search all the partitions though, only those marked with
> the bootable flag. That's why I'm trying to drive the standard distro
> boot process (as implemented by U-Boot) to honor the bootable flag
> and ignore other partitions.

Right, bios uses the bootable flag, UEFI uses the ESP partition which
is why I guess GPT has the bootable flag as a legacy option. I'm in
agreement with you on honouring the bootable flag. I was just trying to
point out that if you put say a usb stick in a machine that had a live
image installed on it that's what the x86 system would boot.

Dennis

[U-Boot] [PATCH 3/4] config_distro_bootcmd: Scan all partitions for boot files

[Sjoerd Simons]

On Mon, 2015-01-12 at 12:44 -0600, Dennis Gilmore wrote:
> On Mon, 12 Jan 2015 10:42:27 -0700
> Stephen Warren <swarren@wwwdotorg.org> wrote:
> 
> > On 01/10/2015 11:27 AM, Dennis Gilmore wrote:
> > > On Tue, 06 Jan 2015 17:43:19 -0700
> > > Stephen Warren <swarren@wwwdotorg.org> wrote:
> > >
> > >> (CCing Dennis so he can comment from a distro perspective re:
> > >> partition table bootable flags v.s. scanning all partitions)
> > >>
> > >> On 01/06/2015 10:07 AM, Sjoerd Simons wrote:
> > >>> On Mon, 2015-01-05 at 13:24 -0700, Stephen Warren wrote:
> > >>>> On 01/05/2015 10:13 AM, Sjoerd Simons wrote:
> > >>>>> Not all devices use the convention that the boot scripts are on
> > >>>>> the first partition. For example on chromebooks it seems common
> > >>>>> for the first two partitions to be ChromeOS kernel partitions.
> > >
> > > ChromeOS seems to have adopted its own unique setup. it is not a
> > > typical configuration.

Sure, However most installation guides for linux on chromeos will still
tell you have the first 2 partitions as kernel ones (Which is also where
you put u-boot if you want to load that without writing it to flash).

In any case, this was just an example to indicate why hardcoding
partition one is nasty.

> > >>>>> So instead of just the first partition scan all partitions on a
> > >>>>> device with a filesystem u-boot can recognize.
> > >>>>
> > >>>> I had planned (but obviously never got around to...) enhancing
> > >>>> the scripts to look up the (set of?) bootable partition(s) on
> > >>>> the disk and to attempt to load the boot files from there.
> > >>>> Bootable would be defined as the MBR bootable flag, or GPT
> > >>>> legacy bootable attribute.
> > >>>>
> > >>>> That would allow the code to zero in on the one specific
> > >>>> partition that it was supposed to look at, rather than searching
> > >>>> all partitions.
> > >>>>
> > >>>> Do you have any thoughts re: which option is better?
> > >>>
> > >>> I did wonder about this as well. I do personally consider the
> > >>> bootable flag as a rather obsolete/legacy thing (GPT even
> > >>> specifies it as a legacy flag), so i was wary about using it..
> > >>> Also i've been bitten a few times on systems that did rely on the
> > >>> bootable flag (what, what, why does it not boot, oooooohhhh),
> > >>> which was another reason for heading this route.
> > >
> > > I really like the idea of using the bootable flag and looking at it
> > > but if its legacy in GPT will it go away in some future partition
> > > table layout? UEFI Requires that a ESP exist. I think requiring
> > > that the bootable flag exist is acceptable.
> > 
> > One other alternative for GPT is to invent a new partition type UUID
> > for bootable partitions. This likely has more implications though,
> > since any tool that looks at the partition type UUID would have to be
> > updated. I have no idea how many such tools exist though.
> 
> or perhaps use the ESP flag. though that might be totally confusing for
> all.

That seems quite confusing indeed. Also ESP partitions are required to
be vfat afaik, which is rather unconvenient. At least in the Debian
world, the package management tools get rather annoyed if /boot isn't a
posix compatible filesystem.

Defining a new partition type UUID would be more GPT-style, there is a
bit of prior art here:
http://www.freedesktop.org/wiki/Specifications/DiscoverablePartitionsSpec/

But that doesn't really define a boot partition currently, apart from
suggesting to mount the ESP on /boot. Something to take up with the
systemd folks.

> > >> One issue with this approach is that there's no way for the user to
> > >> short-circuit the scanning. If I put a ChromiumOS install on an SD
> > >> card and leave it plugged into a system that's going to end up
> > >> booting from eMMC since that's where the boot files are, there are
> > >> lots of partitions to scan on that SD card, which will be a bit
> > >> annoying.
> > >
> > > That is what happens on x86 today though. if you had a bootable
> > > cdrom/dvdrom or usb stick it will boot from that before the local
> > > install.
> > 
> > x86 doesn't search all the partitions though, only those marked with
> > the bootable flag. That's why I'm trying to drive the standard distro
> > boot process (as implemented by U-Boot) to honor the bootable flag
> > and ignore other partitions.
> 
> Right, bios uses the bootable flag, UEFI uses the ESP partition which
> is why I guess GPT has the bootable flag as a legacy option. I'm in
> agreement with you on honouring the bootable flag. I was just trying to
> point out that if you put say a usb stick in a machine that had a live
> image installed on it that's what the x86 system would boot.

Ok. To summarize a bit, it seems the overall consensus thusfar is that
u-boot should honor the bootflag both on old msdos style partitions and
GPT partitions. So basically folks don't agree with my dislike of the
boot flag (ah well, such is life).

So the next step for me would be to update this set adding ``part
listbootable'' command, which does the same as part list, but filters on
partitions which are bootable (as defined by the bootable flag). And
change this particular patch to use listbootable instead of list.

Dennis, Ian, should we keep trying partition 1 as a fallback or does the
current Fedora/Debian installers set this flag on new ARM installations
already? (I guess in the Debian world a NEWS entry for the behaviour
change might be enough, as afaik the u-boot package doesn't
automagically reflash itself just yet)


In case a new GPT UUID is defined, this can be added as a partition that
will show up in listbootable in future, it won't require changes to the
boot_cmds themselves.

-- 
Sjoerd Simons <sjoerd.simons@collabora.co.uk>
Collabora Ltd.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 6170 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150113/3f9b9d8f/attachment.bin>

[U-Boot] [PATCH 3/4] config_distro_bootcmd: Scan all partitions for boot files

[Stephen Warren]

On 01/13/2015 01:40 AM, Sjoerd Simons wrote:
> On Mon, 2015-01-12 at 12:44 -0600, Dennis Gilmore wrote:
>> On Mon, 12 Jan 2015 10:42:27 -0700
>> Stephen Warren <swarren@wwwdotorg.org> wrote:
>>
>>> On 01/10/2015 11:27 AM, Dennis Gilmore wrote:
>>>> On Tue, 06 Jan 2015 17:43:19 -0700
>>>> Stephen Warren <swarren@wwwdotorg.org> wrote:
>>>>
>>>>> (CCing Dennis so he can comment from a distro perspective re:
>>>>> partition table bootable flags v.s. scanning all partitions)
>>>>>
>>>>> On 01/06/2015 10:07 AM, Sjoerd Simons wrote:
>>>>>> On Mon, 2015-01-05 at 13:24 -0700, Stephen Warren wrote:
>>>>>>> On 01/05/2015 10:13 AM, Sjoerd Simons wrote:
>>>>>>>> Not all devices use the convention that the boot scripts are on
>>>>>>>> the first partition. For example on chromebooks it seems common
>>>>>>>> for the first two partitions to be ChromeOS kernel partitions.
>>>>
>>>> ChromeOS seems to have adopted its own unique setup. it is not a
>>>> typical configuration.
>
> Sure, However most installation guides for linux on chromeos will still
> tell you have the first 2 partitions as kernel ones (Which is also where
> you put u-boot if you want to load that without writing it to flash).
>
> In any case, this was just an example to indicate why hardcoding
> partition one is nasty.
>
>>>>>>>> So instead of just the first partition scan all partitions on a
>>>>>>>> device with a filesystem u-boot can recognize.
>>>>>>>
>>>>>>> I had planned (but obviously never got around to...) enhancing
>>>>>>> the scripts to look up the (set of?) bootable partition(s) on
>>>>>>> the disk and to attempt to load the boot files from there.
>>>>>>> Bootable would be defined as the MBR bootable flag, or GPT
>>>>>>> legacy bootable attribute.
>>>>>>>
>>>>>>> That would allow the code to zero in on the one specific
>>>>>>> partition that it was supposed to look at, rather than searching
>>>>>>> all partitions.
>>>>>>>
>>>>>>> Do you have any thoughts re: which option is better?
>>>>>>
>>>>>> I did wonder about this as well. I do personally consider the
>>>>>> bootable flag as a rather obsolete/legacy thing (GPT even
>>>>>> specifies it as a legacy flag), so i was wary about using it..
>>>>>> Also i've been bitten a few times on systems that did rely on the
>>>>>> bootable flag (what, what, why does it not boot, oooooohhhh),
>>>>>> which was another reason for heading this route.
>>>>
>>>> I really like the idea of using the bootable flag and looking at it
>>>> but if its legacy in GPT will it go away in some future partition
>>>> table layout? UEFI Requires that a ESP exist. I think requiring
>>>> that the bootable flag exist is acceptable.
>>>
>>> One other alternative for GPT is to invent a new partition type UUID
>>> for bootable partitions. This likely has more implications though,
>>> since any tool that looks at the partition type UUID would have to be
>>> updated. I have no idea how many such tools exist though.
>>
>> or perhaps use the ESP flag. though that might be totally confusing for
>> all.
>
> That seems quite confusing indeed. Also ESP partitions are required to
> be vfat afaik, which is rather unconvenient. At least in the Debian
> world, the package management tools get rather annoyed if /boot isn't a
> posix compatible filesystem.
>
> Defining a new partition type UUID would be more GPT-style, there is a
> bit of prior art here:
> http://www.freedesktop.org/wiki/Specifications/DiscoverablePartitionsSpec/
>
> But that doesn't really define a boot partition currently, apart from
> suggesting to mount the ESP on /boot. Something to take up with the
> systemd folks.

One other issue is that there may be no separate /boot partition. We 
don't want to force the existence of a separate partition just so that 
it can have the bootable type UUID.

>>>>> One issue with this approach is that there's no way for the user to
>>>>> short-circuit the scanning. If I put a ChromiumOS install on an SD
>>>>> card and leave it plugged into a system that's going to end up
>>>>> booting from eMMC since that's where the boot files are, there are
>>>>> lots of partitions to scan on that SD card, which will be a bit
>>>>> annoying.
>>>>
>>>> That is what happens on x86 today though. if you had a bootable
>>>> cdrom/dvdrom or usb stick it will boot from that before the local
>>>> install.
>>>
>>> x86 doesn't search all the partitions though, only those marked with
>>> the bootable flag. That's why I'm trying to drive the standard distro
>>> boot process (as implemented by U-Boot) to honor the bootable flag
>>> and ignore other partitions.
>>
>> Right, bios uses the bootable flag, UEFI uses the ESP partition which
>> is why I guess GPT has the bootable flag as a legacy option. I'm in
>> agreement with you on honouring the bootable flag. I was just trying to
>> point out that if you put say a usb stick in a machine that had a live
>> image installed on it that's what the x86 system would boot.
>
> Ok. To summarize a bit, it seems the overall consensus thusfar is that
> u-boot should honor the bootflag both on old msdos style partitions and
> GPT partitions. So basically folks don't agree with my dislike of the
> boot flag (ah well, such is life).
>
> So the next step for me would be to update this set adding ``part
> listbootable'' command, which does the same as part list, but filters on
> partitions which are bootable (as defined by the bootable flag). And
> change this particular patch to use listbootable instead of list.

I'd suggest a "part list" command that lists all partition IDs, with 
optional options to restrict the set of partitions returned 
"-bootable-flag" would likely be all we implement to start with, but we 
could later implement "-type fat"/"-type ext" etc.

> Dennis, Ian, should we keep trying partition 1 as a fallback or does the
> current Fedora/Debian installers set this flag on new ARM installations
> already? (I guess in the Debian world a NEWS entry for the behaviour
> change might be enough, as afaik the u-boot package doesn't
> automagically reflash itself just yet)

Falling back to partition 1 seems like a good fail-safe, it the list 
returned by "part list" was empty at least.

> In case a new GPT UUID is defined, this can be added as a partition that
> will show up in listbootable in future, it won't require changes to the
> boot_cmds themselves.

Yup.

[U-Boot] [U-Boot, 3/4] config_distro_bootcmd: Scan all partitions for boot files

[Tom Rini]

On Mon, Jan 05, 2015 at 06:13:38PM +0100, Sjoerd Simons wrote:

> Not all devices use the convention that the boot scripts are on the
> first partition. For example on chromebooks it seems common for the
> first two partitions to be ChromeOS kernel partitions.
> 
> So instead of just the first partition scan all partitions on a device
> with a filesystem u-boot can recognize.
> 
> Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150202/4f8288ac/attachment.sig>

[U-Boot] [PATCH 4/4] distro_distro_bootcmd: use CONFIG_BOOTCOMMAND instead of setting bootcmd=

[Sjoerd Simons]

Move the bootcmd commands into a seperate distro_bootcmd environment
variable. Allowing a user to easily launch the distro boot sequence if
the default bootcmd did not default to distro boot commands.

Also set CONFIG_BOOTCOMMAND to "run distro_bootcmd" if it hasn't been
configured yet rather then putting it directly in the environment. This
allows boards to make the distro boot commands available without
necessarily default to them or to use them as a fallback after running
some board specific commands instead.

Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>
---
 include/config_distro_bootcmd.h | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/include/config_distro_bootcmd.h b/include/config_distro_bootcmd.h
index 68f5fcb..793e4d7 100644
--- a/include/config_distro_bootcmd.h
+++ b/include/config_distro_bootcmd.h
@@ -210,9 +210,14 @@
 	\
 	BOOT_TARGET_DEVICES(BOOTENV_DEV)                                  \
 	\
-	"bootcmd=" BOOTENV_SET_USB_NEED_INIT BOOTENV_SET_SCSI_NEED_INIT   \
+	"distro_bootcmd=" BOOTENV_SET_USB_NEED_INIT                       \
+		BOOTENV_SET_SCSI_NEED_INIT                                \
 		"for target in ${boot_targets}; do "                      \
 			"run bootcmd_${target}; "                         \
 		"done\0"
 
+#ifndef CONFIG_BOOTCOMMAND
+#define CONFIG_BOOTCOMMAND "run distro_bootcmd"
+#endif
+
 #endif  /* _CONFIG_CMD_DISTRO_BOOTCMD_H */
-- 
2.1.4

[U-Boot] [PATCH 4/4] distro_distro_bootcmd: use CONFIG_BOOTCOMMAND instead of setting bootcmd=

[Stephen Warren]

On 01/05/2015 10:13 AM, Sjoerd Simons wrote:
> Move the bootcmd commands into a seperate distro_bootcmd environment
> variable. Allowing a user to easily launch the distro boot sequence if
> the default bootcmd did not default to distro boot commands.

Well, the user can just type "boot" or "run bootcmd", so I don't think 
this makes it much easier for the user:-P

> Also set CONFIG_BOOTCOMMAND to "run distro_bootcmd" if it hasn't been
> configured yet rather then putting it directly in the environment. This
> allows boards to make the distro boot commands available without
> necessarily default to them or to use them as a fallback after running
> some board specific commands instead.

Reviewed-by: Stephen Warren <swarren@nvidia.com>

As an aside, I actually have a local commit that amends the bootcmd 
value to include BOOTENV_HOOK_BOOTCMD_PRE before the current value, 
which also allows boards to do arbitrary things before executing the 
distro boot code. Still, I think your way is more flexible, since it 
allows board-provided scripts to invoke $distro_bootcmd when/whereever 
they want. I'll rework my code on top of this.

[U-Boot] [PATCH 4/4] distro_distro_bootcmd: use CONFIG_BOOTCOMMAND instead of setting bootcmd=

[Sjoerd Simons]

On Mon, 2015-01-05 at 13:31 -0700, Stephen Warren wrote:
> On 01/05/2015 10:13 AM, Sjoerd Simons wrote:
> > Move the bootcmd commands into a seperate distro_bootcmd environment
> > variable. Allowing a user to easily launch the distro boot sequence if
> > the default bootcmd did not default to distro boot commands.
> 
> Well, the user can just type "boot" or "run bootcmd", so I don't think 
> this makes it much easier for the user:-P

What i meant was if the bootcmd environment variable is set to something
different then the distro bootcmds, you can now simply type "run
distro_bootcmd" to get that behaviour :)

> > Also set CONFIG_BOOTCOMMAND to "run distro_bootcmd" if it hasn't been
> > configured yet rather then putting it directly in the environment. This
> > allows boards to make the distro boot commands available without
> > necessarily default to them or to use them as a fallback after running
> > some board specific commands instead.
> 
> Reviewed-by: Stephen Warren <swarren@nvidia.com>

Thanks for the review!

> As an aside, I actually have a local commit that amends the bootcmd 
> value to include BOOTENV_HOOK_BOOTCMD_PRE before the current value, 
> which also allows boards to do arbitrary things before executing the 
> distro boot code. Still, I think your way is more flexible, since it 
> allows board-provided scripts to invoke $distro_bootcmd when/whereever 
> they want. I'll rework my code on top of this.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 6170 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150106/a2e365f0/attachment.bin>

[U-Boot] [U-Boot, 4/4] distro_distro_bootcmd: use CONFIG_BOOTCOMMAND instead of setting bootcmd=

[Tom Rini]

On Mon, Jan 05, 2015 at 06:13:39PM +0100, Sjoerd Simons wrote:

> Move the bootcmd commands into a seperate distro_bootcmd environment
> variable. Allowing a user to easily launch the distro boot sequence if
> the default bootcmd did not default to distro boot commands.
> 
> Also set CONFIG_BOOTCOMMAND to "run distro_bootcmd" if it hasn't been
> configured yet rather then putting it directly in the environment. This
> allows boards to make the distro boot commands available without
> necessarily default to them or to use them as a fallback after running
> some board specific commands instead.
> 
> Signed-off-by: Sjoerd Simons <sjoerd.simons@collabora.co.uk>
> Reviewed-by: Stephen Warren <swarren@nvidia.com>

Applied to u-boot/master (with a minor tweak as the USB change isn't in
yet), thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150202/3ccdf710/attachment.sig>