blkid -o value

blkid -o value

[Michael Conrad]

Hi, I've just had a semi-serious problem related to blkid's "-o value" 
format.

When I first discovered that output format, I assumed that it would be a 
stable script-friendly output format intended for people who don't want 
to parse the name/value pairs.  The fields appeared to be
   LABEL
   UUID
   TYPE
and I wrote a script assuming the third one was TYPE.

However, suddenly the output has changed to be
   LABEL
   UUID
   SEC_TYPE <-- "ext2"
   TYPE <-- "ext3"

which caused my script to mount a ext3 partition as ext2, killing the 
journal.

Looking at the code, it appears that it iterates over some list coming 
from libblkid and prints a line with the value only if the value 
exists.  So it appears that there was never any intention of it being a 
stable output.

I would request that either the order of fields become stable by 
printing empty lines for missing fields (and documented), or the "-o 
value" be done away with, since I can't see any reason anyone would want 
random output in a script-friendly format, and the fact that it exists 
is likely to lead people astray.

Thoughts?  Comments?

-Mike Conrad

Re: blkid -o value

[Ted Ts'o]

On Tue, Mar 27, 2012 at 01:26:14PM -0400, Michael Conrad wrote:
> 
> I would request that either the order of fields become stable by
> printing empty lines for missing fields (and documented), or the "-o
> value" be done away with, since I can't see any reason anyone would
> want random output in a script-friendly format, and the fact that it
> exists is likely to lead people astray.

Try using this way, which was its intended usage:

blkid -o value -s TYPE -t UUID=a3dc6f56-da22-4079-bf75-1fa2ce1aa4b0

(replace part following -t with the search criteria you want; for
example, "-t LABEL=backup").

It was intended to be used a script-friendly way, and I believe there
are shell scripts using in this fashion.  So please don't change blkid
-o value.

Best regards,

						- Ted

Re: blkid -o value

[Michael Conrad]

On 3/27/2012 1:31 PM, Ted Ts'o wrote:
> On Tue, Mar 27, 2012 at 01:26:14PM -0400, Michael Conrad wrote:
>> I would request that either the order of fields become stable by
>> printing empty lines for missing fields (and documented), or the "-o
>> value" be done away with, since I can't see any reason anyone would
>> want random output in a script-friendly format, and the fact that it
>> exists is likely to lead people astray.
> Try using this way, which was its intended usage:
>
> blkid -o value -s TYPE -t UUID=a3dc6f56-da22-4079-bf75-1fa2ce1aa4b0
>
> (replace part following -t with the search criteria you want; for
> example, "-t LABEL=backup").
>
> It was intended to be used a script-friendly way, and I believe there
> are shell scripts using in this fashion.  So please don't change blkid
> -o value.

I see how that could be useful if and only if you use exactly one "-s", 
but if you requested more than one "-s" they still come out in 
non-stable order.  It's like running a SQL query and having all the 
columns shifted because the first one was a NULL.  If you're on a system 
where all the volumes have labels, types, and uuids (and nothing else) 
it's very misleading.

At the very least, I would appreciate it if the man page mentioned that 
"-o value" should only ever be used with exactly one "-s".

-Mike

Re: blkid -o value

[Ted Ts'o]

On Tue, Mar 27, 2012 at 01:52:51PM -0400, Michael Conrad wrote:
> 
> I see how that could be useful if and only if you use exactly one
> "-s", but if you requested more than one "-s" they still come out in
> non-stable order.  It's like running a SQL query and having all the
> columns shifted because the first one was a NULL.  

Actually, it's more like running an SQL query without a SORTED BY and
assuming the order of the rows was stable.

> At the very least, I would appreciate it if the man page mentioned
> that "-o value" should only ever be used with exactly one "-s".

What I would suggest is that blkid explicitly state that the order of
fields or file systems (if more than one file system is selected) is
not defined, since that's something which is true in general, and not
just for "-o value".

It might also be useful to have an EXAMPLES section which gives some
sample ways that blkid can be used.

Regards,

						- Ted

Re: blkid -o value

[Karel Zak]

On Tue, Mar 27, 2012 at 01:26:14PM -0400, Michael Conrad wrote:
> which caused my script to mount a ext3 partition as ext2, killing the
> journal.

 BTW, why do you need to call blkid before mount? The command mount(8)
 is able to detect FS type by libblkid.

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: blkid -o value

[Karel Zak]

On Tue, Mar 27, 2012 at 11:43:45AM -0700, Ted Ts'o wrote:
> On Tue, Mar 27, 2012 at 01:52:51PM -0400, Michael Conrad wrote:
> > 
> > I see how that could be useful if and only if you use exactly one
> > "-s", but if you requested more than one "-s" they still come out in
> > non-stable order.  It's like running a SQL query and having all the
> > columns shifted because the first one was a NULL.  
> 
> Actually, it's more like running an SQL query without a SORTED BY and
> assuming the order of the rows was stable.
> 
> > At the very least, I would appreciate it if the man page mentioned
> > that "-o value" should only ever be used with exactly one "-s".
> 
> What I would suggest is that blkid explicitly state that the order of
> fields or file systems (if more than one file system is selected) is
> not defined, since that's something which is true in general, and not
> just for "-o value".
> 
> It might also be useful to have an EXAMPLES section which gives some
> sample ways that blkid can be used.

 Good idea. I'll update the man page.

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: blkid -o value

[Michael Conrad]

[-- Attachment #1: Type: text/plain, Size: 1786 bytes --]

On 03/28/2012 05:00 AM, Karel Zak wrote:
> On Tue, Mar 27, 2012 at 11:43:45AM -0700, Ted Ts'o wrote:
>> On Tue, Mar 27, 2012 at 01:52:51PM -0400, Michael Conrad wrote:
>>> I see how that could be useful if and only if you use exactly one
>>> "-s", but if you requested more than one "-s" they still come out in
>>> non-stable order.  It's like running a SQL query and having all the
>>> columns shifted because the first one was a NULL.  
>> Actually, it's more like running an SQL query without a SORTED BY and
>> assuming the order of the rows was stable.
Only if the data weren't in any normal form.  I say it's like columns
getting shifted because if you ask for 2 values you expect the field
offset to imply the type of the data.  If one field is NULL, the rows
shift upward, resulting in a value interpreted as the wrong type.

This is why I'm saying the "-o values" is only usable if you use exactly
one "-s", because any other scenario means every value returned has an
indeterminate type, which is garbage data.  IMO a tool should never
output garbage.  It should prevent or warn about illegitimate usage. 
Since the tool has been out there and in use for quite a while, I would
suggest a warning.

(patch attached)

>>> At the very least, I would appreciate it if the man page mentioned
>>> that "-o value" should only ever be used with exactly one "-s".
>> What I would suggest is that blkid explicitly state that the order of
>> fields or file systems (if more than one file system is selected) is
>> not defined, since that's something which is true in general, and not
>> just for "-o value".
>>
>> It might also be useful to have an EXAMPLES section which gives some
>> sample ways that blkid can be used.
>  Good idea. I'll update the man page.
>
>     Karel

Thanks!



[-- Attachment #2: blkid.patch --]
[-- Type: text/plain, Size: 1510 bytes --]

>From 0415f6dcb3c5636a69499b0accdd0e2fae9ac0c3 Mon Sep 17 00:00:00 2001
From: Michael Conrad <mconrad@intellitree.com>
Date: Wed, 28 Mar 2012 16:51:12 -0400
Subject: [PATCH] blkid: added warning about unstable output for '-o values'

 * A warning is now emitted when using 'values' without requesting
   a single tag.

 * Updated man page to describe the reason behind it.
---
 misc-utils/blkid.8 |    3 ++-
 misc-utils/blkid.c |    3 +++
 2 files changed, 5 insertions(+), 1 deletions(-)

diff --git a/misc-utils/blkid.8 b/misc-utils/blkid.8
index 6bde6bc..f6175b2 100644
--- a/misc-utils/blkid.8
+++ b/misc-utils/blkid.8
@@ -152,7 +152,8 @@ parameter may be:
 print all tags (the default)
 .TP
 .B value
-print the value of the tags
+print the value of the tags.  The tag order is variable, so you should only
+use this format when selecting a single tag for a single device. (see '-s')
 .TP
 .B list
 print the devices in a user-friendly format; this output format is unsupported
diff --git a/misc-utils/blkid.c b/misc-utils/blkid.c
index 1e8073e..4e5b338 100644
--- a/misc-utils/blkid.c
+++ b/misc-utils/blkid.c
@@ -829,6 +829,9 @@ int main(int argc, char **argv)
 		print_version(stdout);
 		goto exit;
 	}
+	
+	if (output_format == OUTPUT_VALUE_ONLY && numtag != 1)
+		fprintf(stderr, "Warning: '-o value' should only be used when requesting a single tag ('-s')\n");
 
 	/* convert LABEL/UUID lookup to evaluate request */
 	if (lookup && output_format == OUTPUT_DEVICE_ONLY && search_type &&
-- 
1.7.1