test if a subvolume is a snapshot?

test if a subvolume is a snapshot?

[Ulli Horlacher]

How can I test if a subvolume is a snapshot?

Example:

/test/.snapshot/2017-09-08_1037.single is a snapshot (of /test)
/test/data is a regular subvolume

I know this, because I have created them with suitable names :-)

But how can I see/test it?

root@fex:~/bin# btrfs subvol show /test/.snapshot/2017-09-08_1037.single
/test/.snapshot/2017-09-08_1037.single
        Name:                   2017-09-08_1037.single
        UUID:                   eff1cebb-b885-6b47-ae69-36a7c3f266eb
        Parent UUID:            -
        Received UUID:          -
        Creation time:          2017-09-08 10:37:04 +0200
        Subvolume ID:           354
        Generation:             237
        Gen at creation:        237
        Parent ID:              5
        Top level ID:           5
        Flags:                  readonly
        Snapshot(s):

root@fex:~/bin# btrfs subvol show /test/data
/test/data
        Name:                   data
        UUID:                   b32a5949-dfd6-ef45-8616-34ae4cdf6fb8
        Parent UUID:            -
        Received UUID:          -
        Creation time:          2017-09-06 18:30:33 +0200
        Subvolume ID:           257
        Generation:             224
        Gen at creation:        8
        Parent ID:              5
        Top level ID:           5
        Flags:                  -

root@fex:~/bin# btrfs subvol show /test
/test is toplevel subvolume

-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum TIK         
Universitaet Stuttgart         E-Mail: horlacher@tik.uni-stuttgart.de
Allmandring 30a                Tel:    ++49-711-68565868
70569 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/
REF:<20170908085446.GA7876@rus.uni-stuttgart.de>

Re: test if a subvolume is a snapshot?

[Peter Grandi]

> How can I test if a subvolume is a snapshot? [ ... ]

This question is based on the assumption that "snapshot" is a
distinct type of subvolume and not just an operation that
creates a subvolume with reflinked contents.

Unfortunately Btrfs does indeed make snapshots a distinct type
of subvolume... In my 4.4 kernel/progs version of Btrfs it seems
that the 'Parent UUID' is that of the source of the snapshot,
and the source of a snapshot somehow comes with a list to all
the snapshots taken from it:

  #  ls /fs/sda7
  =        @170826  @170829  @170901  @170903  @170905  @170907
  @170825  @170828  @170830  @170902  @170904  @170906  lost+found

  #  btrfs subvolume list /fs/sda7
  ID 431 gen 532441 top level 5 path =
  ID 1619 gen 524915 top level 5 path @170825
  ID 1649 gen 524915 top level 5 path @170826
  ID 1651 gen 524915 top level 5 path @170828
  ID 1652 gen 524915 top level 5 path @170829
  ID 1654 gen 524915 top level 5 path @170830
  ID 1655 gen 523316 top level 5 path @170901
  ID 1656 gen 524034 top level 5 path @170902
  ID 1658 gen 525628 top level 5 path @170903
  ID 1659 gen 527121 top level 5 path @170904
  ID 1660 gen 528719 top level 5 path @170905
  ID 1665 gen 530565 top level 5 path @170906
  ID 1666 gen 532217 top level 5 path @170907

  #  btrfs subvolume show /fs/sda7/= | egrep 'UUID|Parent|Top level|Snap|@'
	  UUID:                   cb99579f-64e5-e94c-b22c-41dcc397c37f
	  Parent UUID:            -
	  Received UUID:          -
	  Parent ID:              5
	  Top level ID:           5
	  Snapshot(s):
				  @170825
				  @170826
				  @170828
				  @170829
				  @170830
				  @170901
				  @170902
				  @170903
				  @170904
				  @170905
				  @170906
				  @170907

  #  btrfs subvolume show /fs/sda7/@170901 | egrep 'UUID|Parent|Top level|Snap|@'
  /fs/sda7/@170901
	  Name:                   @170901
	  UUID:                   851f8ef3-c2af-4b46-89af-0193fd4e6fc4
	  Parent UUID:            cb99579f-64e5-e94c-b22c-41dcc397c37f
	  Received UUID:          -
	  Parent ID:              5
	  Top level ID:           5
	  Snapshot(s):

Note that with typical Btrfs consistency "Parent UUID" is that
the source of the snapshot, while "Parent ID" is that of the
upper level subvolume, and in the "flat" layout for this volume
the snapshot parent is '/fs/sda7/=' and the upper level is
'/fs/sda7' instead.

The different results that you get make me suspect that the
top-level subvolume is "special".

Re: test if a subvolume is a snapshot?

[David Sterba]

On Fri, Sep 08, 2017 at 10:54:46AM +0200, Ulli Horlacher wrote:
> How can I test if a subvolume is a snapshot?

The inode number is 256 on a btrfs filesystem:

if [ stat -f --format=%T $path = btrfs -a stat --format=%i $path = 256 ]; ...

The directory that's result of snapshotting a subvolume, also called
EMPTY_SUBVOL has inode number 2, but that's not considered a normal
subvolume.

Re: test if a subvolume is a snapshot?

[Tomasz Kłoczko]

On 8 September 2017 at 14:10, David Sterba <dsterba@suse.cz> wrote:
> On Fri, Sep 08, 2017 at 10:54:46AM +0200, Ulli Horlacher wrote:
> > How can I test if a subvolume is a snapshot?
>
> The inode number is 256 on a btrfs filesystem:
>
> if [ stat -f --format=%T $path = btrfs -a stat --format=%i $path = 256 ]; ...

This oneliner shows how much really basic btrfs tools commands syntax
is broken by design :(
Looking on how so freakishly overcomplicated btrfs command syntax is
that command like above is completely unintuitive and unreadable
sometimes I'm really thinking about start rewrite btrfs-progs to make
btrfs basic tools syntax as similar as it is only possible to ZFS zfs,
zpool and zdb commands on using which in +90% cases you can guess how
necessary syntax must look like without looking on man pages.

Any volunteers want to join to help implement something like this?
Maybe someone already started doing this?

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH

Re: test if a subvolume is a snapshot?

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 1637 bytes --]

On Fri, Sep 08, 2017 at 04:25:55PM +0100, Tomasz Kłoczko wrote:
> On 8 September 2017 at 14:10, David Sterba <dsterba@suse.cz> wrote:
> > On Fri, Sep 08, 2017 at 10:54:46AM +0200, Ulli Horlacher wrote:
> > > How can I test if a subvolume is a snapshot?
> >
> > The inode number is 256 on a btrfs filesystem:
> >
> > if [ stat -f --format=%T $path = btrfs -a stat --format=%i $path = 256 ]; ...
> 
> This oneliner shows how much really basic btrfs tools commands syntax
> is broken by design :(
> Looking on how so freakishly overcomplicated btrfs command syntax is
> that command like above is completely unintuitive and unreadable

   This is nothing to do with btrfs tooling. The two commands involved
here are test (aka "[") and stat.

> sometimes I'm really thinking about start rewrite btrfs-progs to make
> btrfs basic tools syntax as similar as it is only possible to ZFS zfs,
> zpool and zdb commands on using which in +90% cases you can guess how
> necessary syntax must look like without looking on man pages.
> 
> Any volunteers want to join to help implement something like this?
> Maybe someone already started doing this?

   The main complaint that can be directed at the btrfs command is
that its output is rarely machine-processable. It would therefore make
sense to have a "--table" or "--structured" mode for output, which
would be more trivially parsable by shell tools.

   Hugo.

-- 
Hugo Mills             | Ceci est un travail pour l'Australien.
hugo@... carfax.org.uk |
http://carfax.org.uk/  |
PGP: E2AB1DE4          |                                 Louison, Delicatessen

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: test if a subvolume is a snapshot?

[Tomasz Kłoczko]

On 8 September 2017 at 16:38, Hugo Mills <hugo@carfax.org.uk> wrote:
[..]
>> sometimes I'm really thinking about start rewrite btrfs-progs to make
>> btrfs basic tools syntax as similar as it is only possible to ZFS zfs,
>> zpool and zdb commands on using which in +90% cases you can guess how
>> necessary syntax must look like without looking on man pages.
>>
>> Any volunteers want to join to help implement something like this?
>> Maybe someone already started doing this?
>
>    The main complaint that can be directed at the btrfs command is
> that its output is rarely machine-processable. It would therefore make
> sense to have a "--table" or "--structured" mode for output, which
> would be more trivially parsable by shell tools.

Output of the btrfs command it is coooooompletely different pair of shoes.
On making btrfs tools similar to ZFS analogues *obviously* output
should be as same similar.
By this would possible to solve complains about unreadable output in one go.

For example zfs command parseable output is possible to generate by
add -p switch in those subcommands where it is needed (no --tables or
--structures .. just one switch).

Instead reinventing the wheel just please try to look first how it is
already done in completely predictable/guessable way:
https://www.freebsd.org/cgi/man.cgi?query=zfs&sektion=8
https://www.freebsd.org/cgi/man.cgi?query=zpool&sektion=8

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH

Re: test if a subvolume is a snapshot?

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 2061 bytes --]

On Fri, Sep 08, 2017 at 05:12:11PM +0100, Tomasz Kłoczko wrote:
> On 8 September 2017 at 16:38, Hugo Mills <hugo@carfax.org.uk> wrote:
> [..]
> >> sometimes I'm really thinking about start rewrite btrfs-progs to make
> >> btrfs basic tools syntax as similar as it is only possible to ZFS zfs,
> >> zpool and zdb commands on using which in +90% cases you can guess how
> >> necessary syntax must look like without looking on man pages.
> >>
> >> Any volunteers want to join to help implement something like this?
> >> Maybe someone already started doing this?
> >
> >    The main complaint that can be directed at the btrfs command is
> > that its output is rarely machine-processable. It would therefore make
> > sense to have a "--table" or "--structured" mode for output, which
> > would be more trivially parsable by shell tools.
> 
> Output of the btrfs command it is coooooompletely different pair of shoes.
> On making btrfs tools similar to ZFS analogues *obviously* output
> should be as same similar.
> By this would possible to solve complains about unreadable output in one go.
> 
> For example zfs command parseable output is possible to generate by
> add -p switch in those subcommands where it is needed (no --tables or
> --structures .. just one switch).

   --tables _is_ one switch.

> Instead reinventing the wheel just please try to look first how it is

   What in what I said was reinventing a wheel? Literally the *only*
thing I was suggesting was adding some option to make the btrfs tool
output more machine-parsable.

   Call the option whatever you like. However, note that there are
probably very few single-letter options which are not used in at least
one of the btrfs tool subcommands.

   Hugo.

-- 
Hugo Mills             | How do you become King? You stand in the marketplace
hugo@... carfax.org.uk | and announce you're going to tax everyone. If you
http://carfax.org.uk/  | get out alive, you're King.
PGP: E2AB1DE4          |                                        Harry Harrison

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: test if a subvolume is a snapshot?

[David Sterba]

On Fri, Sep 08, 2017 at 04:25:55PM +0100, Tomasz Kłoczko wrote:
> On 8 September 2017 at 14:10, David Sterba <dsterba@suse.cz> wrote:
> > On Fri, Sep 08, 2017 at 10:54:46AM +0200, Ulli Horlacher wrote:
> > > How can I test if a subvolume is a snapshot?
> >
> > The inode number is 256 on a btrfs filesystem:
> >
> > if [ stat -f --format=%T $path = btrfs -a stat --format=%i $path = 256 ]; ...
> 
> This oneliner shows how much really basic btrfs tools commands syntax
> is broken by design :(

I could have been more explicit that it's a shell snippet that
implements the check, but the original mail contained some commands so I
expected that replying in shell is appropriate.

> Looking on how so freakishly overcomplicated btrfs command syntax is
> that command like above is completely unintuitive and unreadable
> sometimes I'm really thinking about start rewrite btrfs-progs to make
> btrfs basic tools syntax as similar as it is only possible to ZFS zfs,
> zpool and zdb commands on using which in +90% cases you can guess how
> necessary syntax must look like without looking on man pages.

Do you have examples of overcomplicated commands? The command hierarchy
is similar to what I know about /bin/zfs, sometimes it's deeper that
adds some typing but the shortcuts are allowed so it's IMO not that bad.
Also the features do not map 1:1.

A rewrite gives a lot of space to do the interface right, but effectly
means end of backward compatibility. In the past new commands have been
put anywhere the authors saw fit, now it's the opposite when new
commands do not appear.

Starting a new tool could be useful to gather feedback what commands are
actually desired, though not everything could be implemented in progs.

Re: test if a subvolume is a snapshot?

[David Sterba]

On Fri, Sep 08, 2017 at 03:38:16PM +0000, Hugo Mills wrote:
> > sometimes I'm really thinking about start rewrite btrfs-progs to make
> > btrfs basic tools syntax as similar as it is only possible to ZFS zfs,
> > zpool and zdb commands on using which in +90% cases you can guess how
> > necessary syntax must look like without looking on man pages.
> > 
> > Any volunteers want to join to help implement something like this?
> > Maybe someone already started doing this?
> 
>    The main complaint that can be directed at the btrfs command is
> that its output is rarely machine-processable. It would therefore make
> sense to have a "--table" or "--structured" mode for output, which
> would be more trivially parsable by shell tools.

My plan is to introduce a global options to set various this, also the
output format, eg.

 $ btrfs --format=json subvolume list

that would dump the list in json obviously, more formats could follow.
This requires to switch all printf to a wrapper that would select the
format based on global config. Some of the code is there, eg. the global
option parser and the config structure. The printf transitions can be
done incrementally. All of that is easy IMHO, somebody just needs to do
it. I work on that when there are no other more pressing things to do.

Re: test if a subvolume is a snapshot?

[Tomasz Kłoczko]

On 8 September 2017 at 17:39, David Sterba <dsterba@suse.cz> wrote:
[..]
> My plan is to introduce a global options to set various this, also the
> output format, eg.
>
>  $ btrfs -t bell be om -format=json subvolume list
>
> that would dump the list in json obviously, more formats could follow.
> This requires to switch all printf to a wrapper that would select the
> format based on global config. Some of the code is there, eg. the global
> option parser and the config structure. The printf transitions can be
> done incrementally. All of that is easy IMHO, somebody just needs to do
> it. I work on that when there are no other more pressing things to do.

What in the future json will be abandoned and some new/fancy parseable
output will be most desirable?
Hmm ??
Seems some people never saw or already forgot some old jokes.
http://www.webaugur.com/bazaar/53-what-if-operating-systems-were-airlines.html

"Unix Airlines

Each passenger brings a piece of the airplane and a box of tools to
the airport. They gather on the tarmac*, arguing constantly about what
kind of plane they want to build and how to put it together.
Eventually, *they build several different aircraft*, but give them all
the same name. Some passengers actually *reach their destination*"

So how to apply above to generate json output out of simple list? Try
below command on your linux:

$ awk 'BEGIN {print "{\"data\":["; ORS=""} {if (NR!=1) {print ",\n"};
print "{\"{#DISK}\":\"" $3 "\"}"} END {print "\n]}\n"}'
/proc/diskstats

***This is why no one is asking to transform Linux kernel procfs to
provide json output** as standard or an option.
(try to think few seconds on this)

As long as something is able to produce list with comma or spaces
separated fields to process such list and provide some usable json
output you need only:

$  (echo "foo:foo"; echo "bar:bar") | awk -F: 'BEGIN {print
"{\"data\":["; ORS=""} {if (NR!=1) {print ",\n"}; print
"{\"{#FOO}\":{\""$1"\",\""$2"\"}}"} END {print "\n]}\n"}'
{"data":[
{"{#FOO}":{"foo","foo"}},
{"{#FOO}":{"bar","bar"}}
]}

If you don't like awk you can use jq, sed, perl, python, ruby or
whatever you have/like/want.
However on writing fs tools all what you really *must do* is define
*as simple as it is only possible >stable< interface*.
Only this and nothing more.

Really on writing something so simple like fs tools you don't need to
think about future possibility to solve problem of famine on Earth
(using your code).

kloczek

Re: test if a subvolume is a snapshot?

[Ulli Horlacher]

On Fri 2017-09-08 (15:10), David Sterba wrote:
> On Fri, Sep 08, 2017 at 10:54:46AM +0200, Ulli Horlacher wrote:
> 
> > How can I test if a subvolume is a snapshot?
> 
> The inode number is 256 on a btrfs filesystem:
> 
> if [ stat -f --format=%T $path = btrfs -a stat --format=%i $path = 256 ]; ...
> 
> The directory that's result of snapshotting a subvolume, also called
> EMPTY_SUBVOL has inode number 2, but that's not considered a normal
> subvolume.

This is not true.

root@fex:~# mount | grep /mnt/test
/dev/sdc1 on /mnt/test type btrfs (rw,relatime,space_cache,subvolid=5,subvol=/)

root@fex:~# btrfs subvol show /mnt/test
/mnt/test is toplevel subvolume

root@fex:~# btrfs subvolume snapshot -r /mnt/test /mnt/test/.snapshot/test
Create a readonly snapshot of '/mnt/test' in '/mnt/test/.snapshot/test'

root@fex:~# stat --format=%i /mnt/test
256

root@fex:~# stat --format=%i /mnt/test/.snapshot/test
256

No difference: both have inode=256


-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum TIK         
Universitaet Stuttgart         E-Mail: horlacher@tik.uni-stuttgart.de
Allmandring 30a                Tel:    ++49-711-68565868
70569 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/
REF:<20170908131035.GO31874@twin.jikos.cz>

Re: test if a subvolume is a snapshot?

[David Sterba]

On Fri, Sep 08, 2017 at 07:09:55PM +0100, Tomasz Kłoczko wrote:
> On 8 September 2017 at 17:39, David Sterba <dsterba@suse.cz> wrote:
> [..]
> > My plan is to introduce a global options to set various this, also the
> > output format, eg.
> >
> >  $ btrfs -t bell be om -format=json subvolume list
> >
> > that would dump the list in json obviously, more formats could follow.
> > This requires to switch all printf to a wrapper that would select the
> > format based on global config. Some of the code is there, eg. the global
> > option parser and the config structure. The printf transitions can be
> > done incrementally. All of that is easy IMHO, somebody just needs to do
> > it. I work on that when there are no other more pressing things to do.
> 
> What in the future json will be abandoned and some new/fancy parseable
> output will be most desirable?
> Hmm ??

Then we'll add a new format of the day to --format=yaml, as far as
possible we'll use existing code infrastructure, ie. there will be no
changes on the string emmiting side, we'll just change the "backend".

Json was an example because it's available in the util-linux and this is
also output style and capabilities I'm most likely going to copy (see
eg. lsblk --help). There's plain ascii, list, json outputs, or column
selection, sorting. The global option namespace in 'btrfs' is completely
empty now, so we can add one letter options.

> Seems some people never saw or already forgot some old jokes.
> http://www.webaugur.com/bazaar/53-what-if-operating-systems-were-airlines.html
> 
> "Unix Airlines
> 
> Each passenger brings a piece of the airplane and a box of tools to
> the airport. They gather on the tarmac*, arguing constantly about what
> kind of plane they want to build and how to put it together.
> Eventually, *they build several different aircraft*, but give them all
> the same name. Some passengers actually *reach their destination*"
> 
> So how to apply above to generate json output out of simple list? Try
> below command on your linux:
> 
> $ awk 'BEGIN {print "{\"data\":["; ORS=""} {if (NR!=1) {print ",\n"};
> print "{\"{#DISK}\":\"" $3 "\"}"} END {print "\n]}\n"}'
> /proc/diskstats
> 
> ***This is why no one is asking to transform Linux kernel procfs to
> provide json output** as standard or an option.
> (try to think few seconds on this)
> 
> As long as something is able to produce list with comma or spaces
> separated fields to process such list and provide some usable json
> output you need only:
> 
> $  (echo "foo:foo"; echo "bar:bar") | awk -F: 'BEGIN {print
> "{\"data\":["; ORS=""} {if (NR!=1) {print ",\n"}; print
> "{\"{#FOO}\":{\""$1"\",\""$2"\"}}"} END {print "\n]}\n"}'
> {"data":[
> {"{#FOO}":{"foo","foo"}},
> {"{#FOO}":{"bar","bar"}}
> ]}
> 
> If you don't like awk you can use jq, sed, perl, python, ruby or
> whatever you have/like/want.
> However on writing fs tools all what you really *must do* is define
> *as simple as it is only possible >stable< interface*.
> Only this and nothing more.

The tools' output could consumed by humans and scripts, so the idea is
to make the defaults human readlabe but provide a simple (as a
commandline option) way for scripts to parse just the data.

The libsmartcols that provides the columns and output formatting for
util-linux is rich enough to get the same or very similaro output you
get by the awk script above. And that's not supposed to be the only way.
Say I ask the tool to give me all data it has and parse it in the
script.

Stability of the output is a nice goal, hard to achieve in practice. We
dont't know the future needs but can't freeze the output forever. The
scripting tools you mention are powerful and no fancy formatting we
could add to btrfs will replace them. So if there's a terse plain text
output, the chances are high that script writer will be able to get the
information he wants.

I don't disagree with you and don't see what else is there to argue
about.

Re: test if a subvolume is a snapshot?

[David Sterba]

On Fri, Sep 08, 2017 at 08:41:57PM +0200, Ulli Horlacher wrote:
> On Fri 2017-09-08 (15:10), David Sterba wrote:
> > On Fri, Sep 08, 2017 at 10:54:46AM +0200, Ulli Horlacher wrote:
> > 
> > > How can I test if a subvolume is a snapshot?
> > 
> > The inode number is 256 on a btrfs filesystem:
> > 
> > if [ stat -f --format=%T $path = btrfs -a stat --format=%i $path = 256 ]; ...
> > 
> > The directory that's result of snapshotting a subvolume, also called
> > EMPTY_SUBVOL has inode number 2, but that's not considered a normal
> > subvolume.
> 
> This is not true.

Oh I see, I've read the mail too quickly sorry, not just a subvolume but
really a snapshot. Then the simple inode number check does not work of
cours, the answer is more or less what Peter Grandi replied.

Re: test if a subvolume is a snapshot?

[Austin S. Hemmelgarn]

On 2017-09-08 14:09, Tomasz Kłoczko wrote:
> On 8 September 2017 at 17:39, David Sterba <dsterba@suse.cz> wrote:
> [..]
>> My plan is to introduce a global options to set various this, also the
>> output format, eg.
>>
>>   $ btrfs -t bell be om -format=json subvolume list
>>
>> that would dump the list in json obviously, more formats could follow.
>> This requires to switch all printf to a wrapper that would select the
>> format based on global config. Some of the code is there, eg. the global
>> option parser and the config structure. The printf transitions can be
>> done incrementally. All of that is easy IMHO, somebody just needs to do
>> it. I work on that when there are no other more pressing things to do.
> 
> What in the future json will be abandoned and some new/fancy parseable
> output will be most desirable?
> Hmm ??
> Seems some people never saw or already forgot some old jokes.
> http://www.webaugur.com/bazaar/53-what-if-operating-systems-were-airlines.html
> 
> "Unix Airlines
> 
> Each passenger brings a piece of the airplane and a box of tools to
> the airport. They gather on the tarmac*, arguing constantly about what
> kind of plane they want to build and how to put it together.
> Eventually, *they build several different aircraft*, but give them all
> the same name. Some passengers actually *reach their destination*"
> 
> So how to apply above to generate json output out of simple list? Try
> below command on your linux:
> 
> $ awk 'BEGIN {print "{\"data\":["; ORS=""} {if (NR!=1) {print ",\n"};
> print "{\"{#DISK}\":\"" $3 "\"}"} END {print "\n]}\n"}'
> /proc/diskstats
> 
> ***This is why no one is asking to transform Linux kernel procfs to
> provide json output** as standard or an option.
> (try to think few seconds on this)
No, the primary reason that nobody is asking to transform procfs to spit 
out JSON is that it's a pain to parse (in a shell, you need excessive 
stuff like the above command, in almost any other language you have to 
write a special parser or use a separate library) when compared to the 
simple space and newline delimited stuff already in proc (in Python for 
example, you just need str.split() and str.splitlines(), both of which 
are part of the core language), and it's insanely more efficient to just 
spit out the data as-is without all the extra stuff required by JSON 
(it's not as bad as XML, but it's still mostly pointless).  The only 
thing that adding JSON support would give people is a way to avoid 
having to use extra tools or kernel documentation to figure out what 
everything is, which is not seen as a significant benefit since almost 
nobody interacts directly with anything in /proc other than developers 
and people poking at sysctls.
> 
> As long as something is able to produce list with comma or spaces
> separated fields to process such list and provide some usable json
> output you need only:
> 
> $  (echo "foo:foo"; echo "bar:bar") | awk -F: 'BEGIN {print
> "{\"data\":["; ORS=""} {if (NR!=1) {print ",\n"}; print
> "{\"{#FOO}\":{\""$1"\",\""$2"\"}}"} END {print "\n]}\n"}'
> {"data":[
> {"{#FOO}":{"foo","foo"}},
> {"{#FOO}":{"bar","bar"}}
> ]}
> 
> If you don't like awk you can use jq, sed, perl, python, ruby or
> whatever you have/like/want.
And which command is more readable?  Something like the above, or:
btrfs device stats --format=json /

Almost anybody is going to say that the second case (with a simple 
option) is more readable, and it's also almost always going to be more 
efficient too since you don't have to fork() an extra time and deal with 
another pipe.  Even using the other tools you mentioned, none of them is 
going to be as trivially easy for people to understand what's going on 
as just having an extra option to control the output format.
> However on writing fs tools all what you really *must do* is define
> *as simple as it is only possible >stable< interface*.
> Only this and nothing more.
With the added constraint that your output should be self consistent and 
sensibly arranged.  The current output from various btrfs commands is 
designed to be human readable, and does not consistently conform to the 
same formatting.  It's also a serious pain to parse without using a 
language with proper string operations (take a look at the output of 
`btrfs device stats` for an example, it uses runs of spaces to separate 
and align fields, which is ridiculous to try and parse in a shell 
script), and the lack of a standard output format means that anything 
interacting with it is inherently fragile.
> 
> Really on writing something so simple like fs tools you don't need to
> think about future possibility to solve problem of famine on Earth
> (using your code).
This is a case though where having something that's more easily machine 
parseable is a significant benefit, because the command-line is the API 
unless you're willing to fight with ioctls.

Re: test if a subvolume is a snapshot?

[Tomasz Kłoczko]

On 8 September 2017 at 20:06, Austin S. Hemmelgarn <ahferroin7@gmail.com> wrote:
[..]
>> If you don't like awk you can use jq, sed, perl, python, ruby or
>> whatever you have/like/want.
>
> And which command is more readable?  Something like the above, or:
> btrfs device stats --format=json /

How many people you know who are communicating in json or using comma
separated lines of text?
Maybe it is not obvious but such interface is not for the humans.

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH

Re: test if a subvolume is a snapshot?

[Austin S. Hemmelgarn]

On 2017-09-08 16:54, Tomasz Kłoczko wrote:
> On 8 September 2017 at 20:06, Austin S. Hemmelgarn <ahferroin7@gmail.com> wrote:
> [..]
>>> If you don't like awk you can use jq, sed, perl, python, ruby or
>>> whatever you have/like/want.
>>
>> And which command is more readable?  Something like the above, or:
>> btrfs device stats --format=json /
> 
> How many people you know who are communicating in json or using comma
> separated lines of text?
Directly between people?  Probably none other than CS students and 
developers.  JSON is one of the few reliably unambiguous ways of 
representing tables of data in a language agnostic manner while still 
having them be easily machine parseable (and in fact, many programming 
languages use a similar format for defining tables).

Indirectly between people?  Tens of millions at least.  At minimum, 
almost everyone who uses social media is using JSON.

When dealing with administering a computer?  No idea, but at least as 
many people as use LVM and a number of widely used web application 
frameworks (and web applications built from them).  There is quite a lot 
of software that uses JSON files for configuration because it's easy to 
parse and also pretty trivial for most people to understand (and allows 
for certain particularly useful things that the INI format doesn't).

> Maybe it is not obvious but such interface is not for the humans.
But the code communicating using those formats _is_ for humans.  One of 
the most basic rules of software development is to write code that is 
easily maintainable, which in turn means easily readable.  My comment on 
readability was not about the output, but about the means of generating 
that output, which humans (more specifically, programmers) _do_ care about.

Put slightly differently, here's a comparison of getting info about 
device error counters in Python with and without JSON support in 
btrfs-progs:

Without JSON support (and without any error handling:
```
import subprocess

raw = subprocess.check_output([
           '/sbin/btrfs',
           'device',
           'stats',
           '/'
       ])
raw = stats.splitlines()
stats = dict()
for i in range(0, len(raw)):
     line = raw[i].split()
     value = int(line[1])
     device = line[0].split('.')[0].lstrip('[').rstrip(']')
     counter = line[0].split('.')[1]
     if not device in stats.keys():
         stats[device] = dict()
     stats[device][counter] = value
```

With JSON support in btrfs-progs (and also without error handling):
```
import subprocess
import json

raw = subprocess.check_output([
           '/sbin/btrfs',
           'device',
           'stats',
           '--format=json',
           '/'
       ])
stats = json.loads(raw)
```

Notice that the second form is exponentially easier to follow (assuming 
you already know that json.loads() parses a string as JSON, but if 
you're working with Python you probably already know that), and will 
probably run faster too.