How to debug very very slow file delete?

How to debug very very slow file delete?

[Marc MERLIN]

I had a tree with some amount of thousand files (less than 1 million)
on top of md raid5.

It took 18H to rm it in 3 tries:
gargamel:/mnt/dshelf2/backup/polgara# time rm -rf current.todel/
real    1087m26.491s
user    0m2.448s
sys     4m42.012s

gargamel:/mnt/dshelf2/backup/polgara# btrfs fi show /mnt/btrfs_pool2
Label: btrfs_pool2  uuid: cb9df6d3-a528-4afc-9a45-4fed5ec358d6
        Total devices 1 FS bytes used 2.76TiB
        devid    1 size 7.28TiB used 3.43TiB path /dev/mapper/dshelf2

gargamel:/mnt/dshelf2/backup/polgara# btrfs fi df /mnt/btrfs_pool2
Data, single: total=3.28TiB, used=2.70TiB
System, DUP: total=8.00MiB, used=384.00KiB
System, single: total=4.00MiB, used=0.00
Metadata, DUP: total=73.50GiB, used=62.46GiB
Metadata, single: total=8.00MiB, used=0.00

This is running from
md8 : active raid5 sdf1[6] sdb1[5] sda1[3] sde1[2] sdd1[1]
      7814045696 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/5] [UUUUU]
      bitmap: 0/15 pages [0KB], 65536KB chunk

The filesystem is pretty new, it shouldn't be fragmented much.

dmesg shows a bit of this:
INFO: task rm:31885 blocked for more than 120 seconds.
      Not tainted 3.14.0-rc5-amd64-i915-preempt-20140216c #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message
rm              D ffff880117e5a940     0 31885  31280 0x20020080
 ffff8800169f3d10 0000000000000086 ffff8800169f3fd8 ffff880117e5a410
 00000000000141c0 ffff880117e5a410 ffff88011d4f9e90 ffff88020c0109e8
 0000000000000000 ffff88020c010800 ffff8801b6df55a0 ffff8800169f3d20
Call Trace:
 [<ffffffff8160c331>] schedule+0x73/0x75
 [<ffffffff8122a5f9>] wait_current_trans.isra.15+0x98/0xf4
 [<ffffffff810850c9>] ? finish_wait+0x65/0x65
 [<ffffffff8122ba9e>] start_transaction+0x48e/0x4f2
 [<ffffffff8122bb1d>] btrfs_start_transaction+0x1b/0x1d
 [<ffffffff8122cab4>] __unlink_start_trans+0x24/0xaf
 [<ffffffff812327ab>] btrfs_unlink+0x28/0xa0
 [<ffffffff8116176d>] vfs_unlink+0x90/0xdb
 [<ffffffff811618c0>] do_unlinkat+0x108/0x1da
 [<ffffffff810734c8>] ? mmdrop+0x11/0x20
 [<ffffffff8115d2db>] ? path_put+0x1e/0x21
 [<ffffffff811625c7>] SyS_unlinkat+0x22/0x2e
 [<ffffffff816171ac>] sysenter_dispatch+0x7/0x21

That said, it didn't happen much:
[38077.054841] INFO: task btrfs-cleaner:3536 blocked for more than 120 seconds.
[38077.521463] INFO: task rm:31885 blocked for more than 120 seconds.
[38077.926399] INFO: task cp:26645 blocked for more than 120 seconds.
[38078.346885] INFO: task btrfs:7430 blocked for more than 120 seconds.
[38198.921768] INFO: task btrfs-cleaner:3536 blocked for more than 120 seconds.
[38199.357367] INFO: task rm:31885 blocked for more than 120 seconds.
[38199.753897] INFO: task cp:26645 blocked for more than 120 seconds.
[38200.172729] INFO: task btrfs:7430 blocked for more than 120 seconds.
[56696.271850] INFO: task btrfs-transacti:3537 blocked for more than 120 seconds.
[56937.063142] INFO: task btrfs-cleaner:3536 blocked for more than 120 seconds.

Of course, all of those are bad, but I'm unfortunately used to seeing them
It wasn't hanging forever every few minutes, so it looks like it was just working 
very slowly.

The problem does not seem to be just rm though, du is taking way way too
long too. I started one, it's been running for 30mn now.
Interestingly, sending ^C to that du takes 15 seconds to respond, so it
seems that each system call is just slow.

I checked that btrfs scrub is not running.
What else can I check from here?

Thanks,
Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: How to debug very very slow file delete? (btrfs on md-raid5)

[Martin]

On 25/03/14 01:49, Marc MERLIN wrote:
> I had a tree with some amount of thousand files (less than 1 million)
> on top of md raid5.
> 
> It took 18H to rm it in 3 tries:


> Data, single: total=3.28TiB, used=2.70TiB
> System, DUP: total=8.00MiB, used=384.00KiB
> System, single: total=4.00MiB, used=0.00
> Metadata, DUP: total=73.50GiB, used=62.46GiB
> Metadata, single: total=8.00MiB, used=0.00
> 
> This is running from
> md8 : active raid5 sdf1[6] sdb1[5] sda1[3] sde1[2] sdd1[1]
>       7814045696 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/5] [UUUUU]
>       bitmap: 0/15 pages [0KB], 65536KB chunk
> 
> The filesystem is pretty new, it shouldn't be fragmented much.


> The problem does not seem to be just rm though, du is taking way way too
> long too. I started one, it's been running for 30mn now.
> Interestingly, sending ^C to that du takes 15 seconds to respond, so it
> seems that each system call is just slow.
> 
> I checked that btrfs scrub is not running.
> What else can I check from here?

"noatime" set?

What's your cpu hardware wait time?


And is not *the 512kByte raid chunk* going to give you horrendous write
amplification?! For example, rm updates a few bytes in one 4kByte
metadata block and the system has to then do a read-modify-write on
512kBytes...

Also, the 64MByte chunk bit-intent map will add a lot of head seeks to
anything you do on that raid. (The map would be better on a separate SSD
or other separate drive.)

So... That sort of setup is fine for archived data that is effectively
read-only. You'll see poor performance for small writes/changes.


Hope that helps,
Martin

Re: How to debug very very slow file delete? (btrfs on md-raid5)

[Xavier Nicollet]

Le 25 mars 2014 à 12:13, Martin a écrit:
> On 25/03/14 01:49, Marc MERLIN wrote:
> > It took 18H to rm it in 3 tries:

> And is not *the 512kByte raid chunk* going to give you horrendous write
> amplification?! For example, rm updates a few bytes in one 4kByte
> metadata block and the system has to then do a read-modify-write on
> 512kBytes...

My question would be naive, but would it be possible to have a syscall or something to do 
a fast "rm -rf" or du ?

I think ceph might have the later actually.

-- 
Xavier Nicollet

Re: How to debug very very slow file delete? (btrfs on md-raid5)

[Marc MERLIN]

On Tue, Mar 25, 2014 at 12:13:50PM +0000, Martin wrote:
> On 25/03/14 01:49, Marc MERLIN wrote:
> > I had a tree with some amount of thousand files (less than 1 million)
> > on top of md raid5.
> > 
> > It took 18H to rm it in 3 tries:

I ran another test after typing the original Email:
gargamel:/mnt/dshelf2/backup/polgara# time du -sh 20140312-feisty/; time find 20140 312-feisty/ | wc -l
17G     20140312-feisty/
real    245m19.491s
user    0m2.108s
sys     1m0.508s

728507 <- number of files
real    11m41.853s <- 11mn to restat them when they should all be in cache ideally
user    0m1.040s
sys     0m4.360s

4 hours to stat 700K files. That's bad...
Even 11mn to restat them just to count them looks bad too.

> > I checked that btrfs scrub is not running.
> > What else can I check from here?
> 
> "noatime" set?

I have relatime
gargamel:/mnt/dshelf2/backup/polgara# df .
Filesystem           1K-blocks       Used  Available Use% Mounted on
/dev/mapper/dshelf2 7814041600 3026472436 4760588292  39% /mnt/dshelf2/backup

gargamel:/mnt/dshelf2/backup/polgara# grep /mnt/dshelf2/backup /proc/mounts
/dev/mapper/dshelf2 /mnt/dshelf2/backup btrfs rw,relatime,compress=lzo,space_cache 0 0
 
> What's your cpu hardware wait time?
 
Sorry, not sure how to get that.
 
> And is not *the 512kByte raid chunk* going to give you horrendous write
> amplification?! For example, rm updates a few bytes in one 4kByte
> metadata block and the system has to then do a read-modify-write on
> 512kBytes...

That's probably not great, but
1) rm -rf should bunch a lot of writes together before they start
hitting the block layer for writes, so I'm not sure that is too much a
problem with the caching layer in between

2) this does not explain 4H to just run du with relatime, which
shouldn't generate any writing, correct?
iostat seems to confirm:

gargamel:~# iostat /dev/md8 1 20
Linux 3.14.0-rc5-amd64-i915-preempt-20140216c (gargamel.svh.merlins.org)        03/25/2014      _x86_64_        (4 CPU)
avg-cpu:  %user   %nice %system %iowait  %steal   %idle  
          75.19    0.00   10.13    8.61    0.00    6.08
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md8              98.00       392.00         0.00        392          0
md8              96.00       384.00         0.00        384          0
md8              83.00       332.00         0.00        332          0
md8             153.00       612.00         0.00        612          0
md8              82.00       328.00         0.00        328          0
md8              55.00       220.00         0.00        220          0
md8              69.00       276.00         0.00        276          0

> Also, the 64MByte chunk bit-intent map will add a lot of head seeks to
> anything you do on that raid. (The map would be better on a separate SSD
> or other separate drive.)

That's true for writing, but not reading, right?
 
> So... That sort of setup is fine for archived data that is effectively
> read-only. You'll see poor performance for small writes/changes.

So I agree with you that the write case can be improved, especially since I also have a layer
of dmcrypt in the middle
gargamel:/mnt/dshelf2/backup/polgara# cryptsetup luksDump /dev/md8
LUKS header information for /dev/md8
Cipher name:    aes
Cipher mode:    xts-plain64
Hash spec:      sha1
Payload offset: 8192

(I used cryptsetup luksFormat --align-payload=8192 -s 256 -c aes-xts-plain64)

I'm still not convinced that a lot of file IO don't get all collated in memory 
before hitting disk in bigger blocks, but maybe not.

If I were to recreate this array entirely, what would you use for the raid creation
and cryptsetup?

More generally, before I go through all that trouble (it will likely
take 1 week of data copying back and forth), I'd like to debug why my reads are
so slow first.

Thanks,
Marc

On Tue, Mar 25, 2014 at 02:57:57PM +0100, Xavier Nicollet wrote:
> Le 25 mars 2014 à 12:13, Martin a écrit:
> > On 25/03/14 01:49, Marc MERLIN wrote:
> > > It took 18H to rm it in 3 tries:
> 
> > And is not *the 512kByte raid chunk* going to give you horrendous write
> > amplification?! For example, rm updates a few bytes in one 4kByte
> > metadata block and the system has to then do a read-modify-write on
> > 512kBytes...
> 
> My question would be naive, but would it be possible to have a syscall or something to do 
> a fast "rm -rf" or du ?

Well, that wouldn't hurt either, even if it wouldn't address my underlying problem.

Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/                         | PGP 1024R/763BE901

[PATCH 00/27] Replace the old man page with asciidoc and man page for each btrfs subcommand.

[Qu Wenruo]

Convert the old btrfs man pages to new asciidoc and split the huge
btrfs man page into subcommand man page.

The asciidoc style and Makefile things are mostly simplified from git
Documentation, which only supports man page output and remove html output,
since html output is somewhat overkilled for btrfs.

This huge convert and split will bring the following advantages:
1) More human-readable raw documentations.
  The roff grammar is not so easy to read, especially mixed with a lot
  of special words.

2) Easier to modify man page.
  The old huge btrfs man page makes it hard to maintain.
  It's common that some one adds some new options, but only modifies
  the detailed options but not the 'SYNOPSIS' section.

  Now, there is few duplication since btrfs man page is only a overall man
  page, details are all in its individual man page.

I hope this patchset will be the last huge change to man pages and save later
contributors time to maintain the documentations.

Qu Wenruo (27):
  btrfs-progs: Introduce asciidoc based man page and btrfs man page.
  btrfs-progs: Convert man page for btrfs-subvolume
  btrfs-progs: Convert man page for filesystem subcommand.
  btrfs-progs: Convert man page for btrfs-balance.
  btrfs-progs: Convert man page for btrfs-device subcommand.
  btrfs-progs: Convert man page for btrfs-scrub
  btrfs-progs: Convert man page for btrfs-check.
  btrfs-progs: Convert man page for btrfs-rescue
  btrfs-progs: Convert man page for btrfs-inspect-internal
  btrfs-progs: Convert man page for btrfs-send.
  btrfs-progs: Convert man page for btrfs-receive.
  btrfs-progs: Convert man page for btrfs-quota.
  btrfs-progs: Convert and enhance the man page of btrfs-qgroup.
  btrfs-progs: Convert man page for btrfs-replace.
  btrfs-progs: Convert man page for btrfs-dedup.
  btrfs-progs: Convert man page for btrfsck
  btrfs-progs: Convert man page for btrfs-convert.
  btrfs-progs: Convert man page for btrfs-debug-tree.
  btrfs-progs: Convert man page for btrfs-find-root.
  btrfs-progs: Convert man page for btrfs-image.
  btrfs-progs: Convert man page for btrfs-map-logical.
  btrfs-progs: Convert man page for btrfs-show-super.
  btrfs-progs: Convert man page for btrfstune.
  btrfs-progs: Convert man page for btrfs-zero-log
  btrfs-progs: Convert man page for fsck.btrfs.
  btrfs-progs: Convert man page for mkfs.btrfs.
  btrfs-progs: Switch to the new asciidoc Documentation.

 .gitignore                               |   1 +
 Documentation/Makefile                   |  92 +++
 Documentation/asciidoc.conf              |  42 ++
 Documentation/btrfs-balance.txt          |  77 +++
 Documentation/btrfs-check.txt            |  45 ++
 Documentation/btrfs-convert.txt          |  49 ++
 Documentation/btrfs-debug-tree.txt       |  50 ++
 Documentation/btrfs-dedup.txt            |  51 ++
 Documentation/btrfs-device.txt           |  75 +++
 Documentation/btrfs-filesystem.txt       | 163 ++++++
 Documentation/btrfs-find-root.txt        |  45 ++
 Documentation/btrfs-image.txt            |  70 +++
 Documentation/btrfs-inspect-internal.txt |  69 +++
 Documentation/btrfs-map-logical.txt      |  49 ++
 Documentation/btrfs-qgroup.txt           | 110 ++++
 Documentation/btrfs-quota.txt            |  59 ++
 Documentation/btrfs-receive.txt          |  58 ++
 Documentation/btrfs-replace.txt          |  76 +++
 Documentation/btrfs-rescue.txt           |  60 ++
 Documentation/btrfs-scrub.txt            |  98 ++++
 Documentation/btrfs-send.txt             |  60 ++
 Documentation/btrfs-show-super.txt       |  53 ++
 Documentation/btrfs-subvolume.txt        | 172 ++++++
 Documentation/btrfs-zero-log.txt         |  39 ++
 Documentation/btrfs.txt                  | 117 ++++
 Documentation/btrfsck.txt                |  55 ++
 Documentation/btrfstune.txt              |  47 ++
 Documentation/fsck.btrfs.txt             |  51 ++
 Documentation/manpage-base.xsl           |  35 ++
 Documentation/manpage-bold-literal.xsl   |  17 +
 Documentation/manpage-normal.xsl         |  13 +
 Documentation/mkfs.btrfs.txt             | 133 +++++
 Makefile                                 |   4 +-
 man/Makefile                             |  31 -
 man/btrfs-convert.8.in                   |  33 --
 man/btrfs-debug-tree.8.in                |  35 --
 man/btrfs-find-root.8.in                 |  30 -
 man/btrfs-image.8.in                     |  55 --
 man/btrfs-map-logical.8.in               |  33 --
 man/btrfs-show-super.8.in                |  30 -
 man/btrfs-zero-log.8.in                  |  23 -
 man/btrfs.8.in                           | 932 -------------------------------
 man/btrfsck.8.in                         |  29 -
 man/btrfstune.8.in                       |  31 -
 man/fsck.btrfs.8.in                      |  47 --
 man/mkfs.btrfs.8.in                      | 105 ----
 46 files changed, 2133 insertions(+), 1416 deletions(-)
 create mode 100644 Documentation/Makefile
 create mode 100644 Documentation/asciidoc.conf
 create mode 100644 Documentation/btrfs-balance.txt
 create mode 100644 Documentation/btrfs-check.txt
 create mode 100644 Documentation/btrfs-convert.txt
 create mode 100644 Documentation/btrfs-debug-tree.txt
 create mode 100644 Documentation/btrfs-dedup.txt
 create mode 100644 Documentation/btrfs-device.txt
 create mode 100644 Documentation/btrfs-filesystem.txt
 create mode 100644 Documentation/btrfs-find-root.txt
 create mode 100644 Documentation/btrfs-image.txt
 create mode 100644 Documentation/btrfs-inspect-internal.txt
 create mode 100644 Documentation/btrfs-map-logical.txt
 create mode 100644 Documentation/btrfs-qgroup.txt
 create mode 100644 Documentation/btrfs-quota.txt
 create mode 100644 Documentation/btrfs-receive.txt
 create mode 100644 Documentation/btrfs-replace.txt
 create mode 100644 Documentation/btrfs-rescue.txt
 create mode 100644 Documentation/btrfs-scrub.txt
 create mode 100644 Documentation/btrfs-send.txt
 create mode 100644 Documentation/btrfs-show-super.txt
 create mode 100644 Documentation/btrfs-subvolume.txt
 create mode 100644 Documentation/btrfs-zero-log.txt
 create mode 100644 Documentation/btrfs.txt
 create mode 100644 Documentation/btrfsck.txt
 create mode 100644 Documentation/btrfstune.txt
 create mode 100644 Documentation/fsck.btrfs.txt
 create mode 100644 Documentation/manpage-base.xsl
 create mode 100644 Documentation/manpage-bold-literal.xsl
 create mode 100644 Documentation/manpage-normal.xsl
 create mode 100644 Documentation/mkfs.btrfs.txt
 delete mode 100644 man/Makefile
 delete mode 100644 man/btrfs-convert.8.in
 delete mode 100644 man/btrfs-debug-tree.8.in
 delete mode 100644 man/btrfs-find-root.8.in
 delete mode 100644 man/btrfs-image.8.in
 delete mode 100644 man/btrfs-map-logical.8.in
 delete mode 100644 man/btrfs-show-super.8.in
 delete mode 100644 man/btrfs-zero-log.8.in
 delete mode 100644 man/btrfs.8.in
 delete mode 100644 man/btrfsck.8.in
 delete mode 100644 man/btrfstune.8.in
 delete mode 100644 man/fsck.btrfs.8.in
 delete mode 100644 man/mkfs.btrfs.8.in

-- 
1.9.1

[PATCH 01/27] btrfs-progs: Introduce asciidoc based man page and btrfs man page.

[Qu Wenruo]

The old man page of btrfs will grow larger with new functions adding to
btrfs-progs and harder to maintain because the reader-unfriendly roff
grammar and one LARGE btrfs.in.

This patch will introduce the simplified Documentation directory mainly
'stolen' from git and include the first man page for 'btrfs(8)'.
This time, man page will be written in human-friendly asciidoc grammar
and each commands of btrfs will have a separate man page, which I hope
can reduce the effort to maintain the man page.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 .gitignore                             |   1 +
 Documentation/Makefile                 |  91 +++++++++++++++++++++++++
 Documentation/asciidoc.conf            |  42 ++++++++++++
 Documentation/btrfs.txt                | 117 +++++++++++++++++++++++++++++++++
 Documentation/manpage-base.xsl         |  35 ++++++++++
 Documentation/manpage-bold-literal.xsl |  17 +++++
 Documentation/manpage-normal.xsl       |  13 ++++
 7 files changed, 316 insertions(+)
 create mode 100644 Documentation/Makefile
 create mode 100644 Documentation/asciidoc.conf
 create mode 100644 Documentation/btrfs.txt
 create mode 100644 Documentation/manpage-base.xsl
 create mode 100644 Documentation/manpage-bold-literal.xsl
 create mode 100644 Documentation/manpage-normal.xsl

diff --git a/.gitignore b/.gitignore
index ab8b81c..fc8c07a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -5,6 +5,7 @@
 version.h
 version
 man/*.gz
+Documentation/*.gz
 btrfs
 btrfs.static
 btrfs-debug-tree
diff --git a/Documentation/Makefile b/Documentation/Makefile
new file mode 100644
index 0000000..bf38617
--- /dev/null
+++ b/Documentation/Makefile
@@ -0,0 +1,91 @@
+# Guard against environment variables
+MAN8_TXT =
+
+# Top level commands
+MAN8_TXT += btrfs.txt
+#MAN8_TXT += btrfsck.txt
+#MAN8_TXT += btrfs-convert.txt
+#MAN8_TXT += btrfs-debug-tree.txt
+#MAN8_TXT += btrfs-find-root.txt
+#MAN8_TXT += btrfs-image.txt
+#MAN8_TXT += btrfs-map-logical.txt
+#MAN8_TXT += btrfs-show-super.txt
+#MAN8_TXT += btrfstune.txt
+#MAN8_TXT += btrfs-zero-log.txt
+#MAN8_TXT += fsck.btrfs.txt
+#MAN8_TXT += mkfs.btrfs.txt
+
+# Sub commands for btrfs
+#MAN8_TXT += btrfs-subvolume.txt
+#MAN8_TXT += btrfs-filesystem.txt
+#MAN8_TXT += btrfs-balance.txt
+#MAN8_TXT += btrfs-device.txt
+#MAN8_TXT += btrfs-scrub.txt
+#MAN8_TXT += btrfs-check.txt
+#MAN8_TXT += btrfs-rescue.txt
+#MAN8_TXT += btrfs-inspect-internal.txt
+#MAN8_TXT += btrfs-send.txt
+#MAN8_TXT += btrfs-receive.txt
+#MAN8_TXT += btrfs-quota.txt
+#MAN8_TXT += btrfs-replace.txt
+#MAN8_TXT += btrfs-dedup.txt
+
+MAN_TXT = $(MAN8_TXT)
+MAN_XML = $(patsubst %.txt,%.xml,$(MAN_TXT))
+DOC_MAN8 = $(patsubst %.txt,%.8,$(MAN8_TXT))
+GZ_MAN8 = $(patsubst %.txt,%.8.gz,$(MAN8_TXT))
+
+mandir ?= $(prefix)/share/man
+man8dir = $(mandir)/man8
+
+ASCIIDOC = asciidoc
+ASCIIDOC_EXTRA =
+MANPAGE_XSL = manpage-normal.xsl
+XMLTO = xmlto
+XMLTO_EXTRA =
+XMLTO_EXTRA = -m manpage-bold-literal.xsl
+GZIP = gzip
+INSTALL ?= install
+RM ?= rm -f
+BTRFS_VERSION = $(shell sed -n 's/.*BTRFS_BUILD_VERSION "Btrfs \(.*\)"/\1/p'\
+		  ../version.h)
+
+ifneq ($(findstring $(MAKEFLAGS),s),s)
+ifndef V
+	QUIET_ASCIIDOC	= @echo '   ' ASCIIDOC $@;
+	QUIET_XMLTO	= @echo '   ' XMLTO $@;
+	QUIET_GZIP	= @echo '   ' GZIP $@;
+	QUIET_STDERR	= 2> /dev/null
+	QUIET_SUBDIR0	= +@subdir=
+	QUIET_SUBDIR1	= ;$(NO_SUBDIR) echo '   ' SUBDIR $$subdir; \
+			  $(MAKE) $(PRINT_DIR) -C $$subdir
+	export V
+endif
+endif
+
+all: man
+man: man8
+man8: $(GZ_MAN8)
+
+install: install-man
+
+install-man: man
+	$(INSTALL) -d -m 755 $(DESTDIR)$(man8dir)
+	$(INSTALL) -m 644 $(GZ_MAN8) $(DESTDIR)$(man8dir)
+
+clean:
+	$(RM) *.xml *.xml+ *.8 *.8.gz
+
+%.8.gz : %.8
+	$(QUIET_GZIP)$(GZIP) -n -c $< > $@
+
+%.8 : %.xml 
+	$(QUIET_XMLTO)$(RM) $@ && \
+	$(XMLTO) -m $(MANPAGE_XSL) $(XMLTO_EXTRA) man $<
+
+%.xml : %.txt asciidoc.conf
+	$(QUIET_ASCIIDOC)$(RM) $@+ $@ && \
+	$(ASCIIDOC) -b docbook -d manpage -f asciidoc.conf \
+		$(ASCIIDOC_EXTRA) -abtrfs_version=$(BTRFS_VERSION) \
+		-o $@+ $< && \
+	mv $@+ $@
diff --git a/Documentation/asciidoc.conf b/Documentation/asciidoc.conf
new file mode 100644
index 0000000..313f185
--- /dev/null
+++ b/Documentation/asciidoc.conf
@@ -0,0 +1,42 @@
+## linkbtrfs: macro
+#
+# Usage: linkbtrfs:command[manpage-section]
+#
+# Note, {0} is the manpage section, while {target} is the command.
+#
+# Show Btrfslink as: <command>(<section>); if section is defined, else just show
+# the command.
+
+[macros]
+(?su)[\\]?(?P<name>linkbtrfs):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+
+[attributes]
+asterisk=&#42;
+plus=&#43;
+caret=&#94;
+startsb=&#91;
+endsb=&#93;
+backslash=&#92;
+tilde=&#126;
+apostrophe=&#39;
+backtick=&#96;
+litdd=&#45;&#45;
+
+ifdef::doctype-manpage[]
+ifdef::backend-docbook[]
+[header]
+template::[header-declarations]
+<refentry>
+<refmeta>
+<refentrytitle>{mantitle}</refentrytitle>
+<manvolnum>{manvolnum}</manvolnum>
+<refmiscinfo class="source">Btrfs</refmiscinfo>
+<refmiscinfo class="version">{btrfs_version}</refmiscinfo>
+<refmiscinfo class="manual">Btrfs Manual</refmiscinfo>
+</refmeta>
+<refnamediv>
+  <refname>{manname}</refname>
+  <refpurpose>{manpurpose}</refpurpose>
+</refnamediv>
+endif::backend-docbook[]
+endif::doctype-manpage[]
diff --git a/Documentation/btrfs.txt b/Documentation/btrfs.txt
new file mode 100644
index 0000000..c9bed70
--- /dev/null
+++ b/Documentation/btrfs.txt
@@ -0,0 +1,117 @@
+btrfs(8)
+========
+
+NAME
+----
+btrfs - control a btrfs filesystem
+
+SYNOPSIS
+--------
+'btrfs' <command> [<args>]
+
+DESCRIPTION
+-----------
+'btrfs' is used to control the filesystem and the files and directories stored.
+It is the tool to create or destroy a snapshot or a subvolume for the
+filesystem, to defrag a file or a directory, flush the data to the disk,
+to resize the filesystem, to scan the device.
+
+It is possible to abbreviate the commands unless the commands are ambiguous.
+For example: it is possible to run 'btrfs sub snaps' instead of
+'btrfs subvolume snapshot'.
+But 'btrfs file s' is not allowed, because 'file s' may be interpreted
+both as 'filesystem show' and as 'filesystem sync'.
+
+If a command is terminated by '--help', the detailed help is showed.
+If the passed command matches more commands,
+detailed help of all the matched commands is showed. For example
+'btrfs dev --help' shows the help of all 'device*' commands.
+
+COMMANDS
+--------
+'subvolume'::
+	Create/delete/list/manage btrfs subvolume. +
+	See `btrfs-subvolume`(8) for details.
+
+'filesystem'::
+	Manage a btrfs filesystem, including label setting/sync and so on. +
+	See `btrfs-filesystem`(8) for details.
+
+'[filesystem] balance'::
+	Balance btrfs filesystem chunks across single or several devices. +
+	See `btrfs-balance`(8) for details.
+
+'device'::
+	Manage devices managed by btrfs, including add/delete/scan and so
+	on. +
+	See `btrfs-device`(8) for details.
+
+'scrub'::
+	Scrub a btrfs filesystem. +
+	See `btrfs-scrub`(8) for details.
+
+'check'::
+	Do off-line check on a btrfs filesystem. +
+	See `btrfs-check`(8) for details.
+
+'rescue'::
+	Try to rescue damaged btrfs filesystem. +
+	See `btrfs-rescue`(8) for details.
+
+'restore'::
+	Manage a btrfs filesystem, including label setting/sync and so on. +
+	See `btrfs-restore`(8) for details.
+
+'inspect-internal'::
+	Debug tools for developers/hackers. +
+	See `btrfs-inspect-internal`(8) for details.
+
+'send'::
+	Send subvolume data to stdout/file for backup and etc. +
+	See `btrfs-send`(8) for details.
+
+'receive'::
+	Receive subvolume data from stdin/file for restore and etc. +
+	See `btrfs-receive`(8) for details.
+'quota'::
+	Manage quota on btrfs filesystem like enabling/rescan and etc. +
+	See `btrfs-quota`(8) and `btrfs-qgroup`(8) for details.
+
+'qgroup'::
+	Manage quota group(qgroup) for btrfs filesystem. +
+	See `btrfs-qgroup`(8) for details.
+
+'replace'::
+	Replace btrfs devices. +
+	See `btrfs-replace`(8) for details.
+
+EXIT STATUS
+-----------
+'btrfs' returns a zero exist status if it succeeds. Non zero is returned in
+case of failure.
+
+AVAILABILITY
+------------
+'btrfs' is part of btrfs-progs. Btrfs filesystem is currently under heavy
+development,
+and not suitable for any uses other than benchmarking and review.
+Please refer to the btrfs wiki http://btrfs.wiki.kernel.org for
+further details.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8), `ionice`(1),
+`btrfs-subvolume`(8),
+`btrfs-filesystem`(8),
+`btrfs-balance`(8),
+`btrfs-device`(8),
+`btrfs-scrub`(8),
+`btrfs-check`(8),
+`btrfs-rescue`(8),
+`btrfs-restore`(8),
+`btrfs-inspect-internal`(8),
+`btrfs-send`(8),
+`btrfs-receive`(8),
+`btrfs-quota`(8),
+`btrfs-qgroup`(8),
+`btrfs-replace`(8),
diff --git a/Documentation/manpage-base.xsl b/Documentation/manpage-base.xsl
new file mode 100644
index 0000000..a264fa6
--- /dev/null
+++ b/Documentation/manpage-base.xsl
@@ -0,0 +1,35 @@
+<!-- manpage-base.xsl:
+     special formatting for manpages rendered from asciidoc+docbook -->
+<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
+		version="1.0">
+
+<!-- these params silence some output from xmlto -->
+<xsl:param name="man.output.quietly" select="1"/>
+<xsl:param name="refentry.meta.get.quietly" select="1"/>
+
+<!-- convert asciidoc callouts to man page format;
+     git.docbook.backslash and git.docbook.dot params
+     must be supplied by another XSL file or other means -->
+<xsl:template match="co">
+	<xsl:value-of select="concat(
+			      $git.docbook.backslash,'fB(',
+			      substring-after(@id,'-'),')',
+			      $git.docbook.backslash,'fR')"/>
+</xsl:template>
+<xsl:template match="calloutlist">
+	<xsl:value-of select="$git.docbook.dot"/>
+	<xsl:text>sp&#10;</xsl:text>
+	<xsl:apply-templates/>
+	<xsl:text>&#10;</xsl:text>
+</xsl:template>
+<xsl:template match="callout">
+	<xsl:value-of select="concat(
+			      $git.docbook.backslash,'fB',
+			      substring-after(@arearefs,'-'),
+			      '. ',$git.docbook.backslash,'fR')"/>
+	<xsl:apply-templates/>
+	<xsl:value-of select="$git.docbook.dot"/>
+	<xsl:text>br&#10;</xsl:text>
+</xsl:template>
+
+</xsl:stylesheet>
diff --git a/Documentation/manpage-bold-literal.xsl b/Documentation/manpage-bold-literal.xsl
new file mode 100644
index 0000000..608eb5d
--- /dev/null
+++ b/Documentation/manpage-bold-literal.xsl
@@ -0,0 +1,17 @@
+<!-- manpage-bold-literal.xsl:
+     special formatting for manpages rendered from asciidoc+docbook -->
+<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
+		version="1.0">
+
+<!-- render literal text as bold (instead of plain or monospace);
+     this makes literal text easier to distinguish in manpages
+     viewed on a tty -->
+<xsl:template match="literal">
+	<xsl:value-of select="$git.docbook.backslash"/>
+	<xsl:text>fB</xsl:text>
+	<xsl:apply-templates/>
+	<xsl:value-of select="$git.docbook.backslash"/>
+	<xsl:text>fR</xsl:text>
+</xsl:template>
+
+</xsl:stylesheet>
diff --git a/Documentation/manpage-normal.xsl b/Documentation/manpage-normal.xsl
new file mode 100644
index 0000000..a48f5b1
--- /dev/null
+++ b/Documentation/manpage-normal.xsl
@@ -0,0 +1,13 @@
+<!-- manpage-normal.xsl:
+     special settings for manpages rendered from asciidoc+docbook
+     handles anything we want to keep away from docbook-xsl 1.72.0 -->
+<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
+		version="1.0">
+
+<xsl:import href="manpage-base.xsl"/>
+
+<!-- these are the normal values for the roff control characters -->
+<xsl:param name="git.docbook.backslash">\</xsl:param>
+<xsl:param name="git.docbook.dot"	>.</xsl:param>
+
+</xsl:stylesheet>
-- 
1.9.1

[PATCH 02/27] btrfs-progs: Convert man page for btrfs-subvolume

[Qu Wenruo]

Convert man page for btrfs-subvolume.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile            |   2 +-
 Documentation/btrfs-subvolume.txt | 172 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 173 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/btrfs-subvolume.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index bf38617..15c1679 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -16,7 +16,7 @@ MAN8_TXT += btrfs.txt
 #MAN8_TXT += mkfs.btrfs.txt
 
 # Sub commands for btrfs
-#MAN8_TXT += btrfs-subvolume.txt
+MAN8_TXT += btrfs-subvolume.txt
 #MAN8_TXT += btrfs-filesystem.txt
 #MAN8_TXT += btrfs-balance.txt
 #MAN8_TXT += btrfs-device.txt
diff --git a/Documentation/btrfs-subvolume.txt b/Documentation/btrfs-subvolume.txt
new file mode 100644
index 0000000..7f32dbd
--- /dev/null
+++ b/Documentation/btrfs-subvolume.txt
@@ -0,0 +1,172 @@
+btrfs-subvolume(8)
+==================
+
+NAME
+----
+btrfs-subvolume - control btrfs subvolume(s)
+
+SYNOPSIS
+--------
+'btrfs subvolume' <subcommand> [<args>]
+
+DESCRIPTION
+-----------
+'btrfs subvolume' is used to control the filesystem to create/delete/list/show
+subvolumes and snapshots.
+
+SUBVOLUME AND SNAPSHOT
+----------------------
+A subvolume in btrfs is not like an LVM logical volume, which is quite
+independent from each other, a btrfs subvolume has its hierarchy and relations
+between other subvolumes.
+
+A subvolume in btrfs can be accessed in two ways.
+
+1. From the parent subvolume +
+When accessing from the parent subvolume, the subvolume can be used just
+like a directory. It can have child subvolumes and its own files/directories.
+
+2. Separate mounted filesystem +
+When `mount`(8) using 'subvol' or 'subvolid' mount option, one can access
+files/directories/subvolumes inside it, but nothing in parent subvolumes.
+
+Also every btrfs filesystem has a default subvolume as its initially top-level
+subvolume, whose subvolume id is 5(FS_TREE).
+
+A btrfs snapshot is much like a subvolume, but shares its data(and metadata)
+with other subvolume/snapshot. Due to the capabilities of COW, modifications
+inside a snapshot will only show in a snapshot but not in its source subvolume.
+
+Although in btrfs, subvolumes/snapshots are treated as directories, only
+subvolume/snapshot can be the source of a snapshot, snapshot can not be made
+from normal directories.
+
+SUBCOMMAND
+-----------
+'create' [-i <qgroupid>] [<dest>]<name>::
+Create a subvolume <name> in <dest>.
++
+If <dest> is not given, subvolume <name> will be created in the currently
+directory.
++
+`Options`
++
+-i <qgroupid>::::
+Add the newly created subvolume to a qgroup. This option can be given multiple
+times.
+
+'delete' [options] <subvolume> [<subvolume>...]::
+Delete the subvolume(s) from the filesystem.
++
+If <subvolume> is not a subvolume, btrfs returns an error but continues if
+there are more arguments to process.
++
+The corresponding directory is removed instantly but the data blocks are
+removed later.  The deletion does not involve full commit by default due to
+performance reasons (as a consequence, the subvolume may appear again after a
+crash).  Use one of the '--commit' options to wait until the operation is safely
+stored on the media.
++
+`Options`
++
+-c|--commit-after::::
+wait for transaction commit at the end of the operation
++
+-C|--commit-each::::
+wait for transaction commit after delet each subvolume
+
+'list' [options] [-G [\+|-] values] [-C [+|-]value] [--sort=rootid,gen,ogen,path] <path>::
+List the subvolumes present in the filesystem <path>.
++
+For every subvolume the following information is shown by default. +
+ID <ID> top level <ID> path <path> +
+where path is the relative path of the subvolume to the top level subvolume.
+The subvolume's ID may be used by the subvolume set-default command,
+or at mount time via the subvolid= option.
+If `-p` is given, then parent <ID> is added to the output between ID
+and top level. The parent's ID may be used at mount time via the
+`subvolrootid=` option.
++
+`Options`
++
+-p::::
+print parent ID.
+-a::::
+print all the subvolumes in the filesystem and distinguish between
+absolute and relative path with respect to the given <path>.
+-c::::
+print the ogeneration of the subvolume, aliases: ogen or origin generation.
+-g::::
+print the generation of the subvolume.
+-o::::
+print only subvolumes bellow specified <path>.
+-u::::
+print the UUID of the subvolume.
+-q::::
+print the parent uuid of subvolumes (and snapshots).
+-t::::
+print the result as a table.
+-s::::
+only snapshot subvolumes in the filesystem will be listed.
+-r::::
+only readonly subvolumes in the filesystem will be listed.
+-G [+|-]value::::
+list subvolumes in the filesystem that its generation is
+>=, \<= or = value. \'\+' means >= value, \'-' means \<= value, If there is
+neither \'+' nor \'-', it means = value.
+-C [+|-]value::::
+list subvolumes in the filesystem that its ogeneration is
+>=, \<= or = value. The usage is the same to '-g' option.
+--sort=rootid,gen,ogen,path::::
+list subvolumes in order by specified items.
+you can add \'\+' or \'-' in front of each items, \'+' means ascending,
+\'-' means descending. The default is ascending.
++
+for --sort you can combine some items together by \',', just like
+-sort=+ogen,-gen,path,rootid.
+
+'snapshot' [-r] <source> <dest>|[<dest>/]<name>::
+Create a writable/readonly snapshot of the subvolume <source> with the
+name <name> in the <dest> directory.
++
+If only <dest> is given, the subvolume will be named the basename of <source>.
+If <source> is not a subvolume, btrfs returns an error.
+If '-r' is given, the snapshot will be readonly.
+
+'get-default' <path>::
+Get the default subvolume of the filesystem <path>.
++
+The output format is similar to 'subvolume list' command.
+
+'set-default' <id> <path>::
+Set the subvolume of the filesystem <path> which is mounted as
+default.
++
+The subvolume is identified by <id>, which is returned by the 'subvolume list'
+command.
+
+'find-new' <subvolume> <last_gen>::
+List the recently modified files in a subvolume, after <last_gen> ID.
+
+'show' <path>::
+Show information of a given subvolume in the <path>.
+
+EXIT STATUS
+-----------
+'btrfs subvolume' returns a zero exist status if it succeeds. Non zero is
+returned in case of failure.
+
+AVAILABILITY
+------------
+'btrfs' is part of btrfs-progs. Btrfs filesystem is currently under heavy
+development,
+and not suitable for any uses other than benchmarking and review.
+Please refer to the btrfs wiki http://btrfs.wiki.kernel.org for
+further details.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8),
+`btrfs-subvolume`(8),
+`btrfs-quota`(8),
+`btrfs-qgroup`(8),
-- 
1.9.1

[PATCH 03/27] btrfs-progs: Convert man page for filesystem subcommand.

[Qu Wenruo]

Convert man page for filesystem subcommand.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile             |   2 +-
 Documentation/btrfs-filesystem.txt | 163 +++++++++++++++++++++++++++++++++++++
 2 files changed, 164 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/btrfs-filesystem.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 15c1679..1fa2b35 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -17,7 +17,7 @@ MAN8_TXT += btrfs.txt
 
 # Sub commands for btrfs
 MAN8_TXT += btrfs-subvolume.txt
-#MAN8_TXT += btrfs-filesystem.txt
+MAN8_TXT += btrfs-filesystem.txt
 #MAN8_TXT += btrfs-balance.txt
 #MAN8_TXT += btrfs-device.txt
 #MAN8_TXT += btrfs-scrub.txt
diff --git a/Documentation/btrfs-filesystem.txt b/Documentation/btrfs-filesystem.txt
new file mode 100644
index 0000000..de9b3f3
--- /dev/null
+++ b/Documentation/btrfs-filesystem.txt
@@ -0,0 +1,163 @@
+btrfs-filesystem(8)
+===================
+
+NAME
+----
+btrfs-filesystem - control btrfs filesystem
+
+SYNOPSIS
+--------
+'btrfs filesystem' <subcommand> <args>
+
+DESCRIPTION
+-----------
+'btrfs filesystem' is used to do the filesystem level control jobs, including
+all the regular filesystem operations like setting/getting label,
+resizing, defragment.
+
+SUBCOMMAND
+----------
+'df' [-b] path [<path>...]::
+Show space usage information for a mount point.
++
+If '-b' is given, then byte is used as unit. Default unit will be
+human-readable unit such as KiB/MiB/GiB.
++
+The command 'btrfs filesystem df' is used to query how many space on the 
+disk(s) are used and an estimation of the free
+space of the filesystem.
+The output of the command 'btrfs filesystem df' shows:
+
+`Disk size`::::
+the total size of the disks which compose the filesystem.
+
+`Disk allocated`::::
+the size of the area of the disks used by the chunks.
+
+`Disk unallocated`::::
+the size of the area of the disks which is free (i.e.
+the differences of the values above).
+
+`Used`::::
+the portion of the logical space used by the file and metadata.
+
+`Free (estimated)`::::
+the estimated free space available: i.e. how many space can be used
+by the user. The evaluation cannot be rigorous because it depends by the
+allocation policy (DUP, Single, RAID1...) of the metadata and data chunks. +
+If every chunk is stored as "Single" the sum of the free (estimated) space
+and the used space  is equal to the disk size.
+Otherwise if all the chunk are mirrored (raid1 or raid10) or duplicated
+the sum of the free (estimated) space and the used space is
+half of the disk size. Normally the free (estimated) is between
+these two limits.
+
+`Data to disk ratio`::::
+the ratio betwen the logical size (i.e. the space available by
+the chunks) and the disk allocated (by the chunks). Normally it is 
+lower than 100% because the metadata is duplicated for security reasons.
+If all the data and metadata are duplicated (or have a profile like RAID1)
+the Data to disk ratio could be 50%.
+
+'show' [--mounted|--all-devices|<path>|<uuid>|<device>|<lable>]::
+Show the btrfs filesystem with some additional info.
++
+If no option nor <path>|<uuid>|<device>|<lable> is passed, btrfs shows
+information of all the btrfs filesystem both mounted and unmounted.
+If '--mounted' is passed, it would probe btrfs kernel to list mounted btrfs
+filesystem(s);
+If '--all-devices' is passed, all the devices under /dev are scanned;
+otherwise the devices list is extracted from the /proc/partitions file.
+
+'sync' <path>::
+Force a sync for the filesystem identified by <path>.
+
+'defragment' [options] <file>|<dir> [<file>|<dir>...]::
+Defragment file data and/or directory metadata *online*.
++
+If '-r' is passed, files in dir will be defragmented recursively.
+The start position and the number of bytes to defragment can be specified by
+start and len using '-s' and '-l' options below.
+Any extent bigger than threshold given by '-t' option, will be considered
+already defragged.
+Use 0 to take the kernel default, and use 1 to
+say every single extent must be rewritten.
+You can also turn on compression in defragment operations.
++
+`Options`
++
+-v::::
+be verbose
+-c::::
+compress file contents while defragmenting
+-r::::
+defragment files recursively
+-f::::
+flush filesystem after defragmenting
+-s <start>::::
+defragment only from byte <start> onward
+-l <len>::::
+defragment only up to <len> bytes
+-t <size>::::
+defragment only files at least <size> bytes big
++
+For <start>, <len>, <size> it is possible to append a suffix
+like 'k' for 1 KBytes, 'm' for 1 MBytes...
++
+WARNING: defragmenting with kernels up to 2.6.37 will unlink COW-ed copies of data,
+don't use it if you use snapshots, have de-duplicated your data or made
+copies with `cp --reflink`.
+
+// Some wording are extracted by the resize2fs man page
+'resize' [devid:][+/-]<size>[gkm]|[devid:]max <path>::
+Resize a filesystem identified by <path> for the underlying device
+devid *online*. +
+The devid can be found with 'btrfs filesystem show' and
+defaults to 1 if not specified.
+The <size> parameter specifies the new size of the filesystem.
+If the prefix + or - is present the size is increased or decreased
+by the quantity <size>.
+If no units are specified, the unit of the <size> parameter defaults to
+bytes. Optionally, the size parameter may be suffixed by one of the following
+units designators: \'K\', \'M', or \'G', kilobytes, megabytes, or gigabytes,
+respectively.
++
+If \'max' is passed, the filesystem will occupy all available space on the
+device devid.
++
+The resize command does not manipulate the size of underlying
+partition.  If you wish to enlarge/reduce a filesystem, you must make sure you
+can expand the partition before enlarging the filesystem and shrink the
+partition after reducing the size of the filesystem.  This can done using
+`fdisk`(8) or `parted`(8) to delete the existing partition and recreate
+it with the new desired size.  When recreating the partition make sure to use
+the same starting disk cylinder as before.
+
+'label' [<dev>|<mount_point>] [newlabel]::
+Show or update the label of a filesystem.
++
+[<device>|<mountpoint>] is used to identify the filesystem. 
+If a newlabel optional argument is passed, the label is changed.
+NOTE: the maximum allowable length shall be less than 256 chars
+
+'disk-usage' [-tb] path [path...]::
+Show in which disk the chunks are allocated. +
+If '-b' is given,  set byte as unit;
+If '-t' is given, show data in tabular format.
+
+EXIT STATUS
+-----------
+'btrfs filesystem' returns a zero exist status if it succeeds. Non zero is
+returned in case of failure.
+
+AVAILABILITY
+------------
+'btrfs' is part of btrfs-progs. Btrfs filesystem is currently under heavy
+development,
+and not suitable for any uses other than benchmarking and review.
+Please refer to the btrfs wiki http://btrfs.wiki.kernel.org for
+further details.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8),
-- 
1.9.1

[PATCH 04/27] btrfs-progs: Convert man page for btrfs-balance.

[Qu Wenruo]

Convert man page for btrfs-balance.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile          |  2 +-
 Documentation/btrfs-balance.txt | 77 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 78 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/btrfs-balance.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 1fa2b35..4839b6d 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -18,7 +18,7 @@ MAN8_TXT += btrfs.txt
 # Sub commands for btrfs
 MAN8_TXT += btrfs-subvolume.txt
 MAN8_TXT += btrfs-filesystem.txt
-#MAN8_TXT += btrfs-balance.txt
+MAN8_TXT += btrfs-balance.txt
 #MAN8_TXT += btrfs-device.txt
 #MAN8_TXT += btrfs-scrub.txt
 #MAN8_TXT += btrfs-check.txt
diff --git a/Documentation/btrfs-balance.txt b/Documentation/btrfs-balance.txt
new file mode 100644
index 0000000..2289fdf
--- /dev/null
+++ b/Documentation/btrfs-balance.txt
@@ -0,0 +1,77 @@
+btrfs-balance(8)
+================
+
+NAME
+----
+btrfs-balance - balance btrfs filesystem
+
+SYNOPSIS
+--------
+'btrfs [filesystem] balance' <subcommand>|<args>
+
+DESCRIPTION
+-----------
+'btrfs balance' is used to balance chunks in a btrfs filesystem across
+multiple or even single device.
+
+SUBCOMMAND
+----------
+<path>::
+Balance chunks across the devices *online*.
++
+'btrfs balance <path>' is deprecated,
+please use 'btrfs balance start' command instead.
+
+'start' [options] <path>::
+Balance chunks across the devices *online*.
++
+Balance and/or convert (change allocation profile of) chunks that
+passed all filters in a comma-separated list of filters for a
+particular chunk type.
+If filter list is not given balance all chunks of that type.
+In case none of the -d, -m or -s options is
+given balance all chunks in a filesystem.
++
+`Options`
++
+-d[filters]::::
+act on data chunks
+-m[filters]::::
+act on metadata chunks
+-s[filters]::::
+act on system chunks (only under -f)
+-v::::
+be verbose
+-f::::
+force reducing of metadata integrity
+
+'pause' <path>::
+Pause running balance.
+
+'cancel' <path>::
+Cancel running or paused balance.
+
+'resume' <path>::
+Resume interrupted balance.
+
+'status' [-v] <path>::
+Show status of running or paused balance.
++
+If '-v' option is given, output will be verbose.
+
+EXIT STATUS
+-----------
+'btrfs balance' returns a zero exist status if it succeeds. Non zero is
+returned in case of failure.
+
+AVAILABILITY
+------------
+'btrfs' is part of btrfs-progs. Btrfs filesystem is currently under heavy
+development,
+and not suitable for any uses other than benchmarking and review.
+Please refer to the btrfs wiki http://btrfs.wiki.kernel.org for
+further details.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8),
-- 
1.9.1

[PATCH 05/27] btrfs-progs: Convert man page for btrfs-device subcommand.

[Qu Wenruo]

Convert man page for btrfs-device subcommand.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile         |  2 +-
 Documentation/btrfs-device.txt | 75 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 76 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/btrfs-device.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 4839b6d..79ae9de 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -19,7 +19,7 @@ MAN8_TXT += btrfs.txt
 MAN8_TXT += btrfs-subvolume.txt
 MAN8_TXT += btrfs-filesystem.txt
 MAN8_TXT += btrfs-balance.txt
-#MAN8_TXT += btrfs-device.txt
+MAN8_TXT += btrfs-device.txt
 #MAN8_TXT += btrfs-scrub.txt
 #MAN8_TXT += btrfs-check.txt
 #MAN8_TXT += btrfs-rescue.txt
diff --git a/Documentation/btrfs-device.txt b/Documentation/btrfs-device.txt
new file mode 100644
index 0000000..20d7bcd
--- /dev/null
+++ b/Documentation/btrfs-device.txt
@@ -0,0 +1,75 @@
+btrfs-device(8)
+===============
+
+NAME
+----
+btrfs-device - control btrfs devices
+
+SYNOPSIS
+--------
+'btrfs device' <subcommand> <args>
+
+DESCRIPTION
+-----------
+'btrfs device' is used to control the btrfs devices, since btrfs can be used
+across several devices, 'btrfs device' is used for multiple device management.
+
+SUBCOMMAND
+----------
+'add' [-Kf] <dev> [<dev>...] <path>::
+Add device(s) to the filesystem identified by <path>.
++
+If applicable, a whole device discard (TRIM) operation is performed.
++
+`Options`
++
+-K|--nodiscard::::
+do not perform discard by default
+-f|--force::::
+force overwrite of existing filesystem on the given disk(s)
+
+'delete' <dev> [<dev>...] <path>::
+Remove device(s) from a filesystem identified by <path>.
+
+'scan' [(--all-devices|-d)|<device> [<device>...]]::
+Scan devices for a btrfs filesystem.
++
+If one or more devices are passed, these are scanned for a btrfs filesystem. 
+If no devices are passed, btrfs uses block devices containing btrfs
+filesystem as listed by blkid.
+Finally, if '--all-devices' or '-d' is passed, all the devices under /dev are 
+scanned.
+
+'disk-usage' [-b] <path> [<path>..]::
+Show which chunks are in a device.
++
+If '-b' is given, byte will be set as unit.
+
+'ready' <device>::
+Check device to see if it has all of it's devices in cache for mounting.
+
+'stats' [-z] <path>|<device>::
+Read and print the device IO stats for all devices of the filesystem
+identified by <path> or for a single <device>.
++
+`Options`
++
+-z::::
+Reset stats to zero after reading them.
+
+EXIT STATUS
+-----------
+'btrfs device' returns a zero exist status if it succeeds. Non zero is
+returned in case of failure.
+
+AVAILABILITY
+------------
+'btrfs' is part of btrfs-progs. Btrfs filesystem is currently under heavy
+development,
+and not suitable for any uses other than benchmarking and review.
+Please refer to the btrfs wiki http://btrfs.wiki.kernel.org for
+further details.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8),
-- 
1.9.1

[PATCH 06/27] btrfs-progs: Convert man page for btrfs-scrub

[Qu Wenruo]

Convert man page for btrfs-scrub.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile        |  2 +-
 Documentation/btrfs-scrub.txt | 98 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 99 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/btrfs-scrub.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 79ae9de..68b7e1f 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -20,7 +20,7 @@ MAN8_TXT += btrfs-subvolume.txt
 MAN8_TXT += btrfs-filesystem.txt
 MAN8_TXT += btrfs-balance.txt
 MAN8_TXT += btrfs-device.txt
-#MAN8_TXT += btrfs-scrub.txt
+MAN8_TXT += btrfs-scrub.txt
 #MAN8_TXT += btrfs-check.txt
 #MAN8_TXT += btrfs-rescue.txt
 #MAN8_TXT += btrfs-inspect-internal.txt
diff --git a/Documentation/btrfs-scrub.txt b/Documentation/btrfs-scrub.txt
new file mode 100644
index 0000000..3973f31
--- /dev/null
+++ b/Documentation/btrfs-scrub.txt
@@ -0,0 +1,98 @@
+btrfs-scrub(8)
+==============
+
+NAME
+----
+btrfs-scrub - scrub btrfs filesystem
+
+SYNOPSIS
+--------
+'btrfs scrub' <subcommand> <args>
+
+DESCRIPTION
+-----------
+'btrfs scrub' is used to scrub a btrfs filesystem, which will reading all data
+from all disks and verifying checksums.
+
+SUBCOMMAND
+----------
+'start' [-BdqrRf] [-c ioprio_class -n ioprio_classdata] <path>|<device>::
+Start a scrub on all devices of the filesystem identified by <path> or on
+a single <device>.
++
+Without options, scrub is started as a background process.
+Progress can be obtained with the 'scrub status' command. Scrubbing
+involves reading all data from all disks and verifying checksums. Errors are
+corrected along the way if possible.
++
+The default IO priority of scrub is the idle class. The priority can be
+configured similar to the `ionice`(1) syntax using '-c' and '-n' options.
++
+`Options`
++
+-B::::
+Do not background and print scrub statistics when finished.
+-d::::
+Print separate statistics for each device of the filesystem (-B only).
+-q::::
+Quiet. Omit error messages and statistics.
+-r::::
+Read only mode. Do not attempt to correct anything.
+-R::::
+Raw print mode. Print full data instead of summary.
+-c ioprio_class::::
+Set IO priority class (see
+ ionice (1)
+manpage).
+-n ioprio_classdata::::
+Set IO priority classdata (see `ionice`(1) manpage).
+-f::::
+force to check whether scrub has started or resumed in userspace.
+this is useful when scrub stat record file is damaged.
+
+'cancel' <path>|<device>::
+If a scrub is running on the filesystem identified by <path>, cancel it.
++
+Progress is saved in the scrub progress file and scrubbing can be resumed later
+using the scrub resume command.
+If a <device> is given, the corresponding filesystem is found and
+scrub cancel behaves as if it was called on that filesystem.
+
+'resume' [-BdqrR] [-c ioprio_class -n ioprio_classdata] <path>|<device>::
+Resume a canceled or interrupted scrub cycle on the filesystem identified by
+<path> or on a given <device>.
++
+Does not start a new scrub if the last scrub finished successfully.
++
+`Options`
++
+see 'scrub start'.
+
+'status' [-d] <path>|<device>::
+Show status of a running scrub for the filesystem identified by <path> or
+for the specified <device>.
++
+If no scrub is running, show statistics of the last finished or canceled scrub
+for that filesystem or device.
++
+`Options`
++
+-d::::
+Print separate statistics for each device of the filesystem.
+
+EXIT STATUS
+-----------
+'btrfs scrub' returns a zero exist status if it succeeds. Non zero is
+returned in case of failure.
+
+AVAILABILITY
+------------
+'btrfs' is part of btrfs-progs. Btrfs filesystem is currently under heavy
+development,
+and not suitable for any uses other than benchmarking and review.
+Please refer to the btrfs wiki http://btrfs.wiki.kernel.org for
+further details.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8),
-- 
1.9.1

[PATCH 07/27] btrfs-progs: Convert man page for btrfs-check.

[Qu Wenruo]

Convert man page for btrfs-check.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile        |  2 +-
 Documentation/btrfs-check.txt | 45 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 46 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/btrfs-check.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 68b7e1f..a838ca1 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -21,7 +21,7 @@ MAN8_TXT += btrfs-filesystem.txt
 MAN8_TXT += btrfs-balance.txt
 MAN8_TXT += btrfs-device.txt
 MAN8_TXT += btrfs-scrub.txt
-#MAN8_TXT += btrfs-check.txt
+MAN8_TXT += btrfs-check.txt
 #MAN8_TXT += btrfs-rescue.txt
 #MAN8_TXT += btrfs-inspect-internal.txt
 #MAN8_TXT += btrfs-send.txt
diff --git a/Documentation/btrfs-check.txt b/Documentation/btrfs-check.txt
new file mode 100644
index 0000000..accbed4
--- /dev/null
+++ b/Documentation/btrfs-check.txt
@@ -0,0 +1,45 @@
+btrfs-check(8)
+==============
+
+NAME
+----
+btrfs-check - check or repair a btrfs filesystem offline
+
+SYNOPSIS
+--------
+'btrfs check' [options] <device>
+
+DESCRIPTION
+-----------
+'btrfs check' is used to check or repair a btrfs filesystem offline.
+
+OPTIONS
+-------
+-s|--support <superblock>::
+use <superblock>th superblock copy.
+--repair::
+try to repair the filesystem.
+--init-csum-tree::
+create a new CRC tree.
+--init-extent-tree::
+create a new extent tree.
+
+EXIT STATUS
+-----------
+'btrfs check' returns a zero exist status if it succeeds. Non zero is
+returned in case of failure.
+
+AVAILABILITY
+------------
+'btrfs' is part of btrfs-progs. Btrfs filesystem is currently under heavy
+development,
+and not suitable for any uses other than benchmarking and review.
+Please refer to the btrfs wiki http://btrfs.wiki.kernel.org for
+further details.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8),
+`btrfs-scrub`(8),
+`btrfs-rescue`(8)
+`btrfsck`(8)
-- 
1.9.1

[PATCH 08/27] btrfs-progs: Convert man page for btrfs-rescue

[Qu Wenruo]

Convert man page for btrfs-rescue.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile         |  2 +-
 Documentation/btrfs-rescue.txt | 60 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 61 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/btrfs-rescue.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index a838ca1..ebe0455 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -22,7 +22,7 @@ MAN8_TXT += btrfs-balance.txt
 MAN8_TXT += btrfs-device.txt
 MAN8_TXT += btrfs-scrub.txt
 MAN8_TXT += btrfs-check.txt
-#MAN8_TXT += btrfs-rescue.txt
+MAN8_TXT += btrfs-rescue.txt
 #MAN8_TXT += btrfs-inspect-internal.txt
 #MAN8_TXT += btrfs-send.txt
 #MAN8_TXT += btrfs-receive.txt
diff --git a/Documentation/btrfs-rescue.txt b/Documentation/btrfs-rescue.txt
new file mode 100644
index 0000000..f66ca85
--- /dev/null
+++ b/Documentation/btrfs-rescue.txt
@@ -0,0 +1,60 @@
+btrfs-check(8)
+==============
+
+NAME
+----
+btrfs-rescue - Recover a damaged btrfs filesystem
+
+SYNOPSIS
+--------
+'btrfs rescue' <subcommand> <args>
+
+DESCRIPTION
+-----------
+'btrfs rescue' is used to try to recover a damaged btrfs filesystem.
+
+SUBCOMMAND
+----------
+'chunk-recover' [options] <device>::
+Recover the chunk tree by scanning the devices
++
+`Options`
++
+-y::::
+assume an answer of 'yes' to all questions.
+-v::::
+verbose mode.
+-h::::
+help.
+
+NOTE: Since 'chunk-recover' will scan the whole device, it will be *VERY* slow
+especially executed on a large device.
+
+'super-recover' [options] <device>::
+Recover bad superblocks from good copies.
++
+`Options`
++
+-y::::
+assume an answer of 'yes' to all questions.
+-v::::
+verbose mode.
+
+EXIT STATUS
+-----------
+'btrfs rescue' returns a zero exist status if it succeeds. Non zero is
+returned in case of failure.
+
+AVAILABILITY
+------------
+'btrfs' is part of btrfs-progs. Btrfs filesystem is currently under heavy
+development,
+and not suitable for any uses other than benchmarking and review.
+Please refer to the btrfs wiki http://btrfs.wiki.kernel.org for
+further details.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8),
+`btrfs-scrub`(8),
+`btrfs-check`(8)
-- 
1.9.1

[PATCH 09/27] btrfs-progs: Convert man page for btrfs-inspect-internal

[Qu Wenruo]

Convert man page for btrfs-inspect-internal.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile                   |  2 +-
 Documentation/btrfs-inspect-internal.txt | 69 ++++++++++++++++++++++++++++++++
 2 files changed, 70 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/btrfs-inspect-internal.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index ebe0455..6655f06 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -23,7 +23,7 @@ MAN8_TXT += btrfs-device.txt
 MAN8_TXT += btrfs-scrub.txt
 MAN8_TXT += btrfs-check.txt
 MAN8_TXT += btrfs-rescue.txt
-#MAN8_TXT += btrfs-inspect-internal.txt
+MAN8_TXT += btrfs-inspect-internal.txt
 #MAN8_TXT += btrfs-send.txt
 #MAN8_TXT += btrfs-receive.txt
 #MAN8_TXT += btrfs-quota.txt
diff --git a/Documentation/btrfs-inspect-internal.txt b/Documentation/btrfs-inspect-internal.txt
new file mode 100644
index 0000000..4555c70
--- /dev/null
+++ b/Documentation/btrfs-inspect-internal.txt
@@ -0,0 +1,69 @@
+btrfs-inspect-internal(8)
+=========================
+
+NAME
+----
+btrfs-inspect-internal - resolve different btrfs items for debug purpose
+
+SYNOPSIS
+--------
+'btrfs inspect-internal' <subcommand> <args>
+
+DESCRIPTION
+-----------
+'btrfs inspect-internal' is used to resolve different items for debug purpose.
+
+SUBCOMMAND
+----------
+'inode-resolve' [-v] <inode> <path>::
+Resolves an <inode> in subvolume <path> to all filesystem paths.
++
+`Options`
++
+-v::::
+verbose mode. print count of returned paths and ioctl() return value
+
+'logical-resolve' [-Pv] [-s bufsize] <logical> <path>::
+Resolves a <logical> address in the filesystem mounted at <path> to all inodes.
++
+By default, each inode is then resolved to a file system path (similar to the
+inode-resolve subcommand).
++
+`Options`
++
+-P::::
+skip the path resolving and print the inodes instead
+-v::::
+verbose mode. print count of returned paths and all ioctl() return values
+-s <bufsize>::::
+set inode container's size.
++
+This is used to increase inode container's size in case it is
+not enough to read all the resolved results. The max value one can set is 64k.
+
+'subvolid-resolve' <subvolid> <path>::
+Get file system paths for the given subvolume ID.
+
+'rootid' <path>::
+For a given file or directory, return the containing tree root id. For a
+subvolume return it's own tree id.
++
+The result is undefined for the so-called empty subvolumes (identified by inode number 2).
+
+EXIT STATUS
+-----------
+'btrfs inspect-internal' returns a zero exist status if it succeeds. Non zero is
+returned in case of failure.
+
+AVAILABILITY
+------------
+'btrfs' is part of btrfs-progs. Btrfs filesystem is currently under heavy
+development,
+and not suitable for any uses other than benchmarking and review.
+Please refer to the btrfs wiki http://btrfs.wiki.kernel.org for
+further details.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8),
+`btrfs-debug-tree`(8)
-- 
1.9.1

[PATCH 10/27] btrfs-progs: Convert man page for btrfs-send.

[Qu Wenruo]

Convert man page for btrfs-send.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile       |  2 +-
 Documentation/btrfs-send.txt | 60 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 61 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/btrfs-send.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 6655f06..081ec42 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -24,7 +24,7 @@ MAN8_TXT += btrfs-scrub.txt
 MAN8_TXT += btrfs-check.txt
 MAN8_TXT += btrfs-rescue.txt
 MAN8_TXT += btrfs-inspect-internal.txt
-#MAN8_TXT += btrfs-send.txt
+MAN8_TXT += btrfs-send.txt
 #MAN8_TXT += btrfs-receive.txt
 #MAN8_TXT += btrfs-quota.txt
 #MAN8_TXT += btrfs-replace.txt
diff --git a/Documentation/btrfs-send.txt b/Documentation/btrfs-send.txt
new file mode 100644
index 0000000..18a98fa
--- /dev/null
+++ b/Documentation/btrfs-send.txt
@@ -0,0 +1,60 @@
+btrfs-send(8)
+=============
+
+NAME
+----
+btrfs-send - send data of subvolume(s) to stdout/file.
+
+SYNOPSIS
+--------
+'btrfs send' [-ve] [-p <parent>] [-c <clone-src>] [-f <outfile>] <subvol> [<subvol>...]
+
+DESCRIPTION
+-----------
+Sends the subvolume(s) specified by <subvol> to stdout.
+
+By default, this will send the whole subvolume. To do an incremental
+send, use '-p <parent>'.
+
+If you want to allow btrfs to clone from any additional local snapshots,
+use '-c <clone-src>' (multiple times where applicable). 
+
+You must not specify clone sources unless you guarantee that these snapshots
+are exactly in the same state on both sides, the sender and the receiver.
+
+It is allowed to omit the '-p <parent>' option when '-c <clone-src>' options
+are given, in which case 'btrfs send' will determine a suitable parent among the
+clone sources itself.
+
+`Options`
+
+-v::
+Enable verbose debug output. Each occurrence of this option increases the
+verbose level more.
+-e::
+If sending multiple subvols at once, use the new format and omit the <end cmd> between the subvols.
+-p <parent>::
+Send an incremental stream from <parent> to <subvol>.
+-c <clone-src>::
+Use this snapshot as a clone source for an incremental send (multiple allowed).
+-f <outfile>::
+Output is normally written to stdout. To write to a file, use this option.
+An alternative would be to use pipes.
+
+EXIT STATUS
+-----------
+'btrfs send' returns a zero exist status if it succeeds. Non zero is
+returned in case of failure.
+
+AVAILABILITY
+------------
+'btrfs' is part of btrfs-progs. Btrfs filesystem is currently under heavy
+development,
+and not suitable for any uses other than benchmarking and review.
+Please refer to the btrfs wiki http://btrfs.wiki.kernel.org for
+further details.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8),
+`btrfs-receive`(8)
-- 
1.9.1

[PATCH 11/27] btrfs-progs: Convert man page for btrfs-receive.

[Qu Wenruo]

Convert man page for btrfs-receive.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile          |  2 +-
 Documentation/btrfs-receive.txt | 58 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 59 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/btrfs-receive.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 081ec42..2af4d40 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -25,7 +25,7 @@ MAN8_TXT += btrfs-check.txt
 MAN8_TXT += btrfs-rescue.txt
 MAN8_TXT += btrfs-inspect-internal.txt
 MAN8_TXT += btrfs-send.txt
-#MAN8_TXT += btrfs-receive.txt
+MAN8_TXT += btrfs-receive.txt
 #MAN8_TXT += btrfs-quota.txt
 #MAN8_TXT += btrfs-replace.txt
 #MAN8_TXT += btrfs-dedup.txt
diff --git a/Documentation/btrfs-receive.txt b/Documentation/btrfs-receive.txt
new file mode 100644
index 0000000..a67bc66
--- /dev/null
+++ b/Documentation/btrfs-receive.txt
@@ -0,0 +1,58 @@
+btrfs-receive(8)
+================
+
+NAME
+----
+btrfs-receive - receive subvolumes from stdin/file.
+
+SYNOPSIS
+--------
+'btrfs receive' [-ve] [-f <infile>] <mount>
+
+DESCRIPTION
+-----------
+Receives one or more subvolumes that were previously
+sent with 'btrfs send'. The received subvolumes are stored
+into <mount>.
+
+'btrfs receive' will fail with the following case:
+
+1. a receiving subvolume already exists.
+
+2. a previously received subvolume was changed after it was received.
+
+3. default subvolume is changed or you don't mount btrfs filesystem with
+fs tree.
+
+After receiving a subvolume, it is immediately set to read only.
+
+`Options`
+
+-v::
+Enable verbose debug output. Each occurrence of this option increases the
+verbose level more.
+-f <infile>::
+By default, btrfs receive uses stdin to receive the subvolumes.
+Use this option to specify a file to use instead.
+-e::
+Terminate after receiving an <end cmd> in the data stream.
+Without this option, the receiver terminates only if an error is recognized
+or on EOF.
+
+EXIT STATUS
+-----------
+'btrfs receive' returns a zero exist status if it succeeds. Non zero is
+returned in case of failure.
+
+AVAILABILITY
+------------
+'btrfs' is part of btrfs-progs. Btrfs filesystem is currently under heavy
+development,
+and not suitable for any uses other than benchmarking and review.
+Please refer to the btrfs wiki http://btrfs.wiki.kernel.org for
+further details.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8),
+`btrfs-send`(8)
-- 
1.9.1

[PATCH 12/27] btrfs-progs: Convert man page for btrfs-quota.

[Qu Wenruo]

Convert man page for btrfs-quota.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile        |  2 +-
 Documentation/btrfs-quota.txt | 59 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 60 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/btrfs-quota.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 2af4d40..645a8e4 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -26,7 +26,7 @@ MAN8_TXT += btrfs-rescue.txt
 MAN8_TXT += btrfs-inspect-internal.txt
 MAN8_TXT += btrfs-send.txt
 MAN8_TXT += btrfs-receive.txt
-#MAN8_TXT += btrfs-quota.txt
+MAN8_TXT += btrfs-quota.txt
 #MAN8_TXT += btrfs-replace.txt
 #MAN8_TXT += btrfs-dedup.txt
 
diff --git a/Documentation/btrfs-quota.txt b/Documentation/btrfs-quota.txt
new file mode 100644
index 0000000..82f2f49
--- /dev/null
+++ b/Documentation/btrfs-quota.txt
@@ -0,0 +1,59 @@
+btrfs-quota(8)
+==============
+
+NAME
+----
+btrfs-quota - control the quota of a btrfs filesystem
+
+SYNOPSIS
+--------
+'btrfs quota' <subcommand> <args>
+
+DESCRIPTION
+-----------
+'btrfs quota' is used to enable/disable or rescan subvolume quota of a btrfs
+filesystem.
+
+For setting quota or other quota operations on a btrfs filesystem, please see
+`btrfs-qgroup`(8) for details.
+
+WARNING: Quota and qgroup in btrfs filesystem is not stable and impacts
+performance in mainline kernel yet(v3.14 so far).
+
+SUBCOMMAND
+----------
+'enable' <path>::
+Enable subvolume quota support for a filesystem.
+
+
+'disable' <path>::
+Disable subvolume quota support for a filesystem.
+
+'rescan' [-s] <path>::
+Trash all qgroup numbers and scan the metadata again with the current config.
++
+`Options`
++
+-s::::
+show status of a running rescan operation.
+-w::::
+wait for rescan operation to finish(can be already in progress).
+
+EXIT STATUS
+-----------
+'btrfs quota' returns a zero exist status if it succeeds. Non zero is
+returned in case of failure.
+
+AVAILABILITY
+------------
+'btrfs' is part of btrfs-progs. Btrfs filesystem is currently under heavy
+development,
+and not suitable for any uses other than benchmarking and review.
+Please refer to the btrfs wiki http://btrfs.wiki.kernel.org for
+further details.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8),
+`btrfs-subvolume`(8),
+`btrfs-qgroup`(8)
-- 
1.9.1

[PATCH 13/27] btrfs-progs: Convert and enhance the man page of btrfs-qgroup.

[Qu Wenruo]

Convert and enhance the man page of btrfs-qgroup.

The original man page for btrfs-qgroup subcommand is almost useless for
new user(like me), so adds more information on it.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile         |   1 +
 Documentation/btrfs-qgroup.txt | 110 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 111 insertions(+)
 create mode 100644 Documentation/btrfs-qgroup.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 645a8e4..84a4859 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -27,6 +27,7 @@ MAN8_TXT += btrfs-inspect-internal.txt
 MAN8_TXT += btrfs-send.txt
 MAN8_TXT += btrfs-receive.txt
 MAN8_TXT += btrfs-quota.txt
+MAN8_TXT += btrfs-qgroup.txt
 #MAN8_TXT += btrfs-replace.txt
 #MAN8_TXT += btrfs-dedup.txt
 
diff --git a/Documentation/btrfs-qgroup.txt b/Documentation/btrfs-qgroup.txt
new file mode 100644
index 0000000..d054423
--- /dev/null
+++ b/Documentation/btrfs-qgroup.txt
@@ -0,0 +1,110 @@
+btrfs-qgroup(8)
+===============
+
+NAME
+----
+btrfs-qgroup - control the quota group of a btrfs filesystem
+
+SYNOPSIS
+--------
+'btrfs qgroup' <subcommand> <args>
+
+DESCRIPTION
+-----------
+'btrfs qgroup' is used to control quota group(qgroup) of a btrfs filesystem.
+
+NOTE: To use qgroup, it needs to enable quota first using 'btrfs quota'
+command.
+
+WARNING: Qgroup is not stable yet and will impact performance in current mainline
+kernel(v3.14 so far).
+
+QGROUP
+------
+Quota group or qgroup in btrfs has its hierarchy like subvolume.
+One subvolume/snapshot can reach its quota limits if it consumes all the quota
+assigned to it or any of the parent qgroup(s).
+
+Also for snapshot, it consumes no quota initially since all its data
+shares with its parent, so only modification in snapshot consumes quota.
+
+Every subvolume/snapshot will have its own qgroup with id '0/<subvolume id>'
+upon creating, but can be later destroyed by 'btrfs qgroup destroy' command.
+
+NOTE: If the qgroup of a subvolume is destroyed, quota about the subvolume
+will not be functional until qgroup '0/<subvolume id>' is created again.
+
+SUBCOMMAND
+----------
+'assign' <src> <dst> <path>::
+Assign qgroup <src> as the child qgroup of <dst> in the btrfs filesystem
+identified by <path>.
+
+'remove' <src> <dst> <path>::
+Remove the relationship between child qgroup <src> and parent qgroup <dst> in
+the btrfs filesystem identified by <path>.
+
+'create' <qgroupid> <path>::
+Create a subvolume quota group.
++
+For the '0/<subvolume id>' qgroup, a qgroup can be created even before the
+subvolume created.
+
+'destroy' <qgroupid> <path>::
+Destroy a qgroup.
++
+If a qgroup is no isolated,which means it is a parent or child qgroup, it
+can't be destroyed.
+
+'show' [options] <path>::
+Show all qgroups in the btrfs filesystem identified by <path>.
++
+`Options`
++
+-p::::
+print parent qgroup id.
+-c::::
+print child qgroup id.
+-r::::
+print max referenced size of qgroup.
+-e::::
+print max exclusive size of qgroup.
+-F::::
+list all qgroups which impact the given path(include ancestral qgroups)
+-f::::
+list all qgroups which impact the given path(exclude ancestral qgroups)
+--sort=[\+/-]ATTR[,[+/-]ATTR]...::::
+list qgroups in order of ATTR.
++
+ATTR can be one or more of qgroupid,rfer,excl,max_rfer,max_excl.
++
+Prefix \'+' means ascending order and \'-' means descending order of ATTR.
+If no prefix is given, use ascending order by default.
++
+If multiple ATTRs is given, use comma to separate.
+
+'limit' [options] <size>|none [<qgroupid>] <path>::
+Limit the size of a qgroup to <size> or no limit in the btrfs filesystem
+identified by <path>.
++
+If <qgroupid> is not given, qgroup of the subvolume identified by <path>
+is used if possible.
+
+EXIT STATUS
+-----------
+'btrfs qgroup' returns a zero exist status if it succeeds. Non zero is
+returned in case of failure.
+
+AVAILABILITY
+------------
+'btrfs' is part of btrfs-progs. Btrfs filesystem is currently under heavy
+development,
+and not suitable for any uses other than benchmarking and review.
+Please refer to the btrfs wiki http://btrfs.wiki.kernel.org for
+further details.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8),
+`btrfs-subvolume`(8),
+`btrfs-quota`(8),
-- 
1.9.1

[PATCH 14/27] btrfs-progs: Convert man page for btrfs-replace.

[Qu Wenruo]

Convert man page for btrfs-replace.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile          |  2 +-
 Documentation/btrfs-replace.txt | 76 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 77 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/btrfs-replace.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 84a4859..a4f7ef0 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -28,7 +28,7 @@ MAN8_TXT += btrfs-send.txt
 MAN8_TXT += btrfs-receive.txt
 MAN8_TXT += btrfs-quota.txt
 MAN8_TXT += btrfs-qgroup.txt
-#MAN8_TXT += btrfs-replace.txt
+MAN8_TXT += btrfs-replace.txt
 #MAN8_TXT += btrfs-dedup.txt
 
 MAN_TXT = $(MAN8_TXT)
diff --git a/Documentation/btrfs-replace.txt b/Documentation/btrfs-replace.txt
new file mode 100644
index 0000000..8c5dcc4
--- /dev/null
+++ b/Documentation/btrfs-replace.txt
@@ -0,0 +1,76 @@
+btrfs-replace(8)
+===============
+
+NAME
+----
+btrfs-replace - replace devices managed by btrfs with other device.
+
+SYNOPSIS
+--------
+'btrfs replace' <subcommand> <args>
+
+DESCRIPTION
+-----------
+'btrfs replace' is used to replace btrfs managed devices with other device.
+
+SUBCOMMAND
+----------
+'start' [-Bfr] <srcdev>|<devid> <targetdev> <path>::
+Replace device of a btrfs filesystem.
++
+On a live filesystem, duplicate the data to the target device which
+is currently stored on the source device.
+If the source device is not available anymore, or if the -r option is set,
+the data is built only using the RAID redundancy mechanisms.
+After completion of the operation, the source device is removed from the
+filesystem.
+If the <srcdev> is a numerical value, it is assumed to be the device id
+of the filesystem which is mounted at <path>, otherwise is is
+the path to the source device. If the source device is disconnected,
+from the system, you have to use the devid parameter format.
+The <targetdev> needs to be same size or larger than the <srcdev>.
++
+`Options`
++
+-r::::
+only read from <srcdev> if no other zero-defect mirror exists.
+(enable this if your drive has lots of read errors, the access would be very
+slow)
+-f::::
+force using and overwriting <targetdev> even if it looks like
+containing a valid btrfs filesystem.
++
+A valid filesystem is assumed if a btrfs superblock is found which contains a
+correct checksum. Devices which are currently mounted are
+never allowed to be used as the <targetdev>.
+-B::::
+no background replace.
+
+'status' [-1] <mount_point>::
+Print status and progress information of a running device replace operation.
++
+`Options`
++
+-1::::
+print once instead of print continuously until the replace
+operation finishes (or is canceled)
+
+'cancel' <mount_point>::
+Cancel a running device replace operation.
+
+EXIT STATUS
+-----------
+'btrfs replace' returns a zero exist status if it succeeds. Non zero is
+returned in case of failure.
+
+AVAILABILITY
+------------
+'btrfs' is part of btrfs-progs. Btrfs filesystem is currently under heavy
+development,
+and not suitable for any uses other than benchmarking and review.
+Please refer to the btrfs wiki http://btrfs.wiki.kernel.org for
+further details.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8),
-- 
1.9.1

[PATCH 15/27] btrfs-progs: Convert man page for btrfs-dedup.

[Qu Wenruo]

Although the btrfs-dedup function is not available in mainline kernel,
add the man page any way and add a warning into it.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile        |  2 +-
 Documentation/btrfs-dedup.txt | 51 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 52 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/btrfs-dedup.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index a4f7ef0..4040951 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -29,7 +29,7 @@ MAN8_TXT += btrfs-receive.txt
 MAN8_TXT += btrfs-quota.txt
 MAN8_TXT += btrfs-qgroup.txt
 MAN8_TXT += btrfs-replace.txt
-#MAN8_TXT += btrfs-dedup.txt
+MAN8_TXT += btrfs-dedup.txt
 
 MAN_TXT = $(MAN8_TXT)
 MAN_XML = $(patsubst %.txt,%.xml,$(MAN_TXT))
diff --git a/Documentation/btrfs-dedup.txt b/Documentation/btrfs-dedup.txt
new file mode 100644
index 0000000..d42cee6
--- /dev/null
+++ b/Documentation/btrfs-dedup.txt
@@ -0,0 +1,51 @@
+btrfs-dedup(8)
+==============
+
+NAME
+----
+btrfs-dedup - control the deduplication in a btrfs filesystem
+
+SYNOPSIS
+--------
+'btrfs dedup' <subcommand> <args>
+
+DESCRIPTION
+-----------
+WARNING: Deduplication function is not implemented in mainline kernel(v3.14
+yet). So most user may find this function not usable.
+
+'btrfs dedup' is used to control deduplication in a bttrfs filesystem.
+
+SUBCOMMAND
+----------
+'enable' <path>::
+Enable data deduplication support for a filesystem.
+
+
+'disable' <path>::
+Disable data deduplication support for a filesystem.
+
+
+'on' [-b|--bs size] <path>::
+Switch on data deduplication or change the dedup blocksize.
+
+
+'off' <path>::
+Switch off data deduplication.
+
+EXIT STATUS
+-----------
+'btrfs dedup' returns a zero exist status if it succeeds. Non zero is
+returned in case of failure.
+
+AVAILABILITY
+------------
+'btrfs' is part of btrfs-progs. Btrfs filesystem is currently under heavy
+development,
+and not suitable for any uses other than benchmarking and review.
+Please refer to the btrfs wiki http://btrfs.wiki.kernel.org for
+further details.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8),
-- 
1.9.1

[PATCH 16/27] btrfs-progs: Convert man page for btrfsck

[Qu Wenruo]

Convert man page for btrfsck.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile    |  2 +-
 Documentation/btrfsck.txt | 55 +++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 56 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/btrfsck.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 4040951..b4d8281 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -3,7 +3,7 @@ MAN8_TXT =
 
 # Top level commands
 MAN8_TXT += btrfs.txt
-#MAN8_TXT += btrfsck.txt
+MAN8_TXT += btrfsck.txt
 #MAN8_TXT += btrfs-convert.txt
 #MAN8_TXT += btrfs-debug-tree.txt
 #MAN8_TXT += btrfs-find-root.txt
diff --git a/Documentation/btrfsck.txt b/Documentation/btrfsck.txt
new file mode 100644
index 0000000..751c4d0
--- /dev/null
+++ b/Documentation/btrfsck.txt
@@ -0,0 +1,55 @@
+btrfsck(8)
+==========
+
+NAME
+----
+btrfsck - check or repair a btrfs filesystem offline
+
+SYNOPSIS
+--------
+'btrfsck' [options] <device>
+
+DESCRIPTION
+-----------
+'btrfsck' is used to check and optionally repair of a Btrfs filesystem.
+
+Now, it can only be run on an unmounted FS.
+<device> is the device file where the filesystem is stored.
+
+'btrfsck' is just an alias of 'btrfs check' command.
+
+WARNING: Considering it is not well-tested in real-life situations yet.
+If you have a broken Btrfs filesystem, btrfsck may not repair but cause
+additional damages. 
+
+
+OPTIONS
+-------
+-s|--support <superblock>::
+use <superblock>th superblock copy.
+--repair::
+try to repair the filesystem.
+--init-csum-tree::
+create a new CRC tree.
+--init-extent-tree::
+create a new extent tree.
+
+EXIT STATUS
+-----------
+'btrfsck' returns a zero exist status if it succeeds. Non zero is
+returned in case of failure.
+
+AVAILABILITY
+------------
+'btrfs' is part of btrfs-progs. Btrfs filesystem is currently under heavy
+development,
+and not suitable for any uses other than benchmarking and review.
+Please refer to the btrfs wiki http://btrfs.wiki.kernel.org for
+further details.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8),
+`btrfs-scrub`(8),
+`btrfs-rescue`(8)
+`btrfs-check`(8)
-- 
1.9.1

[PATCH 17/27] btrfs-progs: Convert man page for btrfs-convert.

[Qu Wenruo]

Convert man page for btrfs-convert.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile          |  2 +-
 Documentation/btrfs-convert.txt | 49 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 50 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/btrfs-convert.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index b4d8281..b2afc7a 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -4,7 +4,7 @@ MAN8_TXT =
 # Top level commands
 MAN8_TXT += btrfs.txt
 MAN8_TXT += btrfsck.txt
-#MAN8_TXT += btrfs-convert.txt
+MAN8_TXT += btrfs-convert.txt
 #MAN8_TXT += btrfs-debug-tree.txt
 #MAN8_TXT += btrfs-find-root.txt
 #MAN8_TXT += btrfs-image.txt
diff --git a/Documentation/btrfs-convert.txt b/Documentation/btrfs-convert.txt
new file mode 100644
index 0000000..780ca9e
--- /dev/null
+++ b/Documentation/btrfs-convert.txt
@@ -0,0 +1,49 @@
+btrfs-convert(8)
+================
+
+NAME
+----
+btrfs-convert - convert from ext2/3/4 filesystem to btrfs or rollback
+
+SYNOPSIS
+--------
+'btrfs-convert' [options] <device>
+
+DESCRIPTION
+-----------
+'btrfs-convert' is used to convert existed ext2/3/4 to btrfs filesystem,
+and the original filesystem image is accessible as from separate subvolume
+named 'ext2_subvol' as file image.
+
+OPTIONS
+-------
+-d::
+Disable data checksum.
+-i::
+Ignore xattrs and ACLs.
+-n::
+Disable packing of small files.
+-r::
+Roll back to ext2fs.
+
+EXIT STATUS
+-----------
+'btrfs-convert' will return 0 if no error happened.
+If any problems happened, 1 will be returned.
+
+AUTHOR
+------
+Written by Shilong Wang and Wenruo Qu.
+
+COPYRIGHT
+---------
+Copyright (C) 2013 FUJITSU LIMITED.
+
+License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
+
+This is free software: you are free  to  change  and  redistribute  it. There is NO WARRANTY, to the extent permitted by law.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8)
+
-- 
1.9.1

[PATCH 18/27] btrfs-progs: Convert man page for btrfs-debug-tree.

[Qu Wenruo]

Convert man page for btrfs-debug-tree.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile             |  2 +-
 Documentation/btrfs-debug-tree.txt | 50 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 51 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/btrfs-debug-tree.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index b2afc7a..460c14f 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -5,7 +5,7 @@ MAN8_TXT =
 MAN8_TXT += btrfs.txt
 MAN8_TXT += btrfsck.txt
 MAN8_TXT += btrfs-convert.txt
-#MAN8_TXT += btrfs-debug-tree.txt
+MAN8_TXT += btrfs-debug-tree.txt
 #MAN8_TXT += btrfs-find-root.txt
 #MAN8_TXT += btrfs-image.txt
 #MAN8_TXT += btrfs-map-logical.txt
diff --git a/Documentation/btrfs-debug-tree.txt b/Documentation/btrfs-debug-tree.txt
new file mode 100644
index 0000000..80a85d4
--- /dev/null
+++ b/Documentation/btrfs-debug-tree.txt
@@ -0,0 +1,50 @@
+btrfs-debug-tree(8)
+===================
+
+NAME
+----
+btrfs-debug-tree - dump btrfs filesystem metadata into stdout
+
+SYNOPSIS
+--------
+'btrfs-debug-tree' [options] <device>
+
+DESCRIPTION
+-----------
+'btrfs-debug-tree' is used to dump the whole tree of the given device.
+
+This is maybe useful for analyzing filesystem state or inconsistence and has
+a positive educational effect on understanding the internal structure.
+<device> is the device file where the filesystem is stored.
+
+OPTIONS
+-------
+-e::
+Print detailed extents info.
+-d::
+Print info of btrfs device and root tree dirs only.
+-r::
+Print info of roots only.
+-b <block_num>::
+Print info of the specified block only.
+
+EXIT STATUS
+-----------
+'btrfs-debug-tree' will return 0 if no error happened.
+If any problems happened, 1 will be returned.
+
+AUTHOR
+------
+Written by Shilong Wang and Wenruo Qu.
+
+COPYRIGHT
+---------
+Copyright (C) 2013 FUJITSU LIMITED.
+
+License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
+
+This is free software: you are free  to  change  and  redistribute  it. There is NO WARRANTY, to the extent permitted by law.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8)
-- 
1.9.1

[PATCH 19/27] btrfs-progs: Convert man page for btrfs-find-root.

[Qu Wenruo]

Convert man page for btrfs-find-root.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile            |  2 +-
 Documentation/btrfs-find-root.txt | 45 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 46 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/btrfs-find-root.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 460c14f..7a38df9 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -6,7 +6,7 @@ MAN8_TXT += btrfs.txt
 MAN8_TXT += btrfsck.txt
 MAN8_TXT += btrfs-convert.txt
 MAN8_TXT += btrfs-debug-tree.txt
-#MAN8_TXT += btrfs-find-root.txt
+MAN8_TXT += btrfs-find-root.txt
 #MAN8_TXT += btrfs-image.txt
 #MAN8_TXT += btrfs-map-logical.txt
 #MAN8_TXT += btrfs-show-super.txt
diff --git a/Documentation/btrfs-find-root.txt b/Documentation/btrfs-find-root.txt
new file mode 100644
index 0000000..18b1e84
--- /dev/null
+++ b/Documentation/btrfs-find-root.txt
@@ -0,0 +1,45 @@
+btrfs-find-root(8)
+==================
+
+NAME
+----
+btrfs-find-root - filter to find btrfs root
+
+SYNOPSIS
+--------
+'btrfs-find-root' [options] <dev>
+
+DESCRIPTION
+-----------
+'btrfs-find-root' is used to find the satisfied root, you can filter by
+root tree's objectid, generation, level.
+
+OPTIONS
+-------
+-g <generation>::
+Filter root tree by it's original transaction id, tree root's generation in default.
+-o <objectid>::
+Filter root tree by it's objectid,tree root's objectid in default.
+-l <level>::
+Filter root tree by B-+ tree's level, level 0 in default.
+
+EXIT STATUS
+-----------
+'btrfs-find-root' will return 0 if no error happened.
+If any problems happened, 1 will be returned.
+
+AUTHOR
+------
+Written by Shilong Wang and Wenruo Qu.
+
+COPYRIGHT
+---------
+Copyright (C) 2013 FUJITSU LIMITED.
+
+License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
+
+This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8)
-- 
1.9.1

[PATCH 20/27] btrfs-progs: Convert man page for btrfs-image.

[Qu Wenruo]

Convert man page for btrfs-image.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile        |  2 +-
 Documentation/btrfs-image.txt | 70 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 71 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/btrfs-image.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 7a38df9..6954a51 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -7,7 +7,7 @@ MAN8_TXT += btrfsck.txt
 MAN8_TXT += btrfs-convert.txt
 MAN8_TXT += btrfs-debug-tree.txt
 MAN8_TXT += btrfs-find-root.txt
-#MAN8_TXT += btrfs-image.txt
+MAN8_TXT += btrfs-image.txt
 #MAN8_TXT += btrfs-map-logical.txt
 #MAN8_TXT += btrfs-show-super.txt
 #MAN8_TXT += btrfstune.txt
diff --git a/Documentation/btrfs-image.txt b/Documentation/btrfs-image.txt
new file mode 100644
index 0000000..bd74a86
--- /dev/null
+++ b/Documentation/btrfs-image.txt
@@ -0,0 +1,70 @@
+btrfs-image(8)
+==============
+
+NAME
+----
+btrfs-image - create/restore an image of the filesystem
+
+SYNOPSIS
+--------
+'btrfs-image' [options] <source> <target>
+
+DESCRIPTION
+-----------
+'btrfs-image' is used to create an image of a btrfs filesystem.
+All data will be zeroed, but metadata and the like is preserved.
+
+Mainly used for debug purpose.
+
+OPTIONS
+-------
+-r::
+Restore metadump image. By default, this fixes super's chunk tree, by
+using 1 stripe pointing to primary device, so that file system can be
+restored by running tree log reply if possible. To restore without
+changing number of stripes in chunk tree check -o option.
+
+-c value::
+Compression level (0 ~ 9).
+
+-t value::
+Number of threads (1 ~ 32) to be used to process the image dump or restore.
+
+-o::
+Use the old restore method, this does not fixup the chunk tree so the restored
+file system will not be able to be mounted.
+
+-s::
+Sanitize the file names when generating the image. One -s means just
+generate random garbage, which means that the directory indexes won't match up
+since the hashes won't match with the garbage filenames. Using -ss will
+calculate a collision for the filename so that the hashes match, and if it
+can't calculate a collision then it will just generate garbage.  The collision
+calculator is very time and CPU intensive so only use it if you are having
+problems with your file system tree and need to have it mostly working.
+
+-w::
+Walk all the trees manually and copy any blocks that are referenced. Use this
+option if your extent tree is corrupted to make sure that all of the metadata is
+captured.
+
+EXIT STATUS
+-----------
+'btrfs-image' will return 0 if no error happened.
+If any problems happened, 1 will be returned.
+
+AUTHOR
+------
+Written by Shilong Wang and Wenruo Qu.
+
+COPYRIGHT
+---------
+Copyright (C) 2013 FUJITSU LIMITED.
+
+License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
+
+This is free software: you are free  to  change  and  redistribute  it. There is NO WARRANTY, to the extent permitted by law.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8)
-- 
1.9.1

[PATCH 21/27] btrfs-progs: Convert man page for btrfs-map-logical.

[Qu Wenruo]

Convert man page for btrfs-map-logical.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile              |  2 +-
 Documentation/btrfs-map-logical.txt | 49 +++++++++++++++++++++++++++++++++++++
 2 files changed, 50 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/btrfs-map-logical.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 6954a51..d418681 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -8,7 +8,7 @@ MAN8_TXT += btrfs-convert.txt
 MAN8_TXT += btrfs-debug-tree.txt
 MAN8_TXT += btrfs-find-root.txt
 MAN8_TXT += btrfs-image.txt
-#MAN8_TXT += btrfs-map-logical.txt
+MAN8_TXT += btrfs-map-logical.txt
 #MAN8_TXT += btrfs-show-super.txt
 #MAN8_TXT += btrfstune.txt
 #MAN8_TXT += btrfs-zero-log.txt
diff --git a/Documentation/btrfs-map-logical.txt b/Documentation/btrfs-map-logical.txt
new file mode 100644
index 0000000..2a93fb0
--- /dev/null
+++ b/Documentation/btrfs-map-logical.txt
@@ -0,0 +1,49 @@
+btrfs-map-logical(8)
+====================
+
+NAME
+----
+btrfs-map-logical - map btrfs logical extent to physical extent
+
+SYNOPSIS
+--------
+'btrfs-map-logical' <options> <device>
+
+DESCRIPTION
+-----------
+'btrfs-map-logical' can be used to find out what the physical offsets are
+on the mirrors, the result is dumped into stdout in default.
+
+Mainly used for debug purpose.
+
+OPTIONS
+-------
+-l|--logical <logical_num>::
+Logical extent to map.
+-c|--copy <copy>::
+Copy of the extent to read(usually 1 or 2).
+-o|--output <filename>::
+Output file to hold the extent.
+-b|--bytes <bytes>::
+Number of bytes to read.
+
+EXIT STATUS
+-----------
+'btrfs-map-logical' will return 0 if no error happened.
+If any problems happened, 1 will be returned.
+
+AUTHOR
+------
+Written by Shilong Wang and Wenruo Qu.
+
+COPYRIGHT
+---------
+Copyright (C) 2013 FUJITSU LIMITED.
+
+License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
+
+This is free software: you are free  to  change  and  redistribute  it. There is NO WARRANTY, to the extent permitted by law.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8)
-- 
1.9.1

[PATCH 22/27] btrfs-progs: Convert man page for btrfs-show-super.

[Qu Wenruo]

Convert man page for btrfs-show-super.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile             |  2 +-
 Documentation/btrfs-show-super.txt | 53 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 54 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/btrfs-show-super.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index d418681..2013d65 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -9,7 +9,7 @@ MAN8_TXT += btrfs-debug-tree.txt
 MAN8_TXT += btrfs-find-root.txt
 MAN8_TXT += btrfs-image.txt
 MAN8_TXT += btrfs-map-logical.txt
-#MAN8_TXT += btrfs-show-super.txt
+MAN8_TXT += btrfs-show-super.txt
 #MAN8_TXT += btrfstune.txt
 #MAN8_TXT += btrfs-zero-log.txt
 #MAN8_TXT += fsck.btrfs.txt
diff --git a/Documentation/btrfs-show-super.txt b/Documentation/btrfs-show-super.txt
new file mode 100644
index 0000000..e8e17ab
--- /dev/null
+++ b/Documentation/btrfs-show-super.txt
@@ -0,0 +1,53 @@
+btrfs-show-super(8)
+====================
+
+NAME
+----
+btrfs-show-super - show btrfs superblock information stored in devices
+
+SYNOPSIS
+--------
+'btrfs-show-super' [options] <dev> [<dev>...]
+
+DESCRIPTION
+-----------
+'btrfs-show-super' is used to print the information of superblock,
+you can specify which mirror to print out.
+
+By default, every device's first superblock will be printed out.
+
+Mainly used for debug purpose.
+
+OPTIONS
+-------
+-a::
+Print all the superblock information.
++
+If this option is given, '-i' option will be ignored.
+
+-i <super_mirror>::
+Specify which mirror to print out.
++
+<super_mirror> is between 0 and 2.
+If several '-i <super_mirror>' are given, only the last one is valid.
+
+EXIT STATUS
+-----------
+'btrfs-show-super' will return 0 if no error happened.
+If any problems happened, 1 will be returned.
+
+AUTHOR
+------
+Written by Shilong Wang and Wenruo Qu.
+
+COPYRIGHT
+---------
+Copyright (C) 2013 FUJITSU LIMITED.
+
+License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
+
+This is free software: you are free  to  change  and  redistribute  it. There is NO WARRANTY, to the extent permitted by law.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8)
-- 
1.9.1

[PATCH 23/27] btrfs-progs: Convert man page for btrfstune.

[Qu Wenruo]

Convert man page for btrfstune.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile      |  2 +-
 Documentation/btrfstune.txt | 47 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 48 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/btrfstune.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 2013d65..e002d53 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -10,7 +10,7 @@ MAN8_TXT += btrfs-find-root.txt
 MAN8_TXT += btrfs-image.txt
 MAN8_TXT += btrfs-map-logical.txt
 MAN8_TXT += btrfs-show-super.txt
-#MAN8_TXT += btrfstune.txt
+MAN8_TXT += btrfstune.txt
 #MAN8_TXT += btrfs-zero-log.txt
 #MAN8_TXT += fsck.btrfs.txt
 #MAN8_TXT += mkfs.btrfs.txt
diff --git a/Documentation/btrfstune.txt b/Documentation/btrfstune.txt
new file mode 100644
index 0000000..bf4bfce
--- /dev/null
+++ b/Documentation/btrfstune.txt
@@ -0,0 +1,47 @@
+btrfstune(8)
+============
+
+NAME
+----
+btrfstune - tune various btrfs filesystem parameters
+
+SYNOPSIS
+--------
+'btrfstune' [options] <dev> [<dev>...]
+
+DESCRIPTION
+-----------
+'btrfstune' is used to tune various btrfs filesystem parameters,you can
+enable/disable some extended features for btrfs.
+
+OPTIONS
+-------
+-S <value>::
+Updates the seeding value, it forces a fs readonly so that you can use it to
+build other filesystems.
+-r::
+Enable extended inode refs.
+-x::
+Enable skinny metadata extent refs.
+
+
+EXIT STATUS
+-----------
+'btrfstune' will return 0 if no error happened.
+If any problems happened, 1 will be returned.
+
+AUTHOR
+------
+Written by Shilong Wang and Wenruo Qu.
+
+COPYRIGHT
+---------
+Copyright (C) 2013 FUJITSU LIMITED.
+
+License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
+
+This is free software: you are free  to  change  and  redistribute  it. There is NO WARRANTY, to the extent permitted by law.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8)
-- 
1.9.1

[PATCH 24/27] btrfs-progs: Convert man page for btrfs-zero-log

[Qu Wenruo]

Convert man page for btrfs-zero-log

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile           |  2 +-
 Documentation/btrfs-zero-log.txt | 39 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 40 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/btrfs-zero-log.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index e002d53..de06629 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -11,7 +11,7 @@ MAN8_TXT += btrfs-image.txt
 MAN8_TXT += btrfs-map-logical.txt
 MAN8_TXT += btrfs-show-super.txt
 MAN8_TXT += btrfstune.txt
-#MAN8_TXT += btrfs-zero-log.txt
+MAN8_TXT += btrfs-zero-log.txt
 #MAN8_TXT += fsck.btrfs.txt
 #MAN8_TXT += mkfs.btrfs.txt
 
diff --git a/Documentation/btrfs-zero-log.txt b/Documentation/btrfs-zero-log.txt
new file mode 100644
index 0000000..e3041fa
--- /dev/null
+++ b/Documentation/btrfs-zero-log.txt
@@ -0,0 +1,39 @@
+btrfs-zero-log(8)
+=================
+
+NAME
+----
+btrfs-zero-log - clear out log tree
+
+SYNOPSIS
+--------
+'btrfs-zero-log' <dev>
+
+DESCRIPTION
+-----------
+'btrfs-zero-log' will remove the log tree if log tree is corrupt, which will
+allow you to mount the filesystem again.
+
+The common case where this happens has been fixed a long time ago,
+so it is unlikely that you will see this particular problem.
+
+EXIT STATUS
+-----------
+'btrfs-zero-log' will return 0 if no error happened.
+Other exit code means some problems happened.
+
+AUTHOR
+------
+Written by Shilong Wang and Wenruo Qu.
+
+COPYRIGHT
+---------
+Copyright (C) 2013 FUJITSU LIMITED.
+
+License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
+
+This is free software: you are free  to  change  and  redistribute  it. There is NO WARRANTY, to the extent permitted by law.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8)
-- 
1.9.1

[PATCH 25/27] btrfs-progs: Convert man page for fsck.btrfs.

[Qu Wenruo]

Convert man page for fsck.btrfs.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile       |  2 +-
 Documentation/fsck.btrfs.txt | 51 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 52 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/fsck.btrfs.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index de06629..4100ddd 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -12,7 +12,7 @@ MAN8_TXT += btrfs-map-logical.txt
 MAN8_TXT += btrfs-show-super.txt
 MAN8_TXT += btrfstune.txt
 MAN8_TXT += btrfs-zero-log.txt
-#MAN8_TXT += fsck.btrfs.txt
+MAN8_TXT += fsck.btrfs.txt
 #MAN8_TXT += mkfs.btrfs.txt
 
 # Sub commands for btrfs
diff --git a/Documentation/fsck.btrfs.txt b/Documentation/fsck.btrfs.txt
new file mode 100644
index 0000000..52e8e4d
--- /dev/null
+++ b/Documentation/fsck.btrfs.txt
@@ -0,0 +1,51 @@
+fsck.btrfs(8)
+=============
+
+NAME
+----
+fsck.btrfs - do nothing, successfully
+
+SYNOPSIS
+--------
+'fsck.btrfs' [-aApy] [device...]
+
+DESCRIPTION
+-----------
+'fsck.btrfs' is a type of utility that should exist for any filesystem and is
+called during system setup when the corresponding `/etc/fstab` entries
+contain non-zero value for `fs_passno` , see `fstab`(5) for more.
+
+Traditional filesystems need to run their respective fsck utility in case the
+filesystem was not unmounted cleanly and the log needs to be replayed before
+mount. This is not needed for BTRFS. You should set fs_passno to 0.
+
+If you wish to check the consistency of a BTRFS filesystem or repair a damaged
+filesystem, see `btrfs-check`(8). By default the filesystem
+consistency is checked, the repair mode is enabled via '--repair' option (use
+with care!).
+
+OPTIONS
+-------
+The options are all the same and detect if 'fsck.btrfs' is executed in
+non-interactive mode and exits with success,
+otherwise prints a message about btrfs check.
+
+EXIT STATUS
+-----------
+There are two possible exit code returned:
+
+0::
+No error
+
+8::
+Operational error, eg. device does not exist
+
+FILES
+-----
+`/etc/fstab`
+
+SEE ALSO
+--------
+`btrfs`(8),
+`fsck`(8),
+`fstab`(5),
-- 
1.9.1

[PATCH 26/27] btrfs-progs: Convert man page for mkfs.btrfs.

[Qu Wenruo]

Convert man page for mkfs.btrfs.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Documentation/Makefile       |   2 +-
 Documentation/mkfs.btrfs.txt | 133 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 134 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/mkfs.btrfs.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 4100ddd..97a775a 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -13,7 +13,7 @@ MAN8_TXT += btrfs-show-super.txt
 MAN8_TXT += btrfstune.txt
 MAN8_TXT += btrfs-zero-log.txt
 MAN8_TXT += fsck.btrfs.txt
-#MAN8_TXT += mkfs.btrfs.txt
+MAN8_TXT += mkfs.btrfs.txt
 
 # Sub commands for btrfs
 MAN8_TXT += btrfs-subvolume.txt
diff --git a/Documentation/mkfs.btrfs.txt b/Documentation/mkfs.btrfs.txt
new file mode 100644
index 0000000..bafe065
--- /dev/null
+++ b/Documentation/mkfs.btrfs.txt
@@ -0,0 +1,133 @@
+mkfs.btrfs(8)
+=============
+
+NAME
+----
+mkfs.btrfs - create a btrfs filesystem
+
+SYNOPSIS
+--------
+'mkfs.btrfs'
+$$[-A|--alloc-start <alloc-start>]$$
+$$[-b|--byte-count <byte-count>]$$
+$$[-d|--data <data-profile>]$$
+$$[-f|--force]$$
+$$[-n|--nodesize <nodesize>]$$
+$$[-l|--leafsize <leafsize>]$$
+$$[-L|--label <label>]$$
+$$[-m|--metadata <metadata profile>]$$
+$$[-M|--mixed]$$
+$$[-s|--sectorsize <sectorsize>]$$
+$$[-r|--rootdir <rootdir>]$$
+$$[-K|--nodiscard]$$
+$$[-O|--features <feature1>[<,feature2...>]]$$
+$$[-h]$$
+$$[-V|--version]$$
+$$<device> [<device>...]$$
+
+DESCRIPTION
+-----------
+'mkfs.btrfs'
+is used to create a btrfs filesystem (usually in a disk partition, or an array
+of disk partitions).
+
+<device>
+is the special file corresponding to the device (e.g /dev/sdXX ).
+If multiple devices are specified, btrfs is created
+spanning across the specified  devices.
+
+OPTIONS
+-------
+-A|--alloc-start <offset>::
+Specify the offset from the start of the device to start the btrfs filesystem. The default value is zero, or the start of the device.
+
+-b|--byte-count <size>::
+Specify the size of the resultant filesystem. If this option is not used,
+mkfs.btrfs uses all the available storage for the filesystem.
+
+-d|--data <type>::
+Specify how the data must be spanned across the devices specified. Valid
+values are 'raid0', 'raid1', 'raid5', 'raid6', 'raid10' or 'single'.
+
+-f|--force::
+Force overwrite when an existing filesystem is detected on the device.
+By default, mkfs.btrfs will not write to the device if it suspects that 
+there is a filesystem or partition table on the device already.
+
+-n|--nodesize <size>
++
+-l|--leafsize <size>::
+Specify the nodesize, the tree block size in which btrfs stores
+data. The default value is 16KB (16384) or the page size, whichever is
+bigger. Must be a multiple of the sectorsize, but not larger than
+65536. Leafsize always equals nodesize and the options are aliases.
+
+-L|--label <name>::
+Specify a label for the filesystem.
++
+NOTE: <name> should be less than 256 characters.
+
+
+-m|--metadata <profile>::
+Specify how metadata must be spanned across the devices specified. Valid
+values are 'raid0', 'raid1', 'raid5', 'raid6', 'raid10', 'single' or 'dup'.
++
+Single device
+will have dup set by default except in the case of SSDs which will default to
+single. This is because SSDs can remap blocks internally so duplicate blocks
+could end up in the same erase block which negates the benefits of doing
+metadata duplication.
+
+-M|--mixed::
+Mix data and metadata chunks together for more efficient space 
+utilization.  This feature incurs a performance penalty in
+larger filesystems.  It is recommended for use with filesystems
+of 1 GiB or smaller.
+
+-s|--sectorsize <size>::
+Specify the sectorsize, the minimum data block allocation unit.
++
+The default
+value is the page size. If the sectorsize differs from the page size, the
+created filesystem may not be mountable by current kernel. Therefore it is not
+recommended to use this option unless you are going to mount it on a system
+with the appropriate page size.
+
+-r|--rootdir <rootdir>::
+Specify a directory to copy into the newly created btrfs filesystem.
++
+NOTE: '-r' option is done completely in userland, and don't need root
+privilege to mount the filesystem.
+
+-K|--nodiscard::
+Do not perform whole device TRIM operation by default.
+
+-O|--features <feature1>[,<feature2>...]::
+A list of filesystem features turned on at mkfs time. Not all features are
+supported by old kernels.
++
+To see all features run::::
+mkfs.btrfs -O list-all
+
+-V|--version::
+Print the 'mkfs.btrfs' version and exit.
+
+-h::
+Print help.
+
+UNIT
+----
+As default the unit is the byte, however it is possible to append a suffix
+to the arguments like 'k' for KBytes, 'm' for MBytes...
+
+AVAILABILITY
+------------
+'btrfs' is part of btrfs-progs. Btrfs filesystem is currently under heavy
+development,
+and not suitable for any uses other than benchmarking and review.
+Please refer to the btrfs wiki http://btrfs.wiki.kernel.org for
+further details.
+
+SEE ALSO
+--------
+`btrfsck`(8)
-- 
1.9.1

[PATCH 27/27] btrfs-progs: Switch to the new asciidoc Documentation.

[Qu Wenruo]

Since all man page are converted to the new asciidoc, the old man page
can be removed.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 Makefile                   |   4 +-
 man/Makefile               |  31 --
 man/btrfs-convert.8.in     |  33 --
 man/btrfs-debug-tree.8.in  |  35 --
 man/btrfs-find-root.8.in   |  30 --
 man/btrfs-image.8.in       |  55 ---
 man/btrfs-map-logical.8.in |  33 --
 man/btrfs-show-super.8.in  |  30 --
 man/btrfs-zero-log.8.in    |  23 --
 man/btrfs.8.in             | 932 ---------------------------------------------
 man/btrfsck.8.in           |  29 --
 man/btrfstune.8.in         |  31 --
 man/fsck.btrfs.8.in        |  47 ---
 man/mkfs.btrfs.8.in        | 105 -----
 14 files changed, 2 insertions(+), 1416 deletions(-)
 delete mode 100644 man/Makefile
 delete mode 100644 man/btrfs-convert.8.in
 delete mode 100644 man/btrfs-debug-tree.8.in
 delete mode 100644 man/btrfs-find-root.8.in
 delete mode 100644 man/btrfs-image.8.in
 delete mode 100644 man/btrfs-map-logical.8.in
 delete mode 100644 man/btrfs-show-super.8.in
 delete mode 100644 man/btrfs-zero-log.8.in
 delete mode 100644 man/btrfs.8.in
 delete mode 100644 man/btrfsck.8.in
 delete mode 100644 man/btrfstune.8.in
 delete mode 100644 man/fsck.btrfs.8.in
 delete mode 100644 man/mkfs.btrfs.8.in

diff --git a/Makefile b/Makefile
index 636cbf0..70eaf57 100644
--- a/Makefile
+++ b/Makefile
@@ -56,7 +56,7 @@ btrfs_convert_libs = -lext2fs -lcom_err
 btrfs_image_libs = -lpthread
 btrfs_fragment_libs = -lgd -lpng -ljpeg -lfreetype
 
-SUBDIRS = man
+SUBDIRS = Documentation
 BUILDDIRS = $(patsubst %,build-%,$(SUBDIRS))
 INSTALLDIRS = $(patsubst %,install-%,$(SUBDIRS))
 CLEANDIRS = $(patsubst %,clean-%,$(SUBDIRS))
@@ -221,7 +221,7 @@ send-test: $(objects) $(libs) send-test.o
 	$(Q)$(CC) $(CFLAGS) -o send-test $(objects) send-test.o $(LDFLAGS) $(LIBS) -lpthread
 
 manpages:
-	$(Q)$(MAKE) $(MAKEOPTS) -C man
+	$(Q)$(MAKE) $(MAKEOPTS) -C Documentation
 
 clean: $(CLEANDIRS)
 	@echo "Cleaning"
diff --git a/man/Makefile b/man/Makefile
deleted file mode 100644
index 552c9f7..0000000
--- a/man/Makefile
+++ /dev/null
@@ -1,31 +0,0 @@
-GZIPCMD=gzip
-INSTALL= install
-
-prefix ?= /usr/local
-bindir = $(prefix)/bin
-mandir = $(prefix)/man
-man8dir = $(mandir)/man8
-
-# clear out all suffixes
-.SUFFIXES:
-# list only those we use
-.SUFFIXES: .in .gz
-
-MANPAGES = mkfs.btrfs.8.gz btrfsck.8.gz btrfs-image.8.gz btrfs.8.gz \
-	   btrfs-debug-tree.8.gz btrfs-show-super.8.gz btrfs-find-root.8.gz \
-	   btrfs-convert.8.gz btrfstune.8.gz btrfs-zero-log.8.gz btrfs-map-logical.8.gz \
-	   fsck.btrfs.8.gz
-INFILES = ${MANPAGES:.in=.gz}
-
-all: $(MANPAGES)
-
-.in.gz :
-	@echo "    [MAN]    $@"
-	$(Q)$(GZIPCMD) -n -c $< > $@
-
-clean :
-	$(Q)rm -f $(MANPAGES)
-
-install: $(MANPAGES)
-	$(INSTALL) -m755 -d $(DESTDIR)$(man8dir)
-	$(INSTALL) -m 644 $(MANPAGES) $(DESTDIR)$(man8dir)
diff --git a/man/btrfs-convert.8.in b/man/btrfs-convert.8.in
deleted file mode 100644
index 85cfa61..0000000
--- a/man/btrfs-convert.8.in
+++ /dev/null
@@ -1,33 +0,0 @@
-.TH BTRFS-CONVERT 8
-.SH NAME
-btrfs-convert \- convert ext2/3/4 to btrfs.
-.SH SYNOPSIS
-.B btrfs-convert [\fIoptions\fP] \fI<dev>\fP
-.SH DESCRIPTION
-\fBbtrfs-convert\fP is used to convert existed ext2/3/4 to btrfs filesystem, and the original filesystem image is accessible as from separate subvolume named ext2_subvol as file image.
-
-\fIOptions\fP
-.IP "\fB-d\fP" 5
-disable data checksum.
-.IP "\fB-i\fP" 5
-ignore xattrs and ACLs.
-.IP "\fB-n\fP" 5
-disable packing of small files.
-.IP "\fB-r\fP" 5
-roll back to ext2fs.
-
-.SH EXIT CODE
-\fBbtrfs-convert\fP will return 0 if no error happened.
-If any problems happened, 1 will be returned.
-
-.SH AUTHOR
-Written by Shilong Wang and Wenruo Qu.
-
-.SH COPYRIGHT
-Copyright \(co 2013 FUJITSU LIMITED.
-License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
-.br
-This is free software: you are free  to  change  and  redistribute  it. There is NO WARRANTY, to the extent permitted by law.
-
-.SH SEE ALSO
-.BR mkfs.btrfs (8)
diff --git a/man/btrfs-debug-tree.8.in b/man/btrfs-debug-tree.8.in
deleted file mode 100644
index eed7097..0000000
--- a/man/btrfs-debug-tree.8.in
+++ /dev/null
@@ -1,35 +0,0 @@
-.TH BTRFS-DEBUG-TREE 8
-.SH NAME
-btrfs-debug-tree \- dump Btrfs filesystem metadata into stdout.
-.SH SYNOPSIS
-.B btrfs-debug-tree [\fIoptions\fP] \fI<device>\fP
-.SH DESCRIPTION
-\fBbtrfs-debug-tree\fP is used to dump the whole tree of the given device.
-This is maybe useful for analyzing filesystem state or inconsistence and has
-a positive educational effect on understanding the internal structure.
-\fIdevice\fP is the device file where the filesystem is stored.
-
-\fIOptions\fP
-.IP "\fB-e\fP" 5
-print detailed extents info.
-.IP "\fB-d\fP" 5
-print info of btrfs device and root tree dirs only.
-.IP "\fB-r\fP" 5
-print info of roots only.
-.IP "\fB-b \fI<block_num>\fP" 5
-print info of the specified block only.
-
-.SH EXIT CODE
-\fBbtrfs-debug-tree\fP will return 0 if no error happened.
-If any problems happened, 1 will be returned.
-
-.SH AUTHOR
-Written by Shilong Wang and Wenruo Qu.
-
-.SH COPYRIGHT
-Copyright \(co 2013 FUJITSU LIMITED.
-License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
-.br
-This is free software: you are free  to  change  and  redistribute  it. There is NO WARRANTY, to the extent permitted by law.
-.SH SEE ALSO
-.BR mkfs.btrfs (8)
diff --git a/man/btrfs-find-root.8.in b/man/btrfs-find-root.8.in
deleted file mode 100644
index c1f48a0..0000000
--- a/man/btrfs-find-root.8.in
+++ /dev/null
@@ -1,30 +0,0 @@
-.TH BTRFS-FIND-ROOT 8
-.SH NAME
-btrfs-find-root \- filter to find btrfs root.
-.SH SYNOPSIS
-.B btrfs-find-root [\fIoptions\fP] \fI<dev>\fP
-.SH DESCRIPTION
-\fBbtrfs-find-root\fP is used to find the satisfied root, you can filter by root tree's objectid, generation, level.
-
-\fIOptions\fP
-.IP "\fB-g \fI<generation>\fP" 5
-filter root tree by it's original transaction id, tree root's generation in default.
-.IP "\fB-o \fI<objectid>\fP" 5
-filter root tree by it's objectid,tree root's objectid in default.
-.IP "\fB-l \fI<level>\fP" 5
-filter root tree by B-+ tree's level, level 0 in default.
-
-.SH EXIT CODE
-\fBbtrfs-find-root\fP will return 0 if no error happened.
-If any problems happened, 1 will be returned.
-
-.SH AUTHOR
-Written by Shilong Wang and Wenruo Qu.
-
-.SH COPYRIGHT
-Copyright \(co 2013 FUJITSU LIMITED.
-License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
-.br
-This is free software: you are free  to  change  and  redistribute  it. There is NO WARRANTY, to the extent permitted by law.
-.SH SEE ALSO
-.BR mkfs.btrfs (8)
diff --git a/man/btrfs-image.8.in b/man/btrfs-image.8.in
deleted file mode 100644
index 3f51f3f..0000000
--- a/man/btrfs-image.8.in
+++ /dev/null
@@ -1,55 +0,0 @@
-.TH BTRFS-IMAGE 8
-.SH NAME
-btrfs-image \- create/restore an image of the filesystem
-.SH SYNOPSIS
-.B btrfs-image
-[options] \fIsource\fP \fItarget\fP
-.SH DESCRIPTION
-.B btrfs-image
-is used to create an image of a btrfs filesystem. All data will be zeroed,
-but metadata and the like is preserved.
-.I source
-is the special file corresponding to the device containing a btrfs filesystem.
-(e.g \fI/dev/sdXX\fP).
-.I target
-is the image file that btrfs-image creates. When used with \fB-r\fP option,
-\fBbtrfs-image\fP restores the image file from source into target.
-.SH OPTIONS
-.TP
-\fB\-r\fP
-Restore metadump image. By default, this fixes super's chunk tree, by
-using 1 stripe pointing to primary device, so that file system can be
-restored by running tree log reply if possible. To restore without
-changing number of stripes in chunk tree check \fB-o\fP option.
-.TP
-\fB\-c\fR \fIvalue\fP
-compression level (0 ~ 9).
-.TP
-\fB\-t\fR \fIvalue\fP
-number of threads (1 ~ 32) to be used to process the image dump or restore.
-.TP
-\fB\-o\fP
-use the old restore method, this does not fixup the chunk tree so the restored
-file system will not be able to be mounted.
-.TP
-\fB\-s\fP
-Sanitize the file names when generating the image. One \fB-s\fP means just
-generate random garbage, which means that the directory indexes won't match up
-since the hashes won't match with the garbage filenames. Using \fB-ss\fP will
-calculate a collision for the filename so that the hashes match, and if it
-can't calculate a collision then it will just generate garbage.  The collision
-calculator is very time and CPU intensive so only use it if you are having
-problems with your file system tree and need to have it mostly working.
-.TP
-\fB\-w\fP
-Walk all the trees manually and copy any blocks that are referenced. Use this
-option if your extent tree is corrupted to make sure that all of the metadata is
-captured.
-.SH AVAILABILITY
-.B btrfs-image
-is part of btrfs-progs. Btrfs is currently under heavy development,
-and not suitable for any uses other than benchmarking and review.
-Please refer to the btrfs wiki
-http://btrfs.wiki.kernel.org for further details.
-.SH SEE ALSO
-.BR btrfsck (8), mkfs.btrfs (8)
diff --git a/man/btrfs-map-logical.8.in b/man/btrfs-map-logical.8.in
deleted file mode 100644
index a253051..0000000
--- a/man/btrfs-map-logical.8.in
+++ /dev/null
@@ -1,33 +0,0 @@
-.TH BTRFS-MAP-LOGICAL 8
-.SH NAME
-btrfs-map-logical \- map btrfs logical extent to physical extent
-.SH SYNOPSIS
-.B btrfs-map-logical [\fIoptions\fP] \fI<device>\fP
-.SH DESCRIPTION
-\fBbtrfs-map-logical\fP can be used to find out what the physical offsets are
-on the mirrors, the result is dumped into stdout in default.
-
-\fIOptions\fP
-.IP "\fB-l|--logical \fI<logical_num>\fP" 5
-Logical extent to map.
-.IP "\fB-c|--copy \fI<copy>\fP" 5
-Copy of the extent to read(usually 1 or 2).
-.IP "\fB-o|--output \fI<filename>\fP" 5
-Output file to hold the extent.
-.IP "\fB-b|--bytes \fI<bytes>\fP" 5
-Number of bytes to read.
-
-.SH EXIT CODE
-\fBbtrfs-map-logical\fP will return 0 if no error happened.
-If any problems happened, 1 will be returned.
-
-.SH AUTHOR
-Written by Shilong Wang and Wenruo Qu.
-
-.SH COPYRIGHT
-Copyright \(co 2013 FUJITSU LIMITED.
-License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
-.br
-This is free software: you are free  to  change  and  redistribute  it. There is NO WARRANTY, to the extent permitted by law.
-.SH SEE ALSO
-.BR mkfs.btrfs (8)
diff --git a/man/btrfs-show-super.8.in b/man/btrfs-show-super.8.in
deleted file mode 100644
index 3fe8bde..0000000
--- a/man/btrfs-show-super.8.in
+++ /dev/null
@@ -1,30 +0,0 @@
-.TH BTRFS-SHOW-SUPER 8
-.SH NAME
-btrfs-show-super \- show btrfs superblock information stored in devices
-.SH SYNOPSIS
-.B btrfs-show-super [\fIoptions\fP] \fI<dev>\fP [\fI<dev>...\fP]
-.SH DESCRIPTION
-\fBbtrfs-show-super\fP is used to print the information of superblock,
-you can specify which mirror to print out. In default, every device's
-first superblock will be printed out.
-
-\fIOptions\fP
-.IP "\fB-a\fP" 5
-print all the superblock information, if this option is given, '\fB-i\fP' option will be ignored.
-.IP "\fB-i \fI<super_mirror>\fP" 5
-specify which mirror to print out. \fI<super_mirror>\fP is between 0 and 2. if several '\fB-i \fI<super_mirror>\fP'\fR are given, only the last one is valid.
-
-.SH EXIT CODE
-\fBbtrfs-show-super\fR will return 0 exit code if no error happened.
-If any problems happened, 1 will be returned.
-
-.SH AUTHOR
-Written by Shilong Wang and Wenruo Qu.
-
-.SH COPYRIGHT
-Copyright \(co 2013 FUJITSU LIMITED.
-License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
-.br
-This is free software: you are free  to  change  and  redistribute  it. There is NO WARRANTY, to the extent permitted by law.
-.SH SEE ALSO
-.BR mkfs.btrfs (8)
diff --git a/man/btrfs-zero-log.8.in b/man/btrfs-zero-log.8.in
deleted file mode 100644
index 0ccfb0e..0000000
--- a/man/btrfs-zero-log.8.in
+++ /dev/null
@@ -1,23 +0,0 @@
-.TH BTRFS-ZERO-LOG 8
-.SH NAME
-btrfs-zero-log \- clear out log tree.
-.SH SYNOPSIS
-.B btrfs-zero-log \fI <dev>\fP
-.SH DESCRIPTION
-\fBbtrfs-zero-log\fP will remove the log tree if log tree is corrupt, which will allow you to mount the filesystem again. The common case where this
-happens has been fixed a long time ago, so it is unlikely that you will see this particular problem.
-
-.SH EXIT CODE
-\fBbtrfs-zero-log\fR will return 0 exit code if no error happened.
-Other exit code means some problems happened.
-
-.SH AUTHOR
-Written by Shilong Wang and Wenruo Qu.
-
-.SH COPYRIGHT
-Copyright \(co 2013 FUJITSU LIMITED.
-License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
-.br
-This is free software: you are free  to  change  and  redistribute  it. There is NO WARRANTY, to the extent permitted by law.
-.SH SEE ALSO
-.BR mkfs.btrfs (8)
diff --git a/man/btrfs.8.in b/man/btrfs.8.in
deleted file mode 100644
index 2b36b26..0000000
--- a/man/btrfs.8.in
+++ /dev/null
@@ -1,932 +0,0 @@
-.TH BTRFS 8 "" "btrfs" "btrfs"
-.\"
-.\" Man page written by Goffredo Baroncelli <kreijack@inwind.it> (Feb 2010)
-.\" Modified by Qu Wenruo <quwenruo@cn.fujitsu.com> (Jun 2013)
-.\"
-.SH NAME
-btrfs \- control a btrfs filesystem
-.SH SYNOPSIS
-\fBbtrfs\fP \fBsubvolume create\fP [-i \fI<qgroupid>\fP] [\fI<dest>\fP/]\fI<name>\fP
-.PP
-\fBbtrfs\fP \fBsubvolume delete\fP [\fIoptions\fP] \fI<subvolume>\fP [\fI<subvolume>...\fP]
-.PP
-\fBbtrfs\fP \fBsubvolume list\fP [\fIoptions\fP] [-G [+|-]\fIvalue\fP] [-C [+|-]\fIvalue\fP] [--sort=rootid,gen,ogen,path] \fI<path>\fP
-.PP
-\fBbtrfs\fP \fBsubvolume snapshot\fP [-r] [-i qgroupid] \fI<source>\fP \fI<dest>\fP|[\fI<dest>\fP/]\fI<name>\fP
-.PP
-\fBbtrfs\fP \fBsubvolume snapshot\fP [-r] [-i qgroupid] \fI-s <subvolid>\fP \fI<dest>/<name>\fP
-.PP
-\fBbtrfs\fP \fBsubvolume get-default\fP\fI <path>\fP
-.PP
-\fBbtrfs\fP \fBsubvolume set-default\fP\fI <id> <path>\fP
-.PP
-\fBbtrfs\fP \fBsubvolume find-new\fP\fI <subvolume> <lastgen>\fP
-.PP
-\fBbtrfs\fP \fBsubvolume show\fP\fI <path>\fP
-.PP
-.PP
-\fBbtrfs\fP \fBfilesystem df\fP\fI [-b] \fIpath [path..]\fR\fP
-.PP
-\fBbtrfs\fP \fBfilesystem show\fP [\fI--mounted\fP|\fI--all-devices\fP|\fI<path>\fP|\fI<uuid>\fP|\fI<device>\fP|\fI<lable>\fP]\fP
-.PP
-\fBbtrfs\fP \fBfilesystem sync\fP\fI <path> \fP
-.PP
-\fBbtrfs\fP \fBfilesystem defragment\fP [\fIoptions\fP] \fI<file>\fP|\fI<dir>\fP [\fI<file>\fP|\fI<dir>...\fP]\fP
-.PP
-\fBbtrfs\fP \fBfilesystem resize\fP [\fIdevid\fP:][+/\-]\fI<size>\fP[gkm]|[\fIdevid\fP:]\fImax <path>\fP
-.PP
-\fBbtrfs\fP \fBfilesystem label\fP [\fI<device>\fP|\fI<mount_point>\fP] [\fI<newlabel>\fP]
-.PP
-\fBbtrfs\fP \fBfilesystem disk-usage [-tb]\fP\fI <path> 
-[path..]\fP
-.PP
-.PP
-\fBbtrfs\fP \fB[filesystem] balance\fP\fI <path> \fP
-.PP
-\fBbtrfs\fP \fB[filesystem] balance start\fP [\fIoptions\fP] \fI<path>\fP
-.PP
-\fBbtrfs\fP \fB[filesystem] balance pause\fP\fI <path>\fP
-.PP
-\fBbtrfs\fP \fB[filesystem] balance cancel\fP\fI <path>\fP
-.PP
-\fBbtrfs\fP \fB[filesystem] balance resume\fP\fI <path>\fP
-.PP
-\fBbtrfs\fP \fB[filesystem] balance status\fP [-v] \fI<path>\fP
-.PP
-.PP
-\fBbtrfs\fP \fBdevice add\fP [-Kf] \fI<device>\fP [\fI<device>...\fP] \fI<path>\fP
-.PP
-\fBbtrfs\fP \fBdevice delete\fP \fI<device>\fP [\fI<device>...\fP] \fI<path>\fP
-.PP
-\fBbtrfs\fP \fBdevice scan\fP [(\fI-d\fP|\fI--all-devices\fP)|\fI<device>\fP [\fI<device>...\fP]]
-.PP
-\fBbtrfs\fP \fBdevice disk-usage\fP\fI [-b] <path> [<path>...] \fP
-.PP
-\fBbtrfs\fP \fBdevice ready\fP\fI <device>\fP
-.PP
-\fBbtrfs\fP \fBdevice stats\fP [-z] {\fI<path>\fP|\fI<device>\fP}
-.PP
-.PP
-\fBbtrfs\fP \fBscrub start\fP [-BdqrRf] [-c \fIioprio_class\fP -n \fIioprio_classdata\fP] {\fI<path>\fP|\fI<device>\fP}
-.PP
-\fBbtrfs\fP \fBscrub cancel\fP {\fI<path>\fP|\fI<device>\fP}
-.PP
-\fBbtrfs\fP \fBscrub resume\fP [-BdqrR] [-c \fIioprio_class\fP -n \fIioprio_classdata\fP] {\fI<path>\fP|\fI<device>\fP}
-.PP
-\fBbtrfs\fP \fBscrub status\fP [-d] {\fI<path>\fP|\fI<device>\fP}
-.PP
-.PP
-\fBbtrfs\fP \fBcheck\fP [\fIoptions\fP] \fI<device>\fP
-.PP
-\fBbtrfs\fP \fBrescue chunk-recover\fP [\fIoptions\fP] \fI<device>\fP
-.PP
-\fBbtrfs\fP \fBrescue super-recover\fP [\fIoptions\fP] \fI<device>\fP
-.PP
-\fBbtrfs\fP \fBrestore\fP [\fIoptions\fP] \fI<device>\fP \fI<path>\fP | -l \fI<device>\fP
-.PP
-.PP
-\fBbtrfs\fP \fBinspect-internal inode-resolve\fP [-v] \fI<inode>\fP \fI<path>\fP
-.PP
-\fBbtrfs\fP \fBinspect-internal logical-resolve\fP [-Pv] [-s \fI<size>\fP] \fI<logical>\fP \fI<path>\fP
-.PP
-\fBbtrfs\fP \fBinspect-internal subvolid-resolve\fP \fI<subvolid>\fP \fI<path>\fP
-.PP
-\fBbtrfs\fP \fBinspect-internal rootid\fP \fI<path>\fP
-.PP
-.PP
-\fBbtrfs\fP \fBsend\fP [-ve] [-p \fI<parent>\fP] [-c \fI<clone-src>\fP] [-f \fI<outfile>\fP] \fI<subvol>\fP [\fI<subvol>\fP...]
-.PP
-\fBbtrfs\fP \fBreceive\fP [-ve] [-f \fI<infile>\fP] \fI<mount>\fP
-.PP
-.PP
-\fBbtrfs\fP \fBquota enable\fP\fI <path>\fP
-.PP
-\fBbtrfs\fP \fBquota disable\fP\fI <path>\fP
-.PP
-\fBbtrfs\fP \fBquota rescan\fP [-sw] \fI<path>\fP
-.PP
-.PP
-\fBbtrfs\fP \fBqgroup assign\fP \fI<src>\fP \fI<dst>\fP \fI<path>\fP
-.PP
-\fBbtrfs\fP \fBqgroup remove\fP \fI<src>\fP \fI<dst>\fP \fI<path>\fP
-.PP
-\fBbtrfs\fP \fBqgroup create\fP \fI<qgroupid>\fP \fI<path>\fP
-.PP
-\fBbtrfs\fP \fBqgroup destroy\fP \fI<qgroupid>\fP \fI<path>\fP
-.PP
-\fBbtrfs\fP \fBqgroup show\fP [\fIoptions\fP] \fI<path>\fP
-.PP
-\fBbtrfs\fP \fBqgroup limit\fP [\fIoptions\fP] \fI<size>\fP|\fBnone\fP [\fI<qgroupid>\fP] \fI<path>\fP
-.PP
-.PP
-\fBbtrfs\fP \fBreplace start\fP [-Bfr] \fI<srcdev>\fP|\fI<devid> <targetdev> <mount_point>\fP
-.PP
-\fBbtrfs\fP \fBreplace status\fP [-1] \fI<mount_point>\fP
-.PP
-\fBbtrfs\fP \fBreplace cancel\fP \fI<mount_point>\fP
-.PP
-\fBbtrfs\fP \fBdedup enable <path> \fP\fI\fP
-.PP
-\fBbtrfs\fP \fBdedup disable <path> \fP\fI\fP
-.PP
-\fBbtrfs\fP \fBdedup on [-b|--bs size] <path> \fP\fI\fP
-.PP
-\fBbtrfs\fP \fBdedup off <path> \fP\fI\fP
-.PP
-\fBbtrfs\fP \fBhelp|\-\-help \fP
-.PP
-\fBbtrfs\fP \fB<command> \-\-help \fP
-.PP
-.SH DESCRIPTION
-.B btrfs
-is used to control the filesystem and the files and directories stored. It is
-the tool to create or destroy a snapshot or a subvolume for the
-filesystem, to defrag a file or a directory, flush the data to the disk,
-to resize the filesystem, to scan the device.
-
-It is possible to abbreviate the commands unless the commands are ambiguous.
-For example: it is possible to run
-.I btrfs sub snaps
-instead of
-.I btrfs subvolume snapshot.
-But
-.I btrfs file s
-is not allowed, because
-.I file s
-may be interpreted both as
-.I filesystem show
-and as
-.I filesystem sync.
-In this case
-.I btrfs
-returns filesystem sync
-If a command is terminated by
-.I --help
-, the detailed help is showed. If the passed command matches more commands,
-detailed help of all the matched commands is showed. For example
-.I btrfs dev --help
-shows the help of all
-.I device*
-commands.
-
-.SH COMMANDS
-.TP
-
-\fBsubvolume create\fP [-i \fI<qgroupid>\fP] [\fI<dest>\fP/]\fI<name>\fP
-Create a subvolume \fI<name>\fR in \fI<dest>\fR.
-If \fI<dest>\fR is not given subvolume \fI<name>\fR will be created in the
-current directory.
-.RS
-
-\fIOptions\fP
-.IP "\fB-i\fP \fI<qgroupid>\fR" 5
-Add the newly created subvolume to a qgroup. This option can be given multiple
-times.
-.RE
-.TP
-
-\fBsubvolume delete\fR [\fIoptions\fP] \fI<subvolume>\fP [\fI<subvolume>...\fP]\fR
-Delete the subvolume(s) from the filesystem. If \fI<subvolume>\fR is not a
-subvolume, \fBbtrfs\fR returns an error but continues if there are more arguments
-to process.
-
-The corresponding directory is removed instantly but the data blocks are
-removed later.  The deletion does not involve full commit by default due to
-performance reasons (as a consequence, the subvolume may appear again after a
-crash).  Use one of the --commit options to wait until the operation is safely
-stored on the media.
-.RS
-
-\fIOptions\fP
-.IP "\fB-c|--commit-after\fP" 5
-wait for transaction commit at the end of the operation
-.IP "\fB-C|--commit-each\fP" 5
-wait for transaction commit after deleting each subvolume
-.RE
-.TP
-
-\fBsubvolume list\fR [\fIoptions\fP] [-G [+|-]\fIvalue\fP] [-C [+|-]\fIvalue\fP] [--sort=rootid,gen,ogen,path] \fI<path>\fR
-
-List the subvolumes present in the filesystem \fI<path>\fR. For every
-subvolume the following information is shown by default.
-ID \fI<ID>\fP top level \fI<ID>\fP path \fI<path>\fP
-where path is the relative path of the subvolume to the \fItop level\fR
-subvolume.
-
-The subvolume's ID may be used by the \fBsubvolume set-default\fR command, or
-at mount time via the \fIsubvolid=\fR option.
-If \fI-p\fR is given, then \fIparent <ID>\fR is added to the output between ID
-and top level. The parent's ID may be used at mount time via the
-\fIsubvolrootid=\fR option.
-.RS
-
-\fIOptions\fP
-.IP "\fB-p\fP" 5
-print parent ID.
-.IP "\fB-a\fP" 5
-print all the subvolumes in the filesystem and distinguish between
-absolute and relative path with respect to the given \fI<path>\fP.
-.IP "\fB-c\fP" 5
-print the ogeneration of the subvolume, aliases: ogen or origin generation.
-.IP "\fB-g\fP" 5
-print the generation of the subvolume.
-.IP "\fB-o\fP" 5
-print only subvolumes bellow specified <path>.
-.IP "\fB-u\fP" 5
-print the UUID of the subvolume.
-.IP "\fB-q\fP" 5
-print the parent uuid of subvolumes (and snapshots).
-.IP "\fB-t\fP" 5
-print the result as a table.
-.IP "\fB-s\fP" 5
-only snapshot subvolumes in the filesystem will be listed.
-.IP "\fB-r\fP" 5
-only readonly subvolumes in the filesystem will be listed.
-.IP "\fB-G [+|-]\fIvalue\fP\fP" 5
-list subvolumes in the filesystem that its generation is
->=, <= or = \fIvalue\fP. '+' means >= \fIvalue\fP, '-' means <= \fIvalue\fP, If there is
-neither '+' nor '-', it means = \fIvalue\fP.
-.IP "\fB-C [+|-]\fIvalue\fP" 5
-list subvolumes in the filesystem that its ogeneration is
->=, <= or = \fIvalue\fP. The usage is the same to '-g' option.
-.IP "\fB--sort=rootid,gen,ogen,path\fP" 5
-list subvolumes in order by specified items.
-you can add '+' or '-' in front of each items, '+' means ascending, '-'
-means descending. The default is ascending.
-
-for \fB--sort\fP you can combine some items together by ',', just like
-\f--sort=+ogen,-gen,path,rootid\fR.
-.RE
-.TP
-
-\fBsubvolume snapshot\fP [-r] [-i qgroupid] \fI<source>\fP \fI<dest>\fP|[\fI<dest>\fP/]\fI<name>\fP
-Create a writable/readonly snapshot of the subvolume \fI<source>\fR with the
-name \fI<name>\fR in the \fI<dest>\fR directory.
-If only \fI<dest>\fR is given, the subvolume will be named the basename of \fI<source>\fR.
-If \fI<source>\fR is not a subvolume, \fBbtrfs\fR returns an error.
-.RS
-
-\fIOptions\fP
-.IP \fB-r\fP 5
-The newly created snapshot will be readonly.
-.IP "\fB-i\fP \fI<qgroupid>\fR" 5
-Add the newly created subvolume to a qgroup. This option can be given multiple
-times.
-.RE
-.TP
-
-\fBsubvolume snapshot\fP [-r] [-i qgroupid] \fI-s <subvolid>\fP \fI<dest>\fP/\fI<name>\fP
-Create a writable/readonly snapshot of the subvolume \fI<subvolid>\fR with the
-name \fI<name>\fR in the \fI<dest>\fR directory.
-If \fI<subvolid>\fR does not refer to a subvolume, \fBbtrfs\fR returns an error.
-.RS
-
-\fIOptions\fP
-.IP \fB-r\fP 5
-The newly created snapshot will be readonly.
-.IP "\fB-i\fP \fI<qgroupid>\fR" 5
-Add the newly created subvolume to a qgroup. This option can be given multiple
-times.
-.RE
-.TP
-
-\fBsubvolume get-default\fR\fI <path>\fR
-Get the default subvolume of the filesystem \fI<path>\fR. The output format
-is similar to \fBsubvolume list\fR command.
-.TP
-
-\fBsubvolume set-default\fR\fI <id> <path>\fR
-Set the subvolume of the filesystem \fI<path>\fR which is mounted as
-\fIdefault\fR. The subvolume is identified by \fI<id>\fR, which
-is returned by the \fBsubvolume list\fR command.
-.TP
-
-\fBsubvolume find-new\fR\fI <subvolume> <last_gen>\fR
-List the recently modified files in a subvolume, after \fI<last_gen>\fR ID.
-.TP
-
-\fBsubvolume show\fR\fI <path>\fR
-Show information of a given subvolume in the \fI<path>\fR.
-.TP
-
-\fBfilesystem df\fP [-b] \fIpath [path..]\fR
-
-Show space usage information for a mount point.
-
-\fB-b\fP Set byte as unit
-
-The command \fBbtrfs filesystem df\fP is used to query how many space on the 
-disk(s) are used and an estimation of the free
-space of the filesystem.
-The output of the command \fBbtrfs filesystem df\fP shows:
-
-.RS
-.IP \fBDisk\ size\fP
-the total size of the disks which compose the filesystem.
-
-.IP \fBDisk\ allocated\fP
-the size of the area of the disks used by the chunks.
-
-.IP \fBDisk\ unallocated\fP 
-the size of the area of the disks which is free (i.e.
-the differences of the values above).
-
-.IP \fBUsed\fP
-the portion of the logical space used by the file and metadata.
-
-.IP \fBFree\ (estimated)\fP
-the estimated free space available: i.e. how many space can be used
-by the user. The evaluation 
-cannot be rigorous because it depends by the allocation policy (DUP, Single,
-RAID1...) of the metadata and data chunks. If every chunk is stored as
-"Single" the sum of the \fBfree (estimated)\fP space and the \fBused\fP 
-space  is equal to the \fBdisk size\fP.
-Otherwise if all the chunk are mirrored (raid1 or raid10) or duplicated
-the sum of the \fBfree (estimated)\fP space and the \fBused\fP space is
-half of the \fBdisk size\fP. Normally the \fBfree (estimated)\fP is between
-these two limits.
-
-.IP \fBData\ to\ disk\ ratio\fP
-the ratio betwen the \fBlogical size\fP (i.e. the space available by
-the chunks) and the \fBdisk allocated\fP (by the chunks). Normally it is 
-lower than 100% because the metadata is duplicated for security reasons.
-If all the data and metadata are duplicated (or have a profile like RAID1)
-the \fBData\ to\ disk\ ratio\fP could be 50%.
-.RE
-.TP
-
-\fBfilesystem show\fR [\fI--mounted\fP|\fI--all-devices\fP|\fI<path>\fP|\fI<uuid>\fP|\fI<device>\fP|\fI<lable>\fP]\fR
-Show the btrfs filesystem with some additional info. If no option or
-\fIPATH\fP|\fIUUID\fP|\fIDEVICE\fP|\fILABEL\fP
-is passed, \fBbtrfs\fR shows information of all the btrfs filesystem both mounted
-and unmounted.
-If \fB--mounted\fP is passed, it would probe btrfs kernel to list mounted btrfs filesystem(s);
-If \fB--all-devices\fP is passed, all the devices under /dev are scanned;
-otherwise the devices list is extracted from the /proc/partitions file.
-.TP
-
-\fBfilesystem sync\fR\fI <path> \fR
-Force a sync for the filesystem identified by \fI<path>\fR.
-.TP
-
-\fBfilesystem defragment\fP [\fIoptions\fP] \fI<file>\fP|\fI<dir>\fP [\fI<file>\fP|\fI<dir>...\fP]\fP
-Defragment file data and/or directory metadata. If \fB-r\fP is passed, 
-files in \fIdir\fR will be defragmented recursively.
-
-
-The start position and the number of bytes to defragment can be specified by
-\fIstart\fR and \fIlen\fR. Any extent bigger than threshold will be
-considered already defragged. Use 0 to take the kernel default, and use 1 to
-say every single extent must be rewritten. You can also turn on compression in
-defragment operations.
-.RS
-
-\fIOptions\fR
-.IP "\fB-v\fP" 5
-be verbose
-.IP "\fB-c\fP" 5
-compress file contents while defragmenting
-.IP "\fB-r\fP" 5
-defragment files recursively
-.IP "\fB-f\fP" 5
-flush filesystem after defragmenting
-.IP "\fB-s \fIstart\fP\fP" 5
-defragment only from byte \fIstart\fR onward
-.IP "\fB-l \fIlen\fP\fP" 5
-defragment only up to \fIlen\fR bytes
-.IP "\fB-t \fIsize\fP\fP" 5
-defragment only files at least \fIsize\fR bytes big
-
-For \fBstart\fP, \fBlen\fP, \fBsize\fP it is possible to append a suffix
-like \fBk\fP for 1 KBytes, \fBm\fP for 1 MBytes...
-
-NOTE: defragmenting with kernels up to 2.6.37 will unlink COW-ed copies of data,
-don't use it if you use snapshots, have de-duplicated your data or made
-copies with \fBcp --reflink\fP.
-.RE
-.TP
-
-.\"
-.\" Some wording are extracted by the resize2fs man page
-.\"
-\fBfilesystem resize\fP [\fIdevid\fP:][+/\-]\fI<size>\fP[gkm]|[\fIdevid\fP:]\fImax <path>\fR
-Resize a filesystem identified by \fI<path>\fR for the underlying device
-\fIdevid\fR.  The \fIdevid\fR can be found with \fBbtrfs filesystem show\fR and
-defaults to 1 if not specified.
-The \fI<size>\fR parameter specifies the new size of the filesystem.
-If the prefix \fI+\fR or \fI\-\fR is present the size is increased or decreased
-by the quantity \fI<size>\fR.
-If no units are specified, the unit of the \fI<size>\fR parameter defaults to
-bytes. Optionally, the size parameter may be suffixed by one of the following
-units designators: 'K', 'M', or 'G', kilobytes, megabytes, or gigabytes,
-respectively.
-
-If 'max' is passed, the filesystem will occupy all available space on the
-device \fIdevid\fR.
-
-The \fBresize\fR command \fBdoes not\fR manipulate the size of underlying
-partition.  If you wish to enlarge/reduce a filesystem, you must make sure you
-can expand the partition before enlarging the filesystem and shrink the
-partition after reducing the size of the filesystem.  This can done using
-\fBfdisk(8)\fR or \fBparted(8)\fR to delete the existing partition and recreate
-it with the new desired size.  When recreating the partition make sure to use
-the same starting disk cylinder as before.
-.TP
-
-\fBfilesystem label\fP [\fI<dev>\fP|\fI<mount_point>\fP] [\fInewlabel\fP]\fP
-Show or update the label of a filesystem. \fI[<device>|<mountpoint>]\fR is used
-to identify the filesystem. 
-
-If a \fInewlabel\fR optional argument is passed, the label is changed. The
-following constraints exist for a label:
-.IP
-- the maximum allowable length shall be less than 256 chars
-.TP
-\fBfilesystem disk-usage\fP [-tb] \fIpath [path..]\fR
-
-Show in which disk the chunks are allocated.
-
-\fB-b\fP Set byte as unit
-
-\fB-t\fP Show data in tabular format
-
-.TP
-
-\fB[filesystem] balance \fI<path>\fR
-Blanace chunks across the devices
-
-\fBbtrfs balance \fI<path>\fR is deprecated,
-please use \fBbtrfs balance start\fR command instead.
-
-.TP
-\fB[filesystem] balance start\fR [\fIoptions\fP] \fI<path>\fR
-Balance chunks across the devices
-Balance and/or convert (change allocation profile of) chunks that
-passed all \fIfilters\fR in a comma-separated list of filters for a
-particular chunk type.
-If filter list is not given balance all chunks of that type.
-In case none of the \fI-d\fR, \fI-m\fR or \fI-s\fR options is
-given balance all chunks in a filesystem.
-.RS
-
-\fIOptions\fR
-.IP "\fB-d[\fIfilters\fP]\fR" 5
-act on data chunks
-.IP "\fB-m[\fIfilters\fP]\fR" 5
-act on metadata chunks
-.IP "\fB-s[\fIfilters\fP]\fR" 5
-act on system chunks (only under -f)
-.IP "\fB-v\fP" 5
-be verbose
-.IP "\fB-f\fP" 5
-force reducing of metadata integrity
-.RE
-.TP
-
-\fB[filesystem] balance pause\fR\fI <path>\fR
-Pause running balance.
-
-.RE
-.TP
-
-\fB[filesystem] balance cancel\fR\fI <path>\fR
-Cancel running or paused balance.
-.TP
-
-\fB[filesystem] balance resume\fR\fI <path>\fR
-Resume interrupted balance.
-.TP
-
-\fB[filesystem] balance status\fR [-v] \fI<path>\fR
-Show status of running or paused balance.
-.RS
-
-\fIOptions\fR
-.IP "\fB-v\fP" 5
-be verbose
-.RE
-.TP
-
-\fBdevice add\fR [-Kf] \fI<dev> \fP[\fI<dev>...\fP] \fI<path>\fR
-Add device(s) to the filesystem identified by \fI<path>\fR.
-If applicable, a whole device discard (TRIM) operation is performed.
-.RS
-
-\fIOptions\fR
-.IP "\fB-K|--nodiscard\fP" 5
-do not perform discard by default
-.IP "\fB-f|--force\fP" 5
-force overwrite of existing filesystem on the given disk(s)
-.RE
-.TP
-
-\fBdevice delete\fR\fI <dev> \fP[\fI<dev>...\fP] \fI<path>\fR
-Remove device(s) from a filesystem identified by \fI<path>\fR.
-.TP
-
-\fBdevice scan\fR [(--all-devices|-d)|\fI<device> \fP[\fI<device>...\fP]]\fR
-If one or more devices are passed, these are scanned for a btrfs filesystem. 
-If no devices are passed, \fBbtrfs\fR uses block devices containing btrfs
-filesystem as listed by blkid.
-Finally, if \fB--all-devices\fP or \fB-d\fP is passed, all the devices under /dev are 
-scanned.
-.TP
-
-\fBdevice disk-usage\fR\fI [-b] <path> [<path>..]\fR
-Show which chunks are in a device.
-
-\fB-b\fP set byte as unit.
-.TP
-
-\fBdevice ready\fR \fI<device>\fR
-Check device to see if it has all of it's devices in cache for mounting.
-.TP
-
-\fBdevice stats\fP [-z] {\fI<path>\fP|\fI<device>\fP}
-Read and print the device IO stats for all devices of the filesystem
-identified by \fI<path>\fR or for a single \fI<device>\fR.
-.RS
-
-\fIOptions\fR
-.IP "\fB-z\fP" 5
-Reset stats to zero after reading them.
-.RE
-.TP
-
-\fBscrub start\fP [-BdqrRf] [-c \fIioprio_class\fP -n \fIioprio_classdata\fP] {\fI<path>\fP|\fI<device>\fP}
-Start a scrub on all devices of the filesystem identified by \fI<path>\fR or on
-a single \fI<device>\fR. Without options, scrub is started as a background
-process. Progress can be obtained with the \fBscrub status\fR command. Scrubbing
-involves reading all data from all disks and verifying checksums. Errors are
-corrected along the way if possible.
-.IP
-The default IO priority of scrub is the idle class. The priority can be configured similar to the
-.BR ionice (1)
-syntax.
-.RS
-
-\fIOptions\fR
-.IP "\fB-B\fP" 5
-Do not background and print scrub statistics when finished.
-.IP "\fB-d\fP" 5
-Print separate statistics for each device of the filesystem (-B only).
-.IP "\fB-q\fP" 5
-Quiet. Omit error messages and statistics.
-.IP "\fB-r\fP" 5
-Read only mode. Do not attempt to correct anything.
-.IP "\fB-R\fP" 5
-Raw print mode. Print full data instead of summary.
-.IP "\fB-c \fIioprio_class\fP" 5
-Set IO priority class (see
-.BR ionice (1)
-manpage).
-.IP "\fB-n \fIioprio_classdata\fP" 5
-Set IO priority classdata (see
-.BR ionice (1)
-manpage).
-.IP "\fB-f\fP" 5
-force to check whether scrub has started or resumed in userspace.
-this is useful when scrub stat record file is damaged.
-.RE
-.TP
-
-\fBscrub cancel\fP {\fI<path>\fP|\fI<device>\fP}
-If a scrub is running on the filesystem identified by \fI<path>\fR, cancel it.
-Progress is saved in the scrub progress file and scrubbing can be resumed later
-using the \fBscrub resume\fR command.
-If a \fI<device>\fR is given, the corresponding filesystem is found and
-\fBscrub cancel\fP behaves as if it was called on that filesystem.
-.TP
-
-\fBscrub resume\fP [-BdqrR] [-c ioprio_class -n ioprio_classdata] {\fI<path>\fP|\fI<device>\fP}
-Resume a canceled or interrupted scrub cycle on the filesystem identified by
-\fI<path>\fR or on a given \fI<device>\fR. Does not start a new scrub if the
-last scrub finished successfully.
-.RS
-
-\fIOptions\fR
-.TP
-see \fBscrub start\fP.
-.RE
-.TP
-
-\fBscrub status\fP [-d] {\fI<path>\fP|\fI<device>\fP}
-Show status of a running scrub for the filesystem identified by \fI<path>\fR or
-for the specified \fI<device>\fR.
-If no scrub is running, show statistics of the last finished or canceled scrub
-for that filesystem or device.
-.RS
-
-\fIOptions\fR
-.IP "\fB-d\fP" 5
-Print separate statistics for each device of the filesystem.
-.RE
-.TP
-
-\fBcheck\fR [\fIoptions\fP] <device>\fR
-Check an unmounted btrfs filesystem.
-.RS
-
-\fIOptions\fR
-.IP "\fB-s|--support \fI<superblock>\fP\fR" 5
-use this superblock copy.
-.IP "\fB--repair\fP" 5
-try to repair the filesystem.
-.IP "\fB--init-csum-tree\fP" 5
-create a new CRC tree.
-.IP "\fB--init-extent-tree\fP" 5
-create a new extent tree.
-.RE
-.TP
-
-\fBrescue chunk-recover\fR [\fIoptions\fP] <device>\fR
-Recover the chunk tree by scanning the devices one by one.
-.RS
-
-\fIOptions\fR
-.IP "\fB-y\fP" 5
-assume an answer of 'yes' to all questions.
-.IP "\fB-v\fP" 5
-verbose mode.
-.IP "\fB-h\fP" 5
-help.
-.RE
-.TP
-
-\fBrescue super-recover\fR [\fIoptions\fP] <device>\fR
-Recover bad superblocks from good copies.
-.RS
-
-\fIOptions\fR
-.IP "\fB-y\fP" 5
-assume an answer of 'yes' to all questions.
-.IP "\fB-v\fP" 5
-verbose mode.
-.RE
-.TP
-
-\fBrestore\fR [\fIoptions\fP] \fI<device>\fR \fI<path>\fR | -l \fI<device>\fP
-Try to restore files from a damaged filesystem(unmounted) to given \fI<path>\fR or list  tree roots.
-.RS
-
-\fIOptions\fR
-.IP "\fB-s\fP" 5
-get snapshots.
-.IP "\fB-x\fP" 5
-get extended attributes.
-.IP "\fB-v\fP" 5
-verbose.
-.IP "\fB-i\fP" 5
-ignore errors.
-.IP "\fB-o\fP" 5
-overwrite.
-.IP "\fB-t \fI<location>\fP\fP" 5
-tree location.
-.IP "\fB-f \fI<offset>\fP\fP" 5
-filesystem location.
-.IP "\fB-u \fI<block>\fP\fP" 5
-super mirror.
-.IP "\fB-r \fI<rootid>\fP\fP" 5
-root objectid.
-.IP "\fB-d\fP" 5
-find dir.
-.IP "\fB-l\fP" 5
-list tree roots.
-.RE
-.TP
-
-\fBinspect-internal inode-resolve\fP [-v] \fI<inode>\fP \fI<path>\fP
-Resolves an <inode> in subvolume <path> to all filesystem paths.
-.RS
-
-\fIOptions\fR
-.IP "\fB-v\fP" 5
-verbose mode. print count of returned paths and ioctl() return value
-.RE
-.TP
-
-\fBinspect-internal logical-resolve\fP [-Pv] [-s bufsize] \fI<logical>\fP \fI<path>\fP
-Resolves a <logical> address in the filesystem mounted at <path> to all inodes.
-By default, each inode is then resolved to a file system path (similar to the
-\fBinode-resolve\fP subcommand).
-.RS
-
-\fIOptions\fR
-.IP "\fB-P\fP" 5
-skip the path resolving and print the inodes instead
-.IP "\fB-v\fP" 5
-verbose mode. print count of returned paths and all ioctl() return values
-.IP "\fB-s \fI<bufsize>\fP" 5
-set inode container's size. This is used to increase inode container's size in case it is
-not enough to read all the resolved results. The max value one can set is 64k.
-.RE
-.TP
-
-\fBinspect-internal subvolid-resolve\fP \fI<subvolid> <path>\fP
-Get file system paths for the given subvolume ID.
-.TP
-
-\fBinspect-internal rootid\fP \fI<path>\fP
-For a given file or directory, return the containing tree root id. For a
-subvolume return it's own tree id.
-
-The result is undefined for the so-called empty subvolumes (identified by inode number 2).
-.TP
-
-\fBsend\fP [-ve] [-p \fI<parent>\fP] [-c \fI<clone-src>\fP] [-f \fI<outfile>\fP] \fI<subvol>\fP [\fI<subvol>...\fP]
-Send the subvolume(s) to stdout.
-Sends the subvolume(s) specified by \fI<subvol>\fR to stdout.
-By default, this will send the whole subvolume. To do an incremental
-send, use '\fI-p <parent>\fR'. If you want to allow btrfs to clone from
-any additional local snapshots, use '\fI-c <clone-src>\fR' (multiple times
-where applicable). You must not specify clone sources unless you
-guarantee that these snapshots are exactly in the same state on both
-sides, the sender and the receiver. It is allowed to omit the '\fI-p <parent>\fR'
-option when '\fI-c <clone-src>\fR' options are given, in
-which case '\fBbtrfs send\fP' will determine a suitable parent among the
-clone sources itself.
-.RS
-
-\fIOptions\fR
-.IP "\fB-v\fP" 5
-Enable verbose debug output. Each occurrence of this option increases the
-verbose level more.
-.IP "\fB-e\fP" 5
-If sending multiple subvols at once, use the new format and omit the <end cmd> between the subvols.
-.IP "\fB-p \fI<parent>\fP" 5
-Send an incremental stream from \fI<parent>\fR to \fI<subvol>\fR.
-.IP "\fB-c \fI<clone-src>\fP" 5
-Use this snapshot as a clone source for an incremental send (multiple allowed).
-.IP "\fB-f \fI<outfile>\fP" 5
-Output is normally written to stdout. To write to a file, use this option.
-An alternative would be to use pipes.
-.RE
-.TP
-
-\fBreceive\fP [-ve] [-f \fI<infile>\fR] \fI<mount>\fR
-Receive subvolumes from stdin.
-Receives one or more subvolumes that were previously
-sent with btrfs send. The received subvolumes are stored
-into \fI<mount>\fP.
-btrfs receive will fail with the following case:
-
-1.a receiving subvolume already exists.
-
-2.a previously received subvolume was changed after it was received.
-
-3.default subvolume is changed or you don't mount btrfs filesystem with
-fs tree.
-
-After receiving a subvolume, it is immediately set to read only.
-.RS
-
-\fIOptions\fR
-.IP "\fB-v\fP" 5
-Enable verbose debug output. Each occurrence of this option increases the
-verbose level more.
-.IP "\fB-f \fI<infile>\fR" 5
-By default, btrfs receive uses stdin to receive the subvolumes.
-Use this option to specify a file to use instead.
-.IP "\fB-e\fP" 5
-Terminate after receiving an <end cmd> in the data stream.
-Without this option, the receiver terminates only if an error is recognized or on EOF.
-.RE
-.TP
-
-\fBquota enable\fR \fI<path>\fR
-Enable subvolume quota support for a filesystem.
-.TP
-
-\fBquota disable\fR \fI<path>\fR
-Disable subvolume quota support for a filesystem.
-.TP
-
-\fBquota rescan\fR [-s] \fI<path>\fR
-Trash all qgroup numbers and scan the metadata again with the current config.
-.RS
-
-\fIOptions\fR
-.IP "\fB-s\fP" 5
-show status of a running rescan operation.
-.IP "\fB-w\fP" 5
-wait for rescan operation to finish(can be already in progress).
-.RE
-.TP
-
-\fBqgroup assign\fP \fI<src> <dst> <path>\fP
-Enable subvolume qgroup support for a filesystem.
-.TP
-
-\fBqgroup remove\fP \fI<src> <dst> <path>\fP
-Remove a subvol from a quota group.
-.TP
-
-\fBqgroup create\fP \fI<qgroupid> <path>\fP
-Create a subvolume quota group.
-.TP
-
-\fBqgroup destroy\fP \fI<qgroupid> <path>\fP
-Destroy a subvolume quota group.
-.TP
-
-\fBqgroup show\fP [\fIoptions\fP] \fI<path>\fP
-Show all subvolume quota groups.
-.RS
-
-\fIOptions\fR
-.IP "\fB-p\fP" 5
-print parent qgroup id.
-.IP "\fB-c\fP" 5
-print child qgroup id.
-.IP "\fB-r\fP" 5
-print max referenced size of qgroup.
-.IP "\fB-e\fP" 5
-print max exclusive size of qgroup.
-.IP "\fB-F\fP" 5
-list all qgroups which impact the given path(include ancestral qgroups)
-.IP "\fB-f\fP" 5
-list all qgroups which impact the given path(exclude ancestral qgroups)
-.IP "\fB--sort=ATTR\fP" 5
-list qgroups in order of ATTR.
-ATTR can be one or more of \fBqgroupid\fP,\fBrfer\fP,\fBexcl\fP,\fBmax_rfer\fP,\fBmax_excl\fP.
-If multiple ATTRs is given, use comma to seperate.
-.RE
-.TP
-
-\fBqgroup limit\fP [\fIoptions\fP] \fI<size>\fP|\fBnone\fP [\fI<qgroupid>\fP] \fI<path>\fP
-Limit the size of a subvolume quota group.
-.TP
-
-\fBreplace start\fR [-Bfr] \fI<srcdev>\fP|\fI<devid> <targetdev> <path>\fR
-Replace device of a btrfs filesystem.
-On a live filesystem, duplicate the data to the target device which
-is currently stored on the source device. If the source device is not
-available anymore, or if the \fB-r\fR option is set, the data is built
-only using the RAID redundancy mechanisms. After completion of the
-operation, the source device is removed from the filesystem.
-If the \fI<srcdev>\fR is a numerical value, it is assumed to be the device id
-of the filesystem which is mounted at mount_point, otherwise is is
-the path to the source device. If the source device is disconnected,
-from the system, you have to use the \fIdevid\fR parameter format.
-The \fI<targetdev>\fP needs to be same size or larger than the \fI<srcdev>\fR.
-.RS
-
-\fIOptions\fR
-.IP "\fB-r\fP" 5
-only read from \fI<srcdev>\fR if no other zero-defect mirror exists (enable
-this if your drive has lots of read errors, the access would be very slow)
-.IP "\fB-f\fP" 5
-force using and overwriting \fI<targetdev>\fR even if it looks like
-containing a valid btrfs filesystem. A valid filesystem is
-assumed if a btrfs superblock is found which contains a
-correct checksum. Devices which are currently mounted are
-never allowed to be used as the \fI<targetdev>\fR
-.IP "\fB-B\fP" 5
-do not background
-.RE
-.TP
-
-\fBreplace status\fR [-1] \fI<mount_point>\fR
-Print status and progress information of a running device replace operation.
-.RS
-
-\fIOptions\fR
-.IP "\fB-1\fP" 5
-print once instead of print continuously until the replace
-operation finishes (or is canceled)
-.RE
-.TP
-
-\fBreplace cancel\fR \fI<mount_point>\fR
-Cancel a running device replace operation.
-.TP
-
-\fBbtrfs dedup enable\fP \fI<path>\fP
-Enable data deduplication support for a filesystem.
-.TP
-
-\fBbtrfs dedup disable\fP \fI<path>\fP
-Disable data deduplication support for a filesystem.
-.TP
-
-\fBbtrfs dedup on [-b|--bs size]\fP \fI<path>\fP
-Switch on data deduplication or change the dedup blocksize.
-.TP
-
-\fBbtrfs dedup off\fP \fI<path>\fP
-Switch off data deduplication.
-.RE
-
-.SH EXIT STATUS
-\fBbtrfs\fR returns a zero exist status if it succeeds. Non zero is returned in
-case of failure.
-
-.SH AVAILABILITY
-.B btrfs
-is part of btrfs-progs. Btrfs filesystem is currently under heavy development,
-and not suitable for any uses other than benchmarking and review.
-Please refer to the btrfs wiki http://btrfs.wiki.kernel.org for
-further details.
-.SH SEE ALSO
-.BR mkfs.btrfs (8),
-.BR ionice (1)
diff --git a/man/btrfsck.8.in b/man/btrfsck.8.in
deleted file mode 100644
index bf65e3c..0000000
--- a/man/btrfsck.8.in
+++ /dev/null
@@ -1,29 +0,0 @@
-.TH BTRFSCK 8
-.SH NAME
-btrfsck \- check and repair of a Btrfs filesystem
-.SH SYNOPSIS
-.B btrfsck [\fIoptions\fP] \fI<device>\fP
-.SH DESCRIPTION
-\fBbtrfsck\fP is used to check and optionally repair of a Btrfs filesystem. Now, it can only be run on an unmounted FS. Considering it is not well-tested
-in real-life situations yet. if you have a broken Btrfs filesystem, btrfsck may not repair but cause aditional damages. \fI<device>\fP is the device file
-where the filesystem is stored.
-
-\fIOptions\fP
-.IP "\fB-s,--super \fI<superblock>\fP" 5
-specify which superblock copy that you want to use.
-.IP "\fB--repair\fP" 5
-try to repair the filesystem.
-.IP "\fB--init-csum-tree\fP" 5
-create a new CRC tree.
-.IP "\fB--init-extent-tree\fP" 5
-create a new extent tree.
-
-.SH EXIT CODE
-\fBbtrfsck\fR will return 0 exit code if no error happened.
-Other exit code means some problems happened.
-
-.br
-This is free software: you are free  to  change  and  redistribute  it. There is NO WARRANTY, to the extent permitted by law.
-.SH SEE ALSO
-.BR mkfs.btrfs (8),
-.BR btrfs (8)
diff --git a/man/btrfstune.8.in b/man/btrfstune.8.in
deleted file mode 100644
index c7fff9a..0000000
--- a/man/btrfstune.8.in
+++ /dev/null
@@ -1,31 +0,0 @@
-.TH BTRFSTUNE 8
-.SH NAME
-btrfstune \- tune various filesystem parameters.
-.SH SYNOPSIS
-.B btrfstune [\fIoptions\fP] \fI<device>\fP
-.SH DESCRIPTION
-\fBbtrfstune\fP is used to tune various filesystem parameters,you can
-enable/disable some extended features for btrfs.
-
-\fIOptions\fP
-.IP "\fB-S \fI<value>\fP" 5
-updates the seeding value, it forces a fs readonly so that you can use it to build other filesystems.
-.IP "\fB-r\fP" 5
-enable extended inode refs.
-.IP "\fB-x\fP" 5
-enable skinny metadata extent refs.
-
-.SH EXIT CODE
-\fBbtrfstune\fP will return 0 if no error happened.
-If any problems happened, 1 will be returned.
-
-.SH AUTHOR
-Written by Shilong Wang and Wenruo Qu.
-
-.SH COPYRIGHT
-Copyright \(co 2013 FUJITSU LIMITED.
-License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
-.br
-This is free software: you are free  to  change  and  redistribute  it. There is NO WARRANTY, to the extent permitted by law.
-.SH SEE ALSO
-.BR mkfs.btrfs (8)
diff --git a/man/fsck.btrfs.8.in b/man/fsck.btrfs.8.in
deleted file mode 100644
index eb21c89..0000000
--- a/man/fsck.btrfs.8.in
+++ /dev/null
@@ -1,47 +0,0 @@
-.TH fsck.btrfs 8
-.SH NAME
-fsck.btrfs \- do nothing, successfully
-.SH SYNOPSIS
-.B fsck.btrfs
-[\fB-aApy\fP]
-[\fBdevice...\fP]
-.SH DESCRIPTION
-.B fsck.btrfs
-is a type of utility that should exist for any filesystem and is
-called during system setup when the corresponding
-.BR /etc/fstab
-entries contain non-zero value for
-.BR fs_passno
-, see
-.BR fstab(5)
-for more.
-.PP
-Traditional filesystems need to run their respective fsck utility in case the
-filesystem was not unmounted cleanly and the log needs to be replayed before
-mount. This is not needed for BTRFS. You should set fs_passno to 0.
-.PP
-If you wish to check the consistency of a BTRFS filesystem or repair a damaged
-filesystem, see
-.BR btrfs(8)
-subcommand 'check'. By default the filesystem
-consistency is checked, the repair mode is enabled via --repair option (use
-with care!).
-.SH OPTIONS
-The options detect if \fBfsck.btrfs\fP is executed in non-interactive mode and exits
-with success, otherwise prints a message about \fBbtrfs check\fP.
-.SH EXIT CODE
-There are two possible exit code returned:
-.RS
-.IP 0 5
-No errors
-.IP 8 5
-Operational error, eg. device does not exist
-.RE
-.
-.SH FILES
-.IR /etc/fstab .
-.SH SEE ALSO
-.BR btrfs (8),
-.BR fsck (8),
-.BR fstab (5),
-.\" btrfsck is intentionally left out
diff --git a/man/mkfs.btrfs.8.in b/man/mkfs.btrfs.8.in
deleted file mode 100644
index dabeb62..0000000
--- a/man/mkfs.btrfs.8.in
+++ /dev/null
@@ -1,105 +0,0 @@
-.TH MKFS.BTRFS 8
-.SH NAME
-mkfs.btrfs \- create a btrfs filesystem
-.SH SYNOPSIS
-.B mkfs.btrfs
-[ \fB\-A\fP\fI alloc-start\fP ]
-[ \fB\-b\fP\fI byte-count\fP ]
-[ \fB\-d\fP\fI data-profile\fP ]
-[ \fB\-f\fP ]
-[ \fB\-n\fP\fI nodesize\fP ]
-[ \fB\-l\fP\fI leafsize\fP ]
-[ \fB\-L\fP\fI label\fP ]
-[ \fB\-m\fP\fI metadata profile\fP ]
-[ \fB\-M\fP\fI mixed data+metadata\fP ]
-[ \fB\-s\fP\fI sectorsize\fP ]
-[ \fB\-r\fP\fI rootdir\fP ]
-[ \fB\-K\fP ]
-[ \fB\-O\fP\fI feature1,feature2,...\fP ]
-[ \fB\-h\fP ]
-[ \fB\-V\fP ]
-\fI device\fP [ \fIdevice ...\fP ]
-.SH DESCRIPTION
-.B mkfs.btrfs
-is used to create a btrfs filesystem (usually in a disk partition, or an array
-of disk partitions).
-.I device
-is the special file corresponding to the device (e.g \fI/dev/sdXX\fP ).
-If multiple \fI devices \fP are specified, btrfs is created
-spanning across the specified \fI devices\fP.
-.SH OPTIONS
-.TP
-\fB\-A\fR, \fB\-\-alloc\-start \fIoffset\fR
-Specify the offset from the start of the device to start the btrfs filesystem. The default value is zero, or the start of the device.
-.TP
-\fB\-b\fR, \fB\-\-byte\-count \fIsize\fR
-Specify the size of the resultant filesystem. If this option is not used,
-mkfs.btrfs uses all the available storage for the filesystem.
-.TP
-\fB\-d\fR, \fB\-\-data \fItype\fR
-Specify how the data must be spanned across the devices specified. Valid
-values are raid0, raid1, raid5, raid6, raid10 or single.
-.TP
-\fB\-f\fR, \fB\-\-force\fR
-Force overwrite when an existing filesystem is detected on the device.
-By default, mkfs.btrfs will not write to the device if it suspects that 
-there is a filesystem or partition table on the device already.
-.TP
-\fB\-n\fR, \fB\-\-nodesize \fIsize\fR
-\fB\-l\fR, \fB\-\-leafsize \fIsize\fR
-Specify the nodesize, the tree block size in which btrfs stores
-data. The default value is 16KB (16384) or the page size, whichever is
-bigger. Must be a multiple of the sectorsize, but not larger than
-65536. Leafsize always equals nodesize and the options are aliases.
-.TP
-\fB\-L\fR, \fB\-\-label \fIname\fR
-Specify a label for the filesystem.
-.TP
-\fB\-m\fR, \fB\-\-metadata \fIprofile\fR
-Specify how metadata must be spanned across the devices specified. Valid
-values are raid0, raid1, raid5, raid6, raid10, single or dup.  Single device
-will have dup set by default except in the case of SSDs which will default to
-single. This is because SSDs can remap blocks internally so duplicate blocks
-could end up in the same erase block which negates the benefits of doing
-metadata duplication.
-.TP
-\fB\-M\fR, \fB\-\-mixed\fR
-Mix data and metadata chunks together for more efficient space 
-utilization.  This feature incurs a performance penalty in
-larger filesystems.  It is recommended for use with filesystems
-of 1 GiB or smaller.
-.TP
-\fB\-s\fR, \fB\-\-sectorsize \fIsize\fR
-Specify the sectorsize, the minimum data block allocation unit. The default
-value is the page size. If the sectorsize differs from the page size, the
-created filesystem may not be mountable by current kernel. Therefore it is not
-recommended to use this option unless you are going to mount it on a system
-with the appropriate page size.
-.TP
-\fB\-r\fR, \fB\-\-rootdir \fIrootdir\fR
-Specify a directory to copy into the newly created fs.
-.TP
-\fB\-K\fR, \fB\-\-nodiscard \fR
-Do not perform whole device TRIM operation by default.
-.TP
-\fB\-O\fR, \fB\-\-features \fIfeature1,feature2,...\fR
-A list of filesystem features turned on at mkfs time. Not all features are
-supported by old kernels.
-
-To see all run
-
-\fBmkfs.btrfs -O list-all\fR
-.TP
-\fB\-V\fR, \fB\-\-version\fR
-Print the \fBmkfs.btrfs\fP version and exit.
-.SH UNIT
-As default the unit is the byte, however it is possible to append a suffix
-to the arguments like \fBk\fP for KBytes, \fBm\fP for MBytes...
-.SH AVAILABILITY
-.B mkfs.btrfs
-is part of btrfs-progs. Btrfs is currently under heavy development,
-and not suitable for any uses other than benchmarking and review.
-Please refer to the btrfs wiki
-http://btrfs.wiki.kernel.org for further details.
-.SH SEE ALSO
-.BR btrfsck (8)
-- 
1.9.1

Re: [PATCH 00/27] Replace the old man page with asciidoc and man page for each btrfs subcommand.

[Chris Mason]

On 04/02/2014 04:29 AM, Qu Wenruo wrote:
> Convert the old btrfs man pages to new asciidoc and split the huge
> btrfs man page into subcommand man page.
>
> The asciidoc style and Makefile things are mostly simplified from git
> Documentation, which only supports man page output and remove html output,
> since html output is somewhat overkilled for btrfs.

Thanks for doing this.  I've never liked roff, but I'll give people on 
the list a chance to complain before taking this one.

-chris

Re: [PATCH 00/27] Replace the old man page with asciidoc and man page for each btrfs subcommand.

[Marc MERLIN]

On Wed, Apr 02, 2014 at 09:24:10AM -0400, Chris Mason wrote:
> On 04/02/2014 04:29 AM, Qu Wenruo wrote:
> >Convert the old btrfs man pages to new asciidoc and split the huge
> >btrfs man page into subcommand man page.
> >
> >The asciidoc style and Makefile things are mostly simplified from git
> >Documentation, which only supports man page output and remove html output,
> >since html output is somewhat overkilled for btrfs.
> 
> Thanks for doing this.  I've never liked roff, but I'll give people
> on the list a chance to complain before taking this one.

1970's called and it wants it troff back :)

More seriously, thanks much for this patch, it makes the source man
pages easier to read and to edit, which is a great thing for keeping them
more up to date.

+10 here.

Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/                         | PGP 1024R/763BE901

Re: [PATCH 00/27] Replace the old man page with asciidoc and man page for each btrfs subcommand.

[David Sterba]

On Wed, Apr 02, 2014 at 04:29:11PM +0800, Qu Wenruo wrote:
> Convert the old btrfs man pages to new asciidoc and split the huge
> btrfs man page into subcommand man page.

Excellent!

> The asciidoc style and Makefile things are mostly simplified from git
> Documentation, which only supports man page output and remove html output,
> since html output is somewhat overkilled for btrfs.

Well, I've been thinking about automatically pushing the man pages to
wiki, with asciidoc it would be much easier, the formatting capabilities
are better than what *roff provides.

> 1) More human-readable raw documentations.
>   The roff grammar is not so easy to read, especially mixed with a lot
>   of special words.
> 
> 2) Easier to modify man page.
>   The old huge btrfs man page makes it hard to maintain.
>   It's common that some one adds some new options, but only modifies
>   the detailed options but not the 'SYNOPSIS' section.

I hope both points will encourage more people to contribute to the
documentation.

Re: [PATCH 00/27] Replace the old man page with asciidoc and man page for each btrfs subcommand.

[Zach Brown]

On Wed, Apr 02, 2014 at 09:24:10AM -0400, Chris Mason wrote:
> On 04/02/2014 04:29 AM, Qu Wenruo wrote:
> >Convert the old btrfs man pages to new asciidoc and split the huge
> >btrfs man page into subcommand man page.
> >
> >The asciidoc style and Makefile things are mostly simplified from git
> >Documentation, which only supports man page output and remove html output,
> >since html output is somewhat overkilled for btrfs.
> 
> Thanks for doing this.  I've never liked roff, but I'll give people
> on the list a chance to complain before taking this one.

Moving to asciidoc is definitely the right thing to do.  Dave's also
been moving the xfs docs in this direction.

http://oss.sgi.com/cgi-bin/gitweb.cgi?p=xfs/xfs-documentation.git;a=summary

- z

Re: [PATCH 24/27] btrfs-progs: Convert man page for btrfs-zero-log

[Marc MERLIN]

On Wed, Apr 02, 2014 at 04:29:35PM +0800, Qu Wenruo wrote:
> Convert man page for btrfs-zero-log
> 
> Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
> ---
>  Documentation/Makefile           |  2 +-
>  Documentation/btrfs-zero-log.txt | 39 +++++++++++++++++++++++++++++++++++++++
>  2 files changed, 40 insertions(+), 1 deletion(-)
>  create mode 100644 Documentation/btrfs-zero-log.txt
> 
> diff --git a/Documentation/Makefile b/Documentation/Makefile
> index e002d53..de06629 100644
> --- a/Documentation/Makefile
> +++ b/Documentation/Makefile
> @@ -11,7 +11,7 @@ MAN8_TXT += btrfs-image.txt
>  MAN8_TXT += btrfs-map-logical.txt
>  MAN8_TXT += btrfs-show-super.txt
>  MAN8_TXT += btrfstune.txt
> -#MAN8_TXT += btrfs-zero-log.txt
> +MAN8_TXT += btrfs-zero-log.txt
>  #MAN8_TXT += fsck.btrfs.txt
>  #MAN8_TXT += mkfs.btrfs.txt
>  
> diff --git a/Documentation/btrfs-zero-log.txt b/Documentation/btrfs-zero-log.txt
> new file mode 100644
> index 0000000..e3041fa
> --- /dev/null
> +++ b/Documentation/btrfs-zero-log.txt
> @@ -0,0 +1,39 @@
> +btrfs-zero-log(8)
> +=================
> +
> +NAME
> +----
> +btrfs-zero-log - clear out log tree
> +
> +SYNOPSIS
> +--------
> +'btrfs-zero-log' <dev>
> +
> +DESCRIPTION
> +-----------
> +'btrfs-zero-log' will remove the log tree if log tree is corrupt, which will
> +allow you to mount the filesystem again.
> +
> +The common case where this happens has been fixed a long time ago,
> +so it is unlikely that you will see this particular problem.

A note on this one: this can happen if your SSD rites things in the
wrong order or potentially writes garbage when power is lost, or before
locking up.
I hit this problem about 10 times and it wasn't a btrfs bug, just the
drive doing bad things.

I had debian add this to the initramfs initrd by default so that someone
can recover their root filesystem with this command if it won't mount.

What got fixed is the kernel used to oops and crash, and now it gives a
nice "can't mount" error message.

Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/                         | PGP 1024R/763BE901

Re: [PATCH 14/27] btrfs-progs: Convert man page for btrfs-replace.

[Marc MERLIN]

On Wed, Apr 02, 2014 at 04:29:25PM +0800, Qu Wenruo wrote:
> +If the source device is not available anymore, or if the -r option is set,
> +the data is built only using the RAID redundancy mechanisms.
> +After completion of the operation, the source device is removed from the
> +filesystem.

Woudl it make sense to add a paragraph explaining that for raid5/6,
someone should either:
1) use balance to rebuild on a new drive if one of the drives is missing
2) use btrfs device add of a new drive, then btrfs device delete of the
drive to replace, and effectively btrfs will do the same thing that
replace would

?

Thanks,
Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/                         | PGP 1024R/763BE901

Re: [PATCH 24/27] btrfs-progs: Convert man page for btrfs-zero-log

[cwillu]

On Fri, Apr 4, 2014 at 12:46 PM, Marc MERLIN <marc@merlins.org> wrote:
> On Wed, Apr 02, 2014 at 04:29:35PM +0800, Qu Wenruo wrote:
>> Convert man page for btrfs-zero-log
>>
>> Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
>> ---
>>  Documentation/Makefile           |  2 +-
>>  Documentation/btrfs-zero-log.txt | 39 +++++++++++++++++++++++++++++++++++++++
>>  2 files changed, 40 insertions(+), 1 deletion(-)
>>  create mode 100644 Documentation/btrfs-zero-log.txt
>>
>> diff --git a/Documentation/Makefile b/Documentation/Makefile
>> index e002d53..de06629 100644
>> --- a/Documentation/Makefile
>> +++ b/Documentation/Makefile
>> @@ -11,7 +11,7 @@ MAN8_TXT += btrfs-image.txt
>>  MAN8_TXT += btrfs-map-logical.txt
>>  MAN8_TXT += btrfs-show-super.txt
>>  MAN8_TXT += btrfstune.txt
>> -#MAN8_TXT += btrfs-zero-log.txt
>> +MAN8_TXT += btrfs-zero-log.txt
>>  #MAN8_TXT += fsck.btrfs.txt
>>  #MAN8_TXT += mkfs.btrfs.txt
>>
>> diff --git a/Documentation/btrfs-zero-log.txt b/Documentation/btrfs-zero-log.txt
>> new file mode 100644
>> index 0000000..e3041fa
>> --- /dev/null
>> +++ b/Documentation/btrfs-zero-log.txt
>> @@ -0,0 +1,39 @@
>> +btrfs-zero-log(8)
>> +=================
>> +
>> +NAME
>> +----
>> +btrfs-zero-log - clear out log tree
>> +
>> +SYNOPSIS
>> +--------
>> +'btrfs-zero-log' <dev>
>> +
>> +DESCRIPTION
>> +-----------
>> +'btrfs-zero-log' will remove the log tree if log tree is corrupt, which will
>> +allow you to mount the filesystem again.
>> +
>> +The common case where this happens has been fixed a long time ago,
>> +so it is unlikely that you will see this particular problem.
>
> A note on this one: this can happen if your SSD rites things in the
> wrong order or potentially writes garbage when power is lost, or before
> locking up.
> I hit this problem about 10 times and it wasn't a btrfs bug, just the
> drive doing bad things.

And -o recovery didn't work around it?  My understanding is that -o
recovery will skip reading the log.

Re: [PATCH 24/27] btrfs-progs: Convert man page for btrfs-zero-log

[Marc MERLIN]

On Sat, Apr 05, 2014 at 04:00:27PM -0600, cwillu wrote:
> >> +'btrfs-zero-log' will remove the log tree if log tree is corrupt, which will
> >> +allow you to mount the filesystem again.
> >> +
> >> +The common case where this happens has been fixed a long time ago,
> >> +so it is unlikely that you will see this particular problem.
> >
> > A note on this one: this can happen if your SSD rites things in the
> > wrong order or potentially writes garbage when power is lost, or before
> > locking up.
> > I hit this problem about 10 times and it wasn't a btrfs bug, just the
> > drive doing bad things.
> 
> And -o recovery didn't work around it?  My understanding is that -o
> recovery will skip reading the log.

Maybe it does, but if you're trying to mount your root filesystem to boot
your laptop, that's not super useful since -o recovery is indeed a read only
recovery mode.
btrfs-zero-log just cleans the last log entry and gave me back a fully working
read/write filesystem each time.

Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: [PATCH 24/27] btrfs-progs: Convert man page for btrfs-zero-log

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 2703 bytes --]

On Sat, Apr 05, 2014 at 04:00:27PM -0600, cwillu wrote:
> On Fri, Apr 4, 2014 at 12:46 PM, Marc MERLIN <marc@merlins.org> wrote:
> > On Wed, Apr 02, 2014 at 04:29:35PM +0800, Qu Wenruo wrote:
> >> Convert man page for btrfs-zero-log
> >>
> >> Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
> >> ---
> >>  Documentation/Makefile           |  2 +-
> >>  Documentation/btrfs-zero-log.txt | 39 +++++++++++++++++++++++++++++++++++++++
> >>  2 files changed, 40 insertions(+), 1 deletion(-)
> >>  create mode 100644 Documentation/btrfs-zero-log.txt
> >>
> >> diff --git a/Documentation/Makefile b/Documentation/Makefile
> >> index e002d53..de06629 100644
> >> --- a/Documentation/Makefile
> >> +++ b/Documentation/Makefile
> >> @@ -11,7 +11,7 @@ MAN8_TXT += btrfs-image.txt
> >>  MAN8_TXT += btrfs-map-logical.txt
> >>  MAN8_TXT += btrfs-show-super.txt
> >>  MAN8_TXT += btrfstune.txt
> >> -#MAN8_TXT += btrfs-zero-log.txt
> >> +MAN8_TXT += btrfs-zero-log.txt
> >>  #MAN8_TXT += fsck.btrfs.txt
> >>  #MAN8_TXT += mkfs.btrfs.txt
> >>
> >> diff --git a/Documentation/btrfs-zero-log.txt b/Documentation/btrfs-zero-log.txt
> >> new file mode 100644
> >> index 0000000..e3041fa
> >> --- /dev/null
> >> +++ b/Documentation/btrfs-zero-log.txt
> >> @@ -0,0 +1,39 @@
> >> +btrfs-zero-log(8)
> >> +=================
> >> +
> >> +NAME
> >> +----
> >> +btrfs-zero-log - clear out log tree
> >> +
> >> +SYNOPSIS
> >> +--------
> >> +'btrfs-zero-log' <dev>
> >> +
> >> +DESCRIPTION
> >> +-----------
> >> +'btrfs-zero-log' will remove the log tree if log tree is corrupt, which will
> >> +allow you to mount the filesystem again.
> >> +
> >> +The common case where this happens has been fixed a long time ago,
> >> +so it is unlikely that you will see this particular problem.
> >
> > A note on this one: this can happen if your SSD rites things in the
> > wrong order or potentially writes garbage when power is lost, or before
> > locking up.
> > I hit this problem about 10 times and it wasn't a btrfs bug, just the
> > drive doing bad things.
> 
> And -o recovery didn't work around it?  My understanding is that -o
> recovery will skip reading the log.

   No, I'm pretty sure we've had people with problems with the log
where -orecovery didn't help, but -oro,recovery allowed it to be
mounted, because -ro didn't try to replay the log.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
     --- If the first-ever performance is the première,  is the ---     
                  last-ever performance the derrière?                   

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]

Re: [PATCH 24/27] btrfs-progs: Convert man page for btrfs-zero-log

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 1409 bytes --]

On Sat, Apr 05, 2014 at 03:02:03PM -0700, Marc MERLIN wrote:
> On Sat, Apr 05, 2014 at 04:00:27PM -0600, cwillu wrote:
> > >> +'btrfs-zero-log' will remove the log tree if log tree is corrupt, which will
> > >> +allow you to mount the filesystem again.
> > >> +
> > >> +The common case where this happens has been fixed a long time ago,
> > >> +so it is unlikely that you will see this particular problem.
> > >
> > > A note on this one: this can happen if your SSD rites things in the
> > > wrong order or potentially writes garbage when power is lost, or before
> > > locking up.
> > > I hit this problem about 10 times and it wasn't a btrfs bug, just the
> > > drive doing bad things.
> > 
> > And -o recovery didn't work around it?  My understanding is that -o
> > recovery will skip reading the log.
> 
> Maybe it does, but if you're trying to mount your root filesystem to boot
> your laptop, that's not super useful since -o recovery is indeed a read only
> recovery mode.
> btrfs-zero-log just cleans the last log entry and gave me back a fully working
> read/write filesystem each time.

   As far as I recall, -orecovery is read-write. -oro,recovery is
read-only.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
         --- Dullest spy film ever: The Eastbourne Ultimatum ---         

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]

Re: [PATCH 24/27] btrfs-progs: Convert man page for btrfs-zero-log

[Marc MERLIN]

On Sat, Apr 05, 2014 at 03:02:03PM -0700, Marc MERLIN wrote:
> On Sat, Apr 05, 2014 at 04:00:27PM -0600, cwillu wrote:
> > >> +'btrfs-zero-log' will remove the log tree if log tree is corrupt, which will
> > >> +allow you to mount the filesystem again.
> > >> +
> > >> +The common case where this happens has been fixed a long time ago,
> > >> +so it is unlikely that you will see this particular problem.
> > >
> > > A note on this one: this can happen if your SSD rites things in the
> > > wrong order or potentially writes garbage when power is lost, or before
> > > locking up.
> > > I hit this problem about 10 times and it wasn't a btrfs bug, just the
> > > drive doing bad things.
> > 
> > And -o recovery didn't work around it?  My understanding is that -o
> > recovery will skip reading the log.
> 
> Maybe it does, but if you're trying to mount your root filesystem to boot
> your laptop, that's not super useful since -o recovery is indeed a read only
> recovery mode.

Sorry, -o recovery is not read only, but -o recovery,ro which is usually
what is needed for things to work in my experience, is indeed read only.

Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: [PATCH 24/27] btrfs-progs: Convert man page for btrfs-zero-log

[Marc MERLIN]

[-- Attachment #1: Type: text/plain, Size: 1371 bytes --]

On Sat, Apr 05, 2014 at 11:03:46PM +0100, Hugo Mills wrote:
>    As far as I recall, -orecovery is read-write. -oro,recovery is
> read-only.

Yes, we both corrected my Email at the same time :)

Actually it's better/worse than that. From my notes at
http://marc.merlins.org/perso/btrfs/2014-03.html#Btrfs-Tips_-Btrfs-Scrub-and-Btrfs-Filesystem-Repair

mount -o recovery,ro (for when regular mount isn't working). Note, you have to use ro or it will give you a misleading error:
root@polgara:~# mount -o recovery /dev/mapper/crypt /mnt/mnt8
mount: /dev/mapper/crypt already mounted or /mnt/mnt8 busy
root@polgara:~# mount -o recovery,ro /dev/mapper/crypt /mnt/mnt8
root@polgara:~#

In other words, for me -o recovery has never worked unless I added ,ro.
I'm not saying -o recovery cannot work, but when I needed it, it didn't work
because likely it failed to replay the log as you say, and adding ro fixed
that problem.

That said, returning
mount: /dev/mapper/crypt already mounted or /mnt/mnt8 busy                                                                                
is not ideal :)

Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 308 bytes --]

btrfs on 3.14rc5 stuck on "btrfs_tree_read_lock sync"

[Marc MERLIN]

I was debugging my why backup failed to run, and eventually found it was
stuck on sync:
14080       18:18 btrfs_tree_read_lock           sync

This was hung for hours on this lock.

Strangely, it looks like taking my sysrq-w hung the machine pretty hard for
close to 30sec, but this seems to have unhung sync and in the end btrfs send
completed after that.

Sysqrq-w is here:
http://marc.merlins.org/tmp/sysrq-btrfs-sync-hang.txt

Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: btrfs on 3.14rc5 stuck on "btrfs_tree_read_lock sync"

[Josef Bacik]

On 04/07/2014 12:05 PM, Marc MERLIN wrote:
> I was debugging my why backup failed to run, and eventually found it was
> stuck on sync:
> 14080       18:18 btrfs_tree_read_lock           sync
>
> This was hung for hours on this lock.
>
> Strangely, it looks like taking my sysrq-w hung the machine pretty hard for
> close to 30sec, but this seems to have unhung sync and in the end btrfs send
> completed after that.
>
> Sysqrq-w is here:
> https://urldefense.proofpoint.com/v1/url?u=http://marc.merlins.org/tmp/sysrq-btrfs-sync-hang.txt&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=cKCbChRKsMpTX8ybrSkonQ%3D%3D%0A&m=IHXWC1Chbc0jEiUWu1v4Va9NOphtjPbjYp6yVMdUmXM%3D%0A&s=bd787a3422e9ff0972d2d09de7d424f56589aadc9d6db33e19fc44886dce604f

Try Chris's integration branch in a few hours and see if that fixes it. 
  Thanks,

Josef

Re: btrfs on 3.14rc5 stuck on "btrfs_tree_read_lock sync"

[Marc MERLIN]

On Mon, Apr 07, 2014 at 12:10:52PM -0400, Josef Bacik wrote:
> On 04/07/2014 12:05 PM, Marc MERLIN wrote:
> >I was debugging my why backup failed to run, and eventually found it was
> >stuck on sync:
> >14080       18:18 btrfs_tree_read_lock           sync
> >
> >This was hung for hours on this lock.
> >
> >Strangely, it looks like taking my sysrq-w hung the machine pretty hard for
> >close to 30sec, but this seems to have unhung sync and in the end btrfs send
> >completed after that.
> >
> >Sysqrq-w is here:
> >https://urldefense.proofpoint.com/v1/url?u=http://marc.merlins.org/tmp/sysrq-btrfs-sync-hang.txt&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=cKCbChRKsMpTX8ybrSkonQ%3D%3D%0A&m=IHXWC1Chbc0jEiUWu1v4Va9NOphtjPbjYp6yVMdUmXM%3D%0A&s=bd787a3422e9ff0972d2d09de7d424f56589aadc9d6db33e19fc44886dce604f
> 
> Try Chris's integration branch in a few hours and see if that fixes
> it.  Thanks,

Mmmh, so I rebooted that server with 3.14.0 (no rc), and it was
deadlocked a long time during boot (about 10mn) before it unlocked
itself and finished booting.

This is a bit vexing, I don't yet know which of my 3 btrfs filesystems
is causing this, and how to fix it.
After boot, it seems ok enough.

You're recommending that I try btrfs-next on a 3.15 pre kernel, correct?
If so would it be likely to fix my filesystem and let me go back to a
stable 3.14? (I'm a bit warry about running some unstable 3.15 on it :).

Is there a chance balance or some file system cleaning will fix this?

For now, during boot, I get:
INFO: task btrfs-transacti:3633 blocked for more than 120 seconds.
      Not tainted 3.14.0-rc5-amd64-i915-preempt-20140216c #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
btrfs-transacti D ffff88020d762680     0  3633      2 0x00000000
 ffff88020c6c7dc0 0000000000000046 ffff88020c6c7fd8 ffff88020d762150
 00000000000141c0 ffff88020d762150 ffff88020e11be90 ffff8802106271e8
 0000000000000000 ffff880210627000 ffff8800c5c82740 ffff88020c6c7dd0
Call Trace:
 [<ffffffff8160c331>] schedule+0x73/0x75
 [<ffffffff8122a5f9>] wait_current_trans.isra.15+0x98/0xf4
 [<ffffffff810850c9>] ? finish_wait+0x65/0x65
 [<ffffffff8122b812>] start_transaction+0x202/0x4f2
 [<ffffffff8122bb9e>] btrfs_attach_transaction+0x17/0x19
 [<ffffffff812277a8>] transaction_kthread+0xd6/0x1ab
 [<ffffffff812276d2>] ? btrfs_cleanup_transaction+0x43f/0x43f
 [<ffffffff8106bc56>] kthread+0xae/0xb6
 [<ffffffff8106bba8>] ? __kthread_parkme+0x61/0x61
 [<ffffffff816153fc>] ret_from_fork+0x7c/0xb0
 [<ffffffff8106bba8>] ? __kthread_parkme+0x61/0x61
INFO: task btrfs-transacti:3633 blocked for more than 120 seconds.
      Not tainted 3.14.0-rc5-amd64-i915-preempt-20140216c #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
btrfs-transacti D ffff88020d762680     0  3633      2 0x00000000
 ffff88020c6c7dc0 0000000000000046 ffff88020c6c7fd8 ffff88020d762150
 00000000000141c0 ffff88020d762150 ffff88020e11be90 ffff8802106271e8
 0000000000000000 ffff880210627000 ffff8800c5c82740 ffff88020c6c7dd0
Call Trace:
 [<ffffffff8160c331>] schedule+0x73/0x75
 [<ffffffff8122a5f9>] wait_current_trans.isra.15+0x98/0xf4
 [<ffffffff810850c9>] ? finish_wait+0x65/0x65
 [<ffffffff8122b812>] start_transaction+0x202/0x4f2
 [<ffffffff8122bb9e>] btrfs_attach_transaction+0x17/0x19
 [<ffffffff812277a8>] transaction_kthread+0xd6/0x1ab
 [<ffffffff812276d2>] ? btrfs_cleanup_transaction+0x43f/0x43f
 [<ffffffff8106bc56>] kthread+0xae/0xb6
 [<ffffffff8106bba8>] ? __kthread_parkme+0x61/0x61
 [<ffffffff816153fc>] ret_from_fork+0x7c/0xb0
 [<ffffffff8106bba8>] ? __kthread_parkme+0x61/0x61
INFO: task btrfs-transacti:3633 blocked for more than 120 seconds.
      Not tainted 3.14.0-rc5-amd64-i915-preempt-20140216c #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
btrfs-transacti D ffff88020d762680     0  3633      2 0x00000000
 ffff88020c6c7dc0 0000000000000046 ffff88020c6c7fd8 ffff88020d762150
 00000000000141c0 ffff88020d762150 ffff88020e11be90 ffff8802106271e8
 0000000000000000 ffff880210627000 ffff8800c5c82740 ffff88020c6c7dd0
Call Trace:
 [<ffffffff8160c331>] schedule+0x73/0x75
 [<ffffffff8122a5f9>] wait_current_trans.isra.15+0x98/0xf4
 [<ffffffff810850c9>] ? finish_wait+0x65/0x65
 [<ffffffff8122b812>] start_transaction+0x202/0x4f2
 [<ffffffff8122bb9e>] btrfs_attach_transaction+0x17/0x19
 [<ffffffff812277a8>] transaction_kthread+0xd6/0x1ab
 [<ffffffff812276d2>] ? btrfs_cleanup_transaction+0x43f/0x43f
 [<ffffffff8106bc56>] kthread+0xae/0xb6
 [<ffffffff8106bba8>] ? __kthread_parkme+0x61/0x61
 [<ffffffff816153fc>] ret_from_fork+0x7c/0xb0
 [<ffffffff8106bba8>] ? __kthread_parkme+0x61/0x61

Eventually the boot finishes, but it hangs way too long.

Thanks,
Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/                         | PGP 1024R/763BE901

Re: btrfs on 3.14rc5 stuck on "btrfs_tree_read_lock sync"

[Chris Mason]

On 04/07/2014 02:51 PM, Marc MERLIN wrote:
> On Mon, Apr 07, 2014 at 12:10:52PM -0400, Josef Bacik wrote:
>> On 04/07/2014 12:05 PM, Marc MERLIN wrote:
>>> I was debugging my why backup failed to run, and eventually found it was
>>> stuck on sync:
>>> 14080       18:18 btrfs_tree_read_lock           sync
>>>
>>> This was hung for hours on this lock.
>>>
>>> Strangely, it looks like taking my sysrq-w hung the machine pretty hard for
>>> close to 30sec, but this seems to have unhung sync and in the end btrfs send
>>> completed after that.
>>>
>>> Sysqrq-w is here:
>>> https://urldefense.proofpoint.com/v1/url?u=http://marc.merlins.org/tmp/sysrq-btrfs-sync-hang.txt&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=cKCbChRKsMpTX8ybrSkonQ%3D%3D%0A&m=IHXWC1Chbc0jEiUWu1v4Va9NOphtjPbjYp6yVMdUmXM%3D%0A&s=bd787a3422e9ff0972d2d09de7d424f56589aadc9d6db33e19fc44886dce604f
>>
>> Try Chris's integration branch in a few hours and see if that fixes
>> it.  Thanks,
>
> Mmmh, so I rebooted that server with 3.14.0 (no rc), and it was
> deadlocked a long time during boot (about 10mn) before it unlocked
> itself and finished booting.
>
> This is a bit vexing, I don't yet know which of my 3 btrfs filesystems
> is causing this, and how to fix it.
> After boot, it seems ok enough.
>
> You're recommending that I try btrfs-next on a 3.15 pre kernel, correct?
> If so would it be likely to fix my filesystem and let me go back to a
> stable 3.14? (I'm a bit warry about running some unstable 3.15 on it :).
>

Right now the fixes for this are in the integration branch on my git 
tree.  I think we've shaken  out all the problems, but if you want to 
wait until tomorrow I'll have it in my next branch (for linux-next).

-chris

Re: btrfs on 3.14rc5 stuck on "btrfs_tree_read_lock sync"

[Marc MERLIN]

On Mon, Apr 07, 2014 at 03:32:13PM -0400, Chris Mason wrote:
> >You're recommending that I try btrfs-next on a 3.15 pre kernel, correct?
> >If so would it be likely to fix my filesystem and let me go back to a
> >stable 3.14? (I'm a bit warry about running some unstable 3.15 on it :).
> 
> Right now the fixes for this are in the integration branch on my git 
> tree.  I think we've shaken  out all the problems, but if you want to 
> wait until tomorrow I'll have it in my next branch (for linux-next).

I can wait, even a few more days if needed.
But just to be clear: will this new kernel be something that will be
required for me to run from there on to avoid all those deadlocks and very
poor performance I'm seeing, or the new kernel will fix things up, and then
if other stuff isn't quite stable, I can downgrade back to 3.14 stable?

By the way, I think I know which filesystem is causing this, and one unusual
thing is that it uses a lot of hardlinks.

In case that helps, there are only 40 snapshots on it, but many inodes, of
which many are hardlinked together:

gargamel:/mnt/btrfs_pool2# btrfs filesystem df `pwd`
Data, single: total=3.28TiB, used=2.30TiB
System, DUP: total=8.00MiB, used=384.00KiB
System, single: total=4.00MiB, used=0.00
Metadata, DUP: total=74.50GiB, used=70.11GiB
Metadata, single: total=8.00MiB, used=0.00
gargamel:/mnt/btrfs_pool2# btrfs filesystem show `pwd`
Label: btrfs_pool2  uuid: cb9df6d3-a528-4afc-9a45-4fed5ec358d6
	Total devices 1 FS bytes used 2.37TiB
	devid    1 size 7.28TiB used 3.43TiB path /dev/mapper/dshelf2

Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: [PATCH 14/27] btrfs-progs: Convert man page for btrfs-replace.

[Qu Wenruo]

于 2014年04月05日 04:29, Marc MERLIN 写道:
> On Wed, Apr 02, 2014 at 04:29:25PM +0800, Qu Wenruo wrote:
>> +If the source device is not available anymore, or if the -r option is set,
>> +the data is built only using the RAID redundancy mechanisms.
>> +After completion of the operation, the source device is removed from the
>> +filesystem.
> Woudl it make sense to add a paragraph explaining that for raid5/6,
> someone should either:
> 1) use balance to rebuild on a new drive if one of the drives is missing
> 2) use btrfs device add of a new drive, then btrfs device delete of the
> drive to replace, and effectively btrfs will do the same thing that
> replace would
>
> ?

Nice try, I think it's good to add a note paragraph for the use cases.

I'm also wondering should the same things added to btrfs-balance(8) and 
btrfs-device(8) man pages,
if added, three same paragraph will be added and the duplication 
paragraph may increase the difficulty to maintain
the documentation.

So I prefer to add the explaination about RAID5/6 to btrfs-device(8) and 
adds note on
btrfs-replace(8)/btrfs-balance(8) to refer to btrfs-device(8), since all 
of these things are about btrfs device management.

Would you prefer the modification?

Thanks,
Qu.
>
> Thanks,
> Marc

Re: [PATCH 24/27] btrfs-progs: Convert man page for btrfs-zero-log

[Qu Wenruo]

于 2014年04月05日 02:46, Marc MERLIN 写道:
> On Wed, Apr 02, 2014 at 04:29:35PM +0800, Qu Wenruo wrote:
>> Convert man page for btrfs-zero-log
>>
>> Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
>> ---
>>   Documentation/Makefile           |  2 +-
>>   Documentation/btrfs-zero-log.txt | 39 +++++++++++++++++++++++++++++++++++++++
>>   2 files changed, 40 insertions(+), 1 deletion(-)
>>   create mode 100644 Documentation/btrfs-zero-log.txt
>>
>> diff --git a/Documentation/Makefile b/Documentation/Makefile
>> index e002d53..de06629 100644
>> --- a/Documentation/Makefile
>> +++ b/Documentation/Makefile
>> @@ -11,7 +11,7 @@ MAN8_TXT += btrfs-image.txt
>>   MAN8_TXT += btrfs-map-logical.txt
>>   MAN8_TXT += btrfs-show-super.txt
>>   MAN8_TXT += btrfstune.txt
>> -#MAN8_TXT += btrfs-zero-log.txt
>> +MAN8_TXT += btrfs-zero-log.txt
>>   #MAN8_TXT += fsck.btrfs.txt
>>   #MAN8_TXT += mkfs.btrfs.txt
>>   
>> diff --git a/Documentation/btrfs-zero-log.txt b/Documentation/btrfs-zero-log.txt
>> new file mode 100644
>> index 0000000..e3041fa
>> --- /dev/null
>> +++ b/Documentation/btrfs-zero-log.txt
>> @@ -0,0 +1,39 @@
>> +btrfs-zero-log(8)
>> +=================
>> +
>> +NAME
>> +----
>> +btrfs-zero-log - clear out log tree
>> +
>> +SYNOPSIS
>> +--------
>> +'btrfs-zero-log' <dev>
>> +
>> +DESCRIPTION
>> +-----------
>> +'btrfs-zero-log' will remove the log tree if log tree is corrupt, which will
>> +allow you to mount the filesystem again.
>> +
>> +The common case where this happens has been fixed a long time ago,
>> +so it is unlikely that you will see this particular problem.
> A note on this one: this can happen if your SSD rites things in the
> wrong order or potentially writes garbage when power is lost, or before
> locking up.
> I hit this problem about 10 times and it wasn't a btrfs bug, just the
> drive doing bad things.
>
> I had debian add this to the initramfs initrd by default so that someone
> can recover their root filesystem with this command if it won't mount.
>
> What got fixed is the kernel used to oops and crash, and now it gives a
> nice "can't mount" error message.
>
> Marc
Thanks for the note.

After reading following up mails on the thread, I agree that the note is 
needed.

But I would like to add more method to judge whether to call 
btrfs-zero-log or other tools like btrfsck.

It would be quite nice if any one can provide a log-tree-broken small 
btrfs image for me to test?
(Personally, I would prefer using dmesg warning/backtrace and btrfsck 
result to make sure to call btrfs-zero-log)

P.S: If btrfs can be automatically mounted as RO when log tree is 
broken, it would be better.
But it also seems log tree related things will be improved sooner or 
later, it may not be so urgent.

Thanks,
Qu.

Re: btrfs on 3.14rc5 stuck on "btrfs_tree_read_lock sync"

[Marc MERLIN]

On Mon, Apr 07, 2014 at 01:00:02PM -0700, Marc MERLIN wrote:
> On Mon, Apr 07, 2014 at 03:32:13PM -0400, Chris Mason wrote:
> > >You're recommending that I try btrfs-next on a 3.15 pre kernel, correct?
> > >If so would it be likely to fix my filesystem and let me go back to a
> > >stable 3.14? (I'm a bit warry about running some unstable 3.15 on it :).
> > 
> > Right now the fixes for this are in the integration branch on my git 
> > tree.  I think we've shaken  out all the problems, but if you want to 
> > wait until tomorrow I'll have it in my next branch (for linux-next).
> 
> I can wait, even a few more days if needed.
> But just to be clear: will this new kernel be something that will be
> required for me to run from there on to avoid all those deadlocks and very
> poor performance I'm seeing, or the new kernel will fix things up, and then
> if other stuff isn't quite stable, I can downgrade back to 3.14 stable?
> 
> By the way, I think I know which filesystem is causing this, and one unusual
> thing is that it uses a lot of hardlinks.
> 
> In case that helps, there are only 40 snapshots on it, but many inodes, of
> which many are hardlinked together:
> 
> gargamel:/mnt/btrfs_pool2# btrfs filesystem df `pwd`
> Data, single: total=3.28TiB, used=2.30TiB
> System, DUP: total=8.00MiB, used=384.00KiB
> System, single: total=4.00MiB, used=0.00
> Metadata, DUP: total=74.50GiB, used=70.11GiB
> Metadata, single: total=8.00MiB, used=0.00
> gargamel:/mnt/btrfs_pool2# btrfs filesystem show `pwd`
> Label: btrfs_pool2  uuid: cb9df6d3-a528-4afc-9a45-4fed5ec358d6
> 	Total devices 1 FS bytes used 2.37TiB
> 	devid    1 size 7.28TiB used 3.43TiB path /dev/mapper/dshelf2

Back on that front, while debugging the other problem I sent you, I've been
having more issues with this device too.

At boot time, I've been getting multiple of these after boot:
gargamel login: [ 1328.241302] INFO: task btrfs-cleaner:3571 blocked for more than 120 seconds.
[ 1328.264046]       Not tainted 3.14.0-amd64-i915-preempt-20140216 #2
[ 1328.284413] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1328.309394] btrfs-cleaner   D ffff88020d5ea800     0  3571      2 0x00000000
[ 1328.331996]  ffff8800c8985d00 0000000000000046 ffff8800c8985fd8 ffff88020d5ea2d0
[ 1328.355774]  00000000000141c0 ffff88020d5ea2d0 ffff8801d9a7ee50 ffff880213bfc9e8
[ 1328.379408]  0000000000000000 ffff880213bfc800 ffff88020c5b7ce0 ffff8800c8985d10
[ 1328.403654] Call Trace:
[ 1328.412617]  [<ffffffff8160d2a1>] schedule+0x73/0x75
[ 1328.429189]  [<ffffffff8122aa77>] wait_current_trans.isra.15+0x98/0xf4
[ 1328.450402]  [<ffffffff81085116>] ? finish_wait+0x65/0x65
[ 1328.467957]  [<ffffffff8122bf1c>] start_transaction+0x48e/0x4f2
[ 1328.487315]  [<ffffffff8122c2ff>] ? __btrfs_end_transaction+0x2a1/0x2c6
[ 1328.508614]  [<ffffffff8122bf9b>] btrfs_start_transaction+0x1b/0x1d
[ 1328.528842]  [<ffffffff8121cc7d>] btrfs_drop_snapshot+0x443/0x610
[ 1328.548481]  [<ffffffff8122c73d>] btrfs_clean_one_deleted_snapshot+0x103/0x10f
[ 1328.571518]  [<ffffffff81224f09>] cleaner_kthread+0x103/0x136
[ 1328.590436]  [<ffffffff81224e06>] ? btrfs_alloc_root+0x26/0x26
[ 1328.609348]  [<ffffffff8106bc62>] kthread+0xae/0xb6
[ 1328.625275]  [<ffffffff8106bbb4>] ? __kthread_parkme+0x61/0x61
[ 1328.644406]  [<ffffffff8161637c>] ret_from_fork+0x7c/0xb0
[ 1328.662075]  [<ffffffff8106bbb4>] ? __kthread_parkme+0x61/0x61

But more annoyingly, accessing the mountpoint was hanging, so I've now mounted
it with recovery,ro, and backing up all the data to another device so that I
can destroy/recreate this device that clearly has severe performance issues.

Do you want btrfsck output and an image of that one too?
(this one is not raid0, it's on top of an dm encrypted md raid5 array)

Thanks,
Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: How to debug very very slow file delete? (btrfs on md-raid5 with many files, 70GB metadata)

[Marc MERLIN]

So, since then I found out in the thread
Subject: Re: btrfs on 3.14rc5 stuck on "btrfs_tree_read_lock           sync"
that my btrfs filesystem has a clear problem, which Josef and Chris are
still looking into.

Basically, I've had btrfs near deadlocks on this filesystem:
INFO: task btrfs-transacti:3633 blocked for more than 120 seconds.
Not tainted 3.14.0-rc5-amd64-i915-preempt-20140216c #1
INFO: task btrfs-cleaner:3571 blocked for more than 120 seconds.
Not tainted 3.14.0-amd64-i915-preempt-20140216 #2

They are thinking it's due to a balancing issue that 3.15 might fix.

One interesting piece of information I found out since yesterday is that now
that I mounted the filesystem with -o ro,recovery , its speed as improved
very noticeably.
I'm currently copying my data off it about 10x faster.



What follows is for people interested in optimization.

I have swraid5 with dmcrypt on top, and then btrfs.
http://superuser.com/questions/305716/bad-performance-with-linux-software-raid5-and-luks-encryption
says:
"LUKS has a botleneck, that is it just spawns one thread per block device.

Are you placing the encryption on top of the RAID 5? Then from the point of
view of your OS you just have one device, then it is using just one thread
for all those disks, meaning disks are working in a serial way rather than
parallel."
but it was disputed in a reply.
Does someone know if this is still valid/correct in 3.14?


Since I'm going to recreate the filesystem considering the troubles I've had
with it, I might as well do it better this time :)
(but doing the copy back will take days, so I'd rather get it right the first time)

How would you recommend I create the array when I rebuild it?

This filesystem contains may backup with many files, most small, and ideally
identical stuff is hardlinked together (many files, many hardlinks)
gargamel:~# btrfs fi df /mnt/btrfs_pool2
Data, single: total=3.28TiB, used=2.29TiB
System, DUP: total=8.00MiB, used=384.00KiB
System, single: total=4.00MiB, used=0.00
Metadata, DUP: total=74.50GiB, used=70.11GiB  <<< muchos metadata
Metadata, single: total=8.00MiB, used=0.00

This is my current array:
gargamel:~# mdadm --detail /dev/md8
/dev/md8:
        Version : 1.2
  Creation Time : Thu Mar 25 20:15:00 2010
     Raid Level : raid5
     Array Size : 7814045696 (7452.05 GiB 8001.58 GB)
  Used Dev Size : 1953511424 (1863.01 GiB 2000.40 GB)
    Persistence : Superblock is persistent
  Intent Bitmap : Internal
         Layout : left-symmetric
     Chunk Size : 512K

#1 move the intent bitmap to another device. I have /boot on swraid1 with
   ext4, so I'll likely use this.
#2 change chunk size to something smaller? 128K better?
#3 anything else?

Then, I used this for dmcrypt:
cryptsetup luksFormat --align-payload=8192 -s 256 -c aes-xts-plain64  

The align-payload was good for my SSD, but probably not for a hard drive.
http://wiki.drewhess.com/wiki/Creating_an_encrypted_filesystem_on_a_partition
says
"To calculate this value, multiply your RAID chunk size in bytes by the
number of data disks in the array (N/2 for RAID 1, N-1 for RAID 5 and N-2
for RAID 6), and divide by 512 bytes per sector."

So 512K * 4 / 512 = 4K
In other words, I can do align-payload=4096 for a small reduction of write
amplification, or =1024 if I change my raid chunk size to 128K

Correct?

But from what I can see, those will only be small improvements compared to the 
btrfs performance I've seen which hopefully 3.15 will address in some way.

Other bits I found that can maybe help others:
http://superuser.com/questions/305716/bad-performance-with-linux-software-raid5-and-luks-encryption

This seems to help work around the write amplification a bit:
for i in /sys/block/md*/md/stripe_cache_size; do echo 16384 > $i; done

This looks like an easy thing, done.

If you have other suggestions/comments, please share :)

Thanks,
Marc


On Tue, Mar 25, 2014 at 09:41:42AM -0700, Marc MERLIN wrote:
> On Tue, Mar 25, 2014 at 12:13:50PM +0000, Martin wrote:
> > On 25/03/14 01:49, Marc MERLIN wrote:
> > > I had a tree with some amount of thousand files (less than 1 million)
> > > on top of md raid5.
> > > 
> > > It took 18H to rm it in 3 tries:
> 
> I ran another test after typing the original Email:
> gargamel:/mnt/dshelf2/backup/polgara# time du -sh 20140312-feisty/; time find 20140 312-feisty/ | wc -l
> 17G     20140312-feisty/
> real    245m19.491s
> user    0m2.108s
> sys     1m0.508s
> 
> 728507 <- number of files
> real    11m41.853s <- 11mn to restat them when they should all be in cache ideally
> user    0m1.040s
> sys     0m4.360s
> 
> 4 hours to stat 700K files. That's bad...
> Even 11mn to restat them just to count them looks bad too.
> 
> > > I checked that btrfs scrub is not running.
> > > What else can I check from here?
> > 
> > "noatime" set?
> 
> I have relatime
> gargamel:/mnt/dshelf2/backup/polgara# df .
> Filesystem           1K-blocks       Used  Available Use% Mounted on
> /dev/mapper/dshelf2 7814041600 3026472436 4760588292  39% /mnt/dshelf2/backup
> 
> gargamel:/mnt/dshelf2/backup/polgara# grep /mnt/dshelf2/backup /proc/mounts
> /dev/mapper/dshelf2 /mnt/dshelf2/backup btrfs rw,relatime,compress=lzo,space_cache 0 0
>  
> > What's your cpu hardware wait time?
>  
> Sorry, not sure how to get that.
>  
> > And is not *the 512kByte raid chunk* going to give you horrendous write
> > amplification?! For example, rm updates a few bytes in one 4kByte
> > metadata block and the system has to then do a read-modify-write on
> > 512kBytes...
> 
> That's probably not great, but
> 1) rm -rf should bunch a lot of writes together before they start
> hitting the block layer for writes, so I'm not sure that is too much a
> problem with the caching layer in between
> 
> 2) this does not explain 4H to just run du with relatime, which
> shouldn't generate any writing, correct?
> iostat seems to confirm:
> 
> gargamel:~# iostat /dev/md8 1 20
> Linux 3.14.0-rc5-amd64-i915-preempt-20140216c (gargamel.svh.merlins.org)        03/25/2014      _x86_64_        (4 CPU)
> avg-cpu:  %user   %nice %system %iowait  %steal   %idle  
>           75.19    0.00   10.13    8.61    0.00    6.08
> Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
> md8              98.00       392.00         0.00        392          0
> md8              96.00       384.00         0.00        384          0
> md8              83.00       332.00         0.00        332          0
> md8             153.00       612.00         0.00        612          0
> md8              82.00       328.00         0.00        328          0
> md8              55.00       220.00         0.00        220          0
> md8              69.00       276.00         0.00        276          0
> 
> > Also, the 64MByte chunk bit-intent map will add a lot of head seeks to
> > anything you do on that raid. (The map would be better on a separate SSD
> > or other separate drive.)
> 
> That's true for writing, but not reading, right?
>  
> > So... That sort of setup is fine for archived data that is effectively
> > read-only. You'll see poor performance for small writes/changes.
> 
> So I agree with you that the write case can be improved, especially since I also have a layer
> of dmcrypt in the middle
> gargamel:/mnt/dshelf2/backup/polgara# cryptsetup luksDump /dev/md8
> LUKS header information for /dev/md8
> Cipher name:    aes
> Cipher mode:    xts-plain64
> Hash spec:      sha1
> Payload offset: 8192
> 
> (I used cryptsetup luksFormat --align-payload=8192 -s 256 -c aes-xts-plain64)
> 
> I'm still not convinced that a lot of file IO don't get all collated in memory 
> before hitting disk in bigger blocks, but maybe not.
> 
> If I were to recreate this array entirely, what would you use for the raid creation
> and cryptsetup?
> 
> More generally, before I go through all that trouble (it will likely
> take 1 week of data copying back and forth), I'd like to debug why my reads are
> so slow first.
> 
> Thanks,
> Marc
> 
> On Tue, Mar 25, 2014 at 02:57:57PM +0100, Xavier Nicollet wrote:
> > Le 25 mars 2014 à 12:13, Martin a écrit:
> > > On 25/03/14 01:49, Marc MERLIN wrote:
> > > > It took 18H to rm it in 3 tries:
> > 
> > > And is not *the 512kByte raid chunk* going to give you horrendous write
> > > amplification?! For example, rm updates a few bytes in one 4kByte
> > > metadata block and the system has to then do a read-modify-write on
> > > 512kBytes...
> > 
> > My question would be naive, but would it be possible to have a syscall or something to do 
> > a fast "rm -rf" or du ?
> 
> Well, that wouldn't hurt either, even if it wouldn't address my underlying problem.
> 
> Marc
> -- 
> "A mouse is a device used to point at the xterm you want to type in" - A.S.R.
> Microsoft is to operating systems ....
>                                       .... what McDonalds is to gourmet cooking
> Home page: http://marc.merlins.org/                         | PGP 1024R/763BE901
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: [PATCH 24/27] btrfs-progs: Convert man page for btrfs-zero-log

[Marc MERLIN]

On Tue, Apr 08, 2014 at 09:42:03AM +0800, Qu Wenruo wrote:
> >I had debian add this to the initramfs initrd by default so that someone
> >can recover their root filesystem with this command if it won't mount.
> >
> >What got fixed is the kernel used to oops and crash, and now it gives a
> >nice "can't mount" error message.
> >
> >Marc
> Thanks for the note.
> 
> After reading following up mails on the thread, I agree that the note is 
> needed.
> 
> But I would like to add more method to judge whether to call 
> btrfs-zero-log or other tools like btrfsck.
 
Sounds like a good idea. It may also make sense in places to have the man
page point to the btrfs wiki for more detailled info since it's easier to
update the wiki than to update the man page on thousands or millions of
systems :)

I wrote 
http://marc.merlins.org/perso/btrfs/2014-03.html#Btrfs-Tips_-Btrfs-Scrub-and-Btrfs-Filesystem-Repair
which is linked off
https://btrfs.wiki.kernel.org/index.php/Btrfsck
which I just updated with more details.

It would be great to put a pointer to the official wiki in the man page, and
also in the btrfsck man page which currently doesn't warn people enough that
they have many other options to try before btrfsck.

Now that I updated the official wiki page, would you agree that it would be
good to link to it in the man page?

> It would be quite nice if any one can provide a log-tree-broken small 
> btrfs image for me to test?
> (Personally, I would prefer using dmesg warning/backtrace and btrfsck 
> result to make sure to call btrfs-zero-log)

Unfortunately, my broken SSD that was generating that corruption has been
returned to the vendor, and my new SSD doesn't crash anymore, so I don't
have such broken filesystem anymore.
 
> P.S: If btrfs can be automatically mounted as RO when log tree is 
> broken, it would be better.
> But it also seems log tree related things will be improved sooner or 
> later, it may not be so urgent.

I agree this is a great idea, especially if it's the root filesystem without
which you can't fix the machine that's broken.

Thank you,
Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: How to debug very very slow file delete? (btrfs on md-raid5)

[Chris Samuel]

[-- Attachment #1: Type: text/plain, Size: 451 bytes --]

On Tue, 25 Mar 2014 09:41:42 AM Marc MERLIN wrote:

> 4 hours to stat 700K files. That's bad...
> Even 11mn to restat them just to count them looks bad too.

One way to get an idea of where that's happening is to run the command with 
"strace -c" to build up a table of times spent in each system call, usecs per 
call (average) and number of calls.

Best of luck!
Chris
-- 
 Chris Samuel  :  http://www.csamuel.org/  :  Melbourne, VIC


[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 482 bytes --]

Re: How to debug very very slow file delete? (btrfs on md-raid5)

[Marc MERLIN]

[-- Attachment #1: Type: text/plain, Size: 1204 bytes --]

On Sat, Apr 12, 2014 at 12:15:14AM +1000, Chris Samuel wrote:
> On Tue, 25 Mar 2014 09:41:42 AM Marc MERLIN wrote:
> 
> > 4 hours to stat 700K files. That's bad...
> > Even 11mn to restat them just to count them looks bad too.
> 
> One way to get an idea of where that's happening is to run the command with 
> "strace -c" to build up a table of times spent in each system call, usecs per 
> call (average) and number of calls.

If the devs can use this, I can provide this.

For now, the FS got even worse than that, it just hangs btrfs cleaner
processes without me doing anything, so I think they got enough data on what
is going wrong with it, although I'm not sure if it's fixed in 3.15 or not.

I'm waiting just a bit longer before destroying and rebuilding this
filesystem, hence my other questions on how to it better for next time on
the block level side.

Is anyone else using btrfs on top of dmcrypt and software raid 5?

Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 308 bytes --]

Re: How to debug very very slow file delete? (btrfs on md-raid5)

[Duncan]

Marc MERLIN posted on Fri, 11 Apr 2014 10:23:46 -0700 as excerpted:

> Is anyone else using btrfs on top of dmcrypt and software raid 5?

There may be others, but I haven't seen them active on-list.  For active 
list users, I think you're leading the pack in that area. =:^/

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: How to debug very very slow file delete? (btrfs on md-raid5)

[Roman Mamedov]

[-- Attachment #1: Type: text/plain, Size: 785 bytes --]

On Fri, 11 Apr 2014 10:23:46 -0700
Marc MERLIN <marc@merlins.org> wrote:

> Is anyone else using btrfs on top of dmcrypt and software raid 5?

I use Btrfs accessed via NBD over a LAN, physically stored on mdadm RAID5, a
setup which is similar to yours in that the block device used for Btrfs has a
somewhat limited throughput. But there were no problems with this whatsoever
for the past couple of years (other than caused by unrelated h/w issues).

> gargamel:/mnt/dshelf2/backup/polgara# time rm -rf current.todel/
> real    1087m26.491s
> user    0m2.448s
> sys     4m42.012s

You could check if this slowness is caused by excessive fsync'ing, try running
"time eatmydata rm -rf ....." next time. ( https://launchpad.net/libeatmydata )

-- 
With respect,
Roman

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: very slow btrfs filesystem: any data needed before I wipe it?

[Marc MERLIN]

I've posted about this before, and I'm about to delete and recreate the
filesystem.
I took an image, but the image is 18GB, so not very convenient to give
around, although I can if needed.

As a reminder, the problems are that everything is super slow, including
btrfs's own background processes that hang and create tracebacks.
(initially I complained about rm, but it's just everything that is slow).

If I mount with ro,recovery, then it's actually normal speed to copy data
off it, quite weird. It seems like btrfs' background processes are grinding
the FS down to a halt, and I didn't turn on autodefrag.

I haven't gotten confirmation that this a known problem that has been
root-caused, so I'm a bit hesitant about wiping the filesystem before 
getting potentially useful debug data off it, but honestly I'm not sure what
to take.

(I have a btrfsck running, looks like it may take over an hour, so I'll post
that after it's finished)

Other suggestions welcome.

Thanks,
Marc

On Mon, Mar 24, 2014 at 06:49:56PM -0700, Marc MERLIN wrote:
> I had a tree with some amount of thousand files (less than 1 million)
> on top of md raid5.
> 
> It took 18H to rm it in 3 tries:
> gargamel:/mnt/dshelf2/backup/polgara# time rm -rf current.todel/
> real    1087m26.491s
> user    0m2.448s
> sys     4m42.012s
> 
> gargamel:/mnt/dshelf2/backup/polgara# btrfs fi show /mnt/btrfs_pool2
> Label: btrfs_pool2  uuid: cb9df6d3-a528-4afc-9a45-4fed5ec358d6
>         Total devices 1 FS bytes used 2.76TiB
>         devid    1 size 7.28TiB used 3.43TiB path /dev/mapper/dshelf2
> 
> gargamel:/mnt/dshelf2/backup/polgara# btrfs fi df /mnt/btrfs_pool2
> Data, single: total=3.28TiB, used=2.70TiB
> System, DUP: total=8.00MiB, used=384.00KiB
> System, single: total=4.00MiB, used=0.00
> Metadata, DUP: total=73.50GiB, used=62.46GiB
> Metadata, single: total=8.00MiB, used=0.00
> 
> This is running from
> md8 : active raid5 sdf1[6] sdb1[5] sda1[3] sde1[2] sdd1[1]
>       7814045696 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/5] [UUUUU]
>       bitmap: 0/15 pages [0KB], 65536KB chunk
> 
> The filesystem is pretty new, it shouldn't be fragmented much.
> 
> dmesg shows a bit of this:
> INFO: task rm:31885 blocked for more than 120 seconds.
>       Not tainted 3.14.0-rc5-amd64-i915-preempt-20140216c #1
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message
> rm              D ffff880117e5a940     0 31885  31280 0x20020080
>  ffff8800169f3d10 0000000000000086 ffff8800169f3fd8 ffff880117e5a410
>  00000000000141c0 ffff880117e5a410 ffff88011d4f9e90 ffff88020c0109e8
>  0000000000000000 ffff88020c010800 ffff8801b6df55a0 ffff8800169f3d20
> Call Trace:
>  [<ffffffff8160c331>] schedule+0x73/0x75
>  [<ffffffff8122a5f9>] wait_current_trans.isra.15+0x98/0xf4
>  [<ffffffff810850c9>] ? finish_wait+0x65/0x65
>  [<ffffffff8122ba9e>] start_transaction+0x48e/0x4f2
>  [<ffffffff8122bb1d>] btrfs_start_transaction+0x1b/0x1d
>  [<ffffffff8122cab4>] __unlink_start_trans+0x24/0xaf
>  [<ffffffff812327ab>] btrfs_unlink+0x28/0xa0
>  [<ffffffff8116176d>] vfs_unlink+0x90/0xdb
>  [<ffffffff811618c0>] do_unlinkat+0x108/0x1da
>  [<ffffffff810734c8>] ? mmdrop+0x11/0x20
>  [<ffffffff8115d2db>] ? path_put+0x1e/0x21
>  [<ffffffff811625c7>] SyS_unlinkat+0x22/0x2e
>  [<ffffffff816171ac>] sysenter_dispatch+0x7/0x21
> 
> That said, it didn't happen much:
> [38077.054841] INFO: task btrfs-cleaner:3536 blocked for more than 120 seconds.
> [38077.521463] INFO: task rm:31885 blocked for more than 120 seconds.
> [38077.926399] INFO: task cp:26645 blocked for more than 120 seconds.
> [38078.346885] INFO: task btrfs:7430 blocked for more than 120 seconds.
> [38198.921768] INFO: task btrfs-cleaner:3536 blocked for more than 120 seconds.
> [38199.357367] INFO: task rm:31885 blocked for more than 120 seconds.
> [38199.753897] INFO: task cp:26645 blocked for more than 120 seconds.
> [38200.172729] INFO: task btrfs:7430 blocked for more than 120 seconds.
> [56696.271850] INFO: task btrfs-transacti:3537 blocked for more than 120 seconds.
> [56937.063142] INFO: task btrfs-cleaner:3536 blocked for more than 120 seconds.
> 
> At boot time, I've been getting multiple of these after boot:
> gargamel login: [ 1328.241302] INFO: task btrfs-cleaner:3571 blocked for more than 120 seconds.
> [ 1328.264046]       Not tainted 3.14.0-amd64-i915-preempt-20140216 #2
> [ 1328.284413] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> [ 1328.309394] btrfs-cleaner   D ffff88020d5ea800     0  3571      2 0x00000000
> [ 1328.331996]  ffff8800c8985d00 0000000000000046 ffff8800c8985fd8 ffff88020d5ea2d0
> [ 1328.355774]  00000000000141c0 ffff88020d5ea2d0 ffff8801d9a7ee50 ffff880213bfc9e8
> [ 1328.379408]  0000000000000000 ffff880213bfc800 ffff88020c5b7ce0 ffff8800c8985d10
> [ 1328.403654] Call Trace:
> [ 1328.412617]  [<ffffffff8160d2a1>] schedule+0x73/0x75
> [ 1328.429189]  [<ffffffff8122aa77>] wait_current_trans.isra.15+0x98/0xf4
> [ 1328.450402]  [<ffffffff81085116>] ? finish_wait+0x65/0x65
> [ 1328.467957]  [<ffffffff8122bf1c>] start_transaction+0x48e/0x4f2
> [ 1328.487315]  [<ffffffff8122c2ff>] ? __btrfs_end_transaction+0x2a1/0x2c6
> [ 1328.508614]  [<ffffffff8122bf9b>] btrfs_start_transaction+0x1b/0x1d
> [ 1328.528842]  [<ffffffff8121cc7d>] btrfs_drop_snapshot+0x443/0x610
> [ 1328.548481]  [<ffffffff8122c73d>] btrfs_clean_one_deleted_snapshot+0x103/0x10f
> [ 1328.571518]  [<ffffffff81224f09>] cleaner_kthread+0x103/0x136
> [ 1328.590436]  [<ffffffff81224e06>] ? btrfs_alloc_root+0x26/0x26
> [ 1328.609348]  [<ffffffff8106bc62>] kthread+0xae/0xb6
> [ 1328.625275]  [<ffffffff8106bbb4>] ? __kthread_parkme+0x61/0x61
> [ 1328.644406]  [<ffffffff8161637c>] ret_from_fork+0x7c/0xb0
> [ 1328.662075]  [<ffffffff8106bbb4>] ? __kthread_parkme+0x61/0x61


-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: very slow btrfs filesystem: any data needed before I wipe it?

[Duncan]

Marc MERLIN posted on Sat, 12 Apr 2014 13:25:42 -0700 as excerpted:

> If I mount with ro,recovery, then it's actually normal speed to copy
> data off it, quite weird. It seems like btrfs' background processes are
> grinding the FS down to a halt, and I didn't turn on autodefrag.

What happens if you simply mount it ro, without the recovery option?  Is 
it still normal-speed or is that slow as a rw mount?

Speed hasn't been an issue here, but FWIW I normally keep my rootfs ro 
mounted as I decided that's lower risk in the event of a system crash or 
power outage and I don't actually /need/ it rw most of the time (I have a 
dedicated /var/log, /home on an entirely separate btrfs, and the bits of 
/var that need to be writable in normal operation symlinked into
/home/var/), only mounting the rootfs rw to update or do other system 
maintenance.

If a ro filesystem is normal speed but rw slow, the problem must be in 
the write path, which is what I suspect.  If a ro filesystem is slow and 
only a recover,ro filesystem is fast, then there's something strange 
going on with btrfs internals that recovery obviously bypasses.  Knowing 
one way or the other should definitely help to pin things down.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: very slow btrfs filesystem: any data needed before I wipe it?

[Marc MERLIN]

On Sat, Apr 12, 2014 at 01:25:42PM -0700, Marc MERLIN wrote:
> (I have a btrfsck running, looks like it may take over an hour, so I'll post
> that after it's finished)

I ran out of patience after 18 hours of waiting since it seemed to have not
progressed after 12 hours (it was still doing stuff, but no more output on
the screen).

Let's look at the output:
1794407 lines of lines similar to:
Extent back ref already exists for 2148837945344 parent 0 root 257 

61 lines of 
leaf parent key incorrect 504993210368

67 lines of
bad block 504993210368
=> just curious, what is this one supposed to mean? Is the data checksum
bad, or something else?

Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: what does your btrfsck look like?

[Marc MERLIN]

On Sun, Apr 13, 2014 at 07:57:00AM -0700, Marc MERLIN wrote:
> On Sat, Apr 12, 2014 at 01:25:42PM -0700, Marc MERLIN wrote:
> > (I have a btrfsck running, looks like it may take over an hour, so I'll post
> > that after it's finished)
> 
> I ran out of patience after 18 hours of waiting since it seemed to have not
> progressed after 12 hours (it was still doing stuff, but no more output on
> the screen).
> 
> Let's look at the output:
> 1794407 lines of lines similar to:
> Extent back ref already exists for 2148837945344 parent 0 root 257 
> 
> 61 lines of 
> leaf parent key incorrect 504993210368
> 
> 67 lines of
> bad block 504993210368
> => just curious, what is this one supposed to mean? Is the data checksum
> bad, or something else?

We had a chat about btrfsck running clean in some cases.
Sure enough, it did for me on a filesystem I had just created.

But on existing filesystems I've been using for a while, it never has for me
has per the example above.

I've added some info to:
https://btrfs.wiki.kernel.org/index.php/Btrfsck

Two questions:
1) are others seeing errors like these:
Extent back ref already exists for 2148837945344 parent 0 root 257
leaf parent key incorrect 504993210368
bad block 504993210368

2) do you know how bad each one is, and should they all be picked up by a
scrub? (I'm under the impression that they were not for me)

I'm happy to further update the wiki on btrfsck with info you can
contribute.

Thanks,
Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: very slow btrfs filesystem: any data needed before I wipe it?

[Marc MERLIN]

On Sun, Apr 13, 2014 at 04:02:36AM +0000, Duncan wrote:
> What happens if you simply mount it ro, without the recovery option?  Is 
> it still normal-speed or is that slow as a rw mount?

I just tried in ro without recovery, and I could copy data out at 52MB/s
for a big file, so that's quite good.

As soon as I mount it r/w, then everything goes to crap, even before I start
using it.
So from what I'm seeing, some structures are messed up, btrfs then goes into
some almost infinite loops to fix them if the filesystem is in writeable
mode.

I just tried to mount it and umount it, but I couldn't even unmount it.

I'll wait until tomorrow night to see if the devs want anything else out of
it, but otherwise I'll wipe it and start over.
 
> Speed hasn't been an issue here, but FWIW I normally keep my rootfs ro 
> mounted as I decided that's lower risk in the event of a system crash or 
> power outage and I don't actually /need/ it rw most of the time (I have a 

I'm all with you here, but that's not very practical on an array that is
only there to _receive_ backups from other systems :)

So, I made the FS read-write, I was still able to copy big files at 50MB/s,
and then did:
gargamel:/mnt/btrfs_pool2/backup/polgara# time cp -al ../legolas/current/ .
(copying a full system with many small files)

Within, minutes I got:
[114821.950649] INFO: task btrfs-cleaner:14442 blocked for more than 120 seconds.
[114821.984252]       Not tainted 3.14.0-amd64-i915-preempt-20140216 #2
[114822.007214] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[114822.035962] btrfs-cleaner   D ffff88009ea74640     0 14442      2 0x00000080
[114822.061879]  ffff880046bd7d00 0000000000000046 ffff880046bd7fd8 ffff88009ea74110
[114822.088179]  00000000000141c0 ffff88009ea74110 ffff88019f812350 ffff8801dafaa9e8
[114822.114363]  0000000000000000 ffff8801dafaa800 ffff8801bb957460 ffff880046bd7d10
[114822.140443] Call Trace:
[114822.151120]  [<ffffffff8160d2a1>] schedule+0x73/0x75
[114822.169217]  [<ffffffff8122aa77>] wait_current_trans.isra.15+0x98/0xf4
[114822.192089]  [<ffffffff81085116>] ? finish_wait+0x65/0x65
[114822.211381]  [<ffffffff8122bf1c>] start_transaction+0x48e/0x4f2
[114822.232292]  [<ffffffff8122c2ff>] ? __btrfs_end_transaction+0x2a1/0x2c6
[114822.255258]  [<ffffffff8122bf9b>] btrfs_start_transaction+0x1b/0x1d
[114822.277115]  [<ffffffff8121cc7d>] btrfs_drop_snapshot+0x443/0x610
[114822.298422]  [<ffffffff8122c73d>] btrfs_clean_one_deleted_snapshot+0x103/0x10f
[114822.323108]  [<ffffffff81224f09>] cleaner_kthread+0x103/0x136
[114822.343280]  [<ffffffff81224e06>] ? btrfs_alloc_root+0x26/0x26
[114822.363587]  [<ffffffff8106bc62>] kthread+0xae/0xb6
[114822.381042]  [<ffffffff8106bbb4>] ? __kthread_parkme+0x61/0x61
[114822.401319]  [<ffffffff8161637c>] ret_from_fork+0x7c/0xb0
[114822.420054]  [<ffffffff8106bbb4>] ? __kthread_parkme+0x61/0x61
[114822.440419] INFO: task cp:23902 blocked for more than 120 seconds.
[114822.461583]       Not tainted 3.14.0-amd64-i915-preempt-20140216 #2
[114822.482967] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[114822.509029] cp              D ffff88009e782800     0 23902  19922 0x20020080
[114822.532913]  ffff8802081f7d98 0000000000000082 ffff8802081f7fd8 ffff88009e7822d0
[114822.558306]  00000000000141c0 ffff88009e7822d0 ffff88019f812350 ffff8801d08f81e8
[114822.583381]  0000000000000000 ffff8801d08f8000 ffff880160b29640 ffff8802081f7da8
[114822.608434] Call Trace:
[114822.618046]  [<ffffffff8160d2a1>] schedule+0x73/0x75
[114822.635275]  [<ffffffff8122aa77>] wait_current_trans.isra.15+0x98/0xf4
[114822.657098]  [<ffffffff81085116>] ? finish_wait+0x65/0x65
[114822.675586]  [<ffffffff8122bf1c>] start_transaction+0x48e/0x4f2
[114822.695525]  [<ffffffff8122bf9b>] btrfs_start_transaction+0x1b/0x1d
[114822.716469]  [<ffffffff812364cc>] btrfs_link+0x74/0x18e
[114822.734272]  [<ffffffff811607db>] vfs_link+0x101/0x1a5
[114822.751830]  [<ffffffff811627a7>] SYSC_linkat+0x171/0x213
[114822.770100]  [<ffffffff81162ad4>] SyS_linkat+0xe/0x10
[114822.787314]  [<ffffffff8161812c>] sysenter_dispatch+0x7/0x21

After that, it took a long time for the ^C of my cp to work.
Actually, ^C just didn't work at all, and kill -9 didn't work either.

cp is stuck in kernel state:
23902    01:05:47 wait_current_trans.isra.15     cp -i -al ../legolas/current/ .

I was forced to reboot.
And then as soon as I reboot and mount that partition read/write, I get:

[  485.375223] INFO: task btrfs-cleaner:3552 blocked for more than 120 seconds.
[  485.396430]       Not tainted 3.14.0-amd64-i915-preempt-20140216 #2
[  485.415282] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  485.438832] btrfs-cleaner   D ffff880211d2eb80     0  3552      2 0x00000000
[  485.460134]  ffff8800c7f0fd00 0000000000000046 ffff8800c7f0ffd8 ffff880211d2e650
[  485.482503]  00000000000141c0 ffff880211d2e650 ffff88020cea6130 ffff880213fda1e8
[  485.504934]  0000000000000000 ffff880213fda000 ffff88020cfb2a60 ffff8800c7f0fd10
[  485.527342] Call Trace:
[  485.534706]  [<ffffffff8160d2a1>] schedule+0x73/0x75
[  485.549628]  [<ffffffff8122aa77>] wait_current_trans.isra.15+0x98/0xf4
[  485.569235]  [<ffffffff81085116>] ? finish_wait+0x65/0x65
[  485.585458]  [<ffffffff8122bf1c>] start_transaction+0x48e/0x4f2
[  485.603237]  [<ffffffff8122c2ff>] ? __btrfs_end_transaction+0x2a1/0x2c6
[  485.623096]  [<ffffffff8122bf9b>] btrfs_start_transaction+0x1b/0x1d
[  485.641932]  [<ffffffff8121cc7d>] btrfs_drop_snapshot+0x443/0x610
[  485.660256]  [<ffffffff8122c73d>] btrfs_clean_one_deleted_snapshot+0x103/0x10f
[  485.681961]  [<ffffffff81224f09>] cleaner_kthread+0x103/0x136
[  485.699226]  [<ffffffff81224e06>] ? btrfs_alloc_root+0x26/0x26
[  485.716746]  [<ffffffff8106bc62>] kthread+0xae/0xb6
[  485.731410]  [<ffffffff8106bbb4>] ? __kthread_parkme+0x61/0x61
[  485.748927]  [<ffffffff8161637c>] ret_from_fork+0x7c/0xb0
[  485.765147]  [<ffffffff8106bbb4>] ? __kthread_parkme+0x61/0x61

This is with 0 file IO to it.
Turns out even umount of that filesystem hangs.

Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: How to debug very very slow file delete?

[Liu Bo]

On Mon, Mar 24, 2014 at 06:49:56PM -0700, Marc MERLIN wrote:
> I had a tree with some amount of thousand files (less than 1 million)
> on top of md raid5.
> 
> It took 18H to rm it in 3 tries:
> gargamel:/mnt/dshelf2/backup/polgara# time rm -rf current.todel/
> real    1087m26.491s
> user    0m2.448s
> sys     4m42.012s
> 
> gargamel:/mnt/dshelf2/backup/polgara# btrfs fi show /mnt/btrfs_pool2
> Label: btrfs_pool2  uuid: cb9df6d3-a528-4afc-9a45-4fed5ec358d6
>         Total devices 1 FS bytes used 2.76TiB
>         devid    1 size 7.28TiB used 3.43TiB path /dev/mapper/dshelf2
> 
> gargamel:/mnt/dshelf2/backup/polgara# btrfs fi df /mnt/btrfs_pool2
> Data, single: total=3.28TiB, used=2.70TiB
> System, DUP: total=8.00MiB, used=384.00KiB
> System, single: total=4.00MiB, used=0.00
> Metadata, DUP: total=73.50GiB, used=62.46GiB
> Metadata, single: total=8.00MiB, used=0.00
> 
> This is running from
> md8 : active raid5 sdf1[6] sdb1[5] sda1[3] sde1[2] sdd1[1]
>       7814045696 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/5] [UUUUU]
>       bitmap: 0/15 pages [0KB], 65536KB chunk
> 
> The filesystem is pretty new, it shouldn't be fragmented much.
> 
> dmesg shows a bit of this:
> INFO: task rm:31885 blocked for more than 120 seconds.
>       Not tainted 3.14.0-rc5-amd64-i915-preempt-20140216c #1
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message
> rm              D ffff880117e5a940     0 31885  31280 0x20020080
>  ffff8800169f3d10 0000000000000086 ffff8800169f3fd8 ffff880117e5a410
>  00000000000141c0 ffff880117e5a410 ffff88011d4f9e90 ffff88020c0109e8
>  0000000000000000 ffff88020c010800 ffff8801b6df55a0 ffff8800169f3d20
> Call Trace:
>  [<ffffffff8160c331>] schedule+0x73/0x75
>  [<ffffffff8122a5f9>] wait_current_trans.isra.15+0x98/0xf4
>  [<ffffffff810850c9>] ? finish_wait+0x65/0x65
>  [<ffffffff8122ba9e>] start_transaction+0x48e/0x4f2
>  [<ffffffff8122bb1d>] btrfs_start_transaction+0x1b/0x1d
>  [<ffffffff8122cab4>] __unlink_start_trans+0x24/0xaf
>  [<ffffffff812327ab>] btrfs_unlink+0x28/0xa0
>  [<ffffffff8116176d>] vfs_unlink+0x90/0xdb
>  [<ffffffff811618c0>] do_unlinkat+0x108/0x1da
>  [<ffffffff810734c8>] ? mmdrop+0x11/0x20
>  [<ffffffff8115d2db>] ? path_put+0x1e/0x21
>  [<ffffffff811625c7>] SyS_unlinkat+0x22/0x2e
>  [<ffffffff816171ac>] sysenter_dispatch+0x7/0x21
> 
> That said, it didn't happen much:
> [38077.054841] INFO: task btrfs-cleaner:3536 blocked for more than 120 seconds.
> [38077.521463] INFO: task rm:31885 blocked for more than 120 seconds.
> [38077.926399] INFO: task cp:26645 blocked for more than 120 seconds.
> [38078.346885] INFO: task btrfs:7430 blocked for more than 120 seconds.
> [38198.921768] INFO: task btrfs-cleaner:3536 blocked for more than 120 seconds.
> [38199.357367] INFO: task rm:31885 blocked for more than 120 seconds.
> [38199.753897] INFO: task cp:26645 blocked for more than 120 seconds.
> [38200.172729] INFO: task btrfs:7430 blocked for more than 120 seconds.
> [56696.271850] INFO: task btrfs-transacti:3537 blocked for more than 120 seconds.
> [56937.063142] INFO: task btrfs-cleaner:3536 blocked for more than 120 seconds.
> 
> Of course, all of those are bad, but I'm unfortunately used to seeing them
> It wasn't hanging forever every few minutes, so it looks like it was just working 
> very slowly.
> 
> The problem does not seem to be just rm though, du is taking way way too
> long too. I started one, it's been running for 30mn now.
> Interestingly, sending ^C to that du takes 15 seconds to respond, so it
> seems that each system call is just slow.
> 
> I checked that btrfs scrub is not running.
> What else can I check from here?

Maybe 'sysrq+w' can give an answer?  We need to find what blocks new transaction.

-liubo

Re: How to debug very very slow file delete?

[Liu Bo]

On Mon, Apr 14, 2014 at 10:15:07AM +0800, Liu Bo wrote:
> On Mon, Mar 24, 2014 at 06:49:56PM -0700, Marc MERLIN wrote:
> > I had a tree with some amount of thousand files (less than 1 million)
> > on top of md raid5.
> > 
> > It took 18H to rm it in 3 tries:
> > gargamel:/mnt/dshelf2/backup/polgara# time rm -rf current.todel/
> > real    1087m26.491s
> > user    0m2.448s
> > sys     4m42.012s
> > 
> > gargamel:/mnt/dshelf2/backup/polgara# btrfs fi show /mnt/btrfs_pool2
> > Label: btrfs_pool2  uuid: cb9df6d3-a528-4afc-9a45-4fed5ec358d6
> >         Total devices 1 FS bytes used 2.76TiB
> >         devid    1 size 7.28TiB used 3.43TiB path /dev/mapper/dshelf2
> > 
> > gargamel:/mnt/dshelf2/backup/polgara# btrfs fi df /mnt/btrfs_pool2
> > Data, single: total=3.28TiB, used=2.70TiB
> > System, DUP: total=8.00MiB, used=384.00KiB
> > System, single: total=4.00MiB, used=0.00
> > Metadata, DUP: total=73.50GiB, used=62.46GiB
> > Metadata, single: total=8.00MiB, used=0.00
> > 
> > This is running from
> > md8 : active raid5 sdf1[6] sdb1[5] sda1[3] sde1[2] sdd1[1]
> >       7814045696 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/5] [UUUUU]
> >       bitmap: 0/15 pages [0KB], 65536KB chunk
> > 
> > The filesystem is pretty new, it shouldn't be fragmented much.
> > 
> > dmesg shows a bit of this:
> > INFO: task rm:31885 blocked for more than 120 seconds.
> >       Not tainted 3.14.0-rc5-amd64-i915-preempt-20140216c #1
> > "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message
> > rm              D ffff880117e5a940     0 31885  31280 0x20020080
> >  ffff8800169f3d10 0000000000000086 ffff8800169f3fd8 ffff880117e5a410
> >  00000000000141c0 ffff880117e5a410 ffff88011d4f9e90 ffff88020c0109e8
> >  0000000000000000 ffff88020c010800 ffff8801b6df55a0 ffff8800169f3d20
> > Call Trace:
> >  [<ffffffff8160c331>] schedule+0x73/0x75
> >  [<ffffffff8122a5f9>] wait_current_trans.isra.15+0x98/0xf4
> >  [<ffffffff810850c9>] ? finish_wait+0x65/0x65
> >  [<ffffffff8122ba9e>] start_transaction+0x48e/0x4f2
> >  [<ffffffff8122bb1d>] btrfs_start_transaction+0x1b/0x1d
> >  [<ffffffff8122cab4>] __unlink_start_trans+0x24/0xaf
> >  [<ffffffff812327ab>] btrfs_unlink+0x28/0xa0
> >  [<ffffffff8116176d>] vfs_unlink+0x90/0xdb
> >  [<ffffffff811618c0>] do_unlinkat+0x108/0x1da
> >  [<ffffffff810734c8>] ? mmdrop+0x11/0x20
> >  [<ffffffff8115d2db>] ? path_put+0x1e/0x21
> >  [<ffffffff811625c7>] SyS_unlinkat+0x22/0x2e
> >  [<ffffffff816171ac>] sysenter_dispatch+0x7/0x21
> > 
> > That said, it didn't happen much:
> > [38077.054841] INFO: task btrfs-cleaner:3536 blocked for more than 120 seconds.
> > [38077.521463] INFO: task rm:31885 blocked for more than 120 seconds.
> > [38077.926399] INFO: task cp:26645 blocked for more than 120 seconds.
> > [38078.346885] INFO: task btrfs:7430 blocked for more than 120 seconds.
> > [38198.921768] INFO: task btrfs-cleaner:3536 blocked for more than 120 seconds.
> > [38199.357367] INFO: task rm:31885 blocked for more than 120 seconds.
> > [38199.753897] INFO: task cp:26645 blocked for more than 120 seconds.
> > [38200.172729] INFO: task btrfs:7430 blocked for more than 120 seconds.
> > [56696.271850] INFO: task btrfs-transacti:3537 blocked for more than 120 seconds.
> > [56937.063142] INFO: task btrfs-cleaner:3536 blocked for more than 120 seconds.
> > 
> > Of course, all of those are bad, but I'm unfortunately used to seeing them
> > It wasn't hanging forever every few minutes, so it looks like it was just working 
> > very slowly.
> > 
> > The problem does not seem to be just rm though, du is taking way way too
> > long too. I started one, it's been running for 30mn now.
> > Interestingly, sending ^C to that du takes 15 seconds to respond, so it
> > seems that each system call is just slow.
> > 
> > I checked that btrfs scrub is not running.
> > What else can I check from here?
> 
> Maybe 'sysrq+w' can give an answer?  We need to find what blocks new transaction.

Just misread this thread, sorry...I'm reading the following story of the thread.

-liubo

Re: very slow btrfs filesystem: any data needed before I wipe it?

[Duncan]

Marc MERLIN posted on Sun, 13 Apr 2014 18:43:37 -0700 as excerpted:

> On Sun, Apr 13, 2014 at 04:02:36AM +0000, Duncan wrote:
>> What happens if you simply mount it ro, without the recovery option? 
>> Is it still normal-speed or is that slow as a rw mount?
> 
> I just tried in ro without recovery, and I could copy data out at 52MB/s
> for a big file, so that's quite good.
> 
> As soon as I mount it r/w, then everything goes to crap, even before I
> start using it.
> So from what I'm seeing, some structures are messed up, btrfs then goes
> into some almost infinite loops to fix them if the filesystem is in
> writeable mode.

Pretty much what I expected in terms of ro vs. rw, but I had forgotten 
the details on the why, and your explanation both prompted my memory and 
agrees with what I had seen before.

IIRC, Hugo mentioned others seeing something similar at times, where read-
only forced btrfs to leave alone what it was going into loops over, so 
you're not the only one to have seen this.

But you might well be the first report where the devs have good access to 
enough detail to actually trace down the problem.

> I'll wait until tomorrow night to see if the devs want anything else out
> of it, but otherwise I'll wipe it and start over.

Given that our exchange happened over the weekend and "tomorrow" is 
Monday, I'd consider waiting until Tuesday nite if possible, just to be 
sure.  Because Monday is of course back to work day, and it's possible if 
there's something urgent (or if they're taking a long weekend for some 
reason), they won't get fully caught up from the weekend until Tuesday.  
So if you can wait until Tuesday and you don't get a request for anything 
else by then, it's probably safe to wipe and redo, but I'd try to wait 
until then, just in case you have something useful, and they simply don't 
get to it the first day back after the weekend.  Because once it's gone 
it's gone, and now that I'm remembering what Hugo mentioned, it does seem 
you're not the only one to have come across this, but you might indeed 
have the missing key the others couldn't provide, so...

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [PATCH 00/27] Replace the old man page with asciidoc and man page for each btrfs subcommand.

[David Sterba]

On Wed, Apr 02, 2014 at 04:29:11PM +0800, Qu Wenruo wrote:
> Convert the old btrfs man pages to new asciidoc and split the huge
> btrfs man page into subcommand man page.

I'm merging this patchset into the base series of integration because
several patches need to update the docs and it's no longer feasible to
keep it in a separate branch from the patches.

>   btrfs-progs: Convert man page for btrfs-dedup.

As dedup is not yet merged in kernel, the docs will not be part of the
branch (but will be otherwise present in the integration).

>   btrfs-progs: Convert man page for btrfsck

I've skipped the patch adding btrfsck and linked to 'btrfs-check'
instead, patch will follow.

[PATCH] btrfs-progs: doc: link btrfsck to btrfs-check

[David Sterba]

The 'btrfsck' command has been deprecated in favor of 'btrfs check'. For
compatibility install a symlink to the btrfs-check.8 manpage.

CC: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
---
 Documentation/Makefile        | 2 ++
 Documentation/btrfs-check.txt | 3 ++-
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/Documentation/Makefile b/Documentation/Makefile
index ec8598bb57d3..1eef9fd57da3 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -48,6 +48,7 @@ XMLTO_EXTRA = -m manpage-bold-literal.xsl
 GZIP = gzip
 INSTALL ?= install
 RM ?= rm -f
+LNS ?= ln -sf
 BTRFS_VERSION = $(shell sed -n 's/.*BTRFS_BUILD_VERSION "Btrfs \(.*\)"/\1/p'\
 		  ../version.h)
 
@@ -73,6 +74,7 @@ install: install-man
 install-man: man
 	$(INSTALL) -d -m 755 $(DESTDIR)$(man8dir)
 	$(INSTALL) -m 644 $(GZ_MAN8) $(DESTDIR)$(man8dir)
+	$(LNS) btrfs-check.txt $(DESTDIR)$(man8dir)
 
 clean:
 	$(RM) *.xml *.xml+ *.8 *.8.gz
diff --git a/Documentation/btrfs-check.txt b/Documentation/btrfs-check.txt
index ddd7fe77eca2..485a49cbc3ec 100644
--- a/Documentation/btrfs-check.txt
+++ b/Documentation/btrfs-check.txt
@@ -18,6 +18,8 @@ command, it is *highly* recommended to read the following btrfs wiki before
 executing 'btrfs check' with '--repair' option: +
 https://btrfs.wiki.kernel.org/index.php/Btrfsck
 
+'btrfsck' is an alias of 'btrfs check' command and is now deprecated.
+
 OPTIONS
 -------
 -s|--support <superblock>::
@@ -47,4 +49,3 @@ SEE ALSO
 `mkfs.btrfs`(8),
 `btrfs-scrub`(8),
 `btrfs-rescue`(8)
-`btrfsck`(8)
-- 
1.9.0

Re: very slow btrfs filesystem: any data needed before I wipe it?

[Marc MERLIN]

On Mon, Apr 14, 2014 at 10:28:36AM +0000, Duncan wrote:
> But you might well be the first report where the devs have good access to 
> enough detail to actually trace down the problem.
> 
> > I'll wait until tomorrow night to see if the devs want anything else out
> > of it, but otherwise I'll wipe it and start over.
> 
> Given that our exchange happened over the weekend and "tomorrow" is 
> Monday, I'd consider waiting until Tuesday nite if possible, just to be 
> sure.  Because Monday is of course back to work day, and it's possible if 
> there's something urgent (or if they're taking a long weekend for some 
> reason), they won't get fully caught up from the weekend until Tuesday.  

So it seems that no one is interested in that filesystem.

I probably won't have time to wipe and rebuild it until this weekend, so
I'll see how it works after a rebuild.
Hopefully I won't hit the same problems again.

Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: [PATCH] btrfs-progs: doc: link btrfsck to btrfs-check

[Qu Wenruo]

-------- Original Message --------
Subject: [PATCH] btrfs-progs: doc: link btrfsck to btrfs-check
From: David Sterba <dsterba@suse.cz>
To: linux-btrfs@vger.kernel.org
Date: 2014年04月17日 01:16
> The 'btrfsck' command has been deprecated in favor of 'btrfs check'. For
> compatibility install a symlink to the btrfs-check.8 manpage.
>
> CC: Qu Wenruo <quwenruo@cn.fujitsu.com>
> Signed-off-by: David Sterba <dsterba@suse.cz>
> ---
>   Documentation/Makefile        | 2 ++
>   Documentation/btrfs-check.txt | 3 ++-
>   2 files changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/Makefile b/Documentation/Makefile
> index ec8598bb57d3..1eef9fd57da3 100644
> --- a/Documentation/Makefile
> +++ b/Documentation/Makefile
> @@ -48,6 +48,7 @@ XMLTO_EXTRA = -m manpage-bold-literal.xsl
>   GZIP = gzip
>   INSTALL ?= install
>   RM ?= rm -f
> +LNS ?= ln -sf
>   BTRFS_VERSION = $(shell sed -n 's/.*BTRFS_BUILD_VERSION "Btrfs \(.*\)"/\1/p'\
>   		  ../version.h)
>   
> @@ -73,6 +74,7 @@ install: install-man
>   install-man: man
>   	$(INSTALL) -d -m 755 $(DESTDIR)$(man8dir)
>   	$(INSTALL) -m 644 $(GZ_MAN8) $(DESTDIR)$(man8dir)
> +	$(LNS) btrfs-check.txt $(DESTDIR)$(man8dir)
Shouldn't the source of soft link be btrfs-check.8.gz. ?
>   
>   clean:
>   	$(RM) *.xml *.xml+ *.8 *.8.gz
> diff --git a/Documentation/btrfs-check.txt b/Documentation/btrfs-check.txt
> index ddd7fe77eca2..485a49cbc3ec 100644
> --- a/Documentation/btrfs-check.txt
> +++ b/Documentation/btrfs-check.txt
> @@ -18,6 +18,8 @@ command, it is *highly* recommended to read the following btrfs wiki before
>   executing 'btrfs check' with '--repair' option: +
>   https://btrfs.wiki.kernel.org/index.php/Btrfsck
>   
> +'btrfsck' is an alias of 'btrfs check' command and is now deprecated.
> +
>   OPTIONS
>   -------
>   -s|--support <superblock>::
> @@ -47,4 +49,3 @@ SEE ALSO
>   `mkfs.btrfs`(8),
>   `btrfs-scrub`(8),
>   `btrfs-rescue`(8)
> -`btrfsck`(8)
Sorry to bother you but 'btrfs-scrub'/'btrfs-rescue' and 'btrfs-restore' 
seems also metioning 'btrfsck' and may also needs to remove 'btrfsck'.

Thanks,
Qu

Re: [PATCH] btrfs-progs: doc: link btrfsck to btrfs-check

[David Sterba]

On Thu, Apr 17, 2014 at 08:47:28AM +0800, Qu Wenruo wrote:
> >@@ -73,6 +74,7 @@ install: install-man
> >  install-man: man
> >  	$(INSTALL) -d -m 755 $(DESTDIR)$(man8dir)
> >  	$(INSTALL) -m 644 $(GZ_MAN8) $(DESTDIR)$(man8dir)
> >+	$(LNS) btrfs-check.txt $(DESTDIR)$(man8dir)
> Shouldn't the source of soft link be btrfs-check.8.gz. ?

> >@@ -47,4 +49,3 @@ SEE ALSO
> >  `mkfs.btrfs`(8),
> >  `btrfs-scrub`(8),
> >  `btrfs-rescue`(8)
> >-`btrfsck`(8)
> Sorry to bother you but 'btrfs-scrub'/'btrfs-rescue' and 'btrfs-restore'
> seems also metioning 'btrfsck' and may also needs to remove 'btrfsck'.

Thanks for catching them, I'll fix it up.

Re: [PATCH] btrfs-progs: doc: link btrfsck to btrfs-check

[WorMzy Tykashi]

Hi,

While trying to view the btrfs-check manpage today (using
integration-20140429), I noticed that the current symlink overwrites
the actual btrfs-check manpage, making an unusable, cyclic link:

    $ man btrfs-check
    man: can't resolve /usr/share/man/man8/btrfs-check.8.gz: Too many
levels of symbolic links
    No manual entry for btrfs-check

I believe you forgot to add the file name of the symlink at the end of
the line, e.g.

    $(LNS) btrfs-check.8.gz $(DESTDIR)$(man8dir)

should be

    $(LNS) btrfs-check.8.gz $(DESTDIR)$(man8dir)/btrfsck.8.gz

Cheers,


WorMzy

Re: [PATCH] btrfs-progs: doc: link btrfsck to btrfs-check

[David Sterba]

On Wed, Apr 30, 2014 at 01:14:59PM +0100, WorMzy Tykashi wrote:
>     $(LNS) btrfs-check.8.gz $(DESTDIR)$(man8dir)
> 
> should be
> 
>     $(LNS) btrfs-check.8.gz $(DESTDIR)$(man8dir)/btrfsck.8.gz

Thank you, fix sent.

Re: [PATCH] btrfs-progs: doc: link btrfsck to btrfs-check

[Qu Wenruo]

-------- Original Message --------
Subject: Re: [PATCH] btrfs-progs: doc: link btrfsck to btrfs-check
From: David Sterba <dsterba@suse.cz>
To: Qu Wenruo <quwenruo@cn.fujitsu.com>
Date: 2014年04月18日 22:48
> On Thu, Apr 17, 2014 at 08:47:28AM +0800, Qu Wenruo wrote:
>>> @@ -73,6 +74,7 @@ install: install-man
>>>   install-man: man
>>>   	$(INSTALL) -d -m 755 $(DESTDIR)$(man8dir)
>>>   	$(INSTALL) -m 644 $(GZ_MAN8) $(DESTDIR)$(man8dir)
>>> +	$(LNS) btrfs-check.txt $(DESTDIR)$(man8dir)
>> Shouldn't the source of soft link be btrfs-check.8.gz. ?
Forgot to mention that the dest is also wrong. This will make 
$(DESTDIR)$(man8dir)/btrfs-check.8.gz to be a
infinite loop(pointing to it self).
The correct one should be like the following:
+       $(LNS) btrfs-check.8.gz $(DESTDIR)$(man8dir)/btrfsck.8.gz

Thanks,
Qu

>>> @@ -47,4 +49,3 @@ SEE ALSO
>>>   `mkfs.btrfs`(8),
>>>   `btrfs-scrub`(8),
>>>   `btrfs-rescue`(8)
>>> -`btrfsck`(8)
>> Sorry to bother you but 'btrfs-scrub'/'btrfs-rescue' and 'btrfs-restore'
>> seems also metioning 'btrfsck' and may also needs to remove 'btrfsck'.
> Thanks for catching them, I'll fix it up.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] btrfs-progs: doc: link btrfsck to btrfs-check

[David Sterba]

On Thu, May 08, 2014 at 09:40:03AM +0800, Qu Wenruo wrote:
> -------- Original Message --------
> Subject: Re: [PATCH] btrfs-progs: doc: link btrfsck to btrfs-check
> From: David Sterba <dsterba@suse.cz>
> To: Qu Wenruo <quwenruo@cn.fujitsu.com>
> Date: 2014年04月18日 22:48
> >On Thu, Apr 17, 2014 at 08:47:28AM +0800, Qu Wenruo wrote:
> >>>@@ -73,6 +74,7 @@ install: install-man
> >>>  install-man: man
> >>>  	$(INSTALL) -d -m 755 $(DESTDIR)$(man8dir)
> >>>  	$(INSTALL) -m 644 $(GZ_MAN8) $(DESTDIR)$(man8dir)
> >>>+	$(LNS) btrfs-check.txt $(DESTDIR)$(man8dir)
> >>Shouldn't the source of soft link be btrfs-check.8.gz. ?
> Forgot to mention that the dest is also wrong. This will make
> $(DESTDIR)$(man8dir)/btrfs-check.8.gz to be a
> infinite loop(pointing to it self).
> The correct one should be like the following:
> +       $(LNS) btrfs-check.8.gz $(DESTDIR)$(man8dir)/btrfsck.8.gz

Thanks, it was reported & fixed a few days ago, though it's not in the
integration branch, lag is on my side.

https://patchwork.kernel.org/patch/4115501/

Re: [PATCH 00/27] Replace the old man page with asciidoc and man page for each btrfs subcommand.

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 2125 bytes --]

On Wed, Apr 16, 2014 at 07:12:19PM +0200, David Sterba wrote:
> On Wed, Apr 02, 2014 at 04:29:11PM +0800, Qu Wenruo wrote:
> > Convert the old btrfs man pages to new asciidoc and split the huge
> > btrfs man page into subcommand man page.
> 
> I'm merging this patchset into the base series of integration because
> several patches need to update the docs and it's no longer feasible to
> keep it in a separate branch from the patches.

   I've just been poking around in the docs for a completely different
reason, and I think there's a fairly serious problem (well, as serious
as problems get with documentation).

   Take, for example, the format for btrfs fi resize:

'resize' [devid:][+/-]<size>[gkm]|[devid:]max <path>::

   Now, this has just thrown away all of the useful markup which
indicates the semantics of the command. The asciidoc renders all of
that text literally and unformatted, making alphasymbolic(*) soup of
the docs. Compare this to the old roff man page:

\fBbtrfs\fP \fBfilesystem resize\fP [\fIdevid\fP:][+/\-]\fI<size>\fP[gkm]|[\fIdevid\fP:]\fImax <path>\fP

   This isn't perfect -- we're missing a \fB around the "max" -- but
it has text in bold(⁑) and italics(⁂) and neither(☃). I've just looked
at some of the other pages, and they've also got similar typographical
problems. This is a lot of fiddly tedious work to get it right, and if
it doesn't get done now in the initial commit, then we're going to end
up with poor examples copied for every new feature or docs update,
making the problem worse before anyone does the work to make it
better.

   Hugo.

(*) Or possibly alphashambolic. :)
(⁑) For literal text
(⁂) For variables that require substitution by the user
(☃) For structural syntax indicators such as [] for optional parts, |
    for alternation and ... to indicate optional continuation of a list

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
            --- There isn't a noun that can't be verbed. ---             

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]

Re: [PATCH 00/27] Replace the old man page with asciidoc and man page for each btrfs subcommand.

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 3584 bytes --]

On Sat, May 17, 2014 at 06:43:15PM +0100, Hugo Mills wrote:
> On Wed, Apr 16, 2014 at 07:12:19PM +0200, David Sterba wrote:
> > On Wed, Apr 02, 2014 at 04:29:11PM +0800, Qu Wenruo wrote:
> > > Convert the old btrfs man pages to new asciidoc and split the huge
> > > btrfs man page into subcommand man page.
> > 
> > I'm merging this patchset into the base series of integration because
> > several patches need to update the docs and it's no longer feasible to
> > keep it in a separate branch from the patches.
> 
>    I've just been poking around in the docs for a completely different
> reason, and I think there's a fairly serious problem (well, as serious
> as problems get with documentation).
> 
>    Take, for example, the format for btrfs fi resize:
> 
> 'resize' [devid:][+/-]<size>[gkm]|[devid:]max <path>::
> 
>    Now, this has just thrown away all of the useful markup which
> indicates the semantics of the command. The asciidoc renders all of
> that text literally and unformatted, making alphasymbolic(*) soup of
> the docs. Compare this to the old roff man page:
> 
> \fBbtrfs\fP \fBfilesystem resize\fP [\fIdevid\fP:][+/\-]\fI<size>\fP[gkm]|[\fIdevid\fP:]\fImax <path>\fP
> 
>    This isn't perfect -- we're missing a \fB around the "max" -- but
> it has text in bold(⁑) and italics(⁂) and neither(☃). I've just looked
> at some of the other pages, and they've also got similar typographical
> problems. This is a lot of fiddly tedious work to get it right, and if
> it doesn't get done now in the initial commit, then we're going to end
> up with poor examples copied for every new feature or docs update,
> making the problem worse before anyone does the work to make it
> better.

   Oh, and asciidoc appears to be the most horrible capricious
inconsistent parser in existence. I've just spent 5 minutes getting
this one line of text to do what I want it to:

=====
__N__**c**[\[_Mmin_**-**]__Mmax__**s**[_P_**p**]]
=====

   I had to run through the list of block quote operators one at a
time in order to find the one I needed for this (the =====); it's
still not indenting it correctly on the resulting man page.

   Note also the fun things like the fact that [[]] is special, so you
have to quote the opening part of it -- but if you try quoting the
first [ with a \ you get a literal \[ in the output. You get the right
output from quoting the *second* [ only.

   The Nc can only be italicised and emboldened properly with __ and
** because _ and * require whitespace around them in order to work
(seriously, WTF?). However, we can't be consistent with that in the
_Mmin_**-** because the quoted \[ appears to count as whitespace, so
using __Mmin__ gives us a leading literal _. The closing __ appears to
close the single opening _ correctly in that case, though.

   Seriously, this is meant to be _easy_ to use? I think I'd rather
type docbook by hand that have to struggle with this. Even the troff
macros for man pages are simpler to get right.

   Hugo.

>    Hugo.
> 
> (*) Or possibly alphashambolic. :)
> (⁑) For literal text
> (⁂) For variables that require substitution by the user
> (☃) For structural syntax indicators such as [] for optional parts, |
>     for alternation and ... to indicate optional continuation of a list
> 

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
       --- Great oxymorons of the world, no. 1: Family Holiday ---       

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]

Re: [PATCH 00/27] Replace the old man page with asciidoc and man page for each btrfs subcommand.

[Qu Wenruo]

-------- Original Message --------
Subject: Re: [PATCH 00/27] Replace the old man page with asciidoc and 
man page for each btrfs subcommand.
From: Hugo Mills <hugo@carfax.org.uk>
To: dsterba@suse.cz, Qu Wenruo <quwenruo@cn.fujitsu.com>, 
linux-btrfs@vger.kernel.org, clm@fb.com
Date: 2014年05月18日 01:43
> On Wed, Apr 16, 2014 at 07:12:19PM +0200, David Sterba wrote:
>> On Wed, Apr 02, 2014 at 04:29:11PM +0800, Qu Wenruo wrote:
>>> Convert the old btrfs man pages to new asciidoc and split the huge
>>> btrfs man page into subcommand man page.
>> I'm merging this patchset into the base series of integration because
>> several patches need to update the docs and it's no longer feasible to
>> keep it in a separate branch from the patches.
>     I've just been poking around in the docs for a completely different
> reason, and I think there's a fairly serious problem (well, as serious
> as problems get with documentation).
>
>     Take, for example, the format for btrfs fi resize:
>
> 'resize' [devid:][+/-]<size>[gkm]|[devid:]max <path>::
>
>     Now, this has just thrown away all of the useful markup which
> indicates the semantics of the command. The asciidoc renders all of
> that text literally and unformatted, making alphasymbolic(*) soup of
> the docs. Compare this to the old roff man page:
>
> \fBbtrfs\fP \fBfilesystem resize\fP [\fIdevid\fP:][+/\-]\fI<size>\fP[gkm]|[\fIdevid\fP:]\fImax <path>\fP
Yes, When I convert man pages to asciidoc docs, I have already realized 
the problem.
As mentioned in first patch, most things including the Makefile is 
'stolen' from git,
which means I also apply the git way to deal with the all 'useful' 
markups, *just throw them away* .
>
>     This isn't perfect -- we're missing a \fB around the "max" -- but
> it has text in bold(⁑) and italics(⁂) and neither(☃). I've just looked
> at some of the other pages, and they've also got similar typographical
> problems. This is a lot of fiddly tedious work to get it right, and if
> it doesn't get done now in the initial commit, then we're going to end
> up with poor examples copied for every new feature or docs update,
> making the problem worse before anyone does the work to make it
> better.
As I mentions above, it's meant to be like this, without extra markup 
just like git.

I choose asciidoc and the git documentation style for the following purpose:

1) Split up the huge 'btrfs' man page. (Main purpose of the patchset)

The 'btrfs' man page is so huge that the synopsis is serveral pages long,
force developers to edit man page twice(one for synopsis and one for 
command).
This makes editing frustrating and easy to make things inconsistent.
(serveral synopsis and command description are already inconsistent)

2) Make the documenation more general purpose. (Why choose asciidoc)

Not only generating man pages, but also html/pdf, much like git.

3) Make the original txt more human readable (Why choose git style)

I can use the old markup method but after doing that I realize if you 
read a document full of
markups, the markups have alread lost their meaning.

If using too many markup, there will be other problems:
     3.1) making the synopsis in original txt too long

             This will make both developer and reviewer harder to 
edit/review.
             I choose asciidoc to reduce the hard-to-understand groff 
grammar, if these \fX are
             just converted to ' or `, IMO the cost is still not cut.

     3.2) the markup is not so highlighted if every thing is highlighted

             So I choose only to highlight really important things, 
since with all the bold and itatlic
             things full of the page, there is no difference between all 
normal formatted text.

Due to the above 3 reasons, I think throwing all markup in synopsis is a 
better choice.

Thanks,
Qu
>
>     Hugo.
>
> (*) Or possibly alphashambolic. :)
> (⁑) For literal text
> (⁂) For variables that require substitution by the user
> (☃) For structural syntax indicators such as [] for optional parts, |
>      for alternation and ... to indicate optional continuation of a list
>

Re: [PATCH 00/27] Replace the old man page with asciidoc and man page for each btrfs subcommand.

[Qu Wenruo]

-------- Original Message --------
Subject: Re: [PATCH 00/27] Replace the old man page with asciidoc and 
man page for each btrfs subcommand.
From: Hugo Mills <hugo@carfax.org.uk>
To: dsterba@suse.cz, Qu Wenruo <quwenruo@cn.fujitsu.com>, 
linux-btrfs@vger.kernel.org, clm@fb.com
Date: 2014年05月18日 02:22
> On Sat, May 17, 2014 at 06:43:15PM +0100, Hugo Mills wrote:
>> On Wed, Apr 16, 2014 at 07:12:19PM +0200, David Sterba wrote:
>>> On Wed, Apr 02, 2014 at 04:29:11PM +0800, Qu Wenruo wrote:
>>>> Convert the old btrfs man pages to new asciidoc and split the huge
>>>> btrfs man page into subcommand man page.
>>> I'm merging this patchset into the base series of integration because
>>> several patches need to update the docs and it's no longer feasible to
>>> keep it in a separate branch from the patches.
>>     I've just been poking around in the docs for a completely different
>> reason, and I think there's a fairly serious problem (well, as serious
>> as problems get with documentation).
>>
>>     Take, for example, the format for btrfs fi resize:
>>
>> 'resize' [devid:][+/-]<size>[gkm]|[devid:]max <path>::
>>
>>     Now, this has just thrown away all of the useful markup which
>> indicates the semantics of the command. The asciidoc renders all of
>> that text literally and unformatted, making alphasymbolic(*) soup of
>> the docs. Compare this to the old roff man page:
>>
>> \fBbtrfs\fP \fBfilesystem resize\fP [\fIdevid\fP:][+/\-]\fI<size>\fP[gkm]|[\fIdevid\fP:]\fImax <path>\fP
>>
>>     This isn't perfect -- we're missing a \fB around the "max" -- but
>> it has text in bold(⁑) and italics(⁂) and neither(☃). I've just looked
>> at some of the other pages, and they've also got similar typographical
>> problems. This is a lot of fiddly tedious work to get it right, and if
>> it doesn't get done now in the initial commit, then we're going to end
>> up with poor examples copied for every new feature or docs update,
>> making the problem worse before anyone does the work to make it
>> better.
>     Oh, and asciidoc appears to be the most horrible capricious
> inconsistent parser in existence. I've just spent 5 minutes getting
> this one line of text to do what I want it to:
>
> =====
> __N__**c**[\[_Mmin_**-**]__Mmax__**s**[_P_**p**]]
> =====
>
>     I had to run through the list of block quote operators one at a
> time in order to find the one I needed for this (the =====); it's
> still not indenting it correctly on the resulting man page.
>
>     Note also the fun things like the fact that [[]] is special, so you
> have to quote the opening part of it -- but if you try quoting the
> first [ with a \ you get a literal \[ in the output. You get the right
> output from quoting the *second* [ only.
I have already encountered problems like that, especially when convert 
'btrfs-resize' related things.

But wait for a minute, do we really need the *fascinating* highlight 
things in a user documentation?
The most important thing is the content not the format.
I choose asciidoc to make developers take less effort on the format 
things, not the opposite.
In this point of view, I think asciidoc does things considerately well.

Although the problem you mentioned is true, but it only affects a small 
part of the documentation,
compared to the overall benefits, I still consider converting to 
asciidoc is worthy.

>
>     The Nc can only be italicised and emboldened properly with __ and
> ** because _ and * require whitespace around them in order to work
> (seriously, WTF?). However, we can't be consistent with that in the
> _Mmin_**-** because the quoted \[ appears to count as whitespace, so
> using __Mmin__ gives us a leading literal _. The closing __ appears to
> close the single opening _ correctly in that case, though.
>
>     Seriously, this is meant to be _easy_ to use? I think I'd rather
> type docbook by hand that have to struggle with this. Even the troff
> macros for man pages are simpler to get right.
 From the purpose of documentation and the above explaination, I think 
the answer is *Yes*.
If you really think there is a better choice, I will be very happy to 
listen,
but please consider what I mentioned above and the privious mail first.

Thanks,
Qu.
>
>     Hugo.
>
>>     Hugo.
>>
>> (*) Or possibly alphashambolic. :)
>> (⁑) For literal text
>> (⁂) For variables that require substitution by the user
>> (☃) For structural syntax indicators such as [] for optional parts, |
>>      for alternation and ... to indicate optional continuation of a list
>>

Re: [PATCH 00/27] Replace the old man page with asciidoc and man page for each btrfs subcommand.

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 5729 bytes --]

On Sun, May 18, 2014 at 02:51:39PM +0800, Qu Wenruo wrote:
> 
> -------- Original Message --------
> Subject: Re: [PATCH 00/27] Replace the old man page with asciidoc and man
> page for each btrfs subcommand.
> From: Hugo Mills <hugo@carfax.org.uk>
> To: dsterba@suse.cz, Qu Wenruo <quwenruo@cn.fujitsu.com>,
> linux-btrfs@vger.kernel.org, clm@fb.com
> Date: 2014年05月18日 01:43
> >On Wed, Apr 16, 2014 at 07:12:19PM +0200, David Sterba wrote:
> >>On Wed, Apr 02, 2014 at 04:29:11PM +0800, Qu Wenruo wrote:
> >>>Convert the old btrfs man pages to new asciidoc and split the huge
> >>>btrfs man page into subcommand man page.
> >>I'm merging this patchset into the base series of integration because
> >>several patches need to update the docs and it's no longer feasible to
> >>keep it in a separate branch from the patches.
> >    I've just been poking around in the docs for a completely different
> >reason, and I think there's a fairly serious problem (well, as serious
> >as problems get with documentation).
> >
> >    Take, for example, the format for btrfs fi resize:
> >
> >'resize' [devid:][+/-]<size>[gkm]|[devid:]max <path>::
> >
> >    Now, this has just thrown away all of the useful markup which
> >indicates the semantics of the command. The asciidoc renders all of
> >that text literally and unformatted, making alphasymbolic(*) soup of
> >the docs. Compare this to the old roff man page:
> >
> >\fBbtrfs\fP \fBfilesystem resize\fP [\fIdevid\fP:][+/\-]\fI<size>\fP[gkm]|[\fIdevid\fP:]\fImax <path>\fP
> Yes, When I convert man pages to asciidoc docs, I have already realized the
> problem.
> As mentioned in first patch, most things including the Makefile is 'stolen'
> from git,
> which means I also apply the git way to deal with the all 'useful' markups,
> *just throw them away* .
> >
> >    This isn't perfect -- we're missing a \fB around the "max" -- but
> >it has text in bold(⁑) and italics(⁂) and neither(☃). I've just looked
> >at some of the other pages, and they've also got similar typographical
> >problems. This is a lot of fiddly tedious work to get it right, and if
> >it doesn't get done now in the initial commit, then we're going to end
> >up with poor examples copied for every new feature or docs update,
> >making the problem worse before anyone does the work to make it
> >better.
> As I mentions above, it's meant to be like this, without extra markup just
> like git.
> 
> I choose asciidoc and the git documentation style for the following purpose:
> 
> 1) Split up the huge 'btrfs' man page. (Main purpose of the patchset)
> 
> The 'btrfs' man page is so huge that the synopsis is serveral pages long,
> force developers to edit man page twice(one for synopsis and one for
> command).
> This makes editing frustrating and easy to make things inconsistent.
> (serveral synopsis and command description are already inconsistent)

   I don't have a problem with that, but it's irrelevant to my point.

> 2) Make the documenation more general purpose. (Why choose asciidoc)
> 
> Not only generating man pages, but also html/pdf, much like git.

   I have no objections at all to that either. It's a great idea. But
again, it's orthogonal to my main point here, which is that we've lost
useful semantics, because the markup _is_ part of the meaning of the
document.

> 3) Make the original txt more human readable (Why choose git style)
> 
> I can use the old markup method but after doing that I realize if you read a
> document full of
> markups, the markups have alread lost their meaning.
> 
> If using too many markup, there will be other problems:
>     3.1) making the synopsis in original txt too long
> 
>             This will make both developer and reviewer harder to
> edit/review.
>             I choose asciidoc to reduce the hard-to-understand groff
> grammar, if these \fX are
>             just converted to ' or `, IMO the cost is still not cut.

   I don't think we should be throwing away meaning just because it
makes things shorter in the source.

>     3.2) the markup is not so highlighted if every thing is highlighted
> 
>             So I choose only to highlight really important things, since
> with all the bold and itatlic
>             things full of the page, there is no difference between all
> normal formatted text.

   I'm only talking about two things: the syntactic summaries, where
we need to distinguish between literals, placeholders and descriptive
elements like []; and "command text" where we distinguish between the
descriptive English text and the stuff you type. This second point is
particularly important for us because so many of our commands consist
of recognisable English words strung together, and having the literal
commands shown in a distinctive style (typically a monospace font)
helps enormously with parsing the text when reading at speed.

> Due to the above 3 reasons, I think throwing all markup in synopsis is a
> better choice.

   I disagree strongly.

   Hugo.

> Thanks,
> Qu
> >
> >    Hugo.
> >
> >(*) Or possibly alphashambolic. :)
> >(⁑) For literal text
> >(⁂) For variables that require substitution by the user
> >(☃) For structural syntax indicators such as [] for optional parts, |
> >     for alternation and ... to indicate optional continuation of a list
> >
> 

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
   --- Two things came out of Berkeley in the 1960s: LSD and Unix. ---   
                       This is not a coincidence.                        

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]

Re: [PATCH 00/27] Replace the old man page with asciidoc and man page for each btrfs subcommand.

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 5927 bytes --]

On Sun, May 18, 2014 at 03:04:33PM +0800, Qu Wenruo wrote:
> 
> -------- Original Message --------
> Subject: Re: [PATCH 00/27] Replace the old man page with asciidoc and man
> page for each btrfs subcommand.
> From: Hugo Mills <hugo@carfax.org.uk>
> To: dsterba@suse.cz, Qu Wenruo <quwenruo@cn.fujitsu.com>,
> linux-btrfs@vger.kernel.org, clm@fb.com
> Date: 2014年05月18日 02:22
> >On Sat, May 17, 2014 at 06:43:15PM +0100, Hugo Mills wrote:
> >>On Wed, Apr 16, 2014 at 07:12:19PM +0200, David Sterba wrote:
> >>>On Wed, Apr 02, 2014 at 04:29:11PM +0800, Qu Wenruo wrote:
> >>>>Convert the old btrfs man pages to new asciidoc and split the huge
> >>>>btrfs man page into subcommand man page.
> >>>I'm merging this patchset into the base series of integration because
> >>>several patches need to update the docs and it's no longer feasible to
> >>>keep it in a separate branch from the patches.
> >>    I've just been poking around in the docs for a completely different
> >>reason, and I think there's a fairly serious problem (well, as serious
> >>as problems get with documentation).
> >>
> >>    Take, for example, the format for btrfs fi resize:
> >>
> >>'resize' [devid:][+/-]<size>[gkm]|[devid:]max <path>::
> >>
> >>    Now, this has just thrown away all of the useful markup which
> >>indicates the semantics of the command. The asciidoc renders all of
> >>that text literally and unformatted, making alphasymbolic(*) soup of
> >>the docs. Compare this to the old roff man page:
> >>
> >>\fBbtrfs\fP \fBfilesystem resize\fP [\fIdevid\fP:][+/\-]\fI<size>\fP[gkm]|[\fIdevid\fP:]\fImax <path>\fP
> >>
> >>    This isn't perfect -- we're missing a \fB around the "max" -- but
> >>it has text in bold(⁑) and italics(⁂) and neither(☃). I've just looked
> >>at some of the other pages, and they've also got similar typographical
> >>problems. This is a lot of fiddly tedious work to get it right, and if
> >>it doesn't get done now in the initial commit, then we're going to end
> >>up with poor examples copied for every new feature or docs update,
> >>making the problem worse before anyone does the work to make it
> >>better.
> >    Oh, and asciidoc appears to be the most horrible capricious
> >inconsistent parser in existence. I've just spent 5 minutes getting
> >this one line of text to do what I want it to:
> >
> >=====
> >__N__**c**[\[_Mmin_**-**]__Mmax__**s**[_P_**p**]]
> >=====
> >
> >    I had to run through the list of block quote operators one at a
> >time in order to find the one I needed for this (the =====); it's
> >still not indenting it correctly on the resulting man page.
> >
> >    Note also the fun things like the fact that [[]] is special, so you
> >have to quote the opening part of it -- but if you try quoting the
> >first [ with a \ you get a literal \[ in the output. You get the right
> >output from quoting the *second* [ only.
> I have already encountered problems like that, especially when convert
> 'btrfs-resize' related things.
> 
> But wait for a minute, do we really need the *fascinating* highlight things
> in a user documentation?

   Yes, absolutely. The formatting is a part of the _meaning_ of the
documentation. Otherwise you're left guessing as to which pieces of
the string of characters are meant to be there literally, and which
pieces have to be replaced by suitable text, and which pieces are
optional.

> The most important thing is the content not the format.

   My point is that in this case the formatting _is_ a part of the
content.

> I choose asciidoc to make developers take less effort on the format things,
> not the opposite.
> In this point of view, I think asciidoc does things considerately well.
> 
> Although the problem you mentioned is true, but it only affects a small part
> of the documentation,
> compared to the overall benefits, I still consider converting to asciidoc is
> worthy.

   I'm finding it almost impossible to make it do what I want. I think
in some cases it actually _is_ impossible. This is a truly frustrating
tool that is really not making things simpler, and I can see is going
to lead to even more badly marked up documentation -- simply because
it's too difficult and frustrating to get it right.

> >    The Nc can only be italicised and emboldened properly with __ and
> >** because _ and * require whitespace around them in order to work
> >(seriously, WTF?). However, we can't be consistent with that in the
> >_Mmin_**-** because the quoted \[ appears to count as whitespace, so
> >using __Mmin__ gives us a leading literal _. The closing __ appears to
> >close the single opening _ correctly in that case, though.
> >
> >    Seriously, this is meant to be _easy_ to use? I think I'd rather
> >type docbook by hand that have to struggle with this. Even the troff
> >macros for man pages are simpler to get right.
> From the purpose of documentation and the above explaination, I think the
> answer is *Yes*.

   Only if you don't care about the typography of the resulting document.

> If you really think there is a better choice, I will be very happy
> to listen, but please consider what I mentioned above and the
> privious mail first.

   I don't have any real suggestions for alternatives coming from my
experience, other than "not this". I've used docbook for man pages
briefly, many years ago. Looking around on the web, reStructuredText
might be a good option. Personally, I'd like to write docs in LaTeX,
but I'm not sure how easy it is to convert that to man pages.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
   --- Two things came out of Berkeley in the 1960s: LSD and Unix. ---   
                       This is not a coincidence.                        

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]

Re: [PATCH 00/27] Replace the old man page with asciidoc and man page for each btrfs subcommand.

[Brendan Hide]

On 2014/05/18 02:05 PM, Hugo Mills wrote:
> On Sun, May 18, 2014 at 03:04:33PM +0800, Qu Wenruo wrote:
>     I don't have any real suggestions for alternatives coming from my
> experience, other than "not this". I've used docbook for man pages
> briefly, many years ago. Looking around on the web, reStructuredText
> might be a good option. Personally, I'd like to write docs in LaTeX,
> but I'm not sure how easy it is to convert that to man pages.
>
>     Hugo.
>
What I have read so far indicates that LaTeX is the simplest most 
beautiful way to create portable documentation - and that exporting to a 
man page is simple. I can't vouch for it except to say that it is worth 
investigating.

-- 
__________
Brendan Hide
http://swiftspirit.co.za/
http://www.webafrica.co.za/?AFF1E97

Re: [PATCH 00/27] Replace the old man page with asciidoc and man page for each btrfs subcommand.

[Qu Wenruo]

-------- Original Message --------
Subject: Re: [PATCH 00/27] Replace the old man page with asciidoc and 
man page for each btrfs subcommand.
From: Hugo Mills <hugo@carfax.org.uk>
To: Qu Wenruo <quwenruo@cn.fujitsu.com>
Date: 2014年05月18日 20:05
> On Sun, May 18, 2014 at 03:04:33PM +0800, Qu Wenruo wrote:
>> -------- Original Message --------
>> Subject: Re: [PATCH 00/27] Replace the old man page with asciidoc and man
>> page for each btrfs subcommand.
>> From: Hugo Mills <hugo@carfax.org.uk>
>> To: dsterba@suse.cz, Qu Wenruo <quwenruo@cn.fujitsu.com>,
>> linux-btrfs@vger.kernel.org, clm@fb.com
>> Date: 2014年05月18日 02:22
>>> On Sat, May 17, 2014 at 06:43:15PM +0100, Hugo Mills wrote:
>>>> On Wed, Apr 16, 2014 at 07:12:19PM +0200, David Sterba wrote:
>>>>> On Wed, Apr 02, 2014 at 04:29:11PM +0800, Qu Wenruo wrote:
>>>>>> Convert the old btrfs man pages to new asciidoc and split the huge
>>>>>> btrfs man page into subcommand man page.
>>>>> I'm merging this patchset into the base series of integration because
>>>>> several patches need to update the docs and it's no longer feasible to
>>>>> keep it in a separate branch from the patches.
>>>>     I've just been poking around in the docs for a completely different
>>>> reason, and I think there's a fairly serious problem (well, as serious
>>>> as problems get with documentation).
>>>>
>>>>     Take, for example, the format for btrfs fi resize:
>>>>
>>>> 'resize' [devid:][+/-]<size>[gkm]|[devid:]max <path>::
>>>>
>>>>     Now, this has just thrown away all of the useful markup which
>>>> indicates the semantics of the command. The asciidoc renders all of
>>>> that text literally and unformatted, making alphasymbolic(*) soup of
>>>> the docs. Compare this to the old roff man page:
>>>>
>>>> \fBbtrfs\fP \fBfilesystem resize\fP [\fIdevid\fP:][+/\-]\fI<size>\fP[gkm]|[\fIdevid\fP:]\fImax <path>\fP
>>>>
>>>>     This isn't perfect -- we're missing a \fB around the "max" -- but
>>>> it has text in bold(⁑) and italics(⁂) and neither(☃). I've just looked
>>>> at some of the other pages, and they've also got similar typographical
>>>> problems. This is a lot of fiddly tedious work to get it right, and if
>>>> it doesn't get done now in the initial commit, then we're going to end
>>>> up with poor examples copied for every new feature or docs update,
>>>> making the problem worse before anyone does the work to make it
>>>> better.
>>>     Oh, and asciidoc appears to be the most horrible capricious
>>> inconsistent parser in existence. I've just spent 5 minutes getting
>>> this one line of text to do what I want it to:
>>>
>>> =====
>>> __N__**c**[\[_Mmin_**-**]__Mmax__**s**[_P_**p**]]
>>> =====
>>>
>>>     I had to run through the list of block quote operators one at a
>>> time in order to find the one I needed for this (the =====); it's
>>> still not indenting it correctly on the resulting man page.
>>>
>>>     Note also the fun things like the fact that [[]] is special, so you
>>> have to quote the opening part of it -- but if you try quoting the
>>> first [ with a \ you get a literal \[ in the output. You get the right
>>> output from quoting the *second* [ only.
>> I have already encountered problems like that, especially when convert
>> 'btrfs-resize' related things.
>>
>> But wait for a minute, do we really need the *fascinating* highlight things
>> in a user documentation?
>     Yes, absolutely. The formatting is a part of the _meaning_ of the
> documentation. Otherwise you're left guessing as to which pieces of
> the string of characters are meant to be there literally, and which
> pieces have to be replaced by suitable text, and which pieces are
> optional.
>
>> The most important thing is the content not the format.
>     My point is that in this case the formatting _is_ a part of the
> content.
>
>> I choose asciidoc to make developers take less effort on the format things,
>> not the opposite.
>> In this point of view, I think asciidoc does things considerately well.
>>
>> Although the problem you mentioned is true, but it only affects a small part
>> of the documentation,
>> compared to the overall benefits, I still consider converting to asciidoc is
>> worthy.
>     I'm finding it almost impossible to make it do what I want. I think
> in some cases it actually _is_ impossible. This is a truly frustrating
> tool that is really not making things simpler, and I can see is going
> to lead to even more badly marked up documentation -- simply because
> it's too difficult and frustrating to get it right.
>
>>>     The Nc can only be italicised and emboldened properly with __ and
>>> ** because _ and * require whitespace around them in order to work
>>> (seriously, WTF?). However, we can't be consistent with that in the
>>> _Mmin_**-** because the quoted \[ appears to count as whitespace, so
>>> using __Mmin__ gives us a leading literal _. The closing __ appears to
>>> close the single opening _ correctly in that case, though.
>>>
>>>     Seriously, this is meant to be _easy_ to use? I think I'd rather
>>> type docbook by hand that have to struggle with this. Even the troff
>>> macros for man pages are simpler to get right.
>>  From the purpose of documentation and the above explaination, I think the
>> answer is *Yes*.
>     Only if you don't care about the typography of the resulting document.
Since you seems really unhappy with the git documentation style, I have 
nothing else to say.
I recommended you to send your opinion to git mail list and see what 
they respond.

I think it is now upon David to make the decision.

Thanks,
Qu.
>
>> If you really think there is a better choice, I will be very happy
>> to listen, but please consider what I mentioned above and the
>> privious mail first.
>     I don't have any real suggestions for alternatives coming from my
> experience, other than "not this". I've used docbook for man pages
> briefly, many years ago. Looking around on the web, reStructuredText
> might be a good option. Personally, I'd like to write docs in LaTeX,
> but I'm not sure how easy it is to convert that to man pages.
>
>     Hugo.
>

Re: [PATCH 00/27] Replace the old man page with asciidoc and man page for each btrfs subcommand.

[Chris Mason]

On 05/17/2014 01:43 PM, Hugo Mills wrote:
> On Wed, Apr 16, 2014 at 07:12:19PM +0200, David Sterba wrote:
>> On Wed, Apr 02, 2014 at 04:29:11PM +0800, Qu Wenruo wrote:
>>> Convert the old btrfs man pages to new asciidoc and split the huge
>>> btrfs man page into subcommand man page.
>>
>> I'm merging this patchset into the base series of integration because
>> several patches need to update the docs and it's no longer feasible to
>> keep it in a separate branch from the patches.
> 
>    I've just been poking around in the docs for a completely different
> reason, and I think there's a fairly serious problem (well, as serious
> as problems get with documentation).
> 
>    Take, for example, the format for btrfs fi resize:
> 
> 'resize' [devid:][+/-]<size>[gkm]|[devid:]max <path>::
> 
>    Now, this has just thrown away all of the useful markup which
> indicates the semantics of the command. The asciidoc renders all of
> that text literally and unformatted, making alphasymbolic(*) soup of
> the docs. Compare this to the old roff man page:
> 
> \fBbtrfs\fP \fBfilesystem resize\fP [\fIdevid\fP:][+/\-]\fI<size>\fP[gkm]|[\fIdevid\fP:]\fImax <path>\fP
> 
>    This isn't perfect -- we're missing a \fB around the "max" -- but
> it has text in bold(⁑) and italics(⁂) and neither(☃). I've just looked
> at some of the other pages, and they've also got similar typographical
> problems. This is a lot of fiddly tedious work to get it right, and if
> it doesn't get done now in the initial commit, then we're going to end
> up with poor examples copied for every new feature or docs update,
> making the problem worse before anyone does the work to make it
> better.

Are there issues with the asciidoc form outside of the command summary line?

The reason I ask is that all of these tools have tradeoffs.  If asciidoc
makes our documentation easier to update and easier to keep up to date,
I'm willing to trade that for the perfect summary line.

I think the easiest way to add clarity to the summary (in any markup
language) is by providing examples.  Italics and bolds definitely help,
but examples always win.

-chris

Re: [PATCH 00/27] Replace the old man page with asciidoc and man page for each btrfs subcommand.

[David Sterba]

On Sat, May 17, 2014 at 06:43:15PM +0100, Hugo Mills wrote:
>    I've just been poking around in the docs for a completely different
> reason, and I think there's a fairly serious problem (well, as serious
> as problems get with documentation).
> 
>    Take, for example, the format for btrfs fi resize:
> 
> 'resize' [devid:][+/-]<size>[gkm]|[devid:]max <path>::
> 
>    Now, this has just thrown away all of the useful markup which
> indicates the semantics of the command. The asciidoc renders all of
> that text literally and unformatted, making alphasymbolic(*) soup of
> the docs. Compare this to the old roff man page:
> 
> \fBbtrfs\fP \fBfilesystem resize\fP [\fIdevid\fP:][+/\-]\fI<size>\fP[gkm]|[\fIdevid\fP:]\fImax <path>\fP

I think we can restore the formatting with asciidoc.

The line above would become:

*btrfs* *filesystem resize* ['devid':][+/-]'size'[kgm]|[devid':]'max <path>'

or with bold max

*btrfs* *filesystem resize* ['devid':][+/-]'size'[kgm]|[devid':]*max* '<path>'

I was first worried that this will not be possible due to limitations of
asciidoc markup but as this turned out not be true, I'd rather spend the
boring time to keep the formatting as before.

My personal feeling about the enriched formatting is that the commands
stand out of the text and are easier to catch (as you've mentioned
somewhere in the thread).

Re: [PATCH 00/27] Replace the old man page with asciidoc and man page for each btrfs subcommand.

[David Sterba]

On Mon, May 19, 2014 at 04:01:23PM +0200, David Sterba wrote:
> On Sat, May 17, 2014 at 06:43:15PM +0100, Hugo Mills wrote:
> >    I've just been poking around in the docs for a completely different
> > reason, and I think there's a fairly serious problem (well, as serious
> > as problems get with documentation).
> > 
> >    Take, for example, the format for btrfs fi resize:
> > 
> > 'resize' [devid:][+/-]<size>[gkm]|[devid:]max <path>::
> > 
> >    Now, this has just thrown away all of the useful markup which
> > indicates the semantics of the command. The asciidoc renders all of
> > that text literally and unformatted, making alphasymbolic(*) soup of
> > the docs. Compare this to the old roff man page:
> > 
> > \fBbtrfs\fP \fBfilesystem resize\fP [\fIdevid\fP:][+/\-]\fI<size>\fP[gkm]|[\fIdevid\fP:]\fImax <path>\fP
> 
> I think we can restore the formatting with asciidoc.
> 
> The line above would become:
> 
> *btrfs* *filesystem resize* ['devid':][+/-]'size'[kgm]|[devid':]'max <path>'
> 
> or with bold max
> 
> *btrfs* *filesystem resize* ['devid':][+/-]'size'[kgm]|[devid':]*max* '<path>'

The correct base string should read

  btrfs filesystem resize [<devid>:][+/-]<size>[kgm]|[<devid>:]max <path>

ie. add <..> around devid and size. That way it's copy-paste-ready.
In this case the italic/underlined text does not IMO add much value.

> My personal feeling about the enriched formatting is that the commands
> stand out of the text and are easier to catch (as you've mentioned
> somewhere in the thread).

The bolded subcommand name seems to be sufficent.

The files are processed by XSL, I think it should be possible to apply
some transformation that would add '...' around <...> automatically
instead of making everybody write that.

Proposed changes:
- format all subcommands as bold instead of italic ('' -> **)
- add all missing <...>
- find a way how to add '...' around <...> (xsl or sed or whatever)

Does that work for you?

Re: [PATCH 00/27] Replace the old man page with asciidoc and man page for each btrfs subcommand.

[Qu Wenruo]

-------- Original Message --------
Subject: Re: [PATCH 00/27] Replace the old man page with asciidoc and 
man page for each btrfs subcommand.
From: David Sterba <dsterba@suse.cz>
To: Hugo Mills <hugo@carfax.org.uk>, Qu Wenruo 
<quwenruo@cn.fujitsu.com>, linux-btrfs@vger.kernel.org, clm@fb.com
Date: 2014年05月19日 22:33
> On Mon, May 19, 2014 at 04:01:23PM +0200, David Sterba wrote:
>> On Sat, May 17, 2014 at 06:43:15PM +0100, Hugo Mills wrote:
>>>     I've just been poking around in the docs for a completely different
>>> reason, and I think there's a fairly serious problem (well, as serious
>>> as problems get with documentation).
>>>
>>>     Take, for example, the format for btrfs fi resize:
>>>
>>> 'resize' [devid:][+/-]<size>[gkm]|[devid:]max <path>::
>>>
>>>     Now, this has just thrown away all of the useful markup which
>>> indicates the semantics of the command. The asciidoc renders all of
>>> that text literally and unformatted, making alphasymbolic(*) soup of
>>> the docs. Compare this to the old roff man page:
>>>
>>> \fBbtrfs\fP \fBfilesystem resize\fP [\fIdevid\fP:][+/\-]\fI<size>\fP[gkm]|[\fIdevid\fP:]\fImax <path>\fP
>> I think we can restore the formatting with asciidoc.
>>
>> The line above would become:
>>
>> *btrfs* *filesystem resize* ['devid':][+/-]'size'[kgm]|[devid':]'max <path>'
>>
>> or with bold max
>>
>> *btrfs* *filesystem resize* ['devid':][+/-]'size'[kgm]|[devid':]*max* '<path>'
> The correct base string should read
>
>    btrfs filesystem resize [<devid>:][+/-]<size>[kgm]|[<devid>:]max <path>
>
> ie. add <..> around devid and size. That way it's copy-paste-ready.
> In this case the italic/underlined text does not IMO add much value.
It is completely OK for me.
Since the base string is copy-paste-ready, it would add any extra effort 
to add other markup.
>> My personal feeling about the enriched formatting is that the commands
>> stand out of the text and are easier to catch (as you've mentioned
>> somewhere in the thread).
> The bolded subcommand name seems to be sufficent.
>
> The files are processed by XSL, I think it should be possible to apply
> some transformation that would add '...' around <...> automatically
> instead of making everybody write that.
>
> Proposed changes:
> - format all subcommands as bold instead of italic ('' -> **)
> - add all missing <...>
> - find a way how to add '...' around <...> (xsl or sed or whatever)
>
> Does that work for you?
That is OK for me, I'll investigate it.

Should I send a new patchset or just delta patches upon the current base?

Thanks,
Qu

Re: [PATCH 00/27] Replace the old man page with asciidoc and man page for each btrfs subcommand.

[David Sterba]

On Tue, May 20, 2014 at 08:34:19AM +0800, Qu Wenruo wrote:
> >Proposed changes:
> >- format all subcommands as bold instead of italic ('' -> **)
> >- add all missing <...>
> >- find a way how to add '...' around <...> (xsl or sed or whatever)
> >
> >Does that work for you?
> That is OK for me, I'll investigate it.
> 
> Should I send a new patchset or just delta patches upon the current base?

Don't bother, patches sent yesterday.

Re: [PATCH] btrfs-progs: doc: link btrfsck to btrfs-check

[WorMzy Tykashi]

On 12 May 2014 15:09, David Sterba <dsterba@suse.cz> wrote:
>
> Thanks, it was reported & fixed a few days ago, though it's not in the
> integration branch, lag is on my side.
>
> https://patchwork.kernel.org/patch/4115501/

Hi David,

I noticed this in another thread:

On 3 June 2014 10:14, David Sterba <dsterba@suse.cz> wrote:
> I've assembled a branch containing doc-only fixes, including this one,
> and asked Chris do do a 3.14.3 release.

Does this branch include this fix for this bug? According to
patchwork, it's status is still "under review", but it's been a month
and the bug slipped into the 3.14.2 release. I just want to make sure
it hasn't fallen out of the pile. :)

Cheers,


WorMzy

Re: [PATCH] btrfs-progs: doc: link btrfsck to btrfs-check

[David Sterba]

On Tue, Jun 03, 2014 at 10:38:16AM +0100, WorMzy Tykashi wrote:
> I noticed this in another thread:
> 
> On 3 June 2014 10:14, David Sterba <dsterba@suse.cz> wrote:
> > I've assembled a branch containing doc-only fixes, including this one,
> > and asked Chris do do a 3.14.3 release.
> 
> Does this branch include this fix for this bug? According to
> patchwork, it's status is still "under review", but it's been a month
> and the bug slipped into the 3.14.2 release. I just want to make sure
> it hasn't fallen out of the pile. :)

Yes the branch contains this fix as well,

http://repo.or.cz/w/btrfs-progs-unstable/devel.git/shortlog/refs/heads/v3.14.x

and it almost fell off, I had forgotten to tag it in my mailbox and a
filter did not pick it.

About patchwork, I'm still using mailbox and git for tracking patches
and sync the patchwork status manually on occasion. This is not ideal
and patchwork status is not accurate. I'm aiming for a semi-automated
sync from git/mbox -> patchwork when the patches change their status
regarding stability and upstream readiness.

btrfs balance crash BUG ON fs/btrfs/relocation.c:1062 or RIP build_backref_tree+0x9fc/0xcc4

[Marc MERLIN]

I did a balance on a system that had 3.11 (yes, I know, it's old). It hung.
So, I rebooted with 3.13, and it failed in fs/btrfs/relocation

Problem #1: I cannot stop the relocation. It starts on its own as soon as I
mount the FS, and I can't stop it.
Is there a bug to fix that?

Problem #2: I rebooted with 3.15rc5, and now it's worse.
[ 1569.598026] kernel BUG at fs/btrfs/relocation.c:1064!
then leads to
[ 1569.613240] RIP  [<ffffffff81267ef0>] build_backref_tree+0x9fc/0xcc4
[ 1569.613491]  RSP <ffff880fde599ad8>
[ 1569.614119] ---[ end trace da0f24875bbde960 ]---
[ 1569.614398] Kernel panic - not syncing: Fatal exception
(full trace below)

I'm sure that filesystem is damaged in some way, but the kernel of course
should not crash.

3.15 dies here:
struct backref_node *build_backref_tree(struct reloc_control *rc,
(...)
		if (!RB_EMPTY_NODE(&upper->rb_node)) {
			if (upper->lowest) {
				list_del_init(&upper->lower);
				upper->lowest = 0;
			}

			list_add_tail(&edge->list[UPPER], &upper->lower);
			continue;
		}

		BUG_ON(!upper->checked); <<<< here

So I'm sure I hit a bug and my FS has issues, but can't the kernel do something better
like abort the rebalance instead of crashing?

In the meantime, does anyone want anything off that filesystem before I wipe it?


=> Crash on 3.13:
btrfs: found 4930 extents
btrfs: relocating block group 82699091968 flags 1
btrfs: found 3719 extents
------------[ cut here ]------------
kernel BUG at /build/buildd/linux-lts-trusty-3.13.0/fs/btrfs/relocation.c:1062!
invalid opcode: 0000 [#1] SMP 
Modules linked in: rfcomm parport_pc ppdev bnep binfmt_misc rpcsec_gss_krb5 nfsd nfs_acl auth_rpcgss nfs fscache lockd sunrpc snd_hda_codec_hdmi nvidia(POF) snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_usb_audio snd_pcm snd_hwdep snd_usbmidi_lib snd_seq_midi snd_seq_midi_event snd_seq btusb bluetooth uvcvideo videobuf2_core videodev videobuf2_vmalloc videobuf2_memops snd_rawmidi snd_timer snd_seq_device drm snd psmouse gpio_ich sb_edac hp_wmi serio_raw edac_core mei_me mei mac_hid soundcore sparse_keymap snd_page_alloc lpc_ich wmi tpm_infineon lp parport btrfs raid6_pq xor libcrc32c hid_generic usbhid hid usb_storage firewire_ohci isci e1000e firewire_core libsas crc_itu_t ptp ahci libahci pps_core scsi_transport_sas
CPU: 4 PID: 1710 Comm: btrfs-balance Tainted: PF          O 3.13.0-29-generic #53~precise1-Ubuntu
Hardware name: Hewlett-Packard HP Z620 Workstation/158A, BIOS J61 v01.17 11/05/2012
task: ffff881000bec7d0 ti: ffff8810018a2000 task.ti: ffff8810018a2000
RIP: 0010:[<ffffffffa01c04a8>]  [<ffffffffa01c04a8>] build_backref_tree+0x1228/0x1290 [btrfs]
RSP: 0018:ffff8810018a3ab8  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff880801a8fa20 RCX: ffff880801d2cf50
RDX: ffff880801d2c390 RSI: ffff880801d2c640 RDI: ffff8807ecad9c80
RBP: ffff8810018a3bb8 R08: ffff8807ecad9c80 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000000 R12: ffff880801d2c650
R13: ffff8807ecad9900 R14: 0000000000000000 R15: ffff88003582a800
FS:  0000000000000000(0000) GS:ffff88080fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f10ee50a370 CR3: 0000000001c0d000 CR4: 00000000000407e0
Stack:
 ffff8807ecad9780 01ffffffa01bded0 ffff880801d2c620 ffff88003582a920
 ffff8807ecad9c80 ffff8807ff72e800 ffff8807ecad9240 ffff8807ecad9200
 ffff880801a8fa20 ffff880801a8fab0 ffff88003582a924 ffff88003582a820
Call Trace:
 [<ffffffffa01c064b>] relocate_tree_blocks+0x13b/0x1e0 [btrfs]
 [<ffffffffa01c12b9>] relocate_block_group+0x199/0x550 [btrfs]
 [<ffffffffa01c182a>] btrfs_relocate_block_group+0x1ba/0x300 [btrfs]
 [<ffffffffa01999f6>] btrfs_relocate_chunk.isra.62+0x56/0x3f0 [btrfs]
 [<ffffffffa01556b3>] ? block_group_cache_tree_search+0xb3/0xf0 [btrfs]
 [<ffffffffa018dfe6>] ? release_extent_buffer+0x36/0xe0 [btrfs]
 [<ffffffffa019c60c>] __btrfs_balance+0x32c/0x420 [btrfs]
 [<ffffffffa019ca38>] btrfs_balance+0x338/0x5d0 [btrfs]
 [<ffffffffa019cd54>] balance_kthread+0x84/0x90 [btrfs]
 [<ffffffffa019ccd0>] ? btrfs_balance+0x5d0/0x5d0 [btrfs]
 [<ffffffff8108f9a9>] kthread+0xc9/0xe0
 [<ffffffff8108f8e0>] ? flush_kthread_worker+0xb0/0xb0
 [<ffffffff817665fc>] ret_from_fork+0x7c/0xb0
 [<ffffffff8108f8e0>] ? flush_kthread_worker+0xb0/0xb0
Code: ff ff 48 89 df e8 79 cb f8 ff 48 8b bd 48 ff ff ff e8 6d cb f8 ff 48 83 bd 58 ff ff ff 00 0f 84 62 ef ff ff e9 87 fd ff ff 0f 0b <0f> 0b 48 8b 85 20 ff ff ff 49 8d 7f 20 48 8b 70 18 48 89 c2 e8 
RIP  [<ffffffffa01c04a8>] build_backref_tree+0x1228/0x1290 [btrfs]
 RSP <ffff8810018a3ab8>
---[ end trace 1b7853634ea4bd18 ]---


=> Crash on 3.15:
[ 1565.477358] BTRFS info (device sdb1): disk space caching is enabled
[ 1565.567580] BTRFS: detected SSD devices, enabling SSD mode
[ 1565.739226] BTRFS info (device sdb1): continuing balance
[ 1565.790228] BTRFS info (device sdb1): relocating block group 82699091968 flags 1
[ 1567.768219] BTRFS info (device sdb1): found 3719 extents
[ 1569.597766] ------------[ cut here ]------------
[ 1569.598026] kernel BUG at fs/btrfs/relocation.c:1064!
[ 1569.598269] invalid opcode: 0000 [#1] PREEMPT SMP 
[ 1569.598528] Modules linked in: des_generic nfsv3 nfsv4 fuse autofs4 bnep rfcomm parport_pc ppdev binfmt_misc uvcvideo videobuf2_core videodev media videobuf2_vmalloc videobuf2_memops snd_usb_audio snd_usbmidi_lib ecb btusb bluetooth 6lowpan_iphc snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_controller snd_hda_codec intel_rapl x86_pkg_temp_thermal snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm intel_powerclamp coretemp rpcsec_gss_krb5 nfsd kvm snd_seq_midi snd_rawmidi snd_seq_midi_event nfs_acl auth_rpcgss snd_seq snd_timer snd_seq_device nfs ehci_pci ehci_hcd hp_wmi crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel tpm_infineon snd sparse_keymap rfkill sb_edac psmouse evdev fscache soundcore lockd edac_core serio_raw lpc_ich wmi aesni_intel processor ablk_helper cryptd lrw gf128mul sunrpc tpm_tis glue_helper tpm aes_x86_64 microcode lp parport loop hid_generic usbhid hid uas usb_storage dm_mod firewire_ohci xhci_hcd firewire_core crc_itu_t usbcore e1000e usb_common isci ptp pps_core libsas scsi_transport_sas
[ 1569.602494] CPU: 4 PID: 6244 Comm: btrfs-balance Not tainted 3.15.0-rc5-amd64-i915-preempt-20140216s2 #1
[ 1569.602963] Hardware name: Hewlett-Packard HP Z620 Workstation/158A, BIOS J61 v01.17 11/05/2012
[ 1569.603433] task: ffff880fde596210 ti: ffff880fde598000 task.ti: ffff880fde598000 [ 1569.603898] RIP: 0010:[<ffffffff81267ef0>]  [<ffffffff81267ef0>] build_backref_tree+0x9fc/0xcc4
[ 1569.604382] RSP: 0018:ffff880fde599ad8  EFLAGS: 00010246
[ 1569.604622] RAX: ffff880fde599b00 RBX: ffff880806836c10 RCX: ffff8807d8ea74d0
[ 1569.604867] RDX: ffff8808060d9750 RSI: ffff880fde599b58 RDI: ffff8807d8f51e90
[ 1569.605109] RBP: ffff880fde599bb8 R08: ffff88080682c940 R09: 0000000000001000
[ 1569.605352] R10: 0000160000000000 R11: 6db6db6db6db6db7 R12: ffff8807d8f51e90
[ 1569.605598] R13: ffff880fde599b68 R14: ffff880806836bc0 R15: ffff880805f64800
[ 1569.605842] FS:  0000000000000000(0000) GS:ffff88082fc80000(0000) knlGS:0000000000000000
[ 1569.606310] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1569.606550] CR2: 00007fa936051000 CR3: 0000000001c11000 CR4: 00000000000407e0
[ 1569.606791] Stack:
[ 1569.607025]  ffff8807d8ebc940 ffff8807d8f51820 ffff8807d8ebc940 ffff880807907d00
[ 1569.607512]  01ff881004811580 ffff880fde599b68 ffff880805f64920 0000000000000000
[ 1569.608001]  ffff880f00000005 ffff8807d9a78f50 ffff880fde599b58 0000000000000000
[ 1569.608487] Call Trace:
[ 1569.608728]  [<ffffffff81269bd2>] relocate_tree_blocks+0x16a/0x44c
[ 1569.608972]  [<ffffffff8126aa03>] relocate_block_group+0x239/0x49a
[ 1569.609217]  [<ffffffff8126adbf>] btrfs_relocate_block_group+0x15b/0x26d
[ 1569.609465]  [<ffffffff81249838>] btrfs_relocate_chunk.isra.23+0x5c/0x5e8
[ 1569.609711]  [<ffffffff8161efbb>] ? _raw_spin_unlock+0x17/0x2a
[ 1569.609955]  [<ffffffff81245584>] ? free_extent_buffer+0x8a/0x8d
[ 1569.610200]  [<ffffffff8124c0be>] btrfs_balance+0x9b6/0xb74
[ 1569.610442]  [<ffffffff81615c3d>] ? printk+0x54/0x56
[ 1569.610684]  [<ffffffff8124c27c>] ? btrfs_balance+0xb74/0xb74
[ 1569.610928]  [<ffffffff8124c2d5>] balance_kthread+0x59/0x7b
[ 1569.611173]  [<ffffffff8106b467>] kthread+0xae/0xb6
[ 1569.611413]  [<ffffffff8106b3b9>] ? __kthread_parkme+0x61/0x61
[ 1569.611664]  [<ffffffff81625b3c>] ret_from_fork+0x7c/0xb0
[ 1569.611905]  [<ffffffff8106b3b9>] ? __kthread_parkme+0x61/0x61
[ 1569.612149] Code: 0d 48 89 df e8 f4 e7 ff ff 41 80 66 71 fd 49 8b 46 58
4d 89 66 58 49 89 1c 24 49 89 44 24 08 4c 89 20 e9 80 00 00 00 a8 10 75 02
<0f> 0b 83 e0 01 39 85 78 ff ff ff 74 02 0f 0b 83 bd 78 ff ff ff 
[ 1569.613240] RIP  [<ffffffff81267ef0>] build_backref_tree+0x9fc/0xcc4
[ 1569.613491]  RSP <ffff880fde599ad8>
[ 1569.614119] ---[ end trace da0f24875bbde960 ]---
[ 1569.614398] Kernel panic - not syncing: Fatal exception
[ 1569.614738] Kernel Offset: 0x0 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffff9fffffff)
[ 1569.615245] ---[ end Kernel panic - not syncing: Fatal exception


-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: btrfs balance crash BUG ON fs/btrfs/relocation.c:1062 or RIP build_backref_tree+0x9fc/0xcc4

[Russell Coker]

On Mon, 9 Jun 2014 16:40:07 Marc MERLIN wrote:
> I did a balance on a system that had 3.11 (yes, I know, it's old). It hung.
> So, I rebooted with 3.13, and it failed in fs/btrfs/relocation
> 
> Problem #1: I cannot stop the relocation. It starts on its own as soon as I
> mount the FS, and I can't stop it.
> Is there a bug to fix that?

The skip_balance mount option prevents a balance from being continued.  On all 
my systems I have rootflags=skip_balance in the GRUB configuration because so 
far I have never had a system reboot during a balance for any reason other 
than a system crash caused by the balance.

I think it would be good to have a program comparable to tune2fs that allows 
us to set the state of such things.  It would be a lot easier than modifying 
boot loader configuration on lots of systems.

Also rootflags=skip_balance will abort the boot if you run kernel 3.2.0 and 
that caused me some annoyance when I applied it to all systems including the 
one with a stock Debian/Wheezy configuration.

-- 
My Main Blog         http://etbe.coker.com.au/
My Documents Blog    http://doc.coker.com.au/

Re: btrfs balance crash BUG ON fs/btrfs/relocation.c:1062 or RIP build_backref_tree+0x9fc/0xcc4

[Marc MERLIN]

On Tue, Jun 10, 2014 at 10:32:33AM +1000, Russell Coker wrote:
> On Mon, 9 Jun 2014 16:40:07 Marc MERLIN wrote:
> > I did a balance on a system that had 3.11 (yes, I know, it's old). It hung.
> > So, I rebooted with 3.13, and it failed in fs/btrfs/relocation
> > 
> > Problem #1: I cannot stop the relocation. It starts on its own as soon as I
> > mount the FS, and I can't stop it.
> > Is there a bug to fix that?
> 
> The skip_balance mount option prevents a balance from being continued.  On all 
> my systems I have rootflags=skip_balance in the GRUB configuration because so 
> far I have never had a system reboot during a balance for any reason other 
> than a system crash caused by the balance.
 
Aaah, good idea, thank you.

> I think it would be good to have a program comparable to tune2fs that allows 
> us to set the state of such things.  It would be a lot easier than modifying 
> boot loader configuration on lots of systems.

Totally agreed. 

I'll still wait a bit to see if someone wants me to provide more info
from the FS, but having balance crash the system and hard lock it, is
indeed not good.

Thanks,
Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/                         | PGP 1024R/763BE901

Re: btrfs balance crash BUG ON fs/btrfs/relocation.c:1062 or RIP build_backref_tree+0x9fc/0xcc4

[Marc MERLIN]

On Mon, Jun 09, 2014 at 04:40:07PM -0700, Marc MERLIN wrote:
> I did a balance on a system that had 3.11 (yes, I know, it's old). It hung.
> So, I rebooted with 3.13, and it failed in fs/btrfs/relocation
> 
> Problem #1: I cannot stop the relocation. It starts on its own as soon as I
> mount the FS, and I can't stop it.
> Is there a bug to fix that?

Chris,

As Russell pointed out, I can stop the balance at mount, but back to the
bug, can you make btrfs not crash like this if it gets input it doesn't
expect?

Separately, I'll send one lsat offer to give data off the FS to give you
a chance to figure out how it got into that state and potentially find
another bug.
That is unless there were one or more known corruption bugs that have
been fixed and that there is nothing to be learned from looking into
this.

Thanks,
Marc
 
> Problem #2: I rebooted with 3.15rc5, and now it's worse.
> [ 1569.598026] kernel BUG at fs/btrfs/relocation.c:1064!
> then leads to
> [ 1569.613240] RIP  [<ffffffff81267ef0>] build_backref_tree+0x9fc/0xcc4
> [ 1569.613491]  RSP <ffff880fde599ad8>
> [ 1569.614119] ---[ end trace da0f24875bbde960 ]---
> [ 1569.614398] Kernel panic - not syncing: Fatal exception
> (full trace below)
> 
> I'm sure that filesystem is damaged in some way, but the kernel of course
> should not crash.
> 
> 3.15 dies here:
> struct backref_node *build_backref_tree(struct reloc_control *rc,
> (...)
> 		if (!RB_EMPTY_NODE(&upper->rb_node)) {
> 			if (upper->lowest) {
> 				list_del_init(&upper->lower);
> 				upper->lowest = 0;
> 			}
> 
> 			list_add_tail(&edge->list[UPPER], &upper->lower);
> 			continue;
> 		}
> 
> 		BUG_ON(!upper->checked); <<<< here
> 
> So I'm sure I hit a bug and my FS has issues, but can't the kernel do something better
> like abort the rebalance instead of crashing?
> 
> In the meantime, does anyone want anything off that filesystem before I wipe it?
> 
> 
> => Crash on 3.13:
> btrfs: found 4930 extents
> btrfs: relocating block group 82699091968 flags 1
> btrfs: found 3719 extents
> ------------[ cut here ]------------
> kernel BUG at /build/buildd/linux-lts-trusty-3.13.0/fs/btrfs/relocation.c:1062!
> invalid opcode: 0000 [#1] SMP 
> Modules linked in: rfcomm parport_pc ppdev bnep binfmt_misc rpcsec_gss_krb5 nfsd nfs_acl auth_rpcgss nfs fscache lockd sunrpc snd_hda_codec_hdmi nvidia(POF) snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_usb_audio snd_pcm snd_hwdep snd_usbmidi_lib snd_seq_midi snd_seq_midi_event snd_seq btusb bluetooth uvcvideo videobuf2_core videodev videobuf2_vmalloc videobuf2_memops snd_rawmidi snd_timer snd_seq_device drm snd psmouse gpio_ich sb_edac hp_wmi serio_raw edac_core mei_me mei mac_hid soundcore sparse_keymap snd_page_alloc lpc_ich wmi tpm_infineon lp parport btrfs raid6_pq xor libcrc32c hid_generic usbhid hid usb_storage firewire_ohci isci e1000e firewire_core libsas crc_itu_t ptp ahci libahci pps_core scsi_transport_sas
> CPU: 4 PID: 1710 Comm: btrfs-balance Tainted: PF          O 3.13.0-29-generic #53~precise1-Ubuntu
> Hardware name: Hewlett-Packard HP Z620 Workstation/158A, BIOS J61 v01.17 11/05/2012
> task: ffff881000bec7d0 ti: ffff8810018a2000 task.ti: ffff8810018a2000
> RIP: 0010:[<ffffffffa01c04a8>]  [<ffffffffa01c04a8>] build_backref_tree+0x1228/0x1290 [btrfs]
> RSP: 0018:ffff8810018a3ab8  EFLAGS: 00010246
> RAX: 0000000000000000 RBX: ffff880801a8fa20 RCX: ffff880801d2cf50
> RDX: ffff880801d2c390 RSI: ffff880801d2c640 RDI: ffff8807ecad9c80
> RBP: ffff8810018a3bb8 R08: ffff8807ecad9c80 R09: 0000000000000001
> R10: 0000000000000001 R11: 0000000000000000 R12: ffff880801d2c650
> R13: ffff8807ecad9900 R14: 0000000000000000 R15: ffff88003582a800
> FS:  0000000000000000(0000) GS:ffff88080fc80000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00007f10ee50a370 CR3: 0000000001c0d000 CR4: 00000000000407e0
> Stack:
>  ffff8807ecad9780 01ffffffa01bded0 ffff880801d2c620 ffff88003582a920
>  ffff8807ecad9c80 ffff8807ff72e800 ffff8807ecad9240 ffff8807ecad9200
>  ffff880801a8fa20 ffff880801a8fab0 ffff88003582a924 ffff88003582a820
> Call Trace:
>  [<ffffffffa01c064b>] relocate_tree_blocks+0x13b/0x1e0 [btrfs]
>  [<ffffffffa01c12b9>] relocate_block_group+0x199/0x550 [btrfs]
>  [<ffffffffa01c182a>] btrfs_relocate_block_group+0x1ba/0x300 [btrfs]
>  [<ffffffffa01999f6>] btrfs_relocate_chunk.isra.62+0x56/0x3f0 [btrfs]
>  [<ffffffffa01556b3>] ? block_group_cache_tree_search+0xb3/0xf0 [btrfs]
>  [<ffffffffa018dfe6>] ? release_extent_buffer+0x36/0xe0 [btrfs]
>  [<ffffffffa019c60c>] __btrfs_balance+0x32c/0x420 [btrfs]
>  [<ffffffffa019ca38>] btrfs_balance+0x338/0x5d0 [btrfs]
>  [<ffffffffa019cd54>] balance_kthread+0x84/0x90 [btrfs]
>  [<ffffffffa019ccd0>] ? btrfs_balance+0x5d0/0x5d0 [btrfs]
>  [<ffffffff8108f9a9>] kthread+0xc9/0xe0
>  [<ffffffff8108f8e0>] ? flush_kthread_worker+0xb0/0xb0
>  [<ffffffff817665fc>] ret_from_fork+0x7c/0xb0
>  [<ffffffff8108f8e0>] ? flush_kthread_worker+0xb0/0xb0
> Code: ff ff 48 89 df e8 79 cb f8 ff 48 8b bd 48 ff ff ff e8 6d cb f8 ff 48 83 bd 58 ff ff ff 00 0f 84 62 ef ff ff e9 87 fd ff ff 0f 0b <0f> 0b 48 8b 85 20 ff ff ff 49 8d 7f 20 48 8b 70 18 48 89 c2 e8 
> RIP  [<ffffffffa01c04a8>] build_backref_tree+0x1228/0x1290 [btrfs]
>  RSP <ffff8810018a3ab8>
> ---[ end trace 1b7853634ea4bd18 ]---
> 
> 
> => Crash on 3.15:
> [ 1565.477358] BTRFS info (device sdb1): disk space caching is enabled
> [ 1565.567580] BTRFS: detected SSD devices, enabling SSD mode
> [ 1565.739226] BTRFS info (device sdb1): continuing balance
> [ 1565.790228] BTRFS info (device sdb1): relocating block group 82699091968 flags 1
> [ 1567.768219] BTRFS info (device sdb1): found 3719 extents
> [ 1569.597766] ------------[ cut here ]------------
> [ 1569.598026] kernel BUG at fs/btrfs/relocation.c:1064!
> [ 1569.598269] invalid opcode: 0000 [#1] PREEMPT SMP 
> [ 1569.598528] Modules linked in: des_generic nfsv3 nfsv4 fuse autofs4 bnep rfcomm parport_pc ppdev binfmt_misc uvcvideo videobuf2_core videodev media videobuf2_vmalloc videobuf2_memops snd_usb_audio snd_usbmidi_lib ecb btusb bluetooth 6lowpan_iphc snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_controller snd_hda_codec intel_rapl x86_pkg_temp_thermal snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm intel_powerclamp coretemp rpcsec_gss_krb5 nfsd kvm snd_seq_midi snd_rawmidi snd_seq_midi_event nfs_acl auth_rpcgss snd_seq snd_timer snd_seq_device nfs ehci_pci ehci_hcd hp_wmi crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel tpm_infineon snd sparse_keymap rfkill sb_edac psmouse evdev fscache soundcore lockd edac_core serio_raw lpc_ich wmi aesni_intel processor ablk_helper cryptd lrw gf128mul sunrpc tpm_tis glue_helper tpm aes_x86_64 microcode lp parport loop hid_generic usbhid hid uas usb_storage dm_mod firewire_ohci xhci_hcd firewire_core crc_itu_t usbcore e1000e usb_common isci ptp pps_core libsas scsi_transport_sas
> [ 1569.602494] CPU: 4 PID: 6244 Comm: btrfs-balance Not tainted 3.15.0-rc5-amd64-i915-preempt-20140216s2 #1
> [ 1569.602963] Hardware name: Hewlett-Packard HP Z620 Workstation/158A, BIOS J61 v01.17 11/05/2012
> [ 1569.603433] task: ffff880fde596210 ti: ffff880fde598000 task.ti: ffff880fde598000 [ 1569.603898] RIP: 0010:[<ffffffff81267ef0>]  [<ffffffff81267ef0>] build_backref_tree+0x9fc/0xcc4
> [ 1569.604382] RSP: 0018:ffff880fde599ad8  EFLAGS: 00010246
> [ 1569.604622] RAX: ffff880fde599b00 RBX: ffff880806836c10 RCX: ffff8807d8ea74d0
> [ 1569.604867] RDX: ffff8808060d9750 RSI: ffff880fde599b58 RDI: ffff8807d8f51e90
> [ 1569.605109] RBP: ffff880fde599bb8 R08: ffff88080682c940 R09: 0000000000001000
> [ 1569.605352] R10: 0000160000000000 R11: 6db6db6db6db6db7 R12: ffff8807d8f51e90
> [ 1569.605598] R13: ffff880fde599b68 R14: ffff880806836bc0 R15: ffff880805f64800
> [ 1569.605842] FS:  0000000000000000(0000) GS:ffff88082fc80000(0000) knlGS:0000000000000000
> [ 1569.606310] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 1569.606550] CR2: 00007fa936051000 CR3: 0000000001c11000 CR4: 00000000000407e0
> [ 1569.606791] Stack:
> [ 1569.607025]  ffff8807d8ebc940 ffff8807d8f51820 ffff8807d8ebc940 ffff880807907d00
> [ 1569.607512]  01ff881004811580 ffff880fde599b68 ffff880805f64920 0000000000000000
> [ 1569.608001]  ffff880f00000005 ffff8807d9a78f50 ffff880fde599b58 0000000000000000
> [ 1569.608487] Call Trace:
> [ 1569.608728]  [<ffffffff81269bd2>] relocate_tree_blocks+0x16a/0x44c
> [ 1569.608972]  [<ffffffff8126aa03>] relocate_block_group+0x239/0x49a
> [ 1569.609217]  [<ffffffff8126adbf>] btrfs_relocate_block_group+0x15b/0x26d
> [ 1569.609465]  [<ffffffff81249838>] btrfs_relocate_chunk.isra.23+0x5c/0x5e8
> [ 1569.609711]  [<ffffffff8161efbb>] ? _raw_spin_unlock+0x17/0x2a
> [ 1569.609955]  [<ffffffff81245584>] ? free_extent_buffer+0x8a/0x8d
> [ 1569.610200]  [<ffffffff8124c0be>] btrfs_balance+0x9b6/0xb74
> [ 1569.610442]  [<ffffffff81615c3d>] ? printk+0x54/0x56
> [ 1569.610684]  [<ffffffff8124c27c>] ? btrfs_balance+0xb74/0xb74
> [ 1569.610928]  [<ffffffff8124c2d5>] balance_kthread+0x59/0x7b
> [ 1569.611173]  [<ffffffff8106b467>] kthread+0xae/0xb6
> [ 1569.611413]  [<ffffffff8106b3b9>] ? __kthread_parkme+0x61/0x61
> [ 1569.611664]  [<ffffffff81625b3c>] ret_from_fork+0x7c/0xb0
> [ 1569.611905]  [<ffffffff8106b3b9>] ? __kthread_parkme+0x61/0x61
> [ 1569.612149] Code: 0d 48 89 df e8 f4 e7 ff ff 41 80 66 71 fd 49 8b 46 58
> 4d 89 66 58 49 89 1c 24 49 89 44 24 08 4c 89 20 e9 80 00 00 00 a8 10 75 02
> <0f> 0b 83 e0 01 39 85 78 ff ff ff 74 02 0f 0b 83 bd 78 ff ff ff 
> [ 1569.613240] RIP  [<ffffffff81267ef0>] build_backref_tree+0x9fc/0xcc4
> [ 1569.613491]  RSP <ffff880fde599ad8>
> [ 1569.614119] ---[ end trace da0f24875bbde960 ]---
> [ 1569.614398] Kernel panic - not syncing: Fatal exception
> [ 1569.614738] Kernel Offset: 0x0 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffff9fffffff)
> [ 1569.615245] ---[ end Kernel panic - not syncing: Fatal exception
> 
> 
> -- 
> "A mouse is a device used to point at the xterm you want to type in" - A.S.R.
> Microsoft is to operating systems ....
>                                       .... what McDonalds is to gourmet cooking
> Home page: http://marc.merlins.org/  
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/                         | PGP 1024R/763BE901

Re: btrfs balance crash BUG ON fs/btrfs/relocation.c:1062 or RIP build_backref_tree+0x9fc/0xcc4

[Josef Bacik]

On 06/09/2014 04:40 PM, Marc MERLIN wrote:
> I did a balance on a system that had 3.11 (yes, I know, it's old). It hung.
> So, I rebooted with 3.13, and it failed in fs/btrfs/relocation
>
> Problem #1: I cannot stop the relocation. It starts on its own as soon as I
> mount the FS, and I can't stop it.
> Is there a bug to fix that?
>
> Problem #2: I rebooted with 3.15rc5, and now it's worse.
> [ 1569.598026] kernel BUG at fs/btrfs/relocation.c:1064!
> then leads to
> [ 1569.613240] RIP  [<ffffffff81267ef0>] build_backref_tree+0x9fc/0xcc4
> [ 1569.613491]  RSP <ffff880fde599ad8>
> [ 1569.614119] ---[ end trace da0f24875bbde960 ]---
> [ 1569.614398] Kernel panic - not syncing: Fatal exception
> (full trace below)
>
> I'm sure that filesystem is damaged in some way, but the kernel of course
> should not crash.
>

I don't have this mail client setup right to send patches, can you just 
go into relocation.c in build_backref_tree and remove the need_check = 
false; statement and see if that fixes it.  Thanks,

Josef

Re: btrfs balance crash BUG ON fs/btrfs/relocation.c:1062 or RIP build_backref_tree+0x9fc/0xcc4

[Marc MERLIN]

On Tue, Jun 17, 2014 at 11:29:57AM -0700, Josef Bacik wrote:
> >I'm sure that filesystem is damaged in some way, but the kernel of course
> >should not crash.
> >
> 
> I don't have this mail client setup right to send patches, can you just 
> go into relocation.c in build_backref_tree and remove the need_check = 
> false; statement and see if that fixes it.  Thanks,

Sure, I'll try that tomorrow when I have access to that machine.

Just to be clear
1) You mean this one, right?
				if (!upper->checked && need_check) {
					need_check = false;   <-- delete this
					list_add_tail(&edge->list[UPPER],
						      &list);
				} else

2) is your guess that the BUG_ON I'm getting shouldn't be triggered
and your proposed patch could fix that, or that I do have an FS problem 
but that we're just going to make the kernel more lenient and not crash on it?

Thanks,
Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: btrfs balance crash BUG ON fs/btrfs/relocation.c:1062 or RIP build_backref_tree+0x9fc/0xcc4

[Josef Bacik]

On 06/17/2014 11:55 AM, Marc MERLIN wrote:
> On Tue, Jun 17, 2014 at 11:29:57AM -0700, Josef Bacik wrote:
>>> I'm sure that filesystem is damaged in some way, but the kernel of course
>>> should not crash.
>>>
>>
>> I don't have this mail client setup right to send patches, can you just
>> go into relocation.c in build_backref_tree and remove the need_check =
>> false; statement and see if that fixes it.  Thanks,
>
> Sure, I'll try that tomorrow when I have access to that machine.
>
> Just to be clear
> 1) You mean this one, right?
> 				if (!upper->checked && need_check) {
> 					need_check = false;   <-- delete this
> 					list_add_tail(&edge->list[UPPER],
> 						      &list);
> 				} else
>

Yup that one.

> 2) is your guess that the BUG_ON I'm getting shouldn't be triggered
> and your proposed patch could fix that, or that I do have an FS problem
> but that we're just going to make the kernel more lenient and not crash on it?
>

All of these BUG_ON(!uppper->checked) are logic errors, not problems 
with the fs, which is why I haven't turned them into an abort or 
something else yet.  I'm thinking about chucking this whole function 
anyway and using the backref walking code, but that is going to be a job 
for future josef, present josef doesn't seem to have any time to do 
anything despite sleeping a lot less.  Thanks,

Josef

Re: btrfs balance crash BUG ON fs/btrfs/relocation.c:1062 or RIP build_backref_tree+0x9fc/0xcc4

[Marc MERLIN]

First thanks for your answer and patch. While they didn't help, I'm happy to
try another one or two if you'd like before I wipe the FS to recover.

On Wed, Jun 18, 2014 at 08:26:46AM -0700, Josef Bacik wrote:
> >2) is your guess that the BUG_ON I'm getting shouldn't be triggered
> >and your proposed patch could fix that, or that I do have an FS problem
> >but that we're just going to make the kernel more lenient and not crash on 
> >it?
> >
> 
> All of these BUG_ON(!uppper->checked) are logic errors, not problems 
> with the fs, which is why I haven't turned them into an abort or 

I'm guessing you saw the comment from someone else in the other thread. 
BUG_ON is not helpful when you have end users. It effectively crashes the
system when it can be unnecessary (a very secondary filesystem you don't
depend on), and it tells users that anything wrong with btrfs on any
filesystem can take their entire machine down needlessly
(not counting the fact of course that it makes bug reporting on the list
that much harder since syslog has no chance to capture and forward it)

Can I appeal to you to remove all the ones you work with?
If something is wrong, it should mount the FS read only and/or make it
unaccessible, but not crash the entire system.

> something else yet.  I'm thinking about chucking this whole function 
> anyway and using the backref walking code, but that is going to be a job 
> for future josef, present josef doesn't seem to have any time to do 
> anything despite sleeping a lot less.  Thanks,

Yes, I understand that part. Good luck with getting sleep indeed.

Back to your patch, unfortunately, it did not help.
Let me know if you'd like me to try one more patch or two before I wipe the
FS and start over.
But for my own sanity, can you explain if I'm hitting reallocate bugs, or
whether my filesystem is indeed correct?

Thanks,
Marc


It crashed on another BUG_ON
	if (!list_empty(&cur->upper)) {
		/*
		 * the backref was added previously when processing
		 * backref of type BTRFS_TREE_BLOCK_REF_KEY
		 */
		BUG_ON(!list_is_singular(&cur->upper));
		edge = list_entry(cur->upper.next, struct backref_edge,
				  list[LOWER]);
HERE ->		BUG_ON(!list_empty(&edge->list[UPPER]));
		exist = edge->node[UPPER];
		/*
		 * add the upper level block to pending list if we need
		 * check its backrefs
		 */
		if (!exist->checked)


[  209.223856] BTRFS info (device sdb1): disk space caching is enabled
[  209.283364] BTRFS: detected SSD devices, enabling SSD mode
[  209.369267] BTRFS: checking UUID tree
[  209.369286] BTRFS info (device sdb1): continuing balance
[  209.412421] BTRFS info (device sdb1): relocating block group 82699091968 flags 1
[  211.148796] BTRFS info (device sdb1): found 3719 extents
[  213.351917] ------------[ cut here ]------------
[  213.351941] kernel BUG at fs/btrfs/relocation.c:752!
[  213.351951] invalid opcode: 0000 [#1] PREEMPT SMP 
[  213.351974] Modules linked in: des_generic nfsv3 nfsv4 xt_NFLOG nfnetlink_log nfnetlink xt_tcpudp xt_comment xt_multiport ip6table_filter ip6_tables iptable_filter ip_tables x_tables fuse autofs4 bnep rfcomm parport_pc ppdev binfmt_misc uvcvideo videobuf2_core videodev snd_usb_audio media snd_usbmidi_lib videobuf2_vmalloc videobuf2_memops hid_generic usbhid hid ecb btusb bluetooth rpcsec_gss_krb5 6lowpan_iphc nfsd nfs_acl auth_rpcgss nfs fscache snd_hda_codec_hdmi lockd sunrpc snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_controller snd_hda_codec intel_rapl x86_pkg_temp_thermal intel_powerclamp snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_midi coretemp snd_rawmidi snd_seq_midi_event kvm snd_seq snd_timer crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel snd_seq_device snd aesni_intel sb_edac edac_core soundcore ehci_pci tpm_infineon ablk_helper cryptd lrw ehci_hcd gf128mul glue_helper hp_wmi sparse_keymap rfkill aes_x86_64 tpm_tis lpc_ich psmouse serio_raw tpm evdev microcode processor wmi lp parport loop uas usb_storage dm_mod firewire_ohci xhci_hcd firewire_core crc_itu_t usbcore isci usb_common e1000e libsas ptp pps_core scsi_transport_sas
[  213.352787] CPU: 4 PID: 13538 Comm: btrfs-balance Not tainted 3.15.1-amd64-i915-preempt-20140216jbp #1
[  213.352795] Hardware name: Hewlett-Packard HP Z620 Workstation/158A, BIOS J61 v01.17 11/05/2012
[  213.352881] task: ffff8800c9e30510 ti: ffff8807bc6ec000 task.ti: ffff8807bc6ec000
[  213.352885] RIP: 0010:[<ffffffff812679c1>]  [<ffffffff812679c1>] build_backref_tree+0x174/0xcbe
[  213.352894] RSP: 0018:ffff8807bc6efae0  EFLAGS: 00010283
[  213.352897] RAX: ffff8807dd60b980 RBX: ffff8800c6f6a890 RCX: ffff8807dd60b990
[  213.352900] RDX: ffff8807bc6efb58 RSI: 0000000000000000 RDI: 0000000000000000
[  213.352903] RBP: ffff8807bc6efbb8 R08: ffff8807bc6efa9c R09: ffff8807bc6ef9d8
[  213.352906] R10: ffff8800c6c3ee50 R11: 0000000000000000 R12: ffff88080329c000
[  213.352909] R13: ffff8800354ae1c0 R14: ffff8800c6f6a890 R15: ffff880804ff6000
[  213.352912] FS:  0000000000000000(0000) GS:ffff88082fc80000(0000) knlGS:0000000000000000
[  213.352915] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  213.352918] CR2: 00007f182f620000 CR3: 0000000001c13000 CR4: 00000000000407e0
[  213.352921] Stack:
[  213.352924]  ffff8807dd60bac0 ffff8807c4fd8e50 ffff8800c69e6fa0 ffff880804ff6124
[  213.352935]  ffff8800354ae200 ffff8807bc6efb68 ffff880804ff6120 ffff88003549dac0
[  213.352950]  0000000000000003 ffff8807bc6efb58 ffff8800c6f6a800 ffff8800c6f6a890
[  213.352961] Call Trace:
[  213.352967]  [<ffffffff81269f25>] relocate_tree_blocks+0x16a/0x44c
[  213.352971]  [<ffffffff8126ad56>] relocate_block_group+0x239/0x49a
[  213.352975]  [<ffffffff8126b112>] btrfs_relocate_block_group+0x15b/0x26d
[  213.352981]  [<ffffffff81249b80>] btrfs_relocate_chunk.isra.23+0x5c/0x5e8
[  213.352987]  [<ffffffff8161faeb>] ? _raw_spin_unlock+0x17/0x2a
[  213.352991]  [<ffffffff812458cc>] ? free_extent_buffer+0x8a/0x8d
[  213.352995]  [<ffffffff8124c406>] btrfs_balance+0x9b6/0xb74
[  213.352999]  [<ffffffff8161676d>] ? printk+0x54/0x56
[  213.353003]  [<ffffffff8124c5c4>] ? btrfs_balance+0xb74/0xb74
[  213.353007]  [<ffffffff8124c61d>] balance_kthread+0x59/0x7b
[  213.353012]  [<ffffffff8106b4b4>] kthread+0xae/0xb6
[  213.353015]  [<ffffffff8106b406>] ? __kthread_parkme+0x61/0x61
[  213.353020]  [<ffffffff8162667c>] ret_from_fork+0x7c/0xb0
[  213.353024]  [<ffffffff8106b406>] ? __kthread_parkme+0x61/0x61
[  213.353027] Code: e8 de 84 de ff 49 8d 45 40 48 89 c1 48 89 85 48 ff ff ff 49 8b 45 40 48 39 c8 74 38 49 3b 45 48 0f 85 25 0b 00 00 e9 22 0b 00 00 <0f> 0b 4c 8b 70 28 41 f6 46 71 10 75 1f 48 8b 4d a8 48 8b 9d 70 
[  213.353203] RIP  [<ffffffff812679c1>] build_backref_tree+0x174/0xcbe
[  213.353208]  RSP <ffff8807bc6efae0>
[  213.353213] ---[ end trace 07cbfc43e0fb0ccd ]---
[  213.353216] Kernel panic - not syncing: Fatal exception
[  213.353258] Kernel Offset: 0x0 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffff9fffffff)
[  213.353262] ---[ end Kernel panic - not syncing: Fatal exception

-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: btrfs balance crash BUG ON fs/btrfs/relocation.c:1062 or RIP build_backref_tree+0x9fc/0xcc4

[Josef Bacik]

On 06/18/2014 01:21 PM, Marc MERLIN wrote:
> First thanks for your answer and patch. While they didn't help, I'm happy to
> try another one or two if you'd like before I wipe the FS to recover.
>
> On Wed, Jun 18, 2014 at 08:26:46AM -0700, Josef Bacik wrote:
>>> 2) is your guess that the BUG_ON I'm getting shouldn't be triggered
>>> and your proposed patch could fix that, or that I do have an FS problem
>>> but that we're just going to make the kernel more lenient and not crash on
>>> it?
>>>
>>
>> All of these BUG_ON(!uppper->checked) are logic errors, not problems
>> with the fs, which is why I haven't turned them into an abort or
>
> I'm guessing you saw the comment from someone else in the other thread.
> BUG_ON is not helpful when you have end users. It effectively crashes the
> system when it can be unnecessary (a very secondary filesystem you don't
> depend on), and it tells users that anything wrong with btrfs on any
> filesystem can take their entire machine down needlessly
> (not counting the fact of course that it makes bug reporting on the list
> that much harder since syslog has no chance to capture and forward it)
>
> Can I appeal to you to remove all the ones you work with?
> If something is wrong, it should mount the FS read only and/or make it
> unaccessible, but not crash the entire system.
>
>> something else yet.  I'm thinking about chucking this whole function
>> anyway and using the backref walking code, but that is going to be a job
>> for future josef, present josef doesn't seem to have any time to do
>> anything despite sleeping a lot less.  Thanks,
>
> Yes, I understand that part. Good luck with getting sleep indeed.
>
> Back to your patch, unfortunately, it did not help.
> Let me know if you'd like me to try one more patch or two before I wipe the
> FS and start over.
> But for my own sanity, can you explain if I'm hitting reallocate bugs, or
> whether my filesystem is indeed correct?
>
> Thanks,
> Marc
>
>
> It crashed on another BUG_ON
> 	if (!list_empty(&cur->upper)) {
> 		/*
> 		 * the backref was added previously when processing
> 		 * backref of type BTRFS_TREE_BLOCK_REF_KEY
> 		 */
> 		BUG_ON(!list_is_singular(&cur->upper));
> 		edge = list_entry(cur->upper.next, struct backref_edge,
> 				  list[LOWER]);
> HERE ->		BUG_ON(!list_empty(&edge->list[UPPER]));
> 		exist = edge->node[UPPER];
> 		/*
> 		 * add the upper level block to pending list if we need
> 		 * check its backrefs
> 		 */
> 		if (!exist->checked)
>
>
> [  209.223856] BTRFS info (device sdb1): disk space caching is enabled
> [  209.283364] BTRFS: detected SSD devices, enabling SSD mode
> [  209.369267] BTRFS: checking UUID tree
> [  209.369286] BTRFS info (device sdb1): continuing balance
> [  209.412421] BTRFS info (device sdb1): relocating block group 82699091968 flags 1
> [  211.148796] BTRFS info (device sdb1): found 3719 extents
> [  213.351917] ------------[ cut here ]------------
> [  213.351941] kernel BUG at fs/btrfs/relocation.c:752!
> [  213.351951] invalid opcode: 0000 [#1] PREEMPT SMP
> [  213.351974] Modules linked in: des_generic nfsv3 nfsv4 xt_NFLOG nfnetlink_log nfnetlink xt_tcpudp xt_comment xt_multiport ip6table_filter ip6_tables iptable_filter ip_tables x_tables fuse autofs4 bnep rfcomm parport_pc ppdev binfmt_misc uvcvideo videobuf2_core videodev snd_usb_audio media snd_usbmidi_lib videobuf2_vmalloc videobuf2_memops hid_generic usbhid hid ecb btusb bluetooth rpcsec_gss_krb5 6lowpan_iphc nfsd nfs_acl auth_rpcgss nfs fscache snd_hda_codec_hdmi lockd sunrpc snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_controller snd_hda_codec intel_rapl x86_pkg_temp_thermal intel_powerclamp snd_hwdep snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_midi coretemp snd_rawmidi snd_seq_midi_event kvm snd_seq snd_timer crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel snd_seq_device snd aesni_intel sb_edac edac_core soundcore ehci_pci tpm_infineon ablk_helper cryptd lrw ehci_hcd gf128mul glue_helper hp_wmi sparse_keymap rfkill aes_x86_64 tpm_tis lpc_ich p
smouse serio_raw tpm evdev microcode processor wmi lp parport loop uas usb_storage dm_mod firewire_ohci xhci_hcd firewire_core crc_itu_t usbcore isci usb_common e1000e libsas ptp pps_core scsi_transport_sas
> [  213.352787] CPU: 4 PID: 13538 Comm: btrfs-balance Not tainted 3.15.1-amd64-i915-preempt-20140216jbp #1
> [  213.352795] Hardware name: Hewlett-Packard HP Z620 Workstation/158A, BIOS J61 v01.17 11/05/2012
> [  213.352881] task: ffff8800c9e30510 ti: ffff8807bc6ec000 task.ti: ffff8807bc6ec000
> [  213.352885] RIP: 0010:[<ffffffff812679c1>]  [<ffffffff812679c1>] build_backref_tree+0x174/0xcbe
> [  213.352894] RSP: 0018:ffff8807bc6efae0  EFLAGS: 00010283
> [  213.352897] RAX: ffff8807dd60b980 RBX: ffff8800c6f6a890 RCX: ffff8807dd60b990
> [  213.352900] RDX: ffff8807bc6efb58 RSI: 0000000000000000 RDI: 0000000000000000
> [  213.352903] RBP: ffff8807bc6efbb8 R08: ffff8807bc6efa9c R09: ffff8807bc6ef9d8
> [  213.352906] R10: ffff8800c6c3ee50 R11: 0000000000000000 R12: ffff88080329c000
> [  213.352909] R13: ffff8800354ae1c0 R14: ffff8800c6f6a890 R15: ffff880804ff6000
> [  213.352912] FS:  0000000000000000(0000) GS:ffff88082fc80000(0000) knlGS:0000000000000000
> [  213.352915] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [  213.352918] CR2: 00007f182f620000 CR3: 0000000001c13000 CR4: 00000000000407e0
> [  213.352921] Stack:
> [  213.352924]  ffff8807dd60bac0 ffff8807c4fd8e50 ffff8800c69e6fa0 ffff880804ff6124
> [  213.352935]  ffff8800354ae200 ffff8807bc6efb68 ffff880804ff6120 ffff88003549dac0
> [  213.352950]  0000000000000003 ffff8807bc6efb58 ffff8800c6f6a800 ffff8800c6f6a890
> [  213.352961] Call Trace:
> [  213.352967]  [<ffffffff81269f25>] relocate_tree_blocks+0x16a/0x44c
> [  213.352971]  [<ffffffff8126ad56>] relocate_block_group+0x239/0x49a
> [  213.352975]  [<ffffffff8126b112>] btrfs_relocate_block_group+0x15b/0x26d
> [  213.352981]  [<ffffffff81249b80>] btrfs_relocate_chunk.isra.23+0x5c/0x5e8
> [  213.352987]  [<ffffffff8161faeb>] ? _raw_spin_unlock+0x17/0x2a
> [  213.352991]  [<ffffffff812458cc>] ? free_extent_buffer+0x8a/0x8d
> [  213.352995]  [<ffffffff8124c406>] btrfs_balance+0x9b6/0xb74
> [  213.352999]  [<ffffffff8161676d>] ? printk+0x54/0x56
> [  213.353003]  [<ffffffff8124c5c4>] ? btrfs_balance+0xb74/0xb74
> [  213.353007]  [<ffffffff8124c61d>] balance_kthread+0x59/0x7b
> [  213.353012]  [<ffffffff8106b4b4>] kthread+0xae/0xb6
> [  213.353015]  [<ffffffff8106b406>] ? __kthread_parkme+0x61/0x61
> [  213.353020]  [<ffffffff8162667c>] ret_from_fork+0x7c/0xb0
> [  213.353024]  [<ffffffff8106b406>] ? __kthread_parkme+0x61/0x61
> [  213.353027] Code: e8 de 84 de ff 49 8d 45 40 48 89 c1 48 89 85 48 ff ff ff 49 8b 45 40 48 39 c8 74 38 49 3b 45 48 0f 85 25 0b 00 00 e9 22 0b 00 00 <0f> 0b 4c 8b 70 28 41 f6 46 71 10 75 1f 48 8b 4d a8 48 8b 9d 70
> [  213.353203] RIP  [<ffffffff812679c1>] build_backref_tree+0x174/0xcbe
> [  213.353208]  RSP <ffff8807bc6efae0>
> [  213.353213] ---[ end trace 07cbfc43e0fb0ccd ]---
> [  213.353216] Kernel panic - not syncing: Fatal exception
> [  213.353258] Kernel Offset: 0x0 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffff9fffffff)
> [  213.353262] ---[ end Kernel panic - not syncing: Fatal exception
>

Ok undo what you did and apply this and re-run.  It is going spit out a metric
shittone of data, but all I want is the last chunk of stuff between

running build_backref_tree
<some shit>
block <some more shit> wasn't checked
done building backref tree

I changed it to return an error instead of bugging, so if it still bugs attach
that as well so I can figure out where down the stack we need to fix.  Thanks,

Josef


diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
index 65245a0..915aab4 100644
--- a/fs/btrfs/relocation.c
+++ b/fs/btrfs/relocation.c
@@ -702,6 +702,7 @@ struct backref_node *build_backref_tree(struct reloc_control *rc,
  	int err = 0;
  	bool need_check = true;
  
+	printk(KERN_ERR "running build_backref_tree\n");
  	path1 = btrfs_alloc_path();
  	path2 = btrfs_alloc_path();
  	if (!path1 || !path2) {
@@ -722,6 +723,7 @@ struct backref_node *build_backref_tree(struct reloc_control *rc,
  	node->lowest = 1;
  	cur = node;
  again:
+	printk(KERN_ERR "building backref for bytenr %llu\n", cur->bytenr);
  	end = 0;
  	ptr = 0;
  	key.objectid = cur->bytenr;
@@ -757,6 +759,7 @@ again:
  		 */
  		if (!exist->checked)
  			list_add_tail(&edge->list[UPPER], &list);
+		printk(KERN_ERR "exist is %llu, checked %d\n", exist->bytenr, exist->checked);
  	} else {
  		exist = NULL;
  	}
@@ -865,6 +868,7 @@ again:
  				 *  cached, add the block to pending list
  				 */
  				list_add_tail(&edge->list[UPPER], &list);
+				printk(KERN_ERR "found shared ref %llu, needs checking\n", upper->bytenr);
  			} else {
  				upper = rb_entry(rb_node, struct backref_node,
  						 rb_node);
@@ -962,9 +966,10 @@ again:
  				 * if we know the block isn't shared
  				 * we can void checking its backrefs.
  				 */
-				if (btrfs_block_can_be_shared(root, eb))
+				if (btrfs_block_can_be_shared(root, eb)) {
+					printk(KERN_ERR "adding block in path %llu, level %d, cur level %d, need_check %d\n", upper->bytenr, upper->level, cur->level, need_check);
  					upper->checked = 0;
-				else
+				} else
  					upper->checked = 1;
  
  				/*
@@ -1019,6 +1024,7 @@ next:
  		edge = list_entry(list.next, struct backref_edge, list[UPPER]);
  		list_del_init(&edge->list[UPPER]);
  		cur = edge->node[UPPER];
+		printk(KERN_ERR "doing the checking for block %llu\n", cur->bytenr);
  		goto again;
  	}
  
@@ -1062,7 +1068,12 @@ next:
  			continue;
  		}
  
-		BUG_ON(!upper->checked);
+		if (!upper->checked) {
+			printk(KERN_ERR "block %llu wasn't checked\n",
+			       upper->bytenr);
+			err = -EINVAL;
+			goto out;
+		}
  		BUG_ON(cowonly != upper->cowonly);
  		if (!cowonly) {
  			rb_node = tree_insert(&cache->rb_root, upper->bytenr,
@@ -1114,6 +1125,7 @@ next:
  		}
  	}
  out:
+	printk(KERN_ERR "done building backref tree\n");
  	btrfs_free_path(path1);
  	btrfs_free_path(path2);
  	if (err) {

Re: btrfs balance crash BUG ON fs/btrfs/relocation.c:1062 or RIP build_backref_tree+0x9fc/0xcc4

[Marc MERLIN]

On Thu, Jun 19, 2014 at 09:12:13AM -0700, Josef Bacik wrote:
> Ok undo what you did and apply this and re-run.  It is going spit out a 
> metric
> shittone of data, but all I want is the last chunk of stuff between
> 
> running build_backref_tree
> <some shit>
> block <some more shit> wasn't checked
> done building backref tree
> 
> I changed it to return an error instead of bugging, so if it still bugs 
> attach
> that as well so I can figure out where down the stack we need to fix.  
> Thanks,

Patch applied, here is the new crash. The output is short, so here is all of
it:

BTRFS info (device sdb1): disk space caching is enabled
BTRFS: detected SSD devices, enabling SSD mode
BTRFS info (device sdb1): continuing balance
BTRFS info (device sdb1): relocating block group 82699091968 flags 1
BTRFS info (device sdb1): found 3719 extents
running build_backref_tree
building backref for bytenr 73005293568
adding block in path 173444124672, level 1, cur level 0, need_check 1
adding block in path 2176913408, level 3, cur level 0, need_check 0
doing the checking for block 173444124672
building backref for bytenr 173444124672
exist is 67327229952, checked 1
found shared ref 173244198912, needs checking
doing the checking for block 173244198912
building backref for bytenr 173244198912
found shared ref 2177122304, needs checking
found shared ref 2177081344, needs checking
found shared ref 2176827392, needs checking
doing the checking for block 2177122304
building backref for bytenr 2177122304
doing the checking for block 2177081344
building backref for bytenr 2177081344
doing the checking for block 2176827392
building backref for bytenr 2176827392
block 2176913408 wasn't checked
done building backref tree
------------[ cut here ]------------
kernel BUG at fs/btrfs/relocation.c:443!
invalid opcode: 0000 [#1] PREEMPT SMP 
Modules linked in: des_generic nfsv3 nfsv4 xt_NFLOG nfnetlink_log nfnetlink xt_tcpudp xt_comment xt_multiport ip6table_filter ip6_tables iptable_filter ip_tables x_tables fuse autofs4 bnep rfcomm parport_pc ppdev binfmt_misc ecb intel_rapl rpcsec_gss_krb5 btusb bluetooth 6lowpan_iphc nfsd nfs_acl auth_rpcgss x86_pkg_temp_thermal intel_powerclamp coretemp nfs fscache lockd sunrpc kvm crct10dif_pclmul crc32_pclmul snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic crc32c_intel snd_usb_audio snd_hda_intel ghash_clmulni_intel snd_hda_controller snd_hda_codec snd_hwdep snd_usbmidi_lib snd_pcm_oss snd_mixer_oss snd_pcm uvcvideo videobuf2_core videodev snd_seq_midi snd_seq_midi_event aesni_intel snd_seq snd_rawmidi media videobuf2_vmalloc videobuf2_memops snd_timer snd_seq_device ablk_helper psmouse serio_raw hp_wmi sparse_keymap tpm_infineon cryptd lrw gf128mul glue_helper snd soundcore rfkill sb_edac aes_x86_64 edac_core ehci_pci lpc_ich microcode ehci_hcd tpm_tis evdev wmi tpm processor lp parport loop hid_generic usbhid hid uas usb_storage dm_mod firewire_ohci firewire_core xhci_hcd crc_itu_t usbcore isci usb_common e1000e libsas ptp pps_core scsi_transport_sas
CPU: 4 PID: 14292 Comm: btrfs-balance Not tainted 3.15.1-amd64-i915-preempt-20140216jbp2 #2
Hardware name: Hewlett-Packard HP Z620 Workstation/158A, BIOS J61 v01.17 11/05/2012
task: ffff880fae66e6d0 ti: ffff880fae37c000 task.ti: ffff880fae37c000
RIP: 0010:[<ffffffff81268d33>]  [<ffffffff81268d33>] remove_backref_node+0x64/0xd5
RSP: 0018:ffff880fae37fc10  EFLAGS: 00010283
RAX: 0000000000000047 RBX: ffff8808065863c0 RCX: ffff880803c350e8
RDX: 0000000000000048 RSI: ffff8807e67cc280 RDI: 0000000000000282
RBP: ffff880fae37fc40 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 00000000ffe8f802 R12: ffff8808065865c0
R13: ffff880803c35020 R14: ffff8807e67cc280 R15: ffff880806586600
FS:  0000000000000000(0000) GS:ffff88082fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fff53d50020 CR3: 0000000001c13000 CR4: 00000000000407e0
Stack:
 ffff880803c35124 ffff8807cbe1fd10 ffff880803c35108 ffff880803c350e8
 00000000ffffffea ffff880803c35000 ffff880fae37fcb8 ffffffff8126afa6
 ffff880803c35020 000000011af11358 ffffea001af11390 0000000000000000
Call Trace:
 [<ffffffff8126afa6>] relocate_block_group+0x390/0x49a
 [<ffffffff8126b20b>] btrfs_relocate_block_group+0x15b/0x26d
 [<ffffffff81249b80>] btrfs_relocate_chunk.isra.23+0x5c/0x5e8
 [<ffffffff8161fb8b>] ? _raw_spin_unlock+0x17/0x2a
 [<ffffffff812458cc>] ? free_extent_buffer+0x8a/0x8d
 [<ffffffff8124c406>] btrfs_balance+0x9b6/0xb74
 [<ffffffff8161684d>] ? printk+0x54/0x56
 [<ffffffff8124c5c4>] ? btrfs_balance+0xb74/0xb74
 [<ffffffff8124c61d>] balance_kthread+0x59/0x7b
 [<ffffffff8106b4b4>] kthread+0xae/0xb6
 [<ffffffff8106b406>] ? __kthread_parkme+0x61/0x61
 [<ffffffff816266fc>] ret_from_fork+0x7c/0xb0
 [<ffffffff8106b406>] ? __kthread_parkme+0x61/0x61
Code: f7 49 8b 5e 28 e8 3a c9 ff ff 49 8d 7e 10 e8 31 c9 ff ff 48 8b 7d d0 4c 89 f6 e8 89 d8 ff ff 48 3b 1b 75 1d 4d 39 7c 24 40 74 02 <0f> 0b 4c 89 e6 4c 89 ef 49 89 dc e8 2f ff ff ff 80 4b 71 02 eb 
RIP  [<ffffffff81268d33>] remove_backref_node+0x64/0xd5
 RSP <ffff880fae37fc10>
---[ end trace 3611f7c2174a33f9 ]---
Kernel panic - not syncing: Fatal exception
Kernel Offset: 0x0 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffff9fffffff)
---[ end Kernel panic - not syncing: Fatal exception

-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: btrfs balance crash BUG ON fs/btrfs/relocation.c:1062 or RIP build_backref_tree+0x9fc/0xcc4

[Josef Bacik]

On 06/19/2014 03:25 PM, Marc MERLIN wrote:
> On Thu, Jun 19, 2014 at 09:12:13AM -0700, Josef Bacik wrote:
>> Ok undo what you did and apply this and re-run.  It is going spit out a
>> metric
>> shittone of data, but all I want is the last chunk of stuff between
>>
>> running build_backref_tree
>> <some shit>
>> block <some more shit> wasn't checked
>> done building backref tree
>>
>> I changed it to return an error instead of bugging, so if it still bugs
>> attach
>> that as well so I can figure out where down the stack we need to fix.
>> Thanks,
>
> Patch applied, here is the new crash. The output is short, so here is all of
> it:

Ok same drill as before, reset and apply this, hopefully no panic this time


diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
index 65245a0..bca5240 100644
--- a/fs/btrfs/relocation.c
+++ b/fs/btrfs/relocation.c
@@ -440,7 +440,7 @@ static void remove_backref_node(struct backref_cache *cache,
  		free_backref_edge(cache, edge);
  
  		if (RB_EMPTY_NODE(&upper->rb_node)) {
-			BUG_ON(!list_empty(&node->upper));
+//			BUG_ON(!list_empty(&node->upper));
  			drop_backref_node(cache, node);
  			node = upper;
  			node->lowest = 1;
@@ -702,6 +702,7 @@ struct backref_node *build_backref_tree(struct reloc_control *rc,
  	int err = 0;
  	bool need_check = true;
  
+	printk(KERN_ERR "running build_backref_tree\n");
  	path1 = btrfs_alloc_path();
  	path2 = btrfs_alloc_path();
  	if (!path1 || !path2) {
@@ -722,6 +723,8 @@ struct backref_node *build_backref_tree(struct reloc_control *rc,
  	node->lowest = 1;
  	cur = node;
  again:
+	printk(KERN_ERR "building backref for bytenr %llu level %d\n",
+	       cur->bytenr, cur->level);
  	end = 0;
  	ptr = 0;
  	key.objectid = cur->bytenr;
@@ -757,6 +760,7 @@ again:
  		 */
  		if (!exist->checked)
  			list_add_tail(&edge->list[UPPER], &list);
+		printk(KERN_ERR "exist is %llu, checked %d\n", exist->bytenr, exist->checked);
  	} else {
  		exist = NULL;
  	}
@@ -865,6 +869,7 @@ again:
  				 *  cached, add the block to pending list
  				 */
  				list_add_tail(&edge->list[UPPER], &list);
+				printk(KERN_ERR "found shared ref %llu, needs checking\n", upper->bytenr);
  			} else {
  				upper = rb_entry(rb_node, struct backref_node,
  						 rb_node);
@@ -958,14 +963,30 @@ again:
  					      &root->state))
  					upper->cowonly = 1;
  
+				printk(KERN_ERR "eb in path %llu, level %d, "
+				       "cowonly %d, owner %llu, gen %llu, last "
+				       "snap %llu, reloc %d, root %llu\n",
+				       upper->bytenr, upper->level,
+				       upper->cowonly, upper->owner,
+				       btrfs_header_generation(eb),
+				       btrfs_root_last_snapshot(&root->root_item),
+				       btrfs_header_flag(eb,
+							 BTRFS_HEADER_FLAG_RELOC),
+				       root->objectid);
+
  				/*
  				 * if we know the block isn't shared
  				 * we can void checking its backrefs.
  				 */
-				if (btrfs_block_can_be_shared(root, eb))
+				if (btrfs_block_can_be_shared(root, eb)) {
+					printk(KERN_ERR "is shared, need_check"
+					       " %d\n", need_check);
  					upper->checked = 0;
-				else
+				} else {
+					printk(KERN_ERR "isn't shared, "
+					       "need_check %d\n", need_check);
  					upper->checked = 1;
+				}
  
  				/*
  				 * add the block to pending list if we
@@ -1019,6 +1040,7 @@ next:
  		edge = list_entry(list.next, struct backref_edge, list[UPPER]);
  		list_del_init(&edge->list[UPPER]);
  		cur = edge->node[UPPER];
+		printk(KERN_ERR "doing the checking for block %llu\n", cur->bytenr);
  		goto again;
  	}
  
@@ -1062,7 +1084,12 @@ next:
  			continue;
  		}
  
-		BUG_ON(!upper->checked);
+		if (!upper->checked) {
+			printk(KERN_ERR "block %llu wasn't checked\n",
+			       upper->bytenr);
+			err = -EINVAL;
+			goto out;
+		}
  		BUG_ON(cowonly != upper->cowonly);
  		if (!cowonly) {
  			rb_node = tree_insert(&cache->rb_root, upper->bytenr,
@@ -1114,6 +1141,7 @@ next:
  		}
  	}
  out:
+	printk(KERN_ERR "done building backref tree\n");
  	btrfs_free_path(path1);
  	btrfs_free_path(path2);
  	if (err) {

Re: btrfs balance crash BUG ON fs/btrfs/relocation.c:1062 or RIP build_backref_tree+0x9fc/0xcc4

[Marc MERLIN]

On Thu, Jun 19, 2014 at 03:50:16PM -0700, Josef Bacik wrote:
> Ok same drill as before, reset and apply this, hopefully no panic this time
> 
> 
> diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
> index 65245a0..bca5240 100644

Here's the output
BTRFS info (device sdb1): disk space caching is enabled
BTRFS: detected SSD devices, enabling SSD mode
BTRFS info (device sdb1): continuing balance
BTRFS info (device sdb1): relocating block group 82699091968 flags 1
BTRFS info (device sdb1): found 3719 extents
running build_backref_tree
building backref for bytenr 73005293568 level 0
eb in path 173444124672, level 1, cowonly 0, owner 256, gen 231481, last snap 243545, reloc 0, root 256
is shared, need_check 1
eb in path 67327229952, level 2, cowonly 0, owner 256, gen 243615, last snap 243545, reloc 0, root 256
isn't shared, need_check 0
eb in path 2176913408, level 3, cowonly 0, owner 256, gen 253956, last snap 243545, reloc 1, root 256
is shared, need_check 0
eb in path 2320281600, level 4, cowonly 0, owner 256, gen 253957, last snap 243545, reloc 0, root 256
isn't shared, need_check 0
doing the checking for block 173444124672
building backref for bytenr 173444124672 level 1
exist is 67327229952, checked 1
found shared ref 173244198912, needs checking
doing the checking for block 173244198912
building backref for bytenr 173244198912 level 2
found shared ref 2177122304, needs checking
found shared ref 2177081344, needs checking
found shared ref 2176827392, needs checking
doing the checking for block 2177122304
building backref for bytenr 2177122304 level 3
eb in path 2314657792, level 4, cowonly 0, owner 6125, gen 253957, last snap 243545, reloc 0, root 6125
isn't shared, need_check 1
doing the checking for block 2177081344
building backref for bytenr 2177081344 level 3
eb in path 2320146432, level 4, cowonly 0, owner 6123, gen 253957, last snap 243338, reloc 0, root 6123
isn't shared, need_check 1
doing the checking for block 2176827392
building backref for bytenr 2176827392 level 3
eb in path 2320363520, level 4, cowonly 0, owner 6124, gen 253957, last snap 243441, reloc 0, root 6124
isn't shared, need_check 1
block 2176913408 wasn't checked
done building backref tree
------------[ cut here ]------------
kernel BUG at fs/btrfs/relocation.c:411!
invalid opcode: 0000 [#1] PREEMPT SMP 
Modules linked in: des_generic nfsv3 nfsv4 xt_NFLOG nfnetlink_log nfnetlink xt_tcpudp xt_comment xt_multiport ip6table_filter ip6_tables iptable_filter ip_tables x_tables fuse autofs4 rfcomm parport_pc bnep ppdev binfmt_misc ecb btusb bluetooth intel_rapl 6lowpan_iphc x86_pkg_temp_thermal intel_powerclamp coretemp kvm rpcsec_gss_krb5 nfsd nfs_acl auth_rpcgss nfs snd_hda_codec_hdmi crct10dif_pclmul crc32_pclmul crc32c_intel fscache lockd sunrpc ghash_clmulni_intel snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_controller aesni_intel snd_hda_codec ablk_helper snd_usb_audio snd_pcm_oss snd_mixer_oss snd_pcm cryptd lrw snd_hwdep snd_usbmidi_lib snd_seq_midi snd_seq_midi_event snd_rawmidi uvcvideo gf128mul sb_edac videobuf2_core snd_seq psmouse videodev media videobuf2_vmalloc videobuf2_memops snd_timer snd_seq_device edac_core ehci_pci glue_helper tpm_infineon hp_wmi sparse_keymap snd soundcore serio_raw rfkill aes_x86_64 ehci_hcd microcode lpc_ich tpm_tis tpm evdev wmi processor lp parport loop hid_generic usbhid hid uas usb_storage dm_mod firewire_ohci xhci_hcd firewire_core crc_itu_t usbcore e1000e isci usb_common ptp libsas pps_core scsi_transport_sas
CPU: 5 PID: 17084 Comm: btrfs-balance Not tainted 3.15.1-amd64-i915-preempt-20140216jbp3 #3
Hardware name: Hewlett-Packard HP Z620 Workstation/158A, BIOS J61 v01.17 11/05/2012
task: ffff880fcc858190 ti: ffff880fcd030000 task.ti: ffff880fcd030000
RIP: 0010:[<ffffffff81268bfb>]  [<ffffffff81268bfb>] drop_backref_node+0x19/0x5d
RSP: 0018:ffff880fcd033bf8  EFLAGS: 00010287
RAX: ffff8807dc9ce180 RBX: ffff8807dc9ce140 RCX: ffff880807eaf8e8
RDX: 0000000000000043 RSI: ffff8807dc9ce140 RDI: ffff880807eaf820
RBP: ffff880fcd033c08 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 00000000ffe77402 R12: ffff880806f7c140
R13: ffff880807eaf820 R14: ffff8807e19fb040 R15: ffff880807eaf924
FS:  0000000000000000(0000) GS:ffff88082fca0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fab45ef3000 CR3: 0000000001c13000 CR4: 00000000000407e0
Stack:
 ffff880806f7c140 ffff880806f7c140 ffff880fcd033c40 ffffffff81268ca4
 ffff8807cb274d10 ffff880807eaf908 ffff880807eaf8e8 00000000ffffffea
 ffff880807eaf800 ffff880fcd033cb8 ffffffff8126af02 ffff880807eaf820
Call Trace:
 [<ffffffff81268ca4>] remove_backref_node+0x65/0xc1
 [<ffffffff8126af02>] relocate_block_group+0x390/0x49a
 [<ffffffff8126b167>] btrfs_relocate_block_group+0x15b/0x26d
 [<ffffffff81249b80>] btrfs_relocate_chunk.isra.23+0x5c/0x5e8
 [<ffffffff8161fbfb>] ? _raw_spin_unlock+0x17/0x2a
 [<ffffffff812458cc>] ? free_extent_buffer+0x8a/0x8d
 [<ffffffff8124c406>] btrfs_balance+0x9b6/0xb74
 [<ffffffff816167ad>] ? printk+0x54/0x56
 [<ffffffff8124c5c4>] ? btrfs_balance+0xb74/0xb74
 [<ffffffff8124c61d>] balance_kthread+0x59/0x7b
 [<ffffffff8106b4b4>] kthread+0xae/0xb6
 [<ffffffff8106b406>] ? __kthread_parkme+0x61/0x61
 [<ffffffff8162677c>] ret_from_fork+0x7c/0xb0
 [<ffffffff8106b406>] ? __kthread_parkme+0x61/0x61
Code: 7b 68 e8 6b cc fd ff 48 c7 43 68 00 00 00 00 5b 5d c3 66 66 66 66 90 55 48 8d 46 40 48 89 e5 41 54 53 48 39 46 40 48 89 f3 74 02 <0f> 0b 49 89 fc 48 89 f7 e8 a1 ff ff ff 48 8d 7b 30 e8 3b ca ff 
RIP  [<ffffffff81268bfb>] drop_backref_node+0x19/0x5d
 RSP <ffff880fcd033bf8>
---[ end trace 539f3f31bdb6112f ]---
Kernel panic - not syncing: Fatal exception
Kernel Offset: 0x0 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffff9fffffff)
---[ end Kernel panic - not syncing: Fatal exception
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: btrfs balance crash BUG ON fs/btrfs/relocation.c:1062 or RIP build_backref_tree+0x9fc/0xcc4

[Josef Bacik]

On 06/19/2014 05:53 PM, Marc MERLIN wrote:
> On Thu, Jun 19, 2014 at 03:50:16PM -0700, Josef Bacik wrote:
>> Ok same drill as before, reset and apply this, hopefully no panic this time
>>
>>
>> diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
>> index 65245a0..bca5240 100644
>

Ok I see what it is but I want to get rid of the panicing so we're going to do
this dance a few more times until it's just failing to mount instead of
panicing, and then we'll fix the actual bug.  Give this a whirl, and I've added
another printk just to make sure what I think is happening is actually what's
happening, so same drill as before.  Thanks,

Josef


diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
index 65245a0..21e8a57 100644
--- a/fs/btrfs/relocation.c
+++ b/fs/btrfs/relocation.c
@@ -702,6 +702,7 @@ struct backref_node *build_backref_tree(struct reloc_control *rc,
  	int err = 0;
  	bool need_check = true;
  
+	printk(KERN_ERR "running build_backref_tree\n");
  	path1 = btrfs_alloc_path();
  	path2 = btrfs_alloc_path();
  	if (!path1 || !path2) {
@@ -722,6 +723,8 @@ struct backref_node *build_backref_tree(struct reloc_control *rc,
  	node->lowest = 1;
  	cur = node;
  again:
+	printk(KERN_ERR "building backref for bytenr %llu level %d\n",
+	       cur->bytenr, cur->level);
  	end = 0;
  	ptr = 0;
  	key.objectid = cur->bytenr;
@@ -757,6 +760,7 @@ again:
  		 */
  		if (!exist->checked)
  			list_add_tail(&edge->list[UPPER], &list);
+		printk(KERN_ERR "exist is %llu, checked %d\n", exist->bytenr, exist->checked);
  	} else {
  		exist = NULL;
  	}
@@ -807,6 +811,8 @@ again:
  		      exist->owner == key.offset) ||
  		     (key.type == BTRFS_SHARED_BLOCK_REF_KEY &&
  		      exist->bytenr == key.offset))) {
+			printk(KERN_ERR "exist is fucking us, bytenr %llu, "
+			       "type %d\n", exist->bytenr, key.type);
  			exist = NULL;
  			goto next;
  		}
@@ -865,6 +871,7 @@ again:
  				 *  cached, add the block to pending list
  				 */
  				list_add_tail(&edge->list[UPPER], &list);
+				printk(KERN_ERR "found shared ref %llu, needs checking\n", upper->bytenr);
  			} else {
  				upper = rb_entry(rb_node, struct backref_node,
  						 rb_node);
@@ -958,14 +965,30 @@ again:
  					      &root->state))
  					upper->cowonly = 1;
  
+				printk(KERN_ERR "eb in path %llu, level %d, "
+				       "cowonly %d, owner %llu, gen %llu, last "
+				       "snap %llu, reloc %d, root %llu\n",
+				       upper->bytenr, upper->level,
+				       upper->cowonly, upper->owner,
+				       btrfs_header_generation(eb),
+				       btrfs_root_last_snapshot(&root->root_item),
+				       btrfs_header_flag(eb,
+							 BTRFS_HEADER_FLAG_RELOC),
+				       root->objectid);
+
  				/*
  				 * if we know the block isn't shared
  				 * we can void checking its backrefs.
  				 */
-				if (btrfs_block_can_be_shared(root, eb))
+				if (btrfs_block_can_be_shared(root, eb)) {
+					printk(KERN_ERR "is shared, need_check"
+					       " %d\n", need_check);
  					upper->checked = 0;
-				else
+				} else {
+					printk(KERN_ERR "isn't shared, "
+					       "need_check %d\n", need_check);
  					upper->checked = 1;
+				}
  
  				/*
  				 * add the block to pending list if we
@@ -1019,6 +1042,7 @@ next:
  		edge = list_entry(list.next, struct backref_edge, list[UPPER]);
  		list_del_init(&edge->list[UPPER]);
  		cur = edge->node[UPPER];
+		printk(KERN_ERR "doing the checking for block %llu\n", cur->bytenr);
  		goto again;
  	}
  
@@ -1062,7 +1086,12 @@ next:
  			continue;
  		}
  
-		BUG_ON(!upper->checked);
+		if (!upper->checked) {
+			printk(KERN_ERR "block %llu wasn't checked\n",
+			       upper->bytenr);
+			err = -EINVAL;
+			goto out;
+		}
  		BUG_ON(cowonly != upper->cowonly);
  		if (!cowonly) {
  			rb_node = tree_insert(&cache->rb_root, upper->bytenr,
@@ -1114,6 +1143,7 @@ next:
  		}
  	}
  out:
+	printk(KERN_ERR "done building backref tree\n");
  	btrfs_free_path(path1);
  	btrfs_free_path(path2);
  	if (err) {
@@ -1123,7 +1153,6 @@ out:
  			list_del_init(&lower->upper);
  		}
  		upper = node;
-		INIT_LIST_HEAD(&list);
  		while (upper) {
  			if (RB_EMPTY_NODE(&upper->rb_node)) {
  				list_splice_tail(&upper->upper, &list);

Re: btrfs balance crash BUG ON fs/btrfs/relocation.c:1062 or RIP build_backref_tree+0x9fc/0xcc4

[Marc MERLIN]

On Fri, Jun 20, 2014 at 08:40:49AM -0700, Josef Bacik wrote:
> On 06/19/2014 05:53 PM, Marc MERLIN wrote:
> >On Thu, Jun 19, 2014 at 03:50:16PM -0700, Josef Bacik wrote:
> >>Ok same drill as before, reset and apply this, hopefully no panic this 
> >>time
> >>
> >>
> >>diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
> >>index 65245a0..bca5240 100644
> >
> 
> Ok I see what it is but I want to get rid of the panicing so we're going
> to do this dance a few more times until it's just failing to mount instead
> of panicing, and then we'll fix the actual bug.  Give this a whirl, and
> I've added another printk just to make sure what I think is happening is
> actually what's happening, so same drill as before.  Thanks,

Patch applied. The panic moved :)

[  313.756971] BTRFS: device label btrfs_pool2 devid 1 transid 254006 /dev/sda1
[  313.757467] BTRFS info (device sda1): disk space caching is enabled
[  313.835538] BTRFS: detected SSD devices, enabling SSD mode
[  313.932327] BTRFS info (device sda1): continuing balance
[  313.990048] BTRFS info (device sda1): relocating block group 82699091968 flags 1
[  316.085055] BTRFS info (device sda1): found 3719 extents
[  317.797058] running build_backref_tree
[  317.797075] building backref for bytenr 73005293568 level 0
[  317.797090] eb in path 173444124672, level 1, cowonly 0, owner 256, gen 231481, last snap 243545, reloc 0, root 256
[  317.797097] is shared, need_check 1
[  317.797104] eb in path 67327229952, level 2, cowonly 0, owner 256, gen 243615, last snap 243545, reloc 0, root 256
[  317.797109] isn't shared, need_check 0
[  317.797117] eb in path 2176913408, level 3, cowonly 0, owner 256, gen 253956, last snap 243545, reloc 1, root 256
[  317.797122] is shared, need_check 0
[  317.797129] eb in path 2320281600, level 4, cowonly 0, owner 256, gen 253957, last snap 243545, reloc 0, root 256
[  317.797134] isn't shared, need_check 0
[  317.797139] doing the checking for block 173444124672
[  317.797144] building backref for bytenr 173444124672 level 1
[  317.797562] exist is 67327229952, checked 1
[  317.797571] exist is fucking us, bytenr 67327229952, type 176
[  317.797578] found shared ref 173244198912, needs checking
[  317.797583] doing the checking for block 173244198912
[  317.797588] building backref for bytenr 173244198912 level 2
[  317.798242] found shared ref 2177122304, needs checking
[  317.798251] found shared ref 2177081344, needs checking
[  317.798257] found shared ref 2176827392, needs checking
[  317.798263] doing the checking for block 2177122304
[  317.798268] building backref for bytenr 2177122304 level 3
[  317.798779] eb in path 2314657792, level 4, cowonly 0, owner 6125, gen 253957, last snap 243545, reloc 0, root 6125
[  317.798787] isn't shared, need_check 1
[  317.798798] doing the checking for block 2177081344
[  317.798804] building backref for bytenr 2177081344 level 3
[  317.798962] eb in path 2320146432, level 4, cowonly 0, owner 6123, gen 253957, last snap 243338, reloc 0, root 6123
[  317.798970] isn't shared, need_check 1
[  317.798976] doing the checking for block 2176827392
[  317.798981] building backref for bytenr 2176827392 level 3
[  317.799144] eb in path 2320363520, level 4, cowonly 0, owner 6124, gen 253957, last snap 243441, reloc 0, root 6124
[  317.799151] isn't shared, need_check 1
[  317.799158] block 2176913408 wasn't checked
[  317.799162] done building backref tree
[  317.799193] general protection fault: 0000 [#1] PREEMPT SMP 
[  317.799207] Modules linked in: xt_NFLOG xt_tcpudp xt_comment xt_multiport ip6table_filter ip6_tables iptable_filter ip_tables x_tables nfnetlink_log nfnetlink fuse autofs4 rfcomm bnep bluetooth 6lowpan_iphc rfkill binfmt_misc snd_hda_codec_hdmi snd_hda_codec_analog snd_hda_codec_generic intel_powerclamp coretemp kvm_intel kvm snd_hda_intel snd_hda_controller snd_hda_codec crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel ehci_pci snd_hwdep rpcsec_gss_krb5 snd_pcm_oss snd_mixer_oss snd_pcm nfsd auth_rpcgss snd_seq_midi snd_seq_midi_event nfs_acl snd_rawmidi nfs lockd sunrpc snd_seq snd_seq_device ppdev aes_x86_64 ehci_hcd snd_timer lrw parport_pc dcdbas i7core_edac lp gf128mul gpio_ich dell_wmi parport snd edac_core acpi_cpufreq soundcore lpc_ich processor loop glue_helper tpm_tis tpm sparse_keymap wmi psmouse serio_raw joydev ablk_helper cryptd evdev fscache microcode hid_generic usbhid hid sr_mod cdrom dm_mod tg3 libphy ptp pps_core uhci_hcd usbcore usb_common
[  317.799543] CPU: 1 PID: 4903 Comm: btrfs-balance Not tainted 3.15.1-amd64-i915-preempt-20140216jbp4 #1
[  317.799548] Hardware name: Dell Inc. Precision WorkStation T3500  /09KPNV, BIOS A10 01/21/2011
[  317.799555] task: ffff8805abd56450 ti: ffff8805abd58000 task.ti: ffff8805abd58000
[  317.799560] RIP: 0010:[<ffffffff81265654>]  [<ffffffff81265654>] list_del+0x8/0x2f
[  317.799573] RSP: 0018:ffff8805abd5bc00  EFLAGS: 00010287
[  317.799579] RAX: dead000000200200 RBX: ffff8805abfb1640 RCX: ffff8805f57b88e8
[  317.799584] RDX: dead000000100100 RSI: ffff8805f6d83940 RDI: ffff8805abff8750
[  317.799589] RBP: ffff8805abd5bc40 R08: 0000000000000000 R09: 0000000000000000
[  317.799594] R10: 00000000ffffffff R11: 0000000000000000 R12: ffff8805f6d83940
[  317.799599] R13: ffff8805f57b8820 R14: ffff8805abff8740 R15: ffff8805f6d83980
[  317.799605] FS:  0000000000000000(0000) GS:ffff880617220000(0000) knlGS:0000000000000000
[  317.799610] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[  317.799615] CR2: 00007f644026f000 CR3: 0000000001c13000 CR4: 00000000000007e0
[  317.799621] Stack:
[  317.799625]  ffff8805abd5bc40 ffffffff81268c9d ffff8805f57b8924 ffff8805ba7f86e0
[  317.799643]  ffff8805f57b8908 ffff8805f57b88e8 00000000ffffffea ffff8805f57b8800
[  317.799659]  ffff8805abd5bcb8 ffffffff8126af28 ffff8805f57b8820 00000001138d93a8
[  317.799674] Call Trace:
[  317.799683]  [<ffffffff81268c9d>] ? remove_backref_node+0x4c/0xd5
[  317.799690]  [<ffffffff8126af28>] relocate_block_group+0x390/0x49a
[  317.799698]  [<ffffffff8126b18d>] btrfs_relocate_block_group+0x15b/0x26d
[  317.799706]  [<ffffffff81249b80>] btrfs_relocate_chunk.isra.23+0x5c/0x5e8
[  317.799715]  [<ffffffff8161fc1b>] ? _raw_spin_unlock+0x17/0x2a
[  317.799722]  [<ffffffff812458cc>] ? free_extent_buffer+0x8a/0x8d
[  317.799729]  [<ffffffff8124c406>] btrfs_balance+0x9b6/0xb74
[  317.799737]  [<ffffffff816167cd>] ? printk+0x54/0x56
[  317.799745]  [<ffffffff8124c5c4>] ? btrfs_balance+0xb74/0xb74
[  317.799752]  [<ffffffff8124c61d>] balance_kthread+0x59/0x7b
[  317.799759]  [<ffffffff8106b4b4>] kthread+0xae/0xb6
[  317.799765]  [<ffffffff8106b406>] ? __kthread_parkme+0x61/0x61
[  317.799774]  [<ffffffff8162677c>] ret_from_fork+0x7c/0xb0
[  317.799780]  [<ffffffff8106b406>] ? __kthread_parkme+0x61/0x61
[  317.799785] Code: 00 00 00 48 c7 c7 fd 89 aa 81 e8 ad 41 eb ff 48 85 c0 48 89 05 6e 6b cb 00 0f 84 7b ff ff ff 31 c0 5d c3 48 8b 47 08 48 8b 17 55 <48> 89 42 08 48 89 10 48 b8 00 01 10 00 00 00 ad de 48 89 07 48 
[  317.799984] RIP  [<ffffffff81265654>] list_del+0x8/0x2f
[  317.799994]  RSP <ffff8805abd5bc00>
[  317.800032] ---[ end trace a9b76875452f420d ]---
[  317.800039] Kernel panic - not syncing: Fatal exception
[  317.800181] Kernel Offset: 0x0 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffff9fffffff)
[  317.800187] ---[ end Kernel panic - not syncing: Fatal exception

-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: btrfs balance crash BUG ON fs/btrfs/relocation.c:1062 or RIP build_backref_tree+0x9fc/0xcc4

[Josef Bacik]

On 06/25/2014 12:40 PM, Marc MERLIN wrote:
> On Fri, Jun 20, 2014 at 08:40:49AM -0700, Josef Bacik wrote:
>> On 06/19/2014 05:53 PM, Marc MERLIN wrote:
>>> On Thu, Jun 19, 2014 at 03:50:16PM -0700, Josef Bacik wrote:
>>>> Ok same drill as before, reset and apply this, hopefully no panic this
>>>> time
>>>>
>>>>
>>>> diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
>>>> index 65245a0..bca5240 100644
>>>
>>
>> Ok I see what it is but I want to get rid of the panicing so we're going
>> to do this dance a few more times until it's just failing to mount instead
>> of panicing, and then we'll fix the actual bug.  Give this a whirl, and
>> I've added another printk just to make sure what I think is happening is
>> actually what's happening, so same drill as before.  Thanks,
>
> Patch applied. The panic moved :)
>

Is it possible for me to get a btrfs-image of this fs?  That would make this a
lot faster.  Thanks,

Josef

kernel BUG at fs/btrfs/relocation.c:1065 in 3.14.16 to 3.17-rc3

[Olivier Bonvalet]

Hi,

I have a btrfs partition which throw kernel BUG, even with linux
3.17-rc3 (I tried 3.14.16, 3.16.1 and 3.17-rc3 kernels) :

[   45.058466] ------------[ cut here ]------------
[   45.058539] kernel BUG at fs/btrfs/relocation.c:1065!
[   45.058578] invalid opcode: 0000 [#1] SMP 
[   45.058655] Modules linked in: nf_conntrack iTCO_wdt iTCO_vendor_support i2c_i801 lpc_ich ehci_pci i2ccore ehci_hcd mfd_core evdev battery ie31200_edac edac_core video button btrfs xor raid6_pq dm_mod raid1 md_mod sg sd_mod crc_t10dif crct10dif_common thermal ahci libahci libata scsi_mod xhci_hcd e1000e fan ptp pps_core
[   45.059500] CPU: 2 PID: 1740 Comm: btrfs-balance Not tainted 3.17-rc3-dae-intel #1
[   45.059550] Hardware name: Digicube sas DediCube/DQ77MK, BIOS MKQ7710H.86A.0058.2013.0226.1541 02/26/2013
[   45.059602] task: ffff8802151c17e0 ti: ffff8802105ec000 task.ti: ffff8802105ec000
[   45.059652] RIP: 0010:[<ffffffffa019fcf8>]  [<ffffffffa019fcf8>] build_backref_tree+0xa3d/0xcf6 [btrfs]
[   45.059739] RSP: 0018:ffff8802105efaf0  EFLAGS: 00010246
[   45.059776] RAX: ffff8802105efb00 RBX: ffff880213b83800 RCX: ffff880210565d10
[   45.059816] RDX: ffff8802105efb68 RSI: ffff8802105efb68 RDI: ffff880210565d10
[   45.059857] RBP: ffff880210565d10 R08: ffff88021313fc40 R09: 0000000000001000
[   45.059896] R10: 0000160000000000 R11: 6db6db6db6db6db7 R12: ffff8800d114d310
[   45.059937] R13: ffff8802105efb78 R14: ffff8800d114d2c0 R15: ffff88021313fc40
[   45.059977] FS:  0000000000000000(0000) GS:ffff88021e280000(0000) knlGS:0000000000000000
[   45.060028] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   45.060066] CR2: 00007f2fc9c649b8 CR3: 0000000001611000 CR4: 00000000001407e0
[   45.060105] Stack:
[   45.060138]  ffff88021d5fc890 ffff88021417d890 0000000000000000 ffff8802105efb68
[   45.060264]  0000000100000005 ffff880213b83920 ffff8802105efb78 00ffffffa015ecd1
[   45.060392]  ffff8800d114d400 ffff8800d114d240 ffff880210464220 ffff880210565d40
[   45.060516] Call Trace:
[   45.060556]  [<ffffffffa01a11ff>] ? relocate_tree_blocks+0x15f/0x430 [btrfs]
[   45.060607]  [<ffffffffa019cd78>] ? tree_insert+0x44/0x47 [btrfs]
[   45.060656]  [<ffffffffa019e2ba>] ? add_tree_block+0x112/0x13c [btrfs]
[   45.060702]  [<ffffffffa01a1ff7>] ? relocate_block_group+0x26d/0x4a6 [btrfs]
[   45.060753]  [<ffffffffa0179014>] ? btrfs_wait_ordered_roots+0x18f/0x1ab [btrfs]
[   45.060812]  [<ffffffffa01a2384>] ? btrfs_relocate_block_group+0x154/0x265 [btrfs]
[   45.060872]  [<ffffffffa0181d01>] ? btrfs_relocate_chunk.isra.29+0x52/0x55d [btrfs]
[   45.060932]  [<ffffffffa018fdc8>] ? btrfs_set_lock_blocking_rw+0xa8/0xaa [btrfs]
[   45.060988]  [<ffffffffa0145680>] ? btrfs_item_key_to_cpu+0x12/0x30 [btrfs]
[   45.061039]  [<ffffffffa017767f>] ? btrfs_get_token_64+0x75/0xcf [btrfs]
[   45.061088]  [<ffffffffa017e201>] ? release_extent_buffer+0x26/0x96 [btrfs]
[   45.061170]  [<ffffffffa0184965>] ? btrfs_balance+0x9e3/0xb78 [btrfs]
[   45.061263]  [<ffffffffa0184afa>] ? btrfs_balance+0xb78/0xb78 [btrfs]
[   45.061314]  [<ffffffffa0184b49>] ? balance_kthread+0x4f/0x6d [btrfs]
[   45.061360]  [<ffffffff8104704b>] ? kthread+0xa7/0xaf
[   45.061420]  [<ffffffff81040000>] ? SyS_old_getrlimit+0x21/0xcb
[   45.061460]  [<ffffffff81046fa4>] ? __kthread_parkme+0x5b/0x5b
[   45.061501]  [<ffffffff813834ec>] ? ret_from_fork+0x7c/0xb0
[   45.061541]  [<ffffffff81046fa4>] ? __kthread_parkme+0x5b/0x5b
[   45.061579] Code: 26 a8 02 74 0d 4c 89 e7 e8 3c e1 ff ff 41 80 66 71 fd 49 8b 46 58 49 89 6e 58 4c 89 65 00 48 89 45 08 48 89 28 eb c0 a8 10 75 02 <0f> 0b 83 e0 01 39 44 24 10 0f 84 20 ff ff ff 0f 0b 49 8b 46 58 
[   45.063148] RIP  [<ffffffffa019fcf8>] build_backref_tree+0xa3d/0xcf6 [btrfs]
[   45.063219]  RSP <ffff8802105efaf0>
[   45.063260] ---[ end trace c396e96e4d1a5697 ]---

I have dump the FS with btrfs-image, but don't know where to push that.
So you can download it at : https://daevel.fr/img/btrfs-image.out
(near 6GB, md5sum ee5559ab31368aba60c259ce3b5b9504)

kernel BUG at fs/btrfs/extent-tree.c:7727! with 3.17-rc3

[Tomasz Chmielewski]

Got the following with 3.17-rc3 and running balance (had to power cycle 
after that):

[ 1329.952600] ------------[ cut here ]------------
[ 1329.952671] WARNING: CPU: 7 PID: 3106 at fs/btrfs/extent-tree.c:876 
btrfs_lookup_extent_info+0x377/0x3eb [btrfs]()
[ 1329.952726] Modules linked in: ipt_MASQUERADE iptable_nat 
nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack 
ip_tables x_tables cpufreq_ondemand cpufreq_conservative 
cpufreq_powersave cpufreq_stats bridge stp llc ipv6 btrfs xor raid6_pq 
zlib_deflate coretemp hwmon loop parport_pc parport pcspkr i2c_i801 
tpm_infineon tpm_tis tpm i2ccore video battery lpc_ich mfd_core ehci_pci 
ehci_hcd button acpi_cpufreq ext4 crc16 jbd2 mbcache raid1 sg sd_mod 
ahci libahci libata scsi_mod r8169 mii
[ 1329.954740] CPU: 7 PID: 3106 Comm: btrfs-balance Not tainted 
3.17.0-rc3 #1
[ 1329.954789] Hardware name: System manufacturer System Product 
Name/P8H77-M PRO, BIOS 1101 02/04/2013
[ 1329.954841]  0000000000000009 ffff880733d4f8d8 ffffffff813ab092 
0000000000000000
[ 1329.955030]  0000000000000000 ffff880733d4f918 ffffffff81039b41 
ffffffff00000007
[ 1329.955219]  ffffffffa02d8560 ffff8807aa536120 0000000000000000 
0000000000000000
[ 1329.955407] Call Trace:
[ 1329.955455]  [<ffffffff813ab092>] dump_stack+0x46/0x58
[ 1329.955503]  [<ffffffff81039b41>] warn_slowpath_common+0x77/0x91
[ 1329.955610]  [<ffffffffa02d8560>] ? 
btrfs_lookup_extent_info+0x377/0x3eb [btrfs]
[ 1329.955758]  [<ffffffff81039b70>] warn_slowpath_null+0x15/0x17
[ 1329.955862]  [<ffffffffa02d8560>] 
btrfs_lookup_extent_info+0x377/0x3eb [btrfs]
[ 1329.956018]  [<ffffffffa02d8c8f>] walk_down_proc+0xc5/0x22b [btrfs]
[ 1329.956128]  [<ffffffffa02ebd20>] ? 
join_transaction.isra.30+0x24/0x309 [btrfs]
[ 1329.956285]  [<ffffffffa02dae92>] walk_down_tree+0x45/0xd5 [btrfs]
[ 1329.956391]  [<ffffffffa02dd89a>] btrfs_drop_snapshot+0x2f5/0x68f 
[btrfs]
[ 1329.956505]  [<ffffffffa032e19d>] merge_reloc_roots+0x139/0x23f 
[btrfs]
[ 1329.956617]  [<ffffffffa032e8f6>] relocate_block_group+0x466/0x4de 
[btrfs]
[ 1329.956728]  [<ffffffffa032eac6>] 
btrfs_relocate_block_group+0x158/0x278 [btrfs]
[ 1329.956890]  [<ffffffffa030b6f0>] 
btrfs_relocate_chunk.isra.62+0x58/0x5f7 [btrfs]
[ 1329.957073]  [<ffffffffa031a8d7>] ? 
btrfs_set_lock_blocking_rw+0x68/0x95 [btrfs]
[ 1329.957214]  [<ffffffffa02cbb04>] ? btrfs_set_path_blocking+0x23/0x54 
[btrfs]
[ 1329.957297]  [<ffffffffa02d0517>] ? btrfs_search_slot+0x7bc/0x816 
[btrfs]
[ 1329.957382]  [<ffffffffa0307bd5>] ? free_extent_buffer+0x6f/0x7c 
[btrfs]
[ 1329.957467]  [<ffffffffa030e5cd>] btrfs_balance+0xa7b/0xc80 [btrfs]
[ 1329.957547]  [<ffffffff813a8e15>] ? printk+0x48/0x4a
[ 1329.957629]  [<ffffffffa030e829>] balance_kthread+0x57/0x7c [btrfs]
[ 1329.957724]  [<ffffffffa030e7d2>] ? btrfs_balance+0xc80/0xc80 [btrfs]
[ 1329.957807]  [<ffffffffa030e7d2>] ? btrfs_balance+0xc80/0xc80 [btrfs]
[ 1329.957887]  [<ffffffff8104e993>] kthread+0xcd/0xd5
[ 1329.957965]  [<ffffffff8104e8c6>] ? 
kthread_freezable_should_stop+0x43/0x43
[ 1329.958045]  [<ffffffff813afcec>] ret_from_fork+0x7c/0xb0
[ 1329.958122]  [<ffffffff8104e8c6>] ? 
kthread_freezable_should_stop+0x43/0x43
[ 1329.958210] ---[ end trace a368b0643f9207e2 ]---
[ 1329.958293] ------------[ cut here ]------------
[ 1329.958378] kernel BUG at fs/btrfs/extent-tree.c:7727!
[ 1329.958455] invalid opcode: 0000 [#1] SMP
[ 1329.958593] Modules linked in: ipt_MASQUERADE iptable_nat 
nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack 
ip_tables x_tables cpufreq_ondemand cpufreq_conservative 
cpufreq_powersave cpufreq_stats bridge stp llc ipv6 btrfs xor raid6_pq 
zlib_deflate coretemp hwmon loop parport_pc parport pcspkr i2c_i801 
tpm_infineon tpm_tis tpm i2ccore video battery lpc_ich mfd_core ehci_pci 
ehci_hcd button acpi_cpufreq ext4 crc16 jbd2 mbcache raid1 sg sd_mod 
ahci libahci libata scsi_mod r8169 mii
[ 1329.960684] CPU: 7 PID: 3106 Comm: btrfs-balance Tainted: G        W  
     3.17.0-rc3 #1
[ 1329.960803] Hardware name: System manufacturer System Product 
Name/P8H77-M PRO, BIOS 1101 02/04/2013
[ 1329.960924] task: ffff8807f18c0000 ti: ffff880733d4c000 task.ti: 
ffff880733d4c000
[ 1329.961043] RIP: 0010:[<ffffffffa02d8ca6>]  [<ffffffffa02d8ca6>] 
walk_down_proc+0xdc/0x22b [btrfs]
[ 1329.961200] RSP: 0018:ffff880733d4f9e8  EFLAGS: 00010246
[ 1329.961277] RAX: 0000000000000000 RBX: 0000000000000002 RCX: 
00000000000f5a50
[ 1329.961356] RDX: 00000000000f5a4f RSI: ffff88081fbd9650 RDI: 
0000000000019650
[ 1329.961436] RBP: ffff880733d4fa38 R08: ffffea001ea94d80 R09: 
00000000000009a2
[ 1329.961515] R10: ffffffffa02cbc38 R11: 0000000000000000 R12: 
ffff8807aa536d80
[ 1329.961594] R13: ffff880733ac5600 R14: ffff880660ba65c8 R15: 
0000000000000002
[ 1329.961674] FS:  0000000000000000(0000) GS:ffff88081fbc0000(0000) 
knlGS:0000000000000000
[ 1329.961794] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1329.961872] CR2: 00007f7fa0c3e000 CR3: 0000000001611000 CR4: 
00000000001407e0
[ 1329.961951] Stack:
[ 1329.962024]  ffff880733ac5650 ffffffffa02ebd20 ffff880732a34820 
ffff8807eb201000
[ 1329.962267]  0000000000000000 ffff8807aa536d80 0000000000000002 
ffff880732a34820
[ 1329.962510]  ffff8807eb201000 ffff880733ac5600 ffff880733d4fa98 
ffffffffa02dae92
[ 1329.962754] Call Trace:
[ 1329.962834]  [<ffffffffa02ebd20>] ? 
join_transaction.isra.30+0x24/0x309 [btrfs]
[ 1329.962957]  [<ffffffffa02dae92>] walk_down_tree+0x45/0xd5 [btrfs]
[ 1329.963040]  [<ffffffffa02dd89a>] btrfs_drop_snapshot+0x2f5/0x68f 
[btrfs]
[ 1329.963126]  [<ffffffffa032e19d>] merge_reloc_roots+0x139/0x23f 
[btrfs]
[ 1329.963210]  [<ffffffffa032e8f6>] relocate_block_group+0x466/0x4de 
[btrfs]
[ 1329.963295]  [<ffffffffa032eac6>] 
btrfs_relocate_block_group+0x158/0x278 [btrfs]
[ 1329.963419]  [<ffffffffa030b6f0>] 
btrfs_relocate_chunk.isra.62+0x58/0x5f7 [btrfs]
[ 1329.963544]  [<ffffffffa031a8d7>] ? 
btrfs_set_lock_blocking_rw+0x68/0x95 [btrfs]
[ 1329.963666]  [<ffffffffa02cbb04>] ? btrfs_set_path_blocking+0x23/0x54 
[btrfs]
[ 1329.963749]  [<ffffffffa02d0517>] ? btrfs_search_slot+0x7bc/0x816 
[btrfs]
[ 1329.963834]  [<ffffffffa0307bd5>] ? free_extent_buffer+0x6f/0x7c 
[btrfs]
[ 1329.963918]  [<ffffffffa030e5cd>] btrfs_balance+0xa7b/0xc80 [btrfs]
[ 1329.963998]  [<ffffffff813a8e15>] ? printk+0x48/0x4a
[ 1329.964080]  [<ffffffffa030e829>] balance_kthread+0x57/0x7c [btrfs]
[ 1329.964163]  [<ffffffffa030e7d2>] ? btrfs_balance+0xc80/0xc80 [btrfs]
[ 1329.964246]  [<ffffffffa030e7d2>] ? btrfs_balance+0xc80/0xc80 [btrfs]
[ 1329.964326]  [<ffffffff8104e993>] kthread+0xcd/0xd5
[ 1329.964404]  [<ffffffff8104e8c6>] ? 
kthread_freezable_should_stop+0x43/0x43
[ 1329.964484]  [<ffffffff813afcec>] ret_from_fork+0x7c/0xb0
[ 1329.964562]  [<ffffffff8104e8c6>] ? 
kthread_freezable_should_stop+0x43/0x43
[ 1329.964641] Code: b8 01 00 00 00 4c 89 f9 48 89 04 24 49 8b 16 e8 5a 
f5 ff ff 83 f8 f4 75 02 0f 0b 85 c0 0f 85 48 01 00 00 4b 83 7c fd 00 00 
75 02 <0f> 0b 41 83 bd 94 00 00 00 01 48 63 c3 75 2d 49 83 7c c5 00 01
[ 1329.966842] RIP  [<ffffffffa02d8ca6>] walk_down_proc+0xdc/0x22b 
[btrfs]
[ 1329.966956]  RSP <ffff880733d4f9e8>
[ 1329.967040] ---[ end trace a368b0643f9207e3 ]---


-- 
Tomasz Chmielewski
http://www.sslrack.com

Re: kernel BUG at fs/btrfs/extent-tree.c:7727! with 3.17-rc3

[Tomasz Chmielewski]

On 2014-09-03 19:42 (Wed), Tomasz Chmielewski wrote:
> Got the following with 3.17-rc3 and running balance (had to power
> cycle after that):

I'm seeing similar BUG with 3.17-rc4:

[ 1049.755843] BTRFS info (device sdb5): found 35715 extents
[ 1050.257075] BTRFS info (device sdb5): relocating block group 
7091332251648 flags 20
[ 2087.976104] BTRFS info (device sdb5): found 40911 extents
[ 2091.357102] BTRFS info (device sdb5): relocating block group 
7006506647552 flags 20
[ 2518.793237] INFO: task btrfs-balance:5688 blocked for more than 120 
seconds.
[ 2518.793263]       Not tainted 3.17.0-rc4 #1
[ 2518.793269] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" 
disables this message.
[ 2518.793279] btrfs-balance   D ffff88081fad15c0     0  5688      2 
0x00000000
[ 2518.793291]  ffff8807dceebac8 0000000000000046 ffff8807dceeb9c8 
ffff8807f0f83020
[ 2518.793303]  00000000000115c0 0000000000004000 ffff8807f4153020 
ffff8807f0f83020
[ 2518.793315]  ffff8807dceeba08 ffffffff8105639d 0000000000000000 
ffff8807f3cd8050
[ 2518.793404] Call Trace:
[ 2518.793452]  [<ffffffff8105639d>] ? wake_up_process+0x31/0x35
[ 2518.793500]  [<ffffffff81049baf>] ? wake_up_worker+0x1f/0x21
[ 2518.793547]  [<ffffffff81049df6>] ? insert_work+0x87/0x94
[ 2518.793605]  [<ffffffffa031724b>] ? free_block_list+0x1f/0x34 [btrfs]
[ 2518.793655]  [<ffffffff813ad75e>] ? wait_for_common+0x118/0x13e
[ 2518.793703]  [<ffffffff813acf2e>] schedule+0x65/0x67
[ 2518.793755]  [<ffffffffa02d934d>] 
wait_current_trans.isra.32+0x94/0xe2 [btrfs]
[ 2518.793843]  [<ffffffff81063cdc>] ? add_wait_queue+0x44/0x44
[ 2518.793895]  [<ffffffffa02da82f>] start_transaction+0x427/0x472 
[btrfs]
[ 2518.793948]  [<ffffffffa02dab02>] btrfs_start_transaction+0x16/0x18 
[btrfs]
[ 2518.794002]  [<ffffffffa031b51a>] relocate_block_group+0x8a/0x4de 
[btrfs]
[ 2518.794055]  [<ffffffffa031bac6>] 
btrfs_relocate_block_group+0x158/0x278 [btrfs]
[ 2518.794148]  [<ffffffffa02f86f0>] 
btrfs_relocate_chunk.isra.62+0x58/0x5f7 [btrfs]
[ 2518.794241]  [<ffffffffa03078d7>] ? 
btrfs_set_lock_blocking_rw+0x68/0x95 [btrfs]
[ 2518.794332]  [<ffffffffa02b8b04>] ? btrfs_set_path_blocking+0x23/0x54 
[btrfs]
[ 2518.794383]  [<ffffffffa02bd517>] ? btrfs_search_slot+0x7bc/0x816 
[btrfs]
[ 2518.794437]  [<ffffffffa02f4bd5>] ? free_extent_buffer+0x6f/0x7c 
[btrfs]
[ 2518.794490]  [<ffffffffa02fb5cd>] btrfs_balance+0xa7b/0xc80 [btrfs]
[ 2518.794538]  [<ffffffff813a9125>] ? printk+0x48/0x4a
[ 2518.794589]  [<ffffffffa02fb829>] balance_kthread+0x57/0x7c [btrfs]
[ 2518.794641]  [<ffffffffa02fb7d2>] ? btrfs_balance+0xc80/0xc80 [btrfs]
[ 2518.794692]  [<ffffffffa02fb7d2>] ? btrfs_balance+0xc80/0xc80 [btrfs]
[ 2518.794741]  [<ffffffff8104e993>] kthread+0xcd/0xd5
[ 2518.794787]  [<ffffffff8104e8c6>] ? 
kthread_freezable_should_stop+0x43/0x43
[ 2518.794836]  [<ffffffff813affec>] ret_from_fork+0x7c/0xb0
[ 2518.794883]  [<ffffffff8104e8c6>] ? 
kthread_freezable_should_stop+0x43/0x43
[ 2518.794935] INFO: task kworker/u16:21:8725 blocked for more than 120 
seconds.
[ 2518.794983]       Not tainted 3.17.0-rc4 #1
[ 2518.795028] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" 
disables this message.
[ 2518.795116] kworker/u16:21  D ffff88081fbd15c0     0  8725      2 
0x00000000
[ 2518.795170] Workqueue: events_unbound 
btrfs_async_reclaim_metadata_space [btrfs]
[ 2518.795258]  ffff8805bf363b48 0000000000000046 ffff8805bf363a88 
ffff8807f3244830
[ 2518.795347]  00000000000115c0 0000000000004000 ffff8807f4171810 
ffff8807f3244830
[ 2518.795436]  ffff8805bf363a88 ffffffff81053484 0000000000200000 
ffff88081fa915c0
[ 2518.795525] Call Trace:
[ 2518.795568]  [<ffffffff81053484>] ? resched_curr+0x47/0x57
[ 2518.795615]  [<ffffffff81053aed>] ? check_preempt_curr+0x3e/0x6d
[ 2518.795663]  [<ffffffff8105634c>] ? try_to_wake_up+0x240/0x251
[ 2518.795710]  [<ffffffff8105639d>] ? wake_up_process+0x31/0x35
[ 2518.795758]  [<ffffffff813acf2e>] schedule+0x65/0x67
[ 2518.795804]  [<ffffffff813af01c>] schedule_timeout+0x26/0x198
[ 2518.795851]  [<ffffffff8104a0df>] ? __queue_work+0x1d2/0x204
[ 2518.795899]  [<ffffffff813ad753>] wait_for_common+0x10d/0x13e
[ 2518.795946]  [<ffffffff8105635d>] ? try_to_wake_up+0x251/0x251
[ 2518.795993]  [<ffffffff813ad79c>] wait_for_completion+0x18/0x1a
[ 2518.796042]  [<ffffffff8111d8aa>] writeback_inodes_sb_nr+0x87/0x8e
[ 2518.796093]  [<ffffffffa02d0101>] ? btrfs_del_csums+0x1b6/0x2ca 
[btrfs]
[ 2518.796145]  [<ffffffffa02c63c9>] flush_space+0x1af/0x407 [btrfs]
[ 2518.796196]  [<ffffffffa02c5e56>] ? btrfs_get_alloc_profile+0x2b/0x2d 
[btrfs]
[ 2518.796248]  [<ffffffffa02c618d>] ? can_overcommit+0x4f/0xdc [btrfs]
[ 2518.796299]  [<ffffffffa02c6a71>] 
btrfs_async_reclaim_metadata_space+0x10c/0x157 [btrfs]
[ 2518.796388]  [<ffffffff8104aadb>] process_one_work+0x188/0x2ab
[ 2518.796435]  [<ffffffff8104ae89>] worker_thread+0x261/0x35e
[ 2518.796482]  [<ffffffff8104ac28>] ? process_scheduled_works+0x2a/0x2a
[ 2518.796530]  [<ffffffff8104e993>] kthread+0xcd/0xd5
[ 2518.796577]  [<ffffffff8104e8c6>] ? 
kthread_freezable_should_stop+0x43/0x43
[ 2518.796625]  [<ffffffff813affec>] ret_from_fork+0x7c/0xb0
[ 2518.796672]  [<ffffffff8104e8c6>] ? 
kthread_freezable_should_stop+0x43/0x43
[ 2638.722499] INFO: task kworker/u16:21:8725 blocked for more than 120 
seconds.
[ 2638.722568]       Not tainted 3.17.0-rc4 #1
[ 2638.722625] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" 
disables this message.
[ 2638.722737] kworker/u16:21  D ffff88081fbd15c0     0  8725      2 
0x00000000
[ 2638.722821] Workqueue: events_unbound 
btrfs_async_reclaim_metadata_space [btrfs]
[ 2638.722935]  ffff8805bf363b48 0000000000000046 ffff8805bf363a88 
ffff8807f3244830
[ 2638.723050]  00000000000115c0 0000000000004000 ffff8807f4171810 
ffff8807f3244830
[ 2638.723165]  ffff8805bf363a88 ffffffff81053484 0000000000200000 
ffff88081fa915c0
[ 2638.723279] Call Trace:
[ 2638.723339]  [<ffffffff81053484>] ? resched_curr+0x47/0x57
[ 2638.723401]  [<ffffffff81053aed>] ? check_preempt_curr+0x3e/0x6d
[ 2638.723465]  [<ffffffff8105634c>] ? try_to_wake_up+0x240/0x251
[ 2638.723526]  [<ffffffff8105639d>] ? wake_up_process+0x31/0x35
[ 2638.723590]  [<ffffffff813acf2e>] schedule+0x65/0x67
[ 2638.723650]  [<ffffffff813af01c>] schedule_timeout+0x26/0x198
[ 2638.723713]  [<ffffffff8104a0df>] ? __queue_work+0x1d2/0x204
[ 2638.723775]  [<ffffffff813ad753>] wait_for_common+0x10d/0x13e
[ 2638.723837]  [<ffffffff8105635d>] ? try_to_wake_up+0x251/0x251
[ 2638.723900]  [<ffffffff813ad79c>] wait_for_completion+0x18/0x1a
[ 2638.723963]  [<ffffffff8111d8aa>] writeback_inodes_sb_nr+0x87/0x8e
[ 2638.724037]  [<ffffffffa02d0101>] ? btrfs_del_csums+0x1b6/0x2ca 
[btrfs]
[ 2638.724109]  [<ffffffffa02c63c9>] flush_space+0x1af/0x407 [btrfs]
[ 2638.724180]  [<ffffffffa02c5e56>] ? btrfs_get_alloc_profile+0x2b/0x2d 
[btrfs]
[ 2638.724252]  [<ffffffffa02c618d>] ? can_overcommit+0x4f/0xdc [btrfs]
[ 2638.724322]  [<ffffffffa02c6a71>] 
btrfs_async_reclaim_metadata_space+0x10c/0x157 [btrfs]
[ 2638.724436]  [<ffffffff8104aadb>] process_one_work+0x188/0x2ab
[ 2638.724498]  [<ffffffff8104ae89>] worker_thread+0x261/0x35e
[ 2638.724559]  [<ffffffff8104ac28>] ? process_scheduled_works+0x2a/0x2a
[ 2638.724622]  [<ffffffff8104e993>] kthread+0xcd/0xd5
[ 2638.724683]  [<ffffffff8104e8c6>] ? 
kthread_freezable_should_stop+0x43/0x43
[ 2638.724747]  [<ffffffff813affec>] ret_from_fork+0x7c/0xb0
[ 2638.724808]  [<ffffffff8104e8c6>] ? 
kthread_freezable_should_stop+0x43/0x43
[ 2758.651776] INFO: task kworker/u16:21:8725 blocked for more than 120 
seconds.
[ 2758.651842]       Not tainted 3.17.0-rc4 #1
[ 2758.651899] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" 
disables this message.
[ 2758.652008] kworker/u16:21  D ffff88081fbd15c0     0  8725      2 
0x00000000
[ 2758.652091] Workqueue: events_unbound 
btrfs_async_reclaim_metadata_space [btrfs]
[ 2758.652201]  ffff8805bf363b48 0000000000000046 ffff8805bf363a88 
ffff8807f3244830
[ 2758.652313]  00000000000115c0 0000000000004000 ffff8807f4171810 
ffff8807f3244830
[ 2758.652425]  ffff8805bf363a88 ffffffff81053484 0000000000200000 
ffff88081fa915c0
[ 2758.652537] Call Trace:
[ 2758.652595]  [<ffffffff81053484>] ? resched_curr+0x47/0x57
[ 2758.652656]  [<ffffffff81053aed>] ? check_preempt_curr+0x3e/0x6d
[ 2758.652717]  [<ffffffff8105634c>] ? try_to_wake_up+0x240/0x251
[ 2758.652779]  [<ffffffff8105639d>] ? wake_up_process+0x31/0x35
[ 2758.652841]  [<ffffffff813acf2e>] schedule+0x65/0x67
[ 2758.652900]  [<ffffffff813af01c>] schedule_timeout+0x26/0x198
[ 2758.652962]  [<ffffffff8104a0df>] ? __queue_work+0x1d2/0x204
[ 2758.653023]  [<ffffffff813ad753>] wait_for_common+0x10d/0x13e
[ 2758.653084]  [<ffffffff8105635d>] ? try_to_wake_up+0x251/0x251
[ 2758.653145]  [<ffffffff813ad79c>] wait_for_completion+0x18/0x1a
[ 2758.653206]  [<ffffffff8111d8aa>] writeback_inodes_sb_nr+0x87/0x8e
[ 2758.653278]  [<ffffffffa02d0101>] ? btrfs_del_csums+0x1b6/0x2ca 
[btrfs]
[ 2758.653347]  [<ffffffffa02c63c9>] flush_space+0x1af/0x407 [btrfs]
[ 2758.653416]  [<ffffffffa02c5e56>] ? btrfs_get_alloc_profile+0x2b/0x2d 
[btrfs]
[ 2758.653486]  [<ffffffffa02c618d>] ? can_overcommit+0x4f/0xdc [btrfs]
[ 2758.653554]  [<ffffffffa02c6a71>] 
btrfs_async_reclaim_metadata_space+0x10c/0x157 [btrfs]
[ 2758.653665]  [<ffffffff8104aadb>] process_one_work+0x188/0x2ab
[ 2758.653726]  [<ffffffff8104ae89>] worker_thread+0x261/0x35e
[ 2758.653786]  [<ffffffff8104ac28>] ? process_scheduled_works+0x2a/0x2a
[ 2758.653848]  [<ffffffff8104e993>] kthread+0xcd/0xd5
[ 2758.653907]  [<ffffffff8104e8c6>] ? 
kthread_freezable_should_stop+0x43/0x43
[ 2758.653969]  [<ffffffff813affec>] ret_from_fork+0x7c/0xb0
[ 2758.654029]  [<ffffffff8104e8c6>] ? 
kthread_freezable_should_stop+0x43/0x43
[ 2878.581047] INFO: task kworker/u16:21:8725 blocked for more than 120 
seconds.
[ 2878.581117]       Not tainted 3.17.0-rc4 #1
[ 2878.581174] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" 
disables this message.
[ 2878.581287] kworker/u16:21  D ffff88081fbd15c0     0  8725      2 
0x00000000
[ 2878.581371] Workqueue: events_unbound 
btrfs_async_reclaim_metadata_space [btrfs]
[ 2878.582875]  ffff8805bf363b48 0000000000000046 ffff8805bf363a88 
ffff8807f3244830
[ 2878.582990]  00000000000115c0 0000000000004000 ffff8807f4171810 
ffff8807f3244830
[ 2878.583105]  ffff8805bf363a88 ffffffff81053484 0000000000200000 
ffff88081fa915c0
[ 2878.583220] Call Trace:
[ 2878.583280]  [<ffffffff81053484>] ? resched_curr+0x47/0x57
[ 2878.583342]  [<ffffffff81053aed>] ? check_preempt_curr+0x3e/0x6d
[ 2878.583404]  [<ffffffff8105634c>] ? try_to_wake_up+0x240/0x251
[ 2878.583467]  [<ffffffff8105639d>] ? wake_up_process+0x31/0x35
[ 2878.583531]  [<ffffffff813acf2e>] schedule+0x65/0x67
[ 2878.583591]  [<ffffffff813af01c>] schedule_timeout+0x26/0x198
[ 2878.583655]  [<ffffffff8104a0df>] ? __queue_work+0x1d2/0x204
[ 2878.583717]  [<ffffffff813ad753>] wait_for_common+0x10d/0x13e
[ 2878.583780]  [<ffffffff8105635d>] ? try_to_wake_up+0x251/0x251
[ 2878.583842]  [<ffffffff813ad79c>] wait_for_completion+0x18/0x1a
[ 2878.583905]  [<ffffffff8111d8aa>] writeback_inodes_sb_nr+0x87/0x8e
[ 2878.583980]  [<ffffffffa02d0101>] ? btrfs_del_csums+0x1b6/0x2ca 
[btrfs]
[ 2878.584052]  [<ffffffffa02c63c9>] flush_space+0x1af/0x407 [btrfs]
[ 2878.584123]  [<ffffffffa02c5e56>] ? btrfs_get_alloc_profile+0x2b/0x2d 
[btrfs]
[ 2878.584195]  [<ffffffffa02c618d>] ? can_overcommit+0x4f/0xdc [btrfs]
[ 2878.584266]  [<ffffffffa02c6a71>] 
btrfs_async_reclaim_metadata_space+0x10c/0x157 [btrfs]
[ 2878.584380]  [<ffffffff8104aadb>] process_one_work+0x188/0x2ab
[ 2878.584442]  [<ffffffff8104ae89>] worker_thread+0x261/0x35e
[ 2878.584503]  [<ffffffff8104ac28>] ? process_scheduled_works+0x2a/0x2a
[ 2878.584566]  [<ffffffff8104e993>] kthread+0xcd/0xd5
[ 2878.584627]  [<ffffffff8104e8c6>] ? 
kthread_freezable_should_stop+0x43/0x43
[ 2878.584691]  [<ffffffff813affec>] ret_from_fork+0x7c/0xb0
[ 2878.584752]  [<ffffffff8104e8c6>] ? 
kthread_freezable_should_stop+0x43/0x43
[ 3384.754985] ------------[ cut here ]------------
[ 3384.755050] WARNING: CPU: 2 PID: 5688 at fs/btrfs/extent-tree.c:876 
btrfs_lookup_extent_info+0x377/0x3eb [btrfs]()
[ 3384.755145] Modules linked in: ipt_MASQUERADE iptable_nat 
nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack 
ip_tables x_tables cpufreq_ondemand cpufreq_conservative 
cpufreq_powersave cpufreq_stats bridge stp llc ipv6 btrfs xor raid6_pq 
zlib_deflate coretemp hwmon loop i2c_i801 i2ccore pcspkr battery 
tpm_infineon tpm_tis tpm parport_pc parport video ehci_pci ehci_hcd 
lpc_ich mfd_core acpi_cpufreq button ext4 crc16 jbd2 mbcache raid1 sg 
sd_mod ahci libahci libata scsi_mod r8169 mii
[ 3384.755547] CPU: 2 PID: 5688 Comm: btrfs-balance Not tainted 
3.17.0-rc4 #1
[ 3384.755597] Hardware name: System manufacturer System Product 
Name/P8H77-M PRO, BIOS 1101 02/04/2013
[ 3384.755691]  0000000000000009 ffff8807dceeb8d8 ffffffff813ab3a2 
0000000000000000
[ 3384.755785]  0000000000000000 ffff8807dceeb918 ffffffff81039b41 
ffffffff00000024
[ 3384.755895]  ffffffffa02c5560 ffff8801b195bc60 0000000000000000 
0000000000000000
[ 3384.755990] Call Trace:
[ 3384.756040]  [<ffffffff813ab3a2>] dump_stack+0x46/0x58
[ 3384.756090]  [<ffffffff81039b41>] warn_slowpath_common+0x77/0x91
[ 3384.756148]  [<ffffffffa02c5560>] ? 
btrfs_lookup_extent_info+0x377/0x3eb [btrfs]
[ 3384.756240]  [<ffffffff81039b70>] warn_slowpath_null+0x15/0x17
[ 3384.756295]  [<ffffffffa02c5560>] 
btrfs_lookup_extent_info+0x377/0x3eb [btrfs]
[ 3384.756393]  [<ffffffffa02c5c8f>] walk_down_proc+0xc5/0x22b [btrfs]
[ 3384.756452]  [<ffffffffa02d8d20>] ? 
join_transaction.isra.30+0x24/0x309 [btrfs]
[ 3384.756545]  [<ffffffffa02c7e92>] walk_down_tree+0x45/0xd5 [btrfs]
[ 3384.756597]  [<ffffffffa02ca89a>] btrfs_drop_snapshot+0x2f5/0x68f 
[btrfs]
[ 3384.756653]  [<ffffffffa031b19d>] merge_reloc_roots+0x139/0x23f 
[btrfs]
[ 3384.756707]  [<ffffffffa031b8f6>] relocate_block_group+0x466/0x4de 
[btrfs]
[ 3384.756760]  [<ffffffffa031bac6>] 
btrfs_relocate_block_group+0x158/0x278 [btrfs]
[ 3384.756854]  [<ffffffffa02f86f0>] 
btrfs_relocate_chunk.isra.62+0x58/0x5f7 [btrfs]
[ 3384.756947]  [<ffffffffa03078d7>] ? 
btrfs_set_lock_blocking_rw+0x68/0x95 [btrfs]
[ 3384.757038]  [<ffffffffa02b8b04>] ? btrfs_set_path_blocking+0x23/0x54 
[btrfs]
[ 3384.757089]  [<ffffffffa02bd517>] ? btrfs_search_slot+0x7bc/0x816 
[btrfs]
[ 3384.757143]  [<ffffffffa02f4bd5>] ? free_extent_buffer+0x6f/0x7c 
[btrfs]
[ 3384.757197]  [<ffffffffa02fb5cd>] btrfs_balance+0xa7b/0xc80 [btrfs]
[ 3384.757246]  [<ffffffff813a9125>] ? printk+0x48/0x4a
[ 3384.757297]  [<ffffffffa02fb829>] balance_kthread+0x57/0x7c [btrfs]
[ 3384.757349]  [<ffffffffa02fb7d2>] ? btrfs_balance+0xc80/0xc80 [btrfs]
[ 3384.757401]  [<ffffffffa02fb7d2>] ? btrfs_balance+0xc80/0xc80 [btrfs]
[ 3384.757450]  [<ffffffff8104e993>] kthread+0xcd/0xd5
[ 3384.757496]  [<ffffffff8104e8c6>] ? 
kthread_freezable_should_stop+0x43/0x43
[ 3384.757545]  [<ffffffff813affec>] ret_from_fork+0x7c/0xb0
[ 3384.757592]  [<ffffffff8104e8c6>] ? 
kthread_freezable_should_stop+0x43/0x43
[ 3384.757640] ---[ end trace 4f32f915eb18c087 ]---
[ 3384.757691] ------------[ cut here ]------------
[ 3384.757736] kernel BUG at fs/btrfs/extent-tree.c:7727!
[ 3384.757782] invalid opcode: 0000 [#1] SMP
[ 3384.757827] Modules linked in: ipt_MASQUERADE iptable_nat 
nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack 
ip_tables x_tables cpufreq_ondemand cpufreq_conservative 
cpufreq_powersave cpufreq_stats bridge stp llc ipv6 btrfs xor raid6_pq 
zlib_deflate coretemp hwmon loop i2c_i801 i2ccore pcspkr battery 
tpm_infineon tpm_tis tpm parport_pc parport video ehci_pci ehci_hcd 
lpc_ich mfd_core acpi_cpufreq button ext4 crc16 jbd2 mbcache raid1 sg 
sd_mod ahci libahci libata scsi_mod r8169 mii
[ 3384.758182] CPU: 2 PID: 5688 Comm: btrfs-balance Tainted: G        W  
     3.17.0-rc4 #1
[ 3384.758270] Hardware name: System manufacturer System Product 
Name/P8H77-M PRO, BIOS 1101 02/04/2013
[ 3384.758360] task: ffff8807f0f83020 ti: ffff8807dcee8000 task.ti: 
ffff8807dcee8000
[ 3384.758448] RIP: 0010:[<ffffffffa02c5ca6>]  [<ffffffffa02c5ca6>] 
walk_down_proc+0xdc/0x22b [btrfs]
[ 3384.758542] RSP: 0018:ffff8807dceeb9e8  EFLAGS: 00010246
[ 3384.758588] RAX: 0000000000000000 RBX: 0000000000000002 RCX: 
0000000000984a1c
[ 3384.758636] RDX: 0000000000984a1b RSI: ffff88081fa99650 RDI: 
0000000000019650
[ 3384.758684] RBP: ffff8807dceeba38 R08: ffffea0006c656c0 R09: 
0000000000000000
[ 3384.758733] R10: ffffffffa02b8c38 R11: 0000000000000000 R12: 
ffff8801b195bbd0
[ 3384.758781] R13: ffff88010d092d80 R14: ffff880102ba33a8 R15: 
0000000000000002
[ 3384.758830] FS:  0000000000000000(0000) GS:ffff88081fa80000(0000) 
knlGS:0000000000000000
[ 3384.758918] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 3384.758965] CR2: 00007f980e95d000 CR3: 0000000001611000 CR4: 
00000000001407e0
[ 3384.759013] Stack:
[ 3384.759054]  ffff88010d092dd0 ffffffffa02d8d20 ffff880102516d20 
ffff8807449a9000
[ 3384.759143]  0000000000000000 ffff8801b195bbd0 0000000000000002 
ffff880102516d20
[ 3384.759232]  ffff8807449a9000 ffff88010d092d80 ffff8807dceeba98 
ffffffffa02c7e92
[ 3384.759321] Call Trace:
[ 3384.759369]  [<ffffffffa02d8d20>] ? 
join_transaction.isra.30+0x24/0x309 [btrfs]
[ 3384.759461]  [<ffffffffa02c7e92>] walk_down_tree+0x45/0xd5 [btrfs]
[ 3384.759513]  [<ffffffffa02ca89a>] btrfs_drop_snapshot+0x2f5/0x68f 
[btrfs]
[ 3384.759567]  [<ffffffffa031b19d>] merge_reloc_roots+0x139/0x23f 
[btrfs]
[ 3384.759620]  [<ffffffffa031b8f6>] relocate_block_group+0x466/0x4de 
[btrfs]
[ 3384.759673]  [<ffffffffa031bac6>] 
btrfs_relocate_block_group+0x158/0x278 [btrfs]
[ 3384.759766]  [<ffffffffa02f86f0>] 
btrfs_relocate_chunk.isra.62+0x58/0x5f7 [btrfs]
[ 3384.759859]  [<ffffffffa03078d7>] ? 
btrfs_set_lock_blocking_rw+0x68/0x95 [btrfs]
[ 3384.759950]  [<ffffffffa02b8b04>] ? btrfs_set_path_blocking+0x23/0x54 
[btrfs]
[ 3384.760001]  [<ffffffffa02bd517>] ? btrfs_search_slot+0x7bc/0x816 
[btrfs]
[ 3384.760055]  [<ffffffffa02f4bd5>] ? free_extent_buffer+0x6f/0x7c 
[btrfs]
[ 3384.760108]  [<ffffffffa02fb5cd>] btrfs_balance+0xa7b/0xc80 [btrfs]
[ 3384.760156]  [<ffffffff813a9125>] ? printk+0x48/0x4a
[ 3384.760207]  [<ffffffffa02fb829>] balance_kthread+0x57/0x7c [btrfs]
[ 3384.760259]  [<ffffffffa02fb7d2>] ? btrfs_balance+0xc80/0xc80 [btrfs]
[ 3384.760310]  [<ffffffffa02fb7d2>] ? btrfs_balance+0xc80/0xc80 [btrfs]
[ 3384.760359]  [<ffffffff8104e993>] kthread+0xcd/0xd5
[ 3384.760405]  [<ffffffff8104e8c6>] ? 
kthread_freezable_should_stop+0x43/0x43
[ 3384.760453]  [<ffffffff813affec>] ret_from_fork+0x7c/0xb0
[ 3384.760500]  [<ffffffff8104e8c6>] ? 
kthread_freezable_should_stop+0x43/0x43
[ 3384.760548] Code: b8 01 00 00 00 4c 89 f9 48 89 04 24 49 8b 16 e8 5a 
f5 ff ff 83 f8 f4 75 02 0f 0b 85 c0 0f 85 48 01 00 00 4b 83 7c fd 00 00 
75 02 <0f> 0b 41 83 bd 94 00 00 00 01 48 63 c3 75 2d 49 83 7c c5 00 01
[ 3384.760721] RIP  [<ffffffffa02c5ca6>] walk_down_proc+0xdc/0x22b 
[btrfs]
[ 3384.760773]  RSP <ffff8807dceeb9e8>
[ 3384.761080] ---[ end trace 4f32f915eb18c088 ]---



-- 
Tomasz Chmielewski
http://www.sslrack.com

Re: kernel BUG at fs/btrfs/relocation.c:1065 in 3.14.16 to 3.17-rc3

[Liu Bo]

On Wed, Sep 03, 2014 at 02:04:29PM +0200, Olivier Bonvalet wrote:
> Hi,
> 
> I have a btrfs partition which throw kernel BUG, even with linux
> 3.17-rc3 (I tried 3.14.16, 3.16.1 and 3.17-rc3 kernels) :
> 
> [   45.058466] ------------[ cut here ]------------
> [   45.058539] kernel BUG at fs/btrfs/relocation.c:1065!
> [   45.058578] invalid opcode: 0000 [#1] SMP 
> [   45.058655] Modules linked in: nf_conntrack iTCO_wdt iTCO_vendor_support i2c_i801 lpc_ich ehci_pci i2ccore ehci_hcd mfd_core evdev battery ie31200_edac edac_core video button btrfs xor raid6_pq dm_mod raid1 md_mod sg sd_mod crc_t10dif crct10dif_common thermal ahci libahci libata scsi_mod xhci_hcd e1000e fan ptp pps_core
> [   45.059500] CPU: 2 PID: 1740 Comm: btrfs-balance Not tainted 3.17-rc3-dae-intel #1
> [   45.059550] Hardware name: Digicube sas DediCube/DQ77MK, BIOS MKQ7710H.86A.0058.2013.0226.1541 02/26/2013
> [   45.059602] task: ffff8802151c17e0 ti: ffff8802105ec000 task.ti: ffff8802105ec000
> [   45.059652] RIP: 0010:[<ffffffffa019fcf8>]  [<ffffffffa019fcf8>] build_backref_tree+0xa3d/0xcf6 [btrfs]
> [   45.059739] RSP: 0018:ffff8802105efaf0  EFLAGS: 00010246
> [   45.059776] RAX: ffff8802105efb00 RBX: ffff880213b83800 RCX: ffff880210565d10
> [   45.059816] RDX: ffff8802105efb68 RSI: ffff8802105efb68 RDI: ffff880210565d10
> [   45.059857] RBP: ffff880210565d10 R08: ffff88021313fc40 R09: 0000000000001000
> [   45.059896] R10: 0000160000000000 R11: 6db6db6db6db6db7 R12: ffff8800d114d310
> [   45.059937] R13: ffff8802105efb78 R14: ffff8800d114d2c0 R15: ffff88021313fc40
> [   45.059977] FS:  0000000000000000(0000) GS:ffff88021e280000(0000) knlGS:0000000000000000
> [   45.060028] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [   45.060066] CR2: 00007f2fc9c649b8 CR3: 0000000001611000 CR4: 00000000001407e0
> [   45.060105] Stack:
> [   45.060138]  ffff88021d5fc890 ffff88021417d890 0000000000000000 ffff8802105efb68
> [   45.060264]  0000000100000005 ffff880213b83920 ffff8802105efb78 00ffffffa015ecd1
> [   45.060392]  ffff8800d114d400 ffff8800d114d240 ffff880210464220 ffff880210565d40
> [   45.060516] Call Trace:
> [   45.060556]  [<ffffffffa01a11ff>] ? relocate_tree_blocks+0x15f/0x430 [btrfs]
> [   45.060607]  [<ffffffffa019cd78>] ? tree_insert+0x44/0x47 [btrfs]
> [   45.060656]  [<ffffffffa019e2ba>] ? add_tree_block+0x112/0x13c [btrfs]
> [   45.060702]  [<ffffffffa01a1ff7>] ? relocate_block_group+0x26d/0x4a6 [btrfs]
> [   45.060753]  [<ffffffffa0179014>] ? btrfs_wait_ordered_roots+0x18f/0x1ab [btrfs]
> [   45.060812]  [<ffffffffa01a2384>] ? btrfs_relocate_block_group+0x154/0x265 [btrfs]
> [   45.060872]  [<ffffffffa0181d01>] ? btrfs_relocate_chunk.isra.29+0x52/0x55d [btrfs]
> [   45.060932]  [<ffffffffa018fdc8>] ? btrfs_set_lock_blocking_rw+0xa8/0xaa [btrfs]
> [   45.060988]  [<ffffffffa0145680>] ? btrfs_item_key_to_cpu+0x12/0x30 [btrfs]
> [   45.061039]  [<ffffffffa017767f>] ? btrfs_get_token_64+0x75/0xcf [btrfs]
> [   45.061088]  [<ffffffffa017e201>] ? release_extent_buffer+0x26/0x96 [btrfs]
> [   45.061170]  [<ffffffffa0184965>] ? btrfs_balance+0x9e3/0xb78 [btrfs]
> [   45.061263]  [<ffffffffa0184afa>] ? btrfs_balance+0xb78/0xb78 [btrfs]
> [   45.061314]  [<ffffffffa0184b49>] ? balance_kthread+0x4f/0x6d [btrfs]
> [   45.061360]  [<ffffffff8104704b>] ? kthread+0xa7/0xaf
> [   45.061420]  [<ffffffff81040000>] ? SyS_old_getrlimit+0x21/0xcb
> [   45.061460]  [<ffffffff81046fa4>] ? __kthread_parkme+0x5b/0x5b
> [   45.061501]  [<ffffffff813834ec>] ? ret_from_fork+0x7c/0xb0
> [   45.061541]  [<ffffffff81046fa4>] ? __kthread_parkme+0x5b/0x5b
> [   45.061579] Code: 26 a8 02 74 0d 4c 89 e7 e8 3c e1 ff ff 41 80 66 71 fd 49 8b 46 58 49 89 6e 58 4c 89 65 00 48 89 45 08 48 89 28 eb c0 a8 10 75 02 <0f> 0b 83 e0 01 39 44 24 10 0f 84 20 ff ff ff 0f 0b 49 8b 46 58 
> [   45.063148] RIP  [<ffffffffa019fcf8>] build_backref_tree+0xa3d/0xcf6 [btrfs]
> [   45.063219]  RSP <ffff8802105efaf0>
> [   45.063260] ---[ end trace c396e96e4d1a5697 ]---
> 
> I have dump the FS with btrfs-image, but don't know where to push that.
> So you can download it at : https://daevel.fr/img/btrfs-image.out
> (near 6GB, md5sum ee5559ab31368aba60c259ce3b5b9504)

Would you please try this patch?

https://patchwork.kernel.org/patch/4939761/

thanks,
-liubo

Re: kernel BUG at fs/btrfs/extent-tree.c:7727! with 3.17-rc3

[Tomasz Chmielewski]

BTW still seeing this with 3.17-rc7 when running balance (it's a 
different filesystem than described in "3.17.0-rc7: kernel BUG at 
fs/btrfs/relocation.c:931!" thread):

[ 6945.014952] BTRFS info (device sdb5): relocating block group 
7079521091584 flags 17
[ 6955.696529] BTRFS info (device sdb5): found 2065 extents
[ 6980.650073] BTRFS info (device sdb5): found 2065 extents
[ 6982.055757] BTRFS info (device sdb5): relocating block group 
7078447349760 flags 17
[ 6993.538142] BTRFS info (device sdb5): found 5959 extents
[ 7120.696355] ------------[ cut here ]------------
[ 7120.696419] WARNING: CPU: 7 PID: 4283 at fs/btrfs/extent-tree.c:876 
btrfs_lookup_extent_info+0x377/0x3eb [btrfs]()
[ 7120.696512] Modules linked in: ipt_MASQUERADE iptable_nat 
nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack 
ip_tables x_tables cpufreq_ondemand cpufreq_conservative 
cpufreq_powersave cpufreq_stats bridge stp llc ipv6 btrfs xor raid6_pq 
zlib_deflate coretemp hwmon loop pcspkr parport_pc parport battery 
i2c_i801 i2c_core tpm_infineon lpc_ich mfd_core tpm_tis tpm video 
ehci_pci ehci_hcd button acpi_cpufreq ext4 crc16 jbd2 mbcache raid1 sg 
sd_mod ahci libahci libata scsi_mod r8169 mii
[ 7120.696897] CPU: 7 PID: 4283 Comm: btrfs Not tainted 3.17.0-rc7 #3
[ 7120.696945] Hardware name: System manufacturer System Product 
Name/P8H77-M PRO, BIOS 1101 02/04/2013
[ 7120.697036]  0000000000000009 ffff8807beddf778 ffffffff813ab972 
0000000000000000
[ 7120.697126]  0000000000000000 ffff8807beddf7b8 ffffffff81039b41 
ffffffff0000001e
[ 7120.697217]  ffffffffa027e560 ffff8806bf8b8510 0000000000000000 
0000000000000000
[ 7120.697307] Call Trace:
[ 7120.697356]  [<ffffffff813ab972>] dump_stack+0x46/0x58
[ 7120.697404]  [<ffffffff81039b41>] warn_slowpath_common+0x77/0x91
[ 7120.697457]  [<ffffffffa027e560>] ? 
btrfs_lookup_extent_info+0x377/0x3eb [btrfs]
[ 7120.697547]  [<ffffffff81039b70>] warn_slowpath_null+0x15/0x17
[ 7120.697607]  [<ffffffffa027e560>] 
btrfs_lookup_extent_info+0x377/0x3eb [btrfs]
[ 7120.697699]  [<ffffffffa027ec8f>] walk_down_proc+0xc5/0x22b [btrfs]
[ 7120.697752]  [<ffffffffa0291d20>] ? 
join_transaction.isra.30+0x24/0x309 [btrfs]
[ 7120.697843]  [<ffffffffa0280e92>] walk_down_tree+0x45/0xd5 [btrfs]
[ 7120.697895]  [<ffffffffa028389a>] btrfs_drop_snapshot+0x2f5/0x68f 
[btrfs]
[ 7120.697951]  [<ffffffffa02d4325>] merge_reloc_roots+0x139/0x23f 
[btrfs]
[ 7120.698005]  [<ffffffffa02d4a7e>] relocate_block_group+0x466/0x4de 
[btrfs]
[ 7120.698059]  [<ffffffffa02d4c4e>] 
btrfs_relocate_block_group+0x158/0x278 [btrfs]
[ 7120.698153]  [<ffffffffa02b179c>] 
btrfs_relocate_chunk.isra.62+0x58/0x5f7 [btrfs]
[ 7120.698246]  [<ffffffffa02c099f>] ? 
btrfs_set_lock_blocking_rw+0x68/0x95 [btrfs]
[ 7120.698337]  [<ffffffffa0271b04>] ? btrfs_set_path_blocking+0x23/0x54 
[btrfs]
[ 7120.698389]  [<ffffffffa0276517>] ? btrfs_search_slot+0x7bc/0x816 
[btrfs]
[ 7120.698443]  [<ffffffffa02adc81>] ? free_extent_buffer+0x6f/0x7c 
[btrfs]
[ 7120.698497]  [<ffffffffa02b4679>] btrfs_balance+0xa7b/0xc80 [btrfs]
[ 7120.698550]  [<ffffffffa02ba177>] btrfs_ioctl_balance+0x220/0x29f 
[btrfs]
[ 7120.698603]  [<ffffffffa02bf1e4>] btrfs_ioctl+0x10bd/0x2281 [btrfs]
[ 7120.698661]  [<ffffffff810d5152>] ? handle_mm_fault+0x44d/0xa00
[ 7120.698756]  [<ffffffff81173e76>] ? avc_has_perm+0x2e/0xf7
[ 7120.698806]  [<ffffffff810d7c6d>] ? __vm_enough_memory+0x25/0x13c
[ 7120.698867]  [<ffffffff8110d72d>] do_vfs_ioctl+0x3f2/0x43c
[ 7120.698915]  [<ffffffff8110d7c5>] SyS_ioctl+0x4e/0x7d
[ 7120.698964]  [<ffffffff81030a71>] ? do_page_fault+0xc/0xf
[ 7120.699015]  [<ffffffff813b0652>] system_call_fastpath+0x16/0x1b
[ 7120.699063] ---[ end trace 5ede8b32160a5b9e ]---
[ 7120.699127] ------------[ cut here ]------------
[ 7120.699174] kernel BUG at fs/btrfs/extent-tree.c:7727!
[ 7120.699231] invalid opcode: 0000 [#1] SMP
[ 7120.699279] Modules linked in: ipt_MASQUERADE iptable_nat 
nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack 
ip_tables x_tables cpufreq_ondemand cpufreq_conservative 
cpufreq_powersave cpufreq_stats bridge stp llc ipv6 btrfs xor raid6_pq 
zlib_deflate coretemp hwmon loop pcspkr parport_pc parport battery 
i2c_i801 i2c_core tpm_infineon lpc_ich mfd_core tpm_tis tpm video 
ehci_pci ehci_hcd button acpi_cpufreq ext4 crc16 jbd2 mbcache raid1 sg 
sd_mod ahci libahci libata scsi_mod r8169 mii
[ 7120.699648] CPU: 7 PID: 4283 Comm: btrfs Tainted: G        W      
3.17.0-rc7 #3
[ 7120.699735] Hardware name: System manufacturer System Product 
Name/P8H77-M PRO, BIOS 1101 02/04/2013
[ 7120.699825] task: ffff8807c0381810 ti: ffff8807beddc000 task.ti: 
ffff8807beddc000
[ 7120.699912] RIP: 0010:[<ffffffffa027eca6>]  [<ffffffffa027eca6>] 
walk_down_proc+0xdc/0x22b [btrfs]
[ 7120.700007] RSP: 0018:ffff8807beddf888  EFLAGS: 00010246
[ 7120.700053] RAX: 0000000000000000 RBX: 0000000000000002 RCX: 
000000000044b954
[ 7120.700101] RDX: 000000000044b953 RSI: ffff88081fbd9650 RDI: 
0000000000019650
[ 7120.700149] RBP: ffff8807beddf8d8 R08: ffffea001afe2e00 R09: 
00000000000008f4
[ 7120.700198] R10: ffffffffa0271c38 R11: 0000000000000000 R12: 
ffff8806bf8b87e0
[ 7120.700246] R13: ffff8807f3ce9780 R14: ffff880009e93ce8 R15: 
0000000000000002
[ 7120.700294] FS:  00007fae1881e840(0000) GS:ffff88081fbc0000(0000) 
knlGS:0000000000000000
[ 7120.700383] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7120.700429] CR2: 00007f7a18042000 CR3: 00000007c0309000 CR4: 
00000000001407e0
[ 7120.700477] Stack:
[ 7120.700519]  ffff8807f3ce97d0 ffffffffa0291d20 ffff88010bf83280 
ffff880745c5a800
[ 7120.700608]  0000000000000000 ffff8806bf8b87e0 0000000000000002 
ffff88010bf83280
[ 7120.700697]  ffff880745c5a800 ffff8807f3ce9780 ffff8807beddf938 
ffffffffa0280e92
[ 7120.700786] Call Trace:
[ 7120.700834]  [<ffffffffa0291d20>] ? 
join_transaction.isra.30+0x24/0x309 [btrfs]
[ 7120.700926]  [<ffffffffa0280e92>] walk_down_tree+0x45/0xd5 [btrfs]
[ 7120.700978]  [<ffffffffa028389a>] btrfs_drop_snapshot+0x2f5/0x68f 
[btrfs]
[ 7120.701032]  [<ffffffffa02d4325>] merge_reloc_roots+0x139/0x23f 
[btrfs]
[ 7120.702196]  [<ffffffffa02d4a7e>] relocate_block_group+0x466/0x4de 
[btrfs]
[ 7120.702249]  [<ffffffffa02d4c4e>] 
btrfs_relocate_block_group+0x158/0x278 [btrfs]
[ 7120.702343]  [<ffffffffa02b179c>] 
btrfs_relocate_chunk.isra.62+0x58/0x5f7 [btrfs]
[ 7120.702436]  [<ffffffffa02c099f>] ? 
btrfs_set_lock_blocking_rw+0x68/0x95 [btrfs]
[ 7120.702526]  [<ffffffffa0271b04>] ? btrfs_set_path_blocking+0x23/0x54 
[btrfs]
[ 7120.702578]  [<ffffffffa0276517>] ? btrfs_search_slot+0x7bc/0x816 
[btrfs]
[ 7120.702632]  [<ffffffffa02adc81>] ? free_extent_buffer+0x6f/0x7c 
[btrfs]
[ 7120.702685]  [<ffffffffa02b4679>] btrfs_balance+0xa7b/0xc80 [btrfs]
[ 7120.702747]  [<ffffffffa02ba177>] btrfs_ioctl_balance+0x220/0x29f 
[btrfs]
[ 7120.702801]  [<ffffffffa02bf1e4>] btrfs_ioctl+0x10bd/0x2281 [btrfs]
[ 7120.702858]  [<ffffffff810d5152>] ? handle_mm_fault+0x44d/0xa00
[ 7120.702905]  [<ffffffff81173e76>] ? avc_has_perm+0x2e/0xf7
[ 7120.702952]  [<ffffffff810d7c6d>] ? __vm_enough_memory+0x25/0x13c
[ 7120.703000]  [<ffffffff8110d72d>] do_vfs_ioctl+0x3f2/0x43c
[ 7120.703047]  [<ffffffff8110d7c5>] SyS_ioctl+0x4e/0x7d
[ 7120.703093]  [<ffffffff81030a71>] ? do_page_fault+0xc/0xf
[ 7120.703140]  [<ffffffff813b0652>] system_call_fastpath+0x16/0x1b
[ 7120.703187] Code: b8 01 00 00 00 4c 89 f9 48 89 04 24 49 8b 16 e8 5a 
f5 ff ff 83 f8 f4 75 02 0f 0b 85 c0 0f 85 48 01 00 00 4b 83 7c fd 00 00 
75 02 <0f> 0b 41 83 bd 94 00 00 00 01 48 63 c3 75 2d 49 83 7c c5 00 01
[ 7120.703358] RIP  [<ffffffffa027eca6>] walk_down_proc+0xdc/0x22b 
[btrfs]
[ 7120.703410]  RSP <ffff8807beddf888>
[ 7120.703716] ---[ end trace 5ede8b32160a5b9f ]---
[ 7120.703909] BUG: unable to handle kernel paging request at 
000000000001f9c6
[ 7120.704050] IP: [<ffffffffa01188ba>] jbd2__journal_start+0x3d/0x151 
[jbd2]
[ 7120.704162] PGD 0
[ 7120.704266] Oops: 0000 [#2] SMP
[ 7120.704402] Modules linked in: ipt_MASQUERADE iptable_nat 
nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack 
ip_tables x_tables cpufreq_ondemand cpufreq_conservative 
cpufreq_powersave cpufreq_stats bridge stp llc ipv6 btrfs xor raid6_pq 
zlib_deflate coretemp hwmon loop pcspkr parport_pc parport battery 
i2c_i801 i2c_core tpm_infineon lpc_ich mfd_core tpm_tis tpm video 
ehci_pci ehci_hcd button acpi_cpufreq ext4 crc16 jbd2 mbcache raid1 sg 
sd_mod ahci libahci libata scsi_mod r8169 mii
[ 7120.706438] CPU: 7 PID: 4283 Comm: btrfs Tainted: G      D W      
3.17.0-rc7 #3
[ 7120.706556] Hardware name: System manufacturer System Product 
Name/P8H77-M PRO, BIOS 1101 02/04/2013
[ 7120.706676] task: ffff8807c0381810 ti: ffff8807beddc000 task.ti: 
ffff8807beddc000
[ 7120.706795] RIP: 0010:[<ffffffffa01188ba>]  [<ffffffffa01188ba>] 
jbd2__journal_start+0x3d/0x151 [jbd2]
[ 7120.706948] RSP: 0018:ffff8807beddf3d8  EFLAGS: 00010282
[ 7120.707025] RAX: 000000000001f9c6 RBX: ffff88010bf83280 RCX: 
0000000000000050
[ 7120.707103] RDX: 0000000000000000 RSI: 000000000000005b RDI: 
ffff8807f156e000
[ 7120.707182] RBP: ffff8807beddf418 R08: 0000000000000005 R09: 
00000000000000f4
[ 7120.707261] R10: 000000000000003c R11: ffff880036e39830 R12: 
ffff8807f156e000
[ 7120.707340] R13: 0000000000000000 R14: 0000000000000005 R15: 
ffff8807f40c0de0
[ 7120.707419] FS:  00007fae1881e840(0000) GS:ffff88081fbc0000(0000) 
knlGS:0000000000000000
[ 7120.707538] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7120.707618] CR2: 000000000001f9c6 CR3: 0000000001611000 CR4: 
00000000001407e0
[ 7120.707697] Stack:
[ 7120.707779]  ffff8807beddf458 00000005000000f4 0000000160f15c0a 
ffff8807f156d000
[ 7120.708021]  000000000000005b 0000000000000000 0000000000000005 
ffff8807f40c0de0
[ 7120.708263]  ffff8807beddf458 ffffffffa0156111 ffff8807beddf438 
ffff8800000000f4
[ 7120.708505] Call Trace:
[ 7120.708585]  [<ffffffffa0156111>] __ext4_journal_start_sb+0x5c/0x67 
[ext4]
[ 7120.708677]  [<ffffffffa013e4cb>] ext4_evict_inode+0x245/0x3d8 [ext4]
[ 7120.708765]  [<ffffffff811137da>] evict+0xa4/0x149
[ 7120.708842]  [<ffffffff81114059>] iput+0x129/0x132
[ 7120.708919]  [<ffffffff81110948>] __dentry_kill+0x145/0x1a4
[ 7120.708996]  [<ffffffff81110f32>] dput+0x130/0x148
[ 7120.709073]  [<ffffffff810fff55>] __fput+0x183/0x1ab
[ 7120.709149]  [<ffffffff810fffab>] ____fput+0x9/0xb
[ 7120.709226]  [<ffffffff8104d8a9>] task_work_run+0x79/0x90
[ 7120.709304]  [<ffffffff8103b9bd>] do_exit+0x3a0/0x8f6
[ 7120.709381]  [<ffffffff810053ae>] oops_end+0xa2/0xab
[ 7120.709457]  [<ffffffff810054de>] die+0x55/0x5f
[ 7120.709535]  [<ffffffff81002c97>] do_trap+0x6a/0x12c
[ 7120.709612]  [<ffffffff8104f593>] ? 
__atomic_notifier_call_chain+0xd/0xf
[ 7120.709691]  [<ffffffff81002e1b>] do_error_trap+0xc2/0xd4
[ 7120.709772]  [<ffffffffa027eca6>] ? walk_down_proc+0xdc/0x22b [btrfs]
[ 7120.709851]  [<ffffffff81063b6e>] ? __wake_up+0x3f/0x48
[ 7120.709935]  [<ffffffffa02adc81>] ? free_extent_buffer+0x6f/0x7c 
[btrfs]
[ 7120.710016]  [<ffffffffa0271bf7>] ? btrfs_release_path+0x6c/0x8b 
[btrfs]
[ 7120.710095]  [<ffffffff81002e9c>] do_invalid_op+0x1b/0x1d
[ 7120.710173]  [<ffffffff813b1928>] invalid_op+0x18/0x20
[ 7120.710252]  [<ffffffffa0271c38>] ? btrfs_free_path+0x22/0x26 [btrfs]
[ 7120.710335]  [<ffffffffa027eca6>] ? walk_down_proc+0xdc/0x22b [btrfs]
[ 7120.710416]  [<ffffffffa027ec8f>] ? walk_down_proc+0xc5/0x22b [btrfs]
[ 7120.710500]  [<ffffffffa0291d20>] ? 
join_transaction.isra.30+0x24/0x309 [btrfs]
[ 7120.710622]  [<ffffffffa0280e92>] walk_down_tree+0x45/0xd5 [btrfs]
[ 7120.710704]  [<ffffffffa028389a>] btrfs_drop_snapshot+0x2f5/0x68f 
[btrfs]
[ 7120.710789]  [<ffffffffa02d4325>] merge_reloc_roots+0x139/0x23f 
[btrfs]
[ 7120.710873]  [<ffffffffa02d4a7e>] relocate_block_group+0x466/0x4de 
[btrfs]
[ 7120.710957]  [<ffffffffa02d4c4e>] 
btrfs_relocate_block_group+0x158/0x278 [btrfs]
[ 7120.711081]  [<ffffffffa02b179c>] 
btrfs_relocate_chunk.isra.62+0x58/0x5f7 [btrfs]
[ 7120.711205]  [<ffffffffa02c099f>] ? 
btrfs_set_lock_blocking_rw+0x68/0x95 [btrfs]
[ 7120.711326]  [<ffffffffa0271b04>] ? btrfs_set_path_blocking+0x23/0x54 
[btrfs]
[ 7120.711408]  [<ffffffffa0276517>] ? btrfs_search_slot+0x7bc/0x816 
[btrfs]
[ 7120.711492]  [<ffffffffa02adc81>] ? free_extent_buffer+0x6f/0x7c 
[btrfs]
[ 7120.711577]  [<ffffffffa02b4679>] btrfs_balance+0xa7b/0xc80 [btrfs]
[ 7120.711660]  [<ffffffffa02ba177>] btrfs_ioctl_balance+0x220/0x29f 
[btrfs]
[ 7120.711744]  [<ffffffffa02bf1e4>] btrfs_ioctl+0x10bd/0x2281 [btrfs]
[ 7120.711823]  [<ffffffff810d5152>] ? handle_mm_fault+0x44d/0xa00
[ 7120.711901]  [<ffffffff81173e76>] ? avc_has_perm+0x2e/0xf7
[ 7120.711979]  [<ffffffff810d7c6d>] ? __vm_enough_memory+0x25/0x13c
[ 7120.712057]  [<ffffffff8110d72d>] do_vfs_ioctl+0x3f2/0x43c
[ 7120.712134]  [<ffffffff8110d7c5>] SyS_ioctl+0x4e/0x7d
[ 7120.712212]  [<ffffffff81030a71>] ? do_page_fault+0xc/0xf
[ 7120.712289]  [<ffffffff813b0652>] system_call_fastpath+0x16/0x1b
[ 7120.712367] Code: 55 41 54 49 89 fc 53 48 83 ec 18 48 85 ff 44 89 45 
cc 44 89 4d c8 48 8b 98 58 07 00 00 0f 84 eb 00 00 00 48 85 db 74 12 48 
8b 03 <48> 39 38 74 02 0f 0b ff 43 14 e9 f3 00 00 00 48 8b 3d 18 9c 00
[ 7120.714556] RIP  [<ffffffffa01188ba>] jbd2__journal_start+0x3d/0x151 
[jbd2]
[ 7120.714668]  RSP <ffff8807beddf3d8>
[ 7120.714742] CR2: 000000000001f9c6
[ 7120.714816] ---[ end trace 5ede8b32160a5ba0 ]---
[ 7120.714892] Fixing recursive fault but reboot is needed!



On 2014-09-08 20:04 (Mon), Tomasz Chmielewski wrote:
> On 2014-09-03 19:42 (Wed), Tomasz Chmielewski wrote:
>> Got the following with 3.17-rc3 and running balance (had to power
>> cycle after that):
> 
> I'm seeing similar BUG with 3.17-rc4:
> 
> [ 1049.755843] BTRFS info (device sdb5): found 35715 extents
> [ 1050.257075] BTRFS info (device sdb5): relocating block group
> 7091332251648 flags 20
> [ 2087.976104] BTRFS info (device sdb5): found 40911 extents
> [ 2091.357102] BTRFS info (device sdb5): relocating block group
> 7006506647552 flags 20
> [ 2518.793237] INFO: task btrfs-balance:5688 blocked for more than 120 
> seconds.
> [ 2518.793263]       Not tainted 3.17.0-rc4 #1
> [ 2518.793269] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
> disables this message.
> [ 2518.793279] btrfs-balance   D ffff88081fad15c0     0  5688      2 
> 0x00000000
> [ 2518.793291]  ffff8807dceebac8 0000000000000046 ffff8807dceeb9c8
> ffff8807f0f83020
> [ 2518.793303]  00000000000115c0 0000000000004000 ffff8807f4153020
> ffff8807f0f83020
> [ 2518.793315]  ffff8807dceeba08 ffffffff8105639d 0000000000000000
> ffff8807f3cd8050
> [ 2518.793404] Call Trace:
> [ 2518.793452]  [<ffffffff8105639d>] ? wake_up_process+0x31/0x35
> [ 2518.793500]  [<ffffffff81049baf>] ? wake_up_worker+0x1f/0x21
> [ 2518.793547]  [<ffffffff81049df6>] ? insert_work+0x87/0x94
> [ 2518.793605]  [<ffffffffa031724b>] ? free_block_list+0x1f/0x34 
> [btrfs]
> [ 2518.793655]  [<ffffffff813ad75e>] ? wait_for_common+0x118/0x13e
> [ 2518.793703]  [<ffffffff813acf2e>] schedule+0x65/0x67
> [ 2518.793755]  [<ffffffffa02d934d>]
> wait_current_trans.isra.32+0x94/0xe2 [btrfs]
> [ 2518.793843]  [<ffffffff81063cdc>] ? add_wait_queue+0x44/0x44
> [ 2518.793895]  [<ffffffffa02da82f>] start_transaction+0x427/0x472 
> [btrfs]
> [ 2518.793948]  [<ffffffffa02dab02>] btrfs_start_transaction+0x16/0x18 
> [btrfs]
> [ 2518.794002]  [<ffffffffa031b51a>] relocate_block_group+0x8a/0x4de 
> [btrfs]
> [ 2518.794055]  [<ffffffffa031bac6>]
> btrfs_relocate_block_group+0x158/0x278 [btrfs]
> [ 2518.794148]  [<ffffffffa02f86f0>]
> btrfs_relocate_chunk.isra.62+0x58/0x5f7 [btrfs]
> [ 2518.794241]  [<ffffffffa03078d7>] ?
> btrfs_set_lock_blocking_rw+0x68/0x95 [btrfs]
> [ 2518.794332]  [<ffffffffa02b8b04>] ? 
> btrfs_set_path_blocking+0x23/0x54 [btrfs]
> [ 2518.794383]  [<ffffffffa02bd517>] ? btrfs_search_slot+0x7bc/0x816 
> [btrfs]
> [ 2518.794437]  [<ffffffffa02f4bd5>] ? free_extent_buffer+0x6f/0x7c 
> [btrfs]
> [ 2518.794490]  [<ffffffffa02fb5cd>] btrfs_balance+0xa7b/0xc80 [btrfs]
> [ 2518.794538]  [<ffffffff813a9125>] ? printk+0x48/0x4a
> [ 2518.794589]  [<ffffffffa02fb829>] balance_kthread+0x57/0x7c [btrfs]
> [ 2518.794641]  [<ffffffffa02fb7d2>] ? btrfs_balance+0xc80/0xc80 
> [btrfs]
> [ 2518.794692]  [<ffffffffa02fb7d2>] ? btrfs_balance+0xc80/0xc80 
> [btrfs]
> [ 2518.794741]  [<ffffffff8104e993>] kthread+0xcd/0xd5
> [ 2518.794787]  [<ffffffff8104e8c6>] ? 
> kthread_freezable_should_stop+0x43/0x43
> [ 2518.794836]  [<ffffffff813affec>] ret_from_fork+0x7c/0xb0
> [ 2518.794883]  [<ffffffff8104e8c6>] ? 
> kthread_freezable_should_stop+0x43/0x43
> [ 2518.794935] INFO: task kworker/u16:21:8725 blocked for more than 120 
> seconds.
> [ 2518.794983]       Not tainted 3.17.0-rc4 #1
> [ 2518.795028] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
> disables this message.
> [ 2518.795116] kworker/u16:21  D ffff88081fbd15c0     0  8725      2 
> 0x00000000
> [ 2518.795170] Workqueue: events_unbound
> btrfs_async_reclaim_metadata_space [btrfs]
> [ 2518.795258]  ffff8805bf363b48 0000000000000046 ffff8805bf363a88
> ffff8807f3244830
> [ 2518.795347]  00000000000115c0 0000000000004000 ffff8807f4171810
> ffff8807f3244830
> [ 2518.795436]  ffff8805bf363a88 ffffffff81053484 0000000000200000
> ffff88081fa915c0
> [ 2518.795525] Call Trace:
> [ 2518.795568]  [<ffffffff81053484>] ? resched_curr+0x47/0x57
> [ 2518.795615]  [<ffffffff81053aed>] ? check_preempt_curr+0x3e/0x6d
> [ 2518.795663]  [<ffffffff8105634c>] ? try_to_wake_up+0x240/0x251
> [ 2518.795710]  [<ffffffff8105639d>] ? wake_up_process+0x31/0x35
> [ 2518.795758]  [<ffffffff813acf2e>] schedule+0x65/0x67
> [ 2518.795804]  [<ffffffff813af01c>] schedule_timeout+0x26/0x198
> [ 2518.795851]  [<ffffffff8104a0df>] ? __queue_work+0x1d2/0x204
> [ 2518.795899]  [<ffffffff813ad753>] wait_for_common+0x10d/0x13e
> [ 2518.795946]  [<ffffffff8105635d>] ? try_to_wake_up+0x251/0x251
> [ 2518.795993]  [<ffffffff813ad79c>] wait_for_completion+0x18/0x1a
> [ 2518.796042]  [<ffffffff8111d8aa>] writeback_inodes_sb_nr+0x87/0x8e
> [ 2518.796093]  [<ffffffffa02d0101>] ? btrfs_del_csums+0x1b6/0x2ca 
> [btrfs]
> [ 2518.796145]  [<ffffffffa02c63c9>] flush_space+0x1af/0x407 [btrfs]
> [ 2518.796196]  [<ffffffffa02c5e56>] ? 
> btrfs_get_alloc_profile+0x2b/0x2d [btrfs]
> [ 2518.796248]  [<ffffffffa02c618d>] ? can_overcommit+0x4f/0xdc [btrfs]
> [ 2518.796299]  [<ffffffffa02c6a71>]
> btrfs_async_reclaim_metadata_space+0x10c/0x157 [btrfs]
> [ 2518.796388]  [<ffffffff8104aadb>] process_one_work+0x188/0x2ab
> [ 2518.796435]  [<ffffffff8104ae89>] worker_thread+0x261/0x35e
> [ 2518.796482]  [<ffffffff8104ac28>] ? 
> process_scheduled_works+0x2a/0x2a
> [ 2518.796530]  [<ffffffff8104e993>] kthread+0xcd/0xd5
> [ 2518.796577]  [<ffffffff8104e8c6>] ? 
> kthread_freezable_should_stop+0x43/0x43
> [ 2518.796625]  [<ffffffff813affec>] ret_from_fork+0x7c/0xb0
> [ 2518.796672]  [<ffffffff8104e8c6>] ? 
> kthread_freezable_should_stop+0x43/0x43
> [ 2638.722499] INFO: task kworker/u16:21:8725 blocked for more than 120 
> seconds.
> [ 2638.722568]       Not tainted 3.17.0-rc4 #1
> [ 2638.722625] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
> disables this message.
> [ 2638.722737] kworker/u16:21  D ffff88081fbd15c0     0  8725      2 
> 0x00000000
> [ 2638.722821] Workqueue: events_unbound
> btrfs_async_reclaim_metadata_space [btrfs]
> [ 2638.722935]  ffff8805bf363b48 0000000000000046 ffff8805bf363a88
> ffff8807f3244830
> [ 2638.723050]  00000000000115c0 0000000000004000 ffff8807f4171810
> ffff8807f3244830
> [ 2638.723165]  ffff8805bf363a88 ffffffff81053484 0000000000200000
> ffff88081fa915c0
> [ 2638.723279] Call Trace:
> [ 2638.723339]  [<ffffffff81053484>] ? resched_curr+0x47/0x57
> [ 2638.723401]  [<ffffffff81053aed>] ? check_preempt_curr+0x3e/0x6d
> [ 2638.723465]  [<ffffffff8105634c>] ? try_to_wake_up+0x240/0x251
> [ 2638.723526]  [<ffffffff8105639d>] ? wake_up_process+0x31/0x35
> [ 2638.723590]  [<ffffffff813acf2e>] schedule+0x65/0x67
> [ 2638.723650]  [<ffffffff813af01c>] schedule_timeout+0x26/0x198
> [ 2638.723713]  [<ffffffff8104a0df>] ? __queue_work+0x1d2/0x204
> [ 2638.723775]  [<ffffffff813ad753>] wait_for_common+0x10d/0x13e
> [ 2638.723837]  [<ffffffff8105635d>] ? try_to_wake_up+0x251/0x251
> [ 2638.723900]  [<ffffffff813ad79c>] wait_for_completion+0x18/0x1a
> [ 2638.723963]  [<ffffffff8111d8aa>] writeback_inodes_sb_nr+0x87/0x8e
> [ 2638.724037]  [<ffffffffa02d0101>] ? btrfs_del_csums+0x1b6/0x2ca 
> [btrfs]
> [ 2638.724109]  [<ffffffffa02c63c9>] flush_space+0x1af/0x407 [btrfs]
> [ 2638.724180]  [<ffffffffa02c5e56>] ? 
> btrfs_get_alloc_profile+0x2b/0x2d [btrfs]
> [ 2638.724252]  [<ffffffffa02c618d>] ? can_overcommit+0x4f/0xdc [btrfs]
> [ 2638.724322]  [<ffffffffa02c6a71>]
> btrfs_async_reclaim_metadata_space+0x10c/0x157 [btrfs]
> [ 2638.724436]  [<ffffffff8104aadb>] process_one_work+0x188/0x2ab
> [ 2638.724498]  [<ffffffff8104ae89>] worker_thread+0x261/0x35e
> [ 2638.724559]  [<ffffffff8104ac28>] ? 
> process_scheduled_works+0x2a/0x2a
> [ 2638.724622]  [<ffffffff8104e993>] kthread+0xcd/0xd5
> [ 2638.724683]  [<ffffffff8104e8c6>] ? 
> kthread_freezable_should_stop+0x43/0x43
> [ 2638.724747]  [<ffffffff813affec>] ret_from_fork+0x7c/0xb0
> [ 2638.724808]  [<ffffffff8104e8c6>] ? 
> kthread_freezable_should_stop+0x43/0x43
> [ 2758.651776] INFO: task kworker/u16:21:8725 blocked for more than 120 
> seconds.
> [ 2758.651842]       Not tainted 3.17.0-rc4 #1
> [ 2758.651899] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
> disables this message.
> [ 2758.652008] kworker/u16:21  D ffff88081fbd15c0     0  8725      2 
> 0x00000000
> [ 2758.652091] Workqueue: events_unbound
> btrfs_async_reclaim_metadata_space [btrfs]
> [ 2758.652201]  ffff8805bf363b48 0000000000000046 ffff8805bf363a88
> ffff8807f3244830
> [ 2758.652313]  00000000000115c0 0000000000004000 ffff8807f4171810
> ffff8807f3244830
> [ 2758.652425]  ffff8805bf363a88 ffffffff81053484 0000000000200000
> ffff88081fa915c0
> [ 2758.652537] Call Trace:
> [ 2758.652595]  [<ffffffff81053484>] ? resched_curr+0x47/0x57
> [ 2758.652656]  [<ffffffff81053aed>] ? check_preempt_curr+0x3e/0x6d
> [ 2758.652717]  [<ffffffff8105634c>] ? try_to_wake_up+0x240/0x251
> [ 2758.652779]  [<ffffffff8105639d>] ? wake_up_process+0x31/0x35
> [ 2758.652841]  [<ffffffff813acf2e>] schedule+0x65/0x67
> [ 2758.652900]  [<ffffffff813af01c>] schedule_timeout+0x26/0x198
> [ 2758.652962]  [<ffffffff8104a0df>] ? __queue_work+0x1d2/0x204
> [ 2758.653023]  [<ffffffff813ad753>] wait_for_common+0x10d/0x13e
> [ 2758.653084]  [<ffffffff8105635d>] ? try_to_wake_up+0x251/0x251
> [ 2758.653145]  [<ffffffff813ad79c>] wait_for_completion+0x18/0x1a
> [ 2758.653206]  [<ffffffff8111d8aa>] writeback_inodes_sb_nr+0x87/0x8e
> [ 2758.653278]  [<ffffffffa02d0101>] ? btrfs_del_csums+0x1b6/0x2ca 
> [btrfs]
> [ 2758.653347]  [<ffffffffa02c63c9>] flush_space+0x1af/0x407 [btrfs]
> [ 2758.653416]  [<ffffffffa02c5e56>] ? 
> btrfs_get_alloc_profile+0x2b/0x2d [btrfs]
> [ 2758.653486]  [<ffffffffa02c618d>] ? can_overcommit+0x4f/0xdc [btrfs]
> [ 2758.653554]  [<ffffffffa02c6a71>]
> btrfs_async_reclaim_metadata_space+0x10c/0x157 [btrfs]
> [ 2758.653665]  [<ffffffff8104aadb>] process_one_work+0x188/0x2ab
> [ 2758.653726]  [<ffffffff8104ae89>] worker_thread+0x261/0x35e
> [ 2758.653786]  [<ffffffff8104ac28>] ? 
> process_scheduled_works+0x2a/0x2a
> [ 2758.653848]  [<ffffffff8104e993>] kthread+0xcd/0xd5
> [ 2758.653907]  [<ffffffff8104e8c6>] ? 
> kthread_freezable_should_stop+0x43/0x43
> [ 2758.653969]  [<ffffffff813affec>] ret_from_fork+0x7c/0xb0
> [ 2758.654029]  [<ffffffff8104e8c6>] ? 
> kthread_freezable_should_stop+0x43/0x43
> [ 2878.581047] INFO: task kworker/u16:21:8725 blocked for more than 120 
> seconds.
> [ 2878.581117]       Not tainted 3.17.0-rc4 #1
> [ 2878.581174] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
> disables this message.
> [ 2878.581287] kworker/u16:21  D ffff88081fbd15c0     0  8725      2 
> 0x00000000
> [ 2878.581371] Workqueue: events_unbound
> btrfs_async_reclaim_metadata_space [btrfs]
> [ 2878.582875]  ffff8805bf363b48 0000000000000046 ffff8805bf363a88
> ffff8807f3244830
> [ 2878.582990]  00000000000115c0 0000000000004000 ffff8807f4171810
> ffff8807f3244830
> [ 2878.583105]  ffff8805bf363a88 ffffffff81053484 0000000000200000
> ffff88081fa915c0
> [ 2878.583220] Call Trace:
> [ 2878.583280]  [<ffffffff81053484>] ? resched_curr+0x47/0x57
> [ 2878.583342]  [<ffffffff81053aed>] ? check_preempt_curr+0x3e/0x6d
> [ 2878.583404]  [<ffffffff8105634c>] ? try_to_wake_up+0x240/0x251
> [ 2878.583467]  [<ffffffff8105639d>] ? wake_up_process+0x31/0x35
> [ 2878.583531]  [<ffffffff813acf2e>] schedule+0x65/0x67
> [ 2878.583591]  [<ffffffff813af01c>] schedule_timeout+0x26/0x198
> [ 2878.583655]  [<ffffffff8104a0df>] ? __queue_work+0x1d2/0x204
> [ 2878.583717]  [<ffffffff813ad753>] wait_for_common+0x10d/0x13e
> [ 2878.583780]  [<ffffffff8105635d>] ? try_to_wake_up+0x251/0x251
> [ 2878.583842]  [<ffffffff813ad79c>] wait_for_completion+0x18/0x1a
> [ 2878.583905]  [<ffffffff8111d8aa>] writeback_inodes_sb_nr+0x87/0x8e
> [ 2878.583980]  [<ffffffffa02d0101>] ? btrfs_del_csums+0x1b6/0x2ca 
> [btrfs]
> [ 2878.584052]  [<ffffffffa02c63c9>] flush_space+0x1af/0x407 [btrfs]
> [ 2878.584123]  [<ffffffffa02c5e56>] ? 
> btrfs_get_alloc_profile+0x2b/0x2d [btrfs]
> [ 2878.584195]  [<ffffffffa02c618d>] ? can_overcommit+0x4f/0xdc [btrfs]
> [ 2878.584266]  [<ffffffffa02c6a71>]
> btrfs_async_reclaim_metadata_space+0x10c/0x157 [btrfs]
> [ 2878.584380]  [<ffffffff8104aadb>] process_one_work+0x188/0x2ab
> [ 2878.584442]  [<ffffffff8104ae89>] worker_thread+0x261/0x35e
> [ 2878.584503]  [<ffffffff8104ac28>] ? 
> process_scheduled_works+0x2a/0x2a
> [ 2878.584566]  [<ffffffff8104e993>] kthread+0xcd/0xd5
> [ 2878.584627]  [<ffffffff8104e8c6>] ? 
> kthread_freezable_should_stop+0x43/0x43
> [ 2878.584691]  [<ffffffff813affec>] ret_from_fork+0x7c/0xb0
> [ 2878.584752]  [<ffffffff8104e8c6>] ? 
> kthread_freezable_should_stop+0x43/0x43
> [ 3384.754985] ------------[ cut here ]------------
> [ 3384.755050] WARNING: CPU: 2 PID: 5688 at fs/btrfs/extent-tree.c:876
> btrfs_lookup_extent_info+0x377/0x3eb [btrfs]()
> [ 3384.755145] Modules linked in: ipt_MASQUERADE iptable_nat
> nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack
> ip_tables x_tables cpufreq_ondemand cpufreq_conservative
> cpufreq_powersave cpufreq_stats bridge stp llc ipv6 btrfs xor raid6_pq
> zlib_deflate coretemp hwmon loop i2c_i801 i2ccore pcspkr battery
> tpm_infineon tpm_tis tpm parport_pc parport video ehci_pci ehci_hcd
> lpc_ich mfd_core acpi_cpufreq button ext4 crc16 jbd2 mbcache raid1 sg
> sd_mod ahci libahci libata scsi_mod r8169 mii
> [ 3384.755547] CPU: 2 PID: 5688 Comm: btrfs-balance Not tainted 
> 3.17.0-rc4 #1
> [ 3384.755597] Hardware name: System manufacturer System Product
> Name/P8H77-M PRO, BIOS 1101 02/04/2013
> [ 3384.755691]  0000000000000009 ffff8807dceeb8d8 ffffffff813ab3a2
> 0000000000000000
> [ 3384.755785]  0000000000000000 ffff8807dceeb918 ffffffff81039b41
> ffffffff00000024
> [ 3384.755895]  ffffffffa02c5560 ffff8801b195bc60 0000000000000000
> 0000000000000000
> [ 3384.755990] Call Trace:
> [ 3384.756040]  [<ffffffff813ab3a2>] dump_stack+0x46/0x58
> [ 3384.756090]  [<ffffffff81039b41>] warn_slowpath_common+0x77/0x91
> [ 3384.756148]  [<ffffffffa02c5560>] ?
> btrfs_lookup_extent_info+0x377/0x3eb [btrfs]
> [ 3384.756240]  [<ffffffff81039b70>] warn_slowpath_null+0x15/0x17
> [ 3384.756295]  [<ffffffffa02c5560>]
> btrfs_lookup_extent_info+0x377/0x3eb [btrfs]
> [ 3384.756393]  [<ffffffffa02c5c8f>] walk_down_proc+0xc5/0x22b [btrfs]
> [ 3384.756452]  [<ffffffffa02d8d20>] ?
> join_transaction.isra.30+0x24/0x309 [btrfs]
> [ 3384.756545]  [<ffffffffa02c7e92>] walk_down_tree+0x45/0xd5 [btrfs]
> [ 3384.756597]  [<ffffffffa02ca89a>] btrfs_drop_snapshot+0x2f5/0x68f 
> [btrfs]
> [ 3384.756653]  [<ffffffffa031b19d>] merge_reloc_roots+0x139/0x23f 
> [btrfs]
> [ 3384.756707]  [<ffffffffa031b8f6>] relocate_block_group+0x466/0x4de 
> [btrfs]
> [ 3384.756760]  [<ffffffffa031bac6>]
> btrfs_relocate_block_group+0x158/0x278 [btrfs]
> [ 3384.756854]  [<ffffffffa02f86f0>]
> btrfs_relocate_chunk.isra.62+0x58/0x5f7 [btrfs]
> [ 3384.756947]  [<ffffffffa03078d7>] ?
> btrfs_set_lock_blocking_rw+0x68/0x95 [btrfs]
> [ 3384.757038]  [<ffffffffa02b8b04>] ? 
> btrfs_set_path_blocking+0x23/0x54 [btrfs]
> [ 3384.757089]  [<ffffffffa02bd517>] ? btrfs_search_slot+0x7bc/0x816 
> [btrfs]
> [ 3384.757143]  [<ffffffffa02f4bd5>] ? free_extent_buffer+0x6f/0x7c 
> [btrfs]
> [ 3384.757197]  [<ffffffffa02fb5cd>] btrfs_balance+0xa7b/0xc80 [btrfs]
> [ 3384.757246]  [<ffffffff813a9125>] ? printk+0x48/0x4a
> [ 3384.757297]  [<ffffffffa02fb829>] balance_kthread+0x57/0x7c [btrfs]
> [ 3384.757349]  [<ffffffffa02fb7d2>] ? btrfs_balance+0xc80/0xc80 
> [btrfs]
> [ 3384.757401]  [<ffffffffa02fb7d2>] ? btrfs_balance+0xc80/0xc80 
> [btrfs]
> [ 3384.757450]  [<ffffffff8104e993>] kthread+0xcd/0xd5
> [ 3384.757496]  [<ffffffff8104e8c6>] ? 
> kthread_freezable_should_stop+0x43/0x43
> [ 3384.757545]  [<ffffffff813affec>] ret_from_fork+0x7c/0xb0
> [ 3384.757592]  [<ffffffff8104e8c6>] ? 
> kthread_freezable_should_stop+0x43/0x43
> [ 3384.757640] ---[ end trace 4f32f915eb18c087 ]---
> [ 3384.757691] ------------[ cut here ]------------
> [ 3384.757736] kernel BUG at fs/btrfs/extent-tree.c:7727!
> [ 3384.757782] invalid opcode: 0000 [#1] SMP
> [ 3384.757827] Modules linked in: ipt_MASQUERADE iptable_nat
> nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack
> ip_tables x_tables cpufreq_ondemand cpufreq_conservative
> cpufreq_powersave cpufreq_stats bridge stp llc ipv6 btrfs xor raid6_pq
> zlib_deflate coretemp hwmon loop i2c_i801 i2ccore pcspkr battery
> tpm_infineon tpm_tis tpm parport_pc parport video ehci_pci ehci_hcd
> lpc_ich mfd_core acpi_cpufreq button ext4 crc16 jbd2 mbcache raid1 sg
> sd_mod ahci libahci libata scsi_mod r8169 mii
> [ 3384.758182] CPU: 2 PID: 5688 Comm: btrfs-balance Tainted: G
> W      3.17.0-rc4 #1
> [ 3384.758270] Hardware name: System manufacturer System Product
> Name/P8H77-M PRO, BIOS 1101 02/04/2013
> [ 3384.758360] task: ffff8807f0f83020 ti: ffff8807dcee8000 task.ti:
> ffff8807dcee8000
> [ 3384.758448] RIP: 0010:[<ffffffffa02c5ca6>]  [<ffffffffa02c5ca6>]
> walk_down_proc+0xdc/0x22b [btrfs]
> [ 3384.758542] RSP: 0018:ffff8807dceeb9e8  EFLAGS: 00010246
> [ 3384.758588] RAX: 0000000000000000 RBX: 0000000000000002 RCX: 
> 0000000000984a1c
> [ 3384.758636] RDX: 0000000000984a1b RSI: ffff88081fa99650 RDI: 
> 0000000000019650
> [ 3384.758684] RBP: ffff8807dceeba38 R08: ffffea0006c656c0 R09: 
> 0000000000000000
> [ 3384.758733] R10: ffffffffa02b8c38 R11: 0000000000000000 R12: 
> ffff8801b195bbd0
> [ 3384.758781] R13: ffff88010d092d80 R14: ffff880102ba33a8 R15: 
> 0000000000000002
> [ 3384.758830] FS:  0000000000000000(0000) GS:ffff88081fa80000(0000)
> knlGS:0000000000000000
> [ 3384.758918] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 3384.758965] CR2: 00007f980e95d000 CR3: 0000000001611000 CR4: 
> 00000000001407e0
> [ 3384.759013] Stack:
> [ 3384.759054]  ffff88010d092dd0 ffffffffa02d8d20 ffff880102516d20
> ffff8807449a9000
> [ 3384.759143]  0000000000000000 ffff8801b195bbd0 0000000000000002
> ffff880102516d20
> [ 3384.759232]  ffff8807449a9000 ffff88010d092d80 ffff8807dceeba98
> ffffffffa02c7e92
> [ 3384.759321] Call Trace:
> [ 3384.759369]  [<ffffffffa02d8d20>] ?
> join_transaction.isra.30+0x24/0x309 [btrfs]
> [ 3384.759461]  [<ffffffffa02c7e92>] walk_down_tree+0x45/0xd5 [btrfs]
> [ 3384.759513]  [<ffffffffa02ca89a>] btrfs_drop_snapshot+0x2f5/0x68f 
> [btrfs]
> [ 3384.759567]  [<ffffffffa031b19d>] merge_reloc_roots+0x139/0x23f 
> [btrfs]
> [ 3384.759620]  [<ffffffffa031b8f6>] relocate_block_group+0x466/0x4de 
> [btrfs]
> [ 3384.759673]  [<ffffffffa031bac6>]
> btrfs_relocate_block_group+0x158/0x278 [btrfs]
> [ 3384.759766]  [<ffffffffa02f86f0>]
> btrfs_relocate_chunk.isra.62+0x58/0x5f7 [btrfs]
> [ 3384.759859]  [<ffffffffa03078d7>] ?
> btrfs_set_lock_blocking_rw+0x68/0x95 [btrfs]
> [ 3384.759950]  [<ffffffffa02b8b04>] ? 
> btrfs_set_path_blocking+0x23/0x54 [btrfs]
> [ 3384.760001]  [<ffffffffa02bd517>] ? btrfs_search_slot+0x7bc/0x816 
> [btrfs]
> [ 3384.760055]  [<ffffffffa02f4bd5>] ? free_extent_buffer+0x6f/0x7c 
> [btrfs]
> [ 3384.760108]  [<ffffffffa02fb5cd>] btrfs_balance+0xa7b/0xc80 [btrfs]
> [ 3384.760156]  [<ffffffff813a9125>] ? printk+0x48/0x4a
> [ 3384.760207]  [<ffffffffa02fb829>] balance_kthread+0x57/0x7c [btrfs]
> [ 3384.760259]  [<ffffffffa02fb7d2>] ? btrfs_balance+0xc80/0xc80 
> [btrfs]
> [ 3384.760310]  [<ffffffffa02fb7d2>] ? btrfs_balance+0xc80/0xc80 
> [btrfs]
> [ 3384.760359]  [<ffffffff8104e993>] kthread+0xcd/0xd5
> [ 3384.760405]  [<ffffffff8104e8c6>] ? 
> kthread_freezable_should_stop+0x43/0x43
> [ 3384.760453]  [<ffffffff813affec>] ret_from_fork+0x7c/0xb0
> [ 3384.760500]  [<ffffffff8104e8c6>] ? 
> kthread_freezable_should_stop+0x43/0x43
> [ 3384.760548] Code: b8 01 00 00 00 4c 89 f9 48 89 04 24 49 8b 16 e8
> 5a f5 ff ff 83 f8 f4 75 02 0f 0b 85 c0 0f 85 48 01 00 00 4b 83 7c fd
> 00 00 75 02 <0f> 0b 41 83 bd 94 00 00 00 01 48 63 c3 75 2d 49 83 7c c5
> 00 01
> [ 3384.760721] RIP  [<ffffffffa02c5ca6>] walk_down_proc+0xdc/0x22b 
> [btrfs]
> [ 3384.760773]  RSP <ffff8807dceeb9e8>
> [ 3384.761080] ---[ end trace 4f32f915eb18c088 ]---

-- 
Tomasz Chmielewski
http://www.sslrack.com

3.19.3, btrfs send/receive error: failed to clone extents

[Marc MERLIN]

I'm getting this from my send/receive job.

I'm just going to delete that one file since I don't need it anyway, but
is this a known problem or something that would need more info from me?


ERROR: failed to clone extents to merlin/.config/google-chrome-beta-42/Default/Application Cache/Cache/data_0
Invalid argument

ABORT: btrfs send -p /mnt/btrfs_pool1/home_ro.20150403_18:01:24 home_ro.20150429_15:01:24 |  btrfs receive /mnt/btrfs_pool2// failed


-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/                         | PGP 1024R/763BE901

Re: 3.19.3: check tree block failed + WARNING: device 0 not present on scrub

[Marc MERLIN]

I'm Cc'ing Christian because he reported the same problem of getting
this during scrub:
WARNING: device 0 not present
scrub device /dev/sda3 (id 1) done
    scrub started at Sat Apr 25 23:55:56 2015 and finished after 233 seconds
    total bytes scrubbed: 100.39GiB with 0 errors
scrub device  (id 0) canceled
    scrub started at Sat Apr 25 23:55:56 2015 and was aborted after 0 seconds
    total bytes scrubbed: 0.00B with 0 errors

Mine was doing it too, but then it stopped out of the blue.

So I was debugging a btrfs send issue with Filipe, and then we found
this:

btrfs-debug-tree -t 2 /dev/mapper/cryptroot 2>&1 | tee /tmp/debug_2.txt

leaf 372551614464 items 125 free space 7363 generation 1911640 owner 2
fs uuid 79a9a6c7-0955-4820-b81e-0c0681a657c1
chunk uuid a45c7819-9aee-4999-8a39-566181679f9c
	item 0 key (291412238336 EXTENT_ITEM 16384) itemoff 16178 itemsize 105
		extent refs 5 gen 688250 flags DATA
		extent data backref root 23942 objectid 2028 offset 27253407744 count 1
		shared data backref parent 919382933504 count 1
		shared data backref parent 510112530432 count 1
		shared data backref parent 510100160512 count 1
		shared data backref parent 230103220224 count 1
(...)
	item 123 key (291431116800 EXTENT_ITEM 4096) itemoff 10554 itemsize 37
		extent refs 1 gen 172150 flags DATA
		shared data backref parent 510226677760 count 1
	item 124 key (291431120896 EXTENT_ITEM 32768) itemoff 10488 itemsize 66
		extent refs 2 gen 334924 flags DATA
		extent data backref root 23942 objectid 2028 offset 85580054528 count 1
		shared data backref parent 504622628864 count 1
Check tree block failed, want=612294639616, have=687330107929042309
Check tree block failed, want=612294639616, have=687330107929042309
Check tree block failed, want=612294639616, have=10656972372562425046
Check tree block failed, want=612294639616, have=10656972372562425046
Check tree block failed, want=612294639616, have=10656972372562425046
read block failed check_tree_block
failed to read 612294639616 in tree 2
parent transid verify failed on 935174963200 wanted 1912353 found 1912435
parent transid verify failed on 935174963200 wanted 1912353 found 1912435
parent transid verify failed on 935174963200 wanted 1912353 found 1912435
parent transid verify failed on 935174963200 wanted 1912353 found 1912435
Ignoring transid failure
print-tree.c:1071: btrfs_print_tree: Assertion failed.
btrfs-debug-tree[0x410489]
btrfs-debug-tree[0x411dbf]
btrfs-debug-tree[0x402adb]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7fc2bc6b9b45]
btrfs-debug-tree[0x402d85]
 

On Sat, May 02, 2015 at 07:11:02AM -0700, Marc MERLIN wrote:
> On Sat, May 02, 2015 at 12:24:14PM +0100, Filipe David Manana wrote:
> > That looks bad.
> > Is this by chance a multi devices fs? Is some device missing?
>  
> It's not. It's a single device.
> 
> I run scrub from cron nightly, no errors.
> 
> > Given that you have such errors reported for the extent tree (id 2),
> > which is a critical tree for the fs to work, I wouldn't expect you to
> > be able to use the fs and maybe not even mount it - but since you
> > never had the fs turn into readonly mode or reported any crashes, etc,
> > lets hope it's just one device missing due to udev rules (I never
> > looked into that part myself).
> 
> It's not, laptop is working just fine (seemingly) and scrub is passing
> nightly.
> 
> btrfs check, I'll have to reboot my laptop from another device before I
> can run this since I can't check my mounted filesystem, correct?
> 
> Oooh, but not that you mention it, I have had scrub return this for a few
> days:
> WARNING: device 0 not present
> during a scrub, even though I had a single device filesystem.
> 
> It then went away on its own without me being able to figure out what it
> was.

Does anyone know what that 'WARNING: device 0 not present' on scrub
could be considering my filesystem is a single device filesystem?

Next, I can't read only btrfs check a mounted filesystem, correct?

If so, should I boot from rescue media, do check and or repair, or try
something else first?

Thanks,
Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/                         | PGP 1024R/763BE901

Re: 3.19.3: check tree block failed + WARNING: device 0 not present on scrub

[Christian Dysthe]

On 05/02/2015 12:30 PM, Marc MERLIN wrote:
> I'm Cc'ing Christian because he reported the same problem of getting
> this during scrub:
> WARNING: device 0 not present
> scrub device /dev/sda3 (id 1) done
>      scrub started at Sat Apr 25 23:55:56 2015 and finished after 233 seconds
>      total bytes scrubbed: 100.39GiB with 0 errors
> scrub device  (id 0) canceled
>      scrub started at Sat Apr 25 23:55:56 2015 and was aborted after 0 seconds
>      total bytes scrubbed: 0.00B with i0 errors
Mine is still doing it. This is from today's log:

/etc/cron.daily/btrfs-scrub:
WARNING: device 0 not present
scrub device /dev/sda3 (id 1) done
	scrub started at Sat May  2 08:03:46 2015 and finished after 241 seconds
	total bytes scrubbed: 109.06GiB with 0 errors
scrub device  (id 0) canceled
	scrub started at Sat May  2 08:03:46 2015 and was aborted after 0 seconds
	total bytes scrubbed: 0.00B with 0 errors

It doesn't seem to cause any problems but it would of course be good to know what it is and even get rid of it. This is my laptop with a SSD containing a small ext4 /boot partition and a latrge btrfs partition / with a /home subvolume (close to Ubuntu standard for btrfs).

Mine was doing it too, but then it stopped out of the blue.

So I was debugging a btrfs send issue with Filipe, and then we found
this:

btrfs-debug-tree -t 2 /dev/mapper/cryptroot 2>&1 | tee /tmp/debug_2.txt

leaf 372551614464 items 125 free space 7363 generation 1911640 owner 2
fs uuid 79a9a6c7-0955-4820-b81e-0c0681a657c1
chunk uuid a45c7819-9aee-4999-8a39-566181679f9c
	item 0 key (291412238336 EXTENT_ITEM 16384) itemoff 16178 itemsize 105
		extent refs 5 gen 688250 flags DATA
		extent data backref root 23942 objectid 2028 offset 27253407744 count 1
		shared data backref parent 919382933504 count 1
		shared data backref parent 510112530432 count 1
		shared data backref parent 510100160512 count 1
		shared data backref parent 230103220224 count 1
(...)
	item 123 key (291431116800 EXTENT_ITEM 4096) itemoff 10554 itemsize 37
		extent refs 1 gen 172150 flags DATA
		shared data backref parent 510226677760 count 1
	item 124 key (291431120896 EXTENT_ITEM 32768) itemoff 10488 itemsize 66
		extent refs 2 gen 334924 flags DATA
		extent data backref root 23942 objectid 2028 offset 85580054528 count 1
		shared data backref parent 504622628864 count 1
Check tree block failed, want=612294639616, have=687330107929042309
Check tree block failed, want=612294639616, have=687330107929042309
Check tree block failed, want=612294639616, have=10656972372562425046
Check tree block failed, want=612294639616, have=10656972372562425046
Check tree block failed, want=612294639616, have=10656972372562425046
read block failed check_tree_block
failed to read 612294639616 in tree 2
parent transid verify failed on 935174963200 wanted 1912353 found 1912435
parent transid verify failed on 935174963200 wanted 1912353 found 1912435
parent transid verify failed on 935174963200 wanted 1912353 found 1912435
parent transid verify failed on 935174963200 wanted 1912353 found 1912435
Ignoring transid failure
print-tree.c:1071: btrfs_print_tree: Assertion failed.
btrfs-debug-tree[0x410489]
btrfs-debug-tree[0x411dbf]
btrfs-debug-tree[0x402adb]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7fc2bc6b9b45]
btrfs-debug-tree[0x402d85]
  

On Sat, May 02, 2015 at 07:11:02AM -0700, Marc MERLIN wrote:

>> On Sat, May 02, 2015 at 12:24:14PM +0100, Filipe David Manana wrote:
>>> That looks bad.
>>> Is this by chance a multi devices fs? Is some device missing?
>>   
>> It's not. It's a single device.
>>
>> I run scrub from cron nightly, no errors.
>>
>>> Given that you have such errors reported for the extent tree (id 2),
>>> which is a critical tree for the fs to work, I wouldn't expect you to
>>> be able to use the fs and maybe not even mount it - but since you
>>> never had the fs turn into readonly mode or reported any crashes, etc,
>>> lets hope it's just one device missing due to udev rules (I never
>>> looked into that part myself).
>> It's not, laptop is working just fine (seemingly) and scrub is passing
>> nightly.
>>
>> btrfs check, I'll have to reboot my laptop from another device before I
>> can run this since I can't check my mounted filesystem, correct?
>>
>> Oooh, but not that you mention it, I have had scrub return this for a few
>> days:
>> WARNING: device 0 not present
>> during a scrub, even though I had a single device filesystem.
>>
>> It then went away on its own without me being able to figure out what it
>> was.
> Does anyone know what that 'WARNING: device 0 not present' on scrub
> could be considering my filesystem is a single device filesystem?
>
> Next, I can't read only btrfs check a mounted filesystem, correct?
>
> If so, should I boot from rescue media, do check and or repair, or try
> something else first?
>
> Thanks,
> Marc

-- 
Regards,
Christian Dysthe - Vivaldi Technlologies
Download Vivaldi at: https://vivaldi.com
Discuss it at: https://vivaldi.net

Re: 3.19.3: check tree block failed + WARNING: device 0 not present on scrub

[Marc MERLIN]

On Sat, May 02, 2015 at 12:50:14PM -0400, Christian Dysthe wrote:
> Mine is still doing it. This is from today's log:
> 
> /etc/cron.daily/btrfs-scrub:
> WARNING: device 0 not present
> scrub device /dev/sda3 (id 1) done
> 	scrub started at Sat May  2 08:03:46 2015 and finished after 241 seconds
> 	total bytes scrubbed: 109.06GiB with 0 errors
> scrub device  (id 0) canceled
> 	scrub started at Sat May  2 08:03:46 2015 and was aborted after 0 seconds
> 	total bytes scrubbed: 0.00B with 0 errors
> 
> It doesn't seem to cause any problems but it would of course be good to know what it is and even get rid of it. This is my laptop with a SSD containing a small ext4 /boot partition and a latrge btrfs partition / with a /home subvolume (close to Ubuntu standard for btrfs).
 
Please run the equivalent of this on your filesystem:
btrfs-debug-tree -t 2 /dev/mapper/cryptroot 2>&1 | tee /tmp/debug_2.txt

And see if it completes or fails in the middle.

Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/                         | PGP 1024R/763BE901

Re: 3.19.3: check tree block failed + WARNING: device 0 not present on scrub

[Christian Dysthe]

On 05/02/2015 01:05 PM, Marc MERLIN wrote:
> On Sat, May 02, 2015 at 12:50:14PM -0400, Christian Dysthe wrote:
>> Mine is still doing it. This is from today's log:
>>
>> /etc/cron.daily/btrfs-scrub:
>> WARNING: device 0 not present
>> scrub device /dev/sda3 (id 1) done
>> 	scrub started at Sat May  2 08:03:46 2015 and finished after 241 seconds
>> 	total bytes scrubbed: 109.06GiB with 0 errors
>> scrub device  (id 0) canceled
>> 	scrub started at Sat May  2 08:03:46 2015 and was aborted after 0 seconds
>> 	total bytes scrubbed: 0.00B with 0 errors
>>
>> It doesn't seem to cause any problems but it would of course be good to know what it is and even get rid of it. This is my laptop with a SSD containing a small ext4 /boot partition and a latrge btrfs partition / with a /home subvolume (close to Ubuntu standard for btrfs).
>   
> Please run the equivalent of this on your filesystem:
> btrfs-debug-tree -t 2 /dev/mapper/cryptroot 2>&1 | tee /tmp/debug_2.txt
>
> And see if it completes or fails in the middle.
All I get is: Could not open /dev/mapper/cryptroot  The only thing in 
that directory is -control I do not have any kinds of encryption on 
anything.
>
> Marc

-- 
Regards,
Christian Dysthe - Vivaldi Technlologies
Download Vivaldi at: https://vivaldi.com
Discuss it at: https://vivaldi.net

Re: 3.19.3: check tree block failed + WARNING: device 0 not present on scrub

[Marc MERLIN]

On Sat, May 02, 2015 at 01:20:00PM -0400, Christian Dysthe wrote:
> 
> 
> On 05/02/2015 01:05 PM, Marc MERLIN wrote:
> >On Sat, May 02, 2015 at 12:50:14PM -0400, Christian Dysthe wrote:
> >>Mine is still doing it. This is from today's log:
> >>
> >>/etc/cron.daily/btrfs-scrub:
> >>WARNING: device 0 not present
> >>scrub device /dev/sda3 (id 1) done
> >>	scrub started at Sat May  2 08:03:46 2015 and finished after 241 seconds
> >>	total bytes scrubbed: 109.06GiB with 0 errors
> >>scrub device  (id 0) canceled
> >>	scrub started at Sat May  2 08:03:46 2015 and was aborted after 0 seconds
> >>	total bytes scrubbed: 0.00B with 0 errors
> >>
> >>It doesn't seem to cause any problems but it would of course be good to know what it is and even get rid of it. This is my laptop with a SSD containing a small ext4 /boot partition and a latrge btrfs partition / with a /home subvolume (close to Ubuntu standard for btrfs).
> >Please run the equivalent of this on your filesystem:
> >btrfs-debug-tree -t 2 /dev/mapper/cryptroot 2>&1 | tee /tmp/debug_2.txt
> >
> >And see if it completes or fails in the middle.
> All I get is: Could not open /dev/mapper/cryptroot  The only thing
> in that directory is -control I do not have any kinds of encryption
> on anything.

I wrote "the equivalent of", so change /dev/mapper/cryptroot for your
device name :)

Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/                         | PGP 1024R/763BE901

Re: 3.19.3: check tree block failed + WARNING: device 0 not present on scrub

[Christian Dysthe]

On 05/02/2015 01:29 PM, Marc MERLIN wrote:
> On Sat, May 02, 2015 at 01:20:00PM -0400, Christian Dysthe wrote:
>>
>> On 05/02/2015 01:05 PM, Marc MERLIN wrote:
>>> On Sat, May 02, 2015 at 12:50:14PM -0400, Christian Dysthe wrote:
>>>> Mine is still doing it. This is from today's log:
>>>>
>>>> /etc/cron.daily/btrfs-scrub:
>>>> WARNING: device 0 not present
>>>> scrub device /dev/sda3 (id 1) done
>>>> 	scrub started at Sat May  2 08:03:46 2015 and finished after 241 seconds
>>>> 	total bytes scrubbed: 109.06GiB with 0 errors
>>>> scrub device  (id 0) canceled
>>>> 	scrub started at Sat May  2 08:03:46 2015 and was aborted after 0 seconds
>>>> 	total bytes scrubbed: 0.00B with 0 errors
>>>>
>>>> It doesn't seem to cause any problems but it would of course be good to know what it is and even get rid of it. This is my laptop with a SSD containing a small ext4 /boot partition and a latrge btrfs partition / with a /home subvolume (close to Ubuntu standard for btrfs).
>>> Please run the equivalent of this on your filesystem:
>>> btrfs-debug-tree -t 2 /dev/mapper/cryptroot 2>&1 | tee /tmp/debug_2.txt
>>>
>>> And see if it completes or fails in the middle.
>> All I get is: Could not open /dev/mapper/cryptroot  The only thing
>> in that directory is -control I do not have any kinds of encryption
>> on anything.
> I wrote "the equivalent of", so change /dev/mapper/cryptroot for your
> device name :)
Okay then! :)

I ran it on my btrfs partition without any error ending with:

total bytes 239269314560
bytes used 116273602560
uuid 3d52dc93-c89f-453f-965d-8601d11e7710
Btrfs v3.17
>
> Marc

-- 
Regards,
Christian Dysthe - Vivaldi Technlologies
Download Vivaldi at: https://vivaldi.com
Discuss it at: https://vivaldi.net

Re: 3.19.3: check tree block failed + WARNING: device 0 not present on scrub

[Marc MERLIN]

On Sat, May 02, 2015 at 09:30:10AM -0700, Marc MERLIN wrote:
> btrfs-debug-tree -t 2 /dev/mapper/cryptroot 2>&1 | tee /tmp/debug_2.txt
> 
> leaf 372551614464 items 125 free space 7363 generation 1911640 owner 2
> fs uuid 79a9a6c7-0955-4820-b81e-0c0681a657c1
> chunk uuid a45c7819-9aee-4999-8a39-566181679f9c
> 	item 0 key (291412238336 EXTENT_ITEM 16384) itemoff 16178 itemsize 105
> 		extent refs 5 gen 688250 flags DATA
> 		extent data backref root 23942 objectid 2028 offset 27253407744 count 1
> 		shared data backref parent 919382933504 count 1
> 		shared data backref parent 510112530432 count 1
> 		shared data backref parent 510100160512 count 1
> 		shared data backref parent 230103220224 count 1
> (...)
> 	item 123 key (291431116800 EXTENT_ITEM 4096) itemoff 10554 itemsize 37
> 		extent refs 1 gen 172150 flags DATA
> 		shared data backref parent 510226677760 count 1
> 	item 124 key (291431120896 EXTENT_ITEM 32768) itemoff 10488 itemsize 66
> 		extent refs 2 gen 334924 flags DATA
> 		extent data backref root 23942 objectid 2028 offset 85580054528 count 1
> 		shared data backref parent 504622628864 count 1
> Check tree block failed, want=612294639616, have=687330107929042309
> Check tree block failed, want=612294639616, have=687330107929042309
> Check tree block failed, want=612294639616, have=10656972372562425046
> Check tree block failed, want=612294639616, have=10656972372562425046
> Check tree block failed, want=612294639616, have=10656972372562425046
> read block failed check_tree_block
> failed to read 612294639616 in tree 2
> parent transid verify failed on 935174963200 wanted 1912353 found 1912435
> parent transid verify failed on 935174963200 wanted 1912353 found 1912435
> parent transid verify failed on 935174963200 wanted 1912353 found 1912435
> parent transid verify failed on 935174963200 wanted 1912353 found 1912435
> Ignoring transid failure
> print-tree.c:1071: btrfs_print_tree: Assertion failed.
> btrfs-debug-tree[0x410489]
> btrfs-debug-tree[0x411dbf]
> btrfs-debug-tree[0x402adb]
> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7fc2bc6b9b45]
> btrfs-debug-tree[0x402d85]

Ok, so I ran btrfs check --repair on it with btrfs 4.0 and got this:
http://marc.merlins.org/tmp/repair.txt

Many lines of 
root 62006 inode 454222 errors 400, nbytes wrong
root 62006 inode 454223 errors 400, nbytes wrong
root 62006 inode 9680874 errors 400, nbytes wrong
and
reset isize for dir 28434 root 394
reset isize for dir 37282 root 394
reset isize for dir 37712 root 394

Not sure when/how I picked those up, but I guess repair was able to handle
them.

But what do they mean? Does size wrong mean that I have files that got reset
with less data that they had before?

Thanks,
Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: 3.19.6: __btrfs_free_extent:5987: errno=-2 No such entry

[Marc MERLIN]

On Mon, May 04, 2015 at 11:32:15PM -0700, Marc MERLIN wrote:
> Ok, so I ran btrfs check --repair on it with btrfs 4.0 and got this:
> http://marc.merlins.org/tmp/repair.txt
> 
> Many lines of 
> root 62006 inode 454222 errors 400, nbytes wrong
> root 62006 inode 454223 errors 400, nbytes wrong
> root 62006 inode 9680874 errors 400, nbytes wrong
> and
> reset isize for dir 28434 root 394
> reset isize for dir 37282 root 394
> reset isize for dir 37712 root 394
> 
> Not sure when/how I picked those up, but I guess repair was able to handle
> them.
> 
> But what do they mean? Does size wrong mean that I have files that got reset
> with less data that they had before?

Mmmh, it looks like repair may have made my FS worse, or uncovered more
problems that just caused my FS to go read only after 4H of use :-/

Can someone tell me if this state is fixable or if I need to destroy the FS
and start over?
[17159.103764] ------------[ cut here ]------------
[17159.103775] WARNING: CPU: 6 PID: 506 at fs/btrfs/extent-tree.c:5981 __btrfs_free_extent+0x3a3/0x81a()
[17159.103777] Modules linked in: cx231xx_alsa cx25840 cx231xx videobuf_vmalloc tveeprom cx2341x rc_core videobuf_core i2c_mux nls_utf8 nls_cp437 vfat fat uas usb_storage rpcsec_gss_krb5 nfsv4 ctr ccm ipt_REJECT nf_reject_ipv4 xt_tcpudp nf_log_ipv4 nf_log_common xt_LOG iptable_mangle ip6table_filter ip6_tables ebtable_nat ebtables rfcomm bnep pci_stub vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) xt_addrtype xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_filter ip_tables x_tables bridge stp llc autofs4 binfmt_misc uinput nfsd auth_rpcgss nfs_acl nfs lockd grace fscache sunrpc ecryptfs configs ppdev parport_pc lp parport input_polldev loop firewire_sbp2 firewire_core crc_itu_t uvcvideo videobuf2_vmalloc videobuf2_memops btusb videobuf2_core v4l2_common bluetooth videodev media joydev arc4 intel_rapl iosf_mbi x86_pkg_temp_thermal intel_powerclamp coretemp snd_hda_codec_realtek kvm_intel snd_hda_codec_generic kvm iwlmvm snd_hda_codec_hdmi mac80211 crct10dif_pclmul crc32_pclmul iTCO_wdt rtsx_pci_ms snd_hda_intel ghash_clmulni_intel iTCO_vendor_support snd_hda_controller memstick rtsx_pci_sdmmc iwlwifi snd_hda_codec snd_pcm_oss snd_hwdep psmouse microcode serio_raw snd_mixer_oss pcspkr i2c_i801 thinkpad_acpi snd_pcm cfg80211 sg tpm_tis rtsx_pci lpc_ich nvram rfkill snd_seq_midi ac tpm wmi battery snd_seq_midi_event ehci_pci xhci_pci ehci_hcd xhci_hcd snd_rawmidi snd_seq snd_seq_device usbcore snd_timer usb_common snd ie31200_edac soundcore edac_core shpchp intel_smartconnect evdev processor sata_sil24 r8169 mii fuse fan raid456 multipath mmc_block mmc_core dm_crypt dm_mod async_raid6_recov async_pq async_xor async_memcpy async_tx blowfish_x86_64 blowfish_common ecb xts crc32c_intel aesni_intel aes_x86_64 e1000e glue_helper lrw gf128mul ptp ablk_helper cryptd pps_core thermal
[17159.103982] CPU: 6 PID: 506 Comm: btrfs-cleaner Tainted: G        W  OE  3.19.6-amd64-i915-volpreempt-s20150421rc3 #2
[17159.103984] Hardware name: LENOVO 20BECT0/20BECT0, BIOS GMET28WW (1.08 ) 09/18/2013
[17159.103986]  0000000000000009 ffff88040669fac8 ffffffff81677ec0 00000000000072d2
[17159.103991]  0000000000000000 ffff88040669fb08 ffffffff810556a7 a800000049e19ce0
[17159.103998]  ffffffff8122ca7d ffff8803d20b6020 00000000fffffffe 0000000000000000
[17159.104003] Call Trace:
[17159.104009]  [<ffffffff81677ec0>] dump_stack+0x45/0x57
[17159.104013]  [<ffffffff810556a7>] warn_slowpath_common+0xa1/0xbb
[17159.104016]  [<ffffffff8122ca7d>] ? __btrfs_free_extent+0x3a3/0x81a
[17159.104018]  [<ffffffff81055764>] warn_slowpath_null+0x1a/0x1c
[17159.104021]  [<ffffffff8122ca7d>] __btrfs_free_extent+0x3a3/0x81a
[17159.104024]  [<ffffffff812323ce>] __btrfs_run_delayed_refs+0xaee/0xc22
[17159.104027]  [<ffffffff81230858>] ? btrfs_free_tree_block+0x18c/0x1c7
[17159.104030]  [<ffffffff81234190>] btrfs_run_delayed_refs+0x6d/0x19e
[17159.104032]  [<ffffffff81230c13>] ? walk_up_tree+0x72/0xf9
[17159.104036]  [<ffffffff81241af6>] btrfs_should_end_transaction+0x52/0x5b
[17159.104038]  [<ffffffff81232d1f>] btrfs_drop_snapshot+0x36f/0x696
[17159.104042]  [<ffffffff81287651>] ? btrfs_kill_all_delayed_nodes+0x43/0xbd
[17159.104046]  [<ffffffff8167ab1a>] ? __schedule+0x472/0x505
[17159.104049]  [<ffffffff81241f3c>] btrfs_clean_one_deleted_snapshot+0xce/0xdb
[17159.104052]  [<ffffffff8123b7c1>] cleaner_kthread+0x112/0x146
[17159.104056]  [<ffffffff8123b6af>] ? atomic_add_unless.constprop.53+0x24/0x24
[17159.104059]  [<ffffffff8106d296>] kthread+0xae/0xb6
[17159.104061]  [<ffffffff8106d1e8>] ? __kthread_parkme+0x61/0x61
[17159.104064]  [<ffffffff8167d818>] ret_from_fork+0x58/0x90
[17159.104068]  [<ffffffff8106d1e8>] ? __kthread_parkme+0x61/0x61
[17159.104070] ---[ end trace d372a2208cf37cec ]---
[17159.104073] BTRFS info (device dm-0): leaf 49860935680 total ptrs 141 free space 5497
[17159.104075]  item 0 key (317305278464 168 77824) itemoff 16246 itemsize 37
[17159.104077]          extent refs 1 gen 1899084 flags 1
[17159.104078]          shared data backref parent 230381830144 count 1
[17159.104080]  item 1 key (317305356288 168 69632) itemoff 16209 itemsize 37
[17159.104082]          extent refs 1 gen 1899084 flags 1
[17159.104083]          shared data backref parent 230381830144 count 1
[17159.104086]  item 2 key (317305757696 168 4096) itemoff 16130 itemsize 79
[17159.104087]          extent refs 3 gen 1899084 flags 1
[17159.104089]          extent data backref root 392 objectid 11041142 offset 421888 count 1
[17159.104090]          shared data backref parent 491878858752 count 1
[17159.104092]          shared data backref parent 327213531136 count 1
(...)
[17159.104836]  item 139 key (317355589632 168 16384) itemoff 9059 itemsize 37
[17159.104837]          extent refs 1 gen 681 flags 1
[17159.104838]          shared data backref parent 230518226944 count 1
[17159.104840]  item 140 key (317356175360 168 49152) itemoff 9022 itemsize 37
[17159.104841]          extent refs 1 gen 1639904 flags 1
[17159.104842]          shared data backref parent 919346233344 count 1
[17159.104844] BTRFS error (device dm-0): unable to find ref byte nr 317317357568 parent 0 root 70041  owner 24331 offset 0
[17159.104846] ------------[ cut here ]------------
[17159.104850] WARNING: CPU: 6 PID: 506 at fs/btrfs/super.c:260 __btrfs_abort_transaction+0x52/0x113()
[17159.104851] BTRFS: Transaction aborted (error -2)
[17159.104852] Modules linked in: cx231xx_alsa cx25840 cx231xx videobuf_vmalloc tveeprom cx2341x rc_core videobuf_core i2c_mux nls_utf8 nls_cp437 vfat fat uas usb_storage rpcsec_gss_krb5 nfsv4 ctr ccm ipt_REJECT nf_reject_ipv4 xt_tcpudp nf_log_ipv4 nf_log_common xt_LOG iptable_mangle ip6table_filter ip6_tables ebtable_nat ebtables rfcomm bnep pci_stub vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) xt_addrtype xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_filter ip_tables x_tables bridge stp llc autofs4 binfmt_misc uinput nfsd auth_rpcgss nfs_acl nfs lockd grace fscache sunrpc ecryptfs configs ppdev parport_pc lp parport input_polldev loop firewire_sbp2 firewire_core crc_itu_t uvcvideo videobuf2_vmalloc videobuf2_memops btusb videobuf2_core v4l2_common bluetooth videodev media joydev arc4 intel_rapl iosf_mbi x86_pkg_temp_thermal intel_powerclamp coretemp snd_hda_codec_realtek kvm_intel snd_hda_codec_generic kvm iwlmvm snd_hda_codec_hdmi mac80211 crct10dif_pclmul crc32_pclmul iTCO_wdt rtsx_pci_ms snd_hda_intel ghash_clmulni_intel iTCO_vendor_support snd_hda_controller memstick rtsx_pci_sdmmc iwlwifi snd_hda_codec snd_pcm_oss snd_hwdep psmouse microcode serio_raw snd_mixer_oss pcspkr i2c_i801 thinkpad_acpi snd_pcm cfg80211 sg tpm_tis rtsx_pci lpc_ich nvram rfkill snd_seq_midi ac tpm wmi battery snd_seq_midi_event ehci_pci xhci_pci ehci_hcd xhci_hcd snd_rawmidi snd_seq snd_seq_device usbcore snd_timer usb_common snd ie31200_edac soundcore edac_core shpchp intel_smartconnect evdev processor sata_sil24 r8169 mii fuse fan raid456 multipath mmc_block mmc_core dm_crypt dm_mod async_raid6_recov async_pq async_xor async_memcpy async_tx blowfish_x86_64 blowfish_common ecb xts crc32c_intel aesni_intel aes_x86_64 e1000e glue_helper lrw gf128mul ptp ablk_helper cryptd pps_core thermal
[17159.105040] CPU: 6 PID: 506 Comm: btrfs-cleaner Tainted: G        W  OE  3.19.6-amd64-i915-volpreempt-s20150421rc3 #2
[17159.105042] Hardware name: LENOVO 20BECT0/20BECT0, BIOS GMET28WW (1.08 ) 09/18/2013
[17159.105043]  0000000000000009 ffff88040669fa38 ffffffff81677ec0 00000000000080e2
[17159.105050]  ffff88040669fa88 ffff88040669fa78 ffffffff810556a7 ffff88040669fa98
[17159.105055]  ffffffff8121e95d 00000000fffffffe ffff88040748b800 ffff880103c7b598
[17159.105059] Call Trace:
[17159.105062]  [<ffffffff81677ec0>] dump_stack+0x45/0x57
[17159.105065]  [<ffffffff810556a7>] warn_slowpath_common+0xa1/0xbb
[17159.105068]  [<ffffffff8121e95d>] ? __btrfs_abort_transaction+0x52/0x113
[17159.105070]  [<ffffffff81055707>] warn_slowpath_fmt+0x46/0x48
[17159.105073]  [<ffffffff8121e95d>] __btrfs_abort_transaction+0x52/0x113
[17159.105075]  [<ffffffff8122ced6>] __btrfs_free_extent+0x7fc/0x81a
[17159.105078]  [<ffffffff812323ce>] __btrfs_run_delayed_refs+0xaee/0xc22
[17159.105081]  [<ffffffff81230858>] ? btrfs_free_tree_block+0x18c/0x1c7
[17159.105083]  [<ffffffff81234190>] btrfs_run_delayed_refs+0x6d/0x19e
[17159.105086]  [<ffffffff81230c13>] ? walk_up_tree+0x72/0xf9
[17159.105089]  [<ffffffff81241af6>] btrfs_should_end_transaction+0x52/0x5b
[17159.105091]  [<ffffffff81232d1f>] btrfs_drop_snapshot+0x36f/0x696
[17159.105095]  [<ffffffff81287651>] ? btrfs_kill_all_delayed_nodes+0x43/0xbd
[17159.105098]  [<ffffffff8167ab1a>] ? __schedule+0x472/0x505
[17159.105100]  [<ffffffff81241f3c>] btrfs_clean_one_deleted_snapshot+0xce/0xdb
[17159.105103]  [<ffffffff8123b7c1>] cleaner_kthread+0x112/0x146
[17159.105105]  [<ffffffff8123b6af>] ? atomic_add_unless.constprop.53+0x24/0x24
[17159.105107]  [<ffffffff8106d296>] kthread+0xae/0xb6
[17159.105110]  [<ffffffff8106d1e8>] ? __kthread_parkme+0x61/0x61
[17159.105113]  [<ffffffff8167d818>] ret_from_fork+0x58/0x90
[17159.105115]  [<ffffffff8106d1e8>] ? __kthread_parkme+0x61/0x61
[17159.105116] ---[ end trace d372a2208cf37ced ]---
[17159.105118] BTRFS: error (device dm-0) in __btrfs_free_extent:5987: errno=-2 No such entry
[17159.105120] BTRFS info (device dm-0): forced readonly
[17159.105123] BTRFS: error (device dm-0) in btrfs_run_delayed_refs:2792: errno=-2 No such entry

-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: 3.19.6: __btrfs_free_extent:5987: errno=-2 No such entry, did btrfs check --repair break it?

[Marc MERLIN]

Dear Btrfs developers :)

I would very much like to start restoring my laptop tonight/tomorrow since
this is my main laptop I use for everything, and I need to get it back to
working state ASAP. Below some details and let me know if I should do
anything before that.

On the plus side, it's actually been a while since I've had to ask for your
help because my main laptop got a messed up btrfs filesystem.

As a 2nd huge plus, btrfs didn't crash my system. It went read only and gave
me logs I was able to easily capture. I know I complained a lot about this
in the past, and wanted to thank all of you who have been removing or
replacing those BUG_ON calls.

As for my problem now:
1) I had a seemingly perfectly working FS but while debugging a btrfs
send/receive bug with Filipe, we noticed that 
btrfs-debug-tree -t 2 /dev/mapper/cryptroot crashed half way.

2) Filipe gave me a fix for the btrfs send issue, and after asking here
about the btrfs-debug-tree issue and not hearing back, I figured I should 
run check --repair (v4.0-dirty compiled from git a few days ago).
Check --repair took about 8H to run for a 800GB filesystem and succeeded
I posted the results here and asked what they meant:
http://marc.merlins.org/tmp/repair.txt

3) I then rebooted to my main drive I had just repaired this morning, and
the filesystem went read only after 4H or so.
Full dmesg:
http://marc.merlins.org/tmp/btrfs-3.19.6-readonly.txt

Summary:
WARNING: CPU: 6 PID: 506 at fs/btrfs/extent-tree.c:5981 __btrfs_free_extent+0x3a3/0x81a()
BTRFS error (device dm-0): unable to find ref byte nr 317317357568 parent 0 root 70041  owner 24331 offset 0
WARNING: CPU: 6 PID: 506 at fs/btrfs/super.c:260 __btrfs_abort_transaction+0x52/0x113()
BTRFS: error (device dm-0) in __btrfs_free_extent:5987: errno=-2 No such entry


Please let me know what I can do for you if anything, before I wipe the
filesystem and recreate it (I'm assuming that running btrfs check --repair
a 2nd time won't help).
If one of you wants a btrfs image, it'll take some hours to generate and
upload, but I can make one.
If I should do anything other than wipe and start over after that, let me
know.


Thanks,
Marc


On Tue, May 05, 2015 at 12:56:10PM -0700, Marc MERLIN wrote:
> On Mon, May 04, 2015 at 11:32:15PM -0700, Marc MERLIN wrote:
> > Ok, so I ran btrfs check --repair on it with btrfs 4.0 and got this:
> > http://marc.merlins.org/tmp/repair.txt
> > 
> > Many lines of 
> > root 62006 inode 454222 errors 400, nbytes wrong
> > root 62006 inode 454223 errors 400, nbytes wrong
> > root 62006 inode 9680874 errors 400, nbytes wrong
> > and
> > reset isize for dir 28434 root 394
> > reset isize for dir 37282 root 394
> > reset isize for dir 37712 root 394
> > 
> > Not sure when/how I picked those up, but I guess repair was able to handle
> > them.
> > 
> > But what do they mean? Does size wrong mean that I have files that got reset
> > with less data that they had before?
> 
> Mmmh, it looks like repair may have made my FS worse, or uncovered more
> problems that just caused my FS to go read only after 4H of use :-/
> 
> Can someone tell me if this state is fixable or if I need to destroy the FS
> and start over?
> [17159.103764] ------------[ cut here ]------------
> [17159.103775] WARNING: CPU: 6 PID: 506 at fs/btrfs/extent-tree.c:5981 __btrfs_free_extent+0x3a3/0x81a()
> [17159.103777] Modules linked in: cx231xx_alsa cx25840 cx231xx videobuf_vmalloc tveeprom cx2341x rc_core videobuf_core i2c_mux nls_utf8 nls_cp437 vfat fat uas usb_storage rpcsec_gss_krb5 nfsv4 ctr ccm ipt_REJECT nf_reject_ipv4 xt_tcpudp nf_log_ipv4 nf_log_common xt_LOG iptable_mangle ip6table_filter ip6_tables ebtable_nat ebtables rfcomm bnep pci_stub vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) xt_addrtype xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_filter ip_tables x_tables bridge stp llc autofs4 binfmt_misc uinput nfsd auth_rpcgss nfs_acl nfs lockd grace fscache sunrpc ecryptfs configs ppdev parport_pc lp parport input_polldev loop firewire_sbp2 firewire_core crc_itu_t uvcvideo videobuf2_vmalloc videobuf2_memops btusb videobuf2_core v4l2_common bluetooth videodev media joydev arc4 intel_rapl iosf_mbi x86_pkg_temp_thermal intel_powerclamp coretemp snd_hda_codec_realtek kvm_intel snd_hda_codec_generic kvm iwlmvm snd_hda_codec_hdmi mac80211 crct10dif_pclmul crc32_pclmul iTCO_wdt rtsx_pci_ms snd_hda_intel ghash_clmulni_intel iTCO_vendor_support snd_hda_controller memstick rtsx_pci_sdmmc iwlwifi snd_hda_codec snd_pcm_oss snd_hwdep psmouse microcode serio_raw snd_mixer_oss pcspkr i2c_i801 thinkpad_acpi snd_pcm cfg80211 sg tpm_tis rtsx_pci lpc_ich nvram rfkill snd_seq_midi ac tpm wmi battery snd_seq_midi_event ehci_pci xhci_pci ehci_hcd xhci_hcd snd_rawmidi snd_seq snd_seq_device usbcore snd_timer usb_common snd ie31200_edac soundcore edac_core shpchp intel_smartconnect evdev processor sata_sil24 r8169 mii fuse fan raid456 multipath mmc_block mmc_core dm_crypt dm_mod async_raid6_recov async_pq async_xor async_memcpy async_tx blowfish_x86_64 blowfish_common ecb xts crc32c_intel aesni_intel aes_x86_64 e1000e glue_helper lrw gf128mul ptp ablk_helper cryptd pps_core thermal
> [17159.103982] CPU: 6 PID: 506 Comm: btrfs-cleaner Tainted: G        W  OE  3.19.6-amd64-i915-volpreempt-s20150421rc3 #2
> [17159.103984] Hardware name: LENOVO 20BECT0/20BECT0, BIOS GMET28WW (1.08 ) 09/18/2013
> [17159.103986]  0000000000000009 ffff88040669fac8 ffffffff81677ec0 00000000000072d2
> [17159.103991]  0000000000000000 ffff88040669fb08 ffffffff810556a7 a800000049e19ce0
> [17159.103998]  ffffffff8122ca7d ffff8803d20b6020 00000000fffffffe 0000000000000000
> [17159.104003] Call Trace:
> [17159.104009]  [<ffffffff81677ec0>] dump_stack+0x45/0x57
> [17159.104013]  [<ffffffff810556a7>] warn_slowpath_common+0xa1/0xbb
> [17159.104016]  [<ffffffff8122ca7d>] ? __btrfs_free_extent+0x3a3/0x81a
> [17159.104018]  [<ffffffff81055764>] warn_slowpath_null+0x1a/0x1c
> [17159.104021]  [<ffffffff8122ca7d>] __btrfs_free_extent+0x3a3/0x81a
> [17159.104024]  [<ffffffff812323ce>] __btrfs_run_delayed_refs+0xaee/0xc22
> [17159.104027]  [<ffffffff81230858>] ? btrfs_free_tree_block+0x18c/0x1c7
> [17159.104030]  [<ffffffff81234190>] btrfs_run_delayed_refs+0x6d/0x19e
> [17159.104032]  [<ffffffff81230c13>] ? walk_up_tree+0x72/0xf9
> [17159.104036]  [<ffffffff81241af6>] btrfs_should_end_transaction+0x52/0x5b
> [17159.104038]  [<ffffffff81232d1f>] btrfs_drop_snapshot+0x36f/0x696
> [17159.104042]  [<ffffffff81287651>] ? btrfs_kill_all_delayed_nodes+0x43/0xbd
> [17159.104046]  [<ffffffff8167ab1a>] ? __schedule+0x472/0x505
> [17159.104049]  [<ffffffff81241f3c>] btrfs_clean_one_deleted_snapshot+0xce/0xdb
> [17159.104052]  [<ffffffff8123b7c1>] cleaner_kthread+0x112/0x146
> [17159.104056]  [<ffffffff8123b6af>] ? atomic_add_unless.constprop.53+0x24/0x24
> [17159.104059]  [<ffffffff8106d296>] kthread+0xae/0xb6
> [17159.104061]  [<ffffffff8106d1e8>] ? __kthread_parkme+0x61/0x61
> [17159.104064]  [<ffffffff8167d818>] ret_from_fork+0x58/0x90
> [17159.104068]  [<ffffffff8106d1e8>] ? __kthread_parkme+0x61/0x61
> [17159.104070] ---[ end trace d372a2208cf37cec ]---
> [17159.104073] BTRFS info (device dm-0): leaf 49860935680 total ptrs 141 free space 5497
> [17159.104075]  item 0 key (317305278464 168 77824) itemoff 16246 itemsize 37
> [17159.104077]          extent refs 1 gen 1899084 flags 1
> [17159.104078]          shared data backref parent 230381830144 count 1
> [17159.104080]  item 1 key (317305356288 168 69632) itemoff 16209 itemsize 37
> [17159.104082]          extent refs 1 gen 1899084 flags 1
> [17159.104083]          shared data backref parent 230381830144 count 1
> [17159.104086]  item 2 key (317305757696 168 4096) itemoff 16130 itemsize 79
> [17159.104087]          extent refs 3 gen 1899084 flags 1
> [17159.104089]          extent data backref root 392 objectid 11041142 offset 421888 count 1
> [17159.104090]          shared data backref parent 491878858752 count 1
> [17159.104092]          shared data backref parent 327213531136 count 1
> (...)
> [17159.104836]  item 139 key (317355589632 168 16384) itemoff 9059 itemsize 37
> [17159.104837]          extent refs 1 gen 681 flags 1
> [17159.104838]          shared data backref parent 230518226944 count 1
> [17159.104840]  item 140 key (317356175360 168 49152) itemoff 9022 itemsize 37
> [17159.104841]          extent refs 1 gen 1639904 flags 1
> [17159.104842]          shared data backref parent 919346233344 count 1
> [17159.104844] BTRFS error (device dm-0): unable to find ref byte nr 317317357568 parent 0 root 70041  owner 24331 offset 0
> [17159.104846] ------------[ cut here ]------------
> [17159.104850] WARNING: CPU: 6 PID: 506 at fs/btrfs/super.c:260 __btrfs_abort_transaction+0x52/0x113()
> [17159.104851] BTRFS: Transaction aborted (error -2)
> [17159.104852] Modules linked in: cx231xx_alsa cx25840 cx231xx videobuf_vmalloc tveeprom cx2341x rc_core videobuf_core i2c_mux nls_utf8 nls_cp437 vfat fat uas usb_storage rpcsec_gss_krb5 nfsv4 ctr ccm ipt_REJECT nf_reject_ipv4 xt_tcpudp nf_log_ipv4 nf_log_common xt_LOG iptable_mangle ip6table_filter ip6_tables ebtable_nat ebtables rfcomm bnep pci_stub vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) xt_addrtype xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_filter ip_tables x_tables bridge stp llc autofs4 binfmt_misc uinput nfsd auth_rpcgss nfs_acl nfs lockd grace fscache sunrpc ecryptfs configs ppdev parport_pc lp parport input_polldev loop firewire_sbp2 firewire_core crc_itu_t uvcvideo videobuf2_vmalloc videobuf2_memops btusb videobuf2_core v4l2_common bluetooth videodev media joydev arc4 intel_rapl iosf_mbi x86_pkg_temp_thermal intel_powerclamp coretemp snd_hda_codec_realtek kvm_intel snd_hda_codec_generic kvm iwlmvm snd_hda_codec_hdmi mac80211 crct10dif_pclmul crc32_pclmul iTCO_wdt rtsx_pci_ms snd_hda_intel ghash_clmulni_intel iTCO_vendor_support snd_hda_controller memstick rtsx_pci_sdmmc iwlwifi snd_hda_codec snd_pcm_oss snd_hwdep psmouse microcode serio_raw snd_mixer_oss pcspkr i2c_i801 thinkpad_acpi snd_pcm cfg80211 sg tpm_tis rtsx_pci lpc_ich nvram rfkill snd_seq_midi ac tpm wmi battery snd_seq_midi_event ehci_pci xhci_pci ehci_hcd xhci_hcd snd_rawmidi snd_seq snd_seq_device usbcore snd_timer usb_common snd ie31200_edac soundcore edac_core shpchp intel_smartconnect evdev processor sata_sil24 r8169 mii fuse fan raid456 multipath mmc_block mmc_core dm_crypt dm_mod async_raid6_recov async_pq async_xor async_memcpy async_tx blowfish_x86_64 blowfish_common ecb xts crc32c_intel aesni_intel aes_x86_64 e1000e glue_helper lrw gf128mul ptp ablk_helper cryptd pps_core thermal
> [17159.105040] CPU: 6 PID: 506 Comm: btrfs-cleaner Tainted: G        W  OE  3.19.6-amd64-i915-volpreempt-s20150421rc3 #2
> [17159.105042] Hardware name: LENOVO 20BECT0/20BECT0, BIOS GMET28WW (1.08 ) 09/18/2013
> [17159.105043]  0000000000000009 ffff88040669fa38 ffffffff81677ec0 00000000000080e2
> [17159.105050]  ffff88040669fa88 ffff88040669fa78 ffffffff810556a7 ffff88040669fa98
> [17159.105055]  ffffffff8121e95d 00000000fffffffe ffff88040748b800 ffff880103c7b598
> [17159.105059] Call Trace:
> [17159.105062]  [<ffffffff81677ec0>] dump_stack+0x45/0x57
> [17159.105065]  [<ffffffff810556a7>] warn_slowpath_common+0xa1/0xbb
> [17159.105068]  [<ffffffff8121e95d>] ? __btrfs_abort_transaction+0x52/0x113
> [17159.105070]  [<ffffffff81055707>] warn_slowpath_fmt+0x46/0x48
> [17159.105073]  [<ffffffff8121e95d>] __btrfs_abort_transaction+0x52/0x113
> [17159.105075]  [<ffffffff8122ced6>] __btrfs_free_extent+0x7fc/0x81a
> [17159.105078]  [<ffffffff812323ce>] __btrfs_run_delayed_refs+0xaee/0xc22
> [17159.105081]  [<ffffffff81230858>] ? btrfs_free_tree_block+0x18c/0x1c7
> [17159.105083]  [<ffffffff81234190>] btrfs_run_delayed_refs+0x6d/0x19e
> [17159.105086]  [<ffffffff81230c13>] ? walk_up_tree+0x72/0xf9
> [17159.105089]  [<ffffffff81241af6>] btrfs_should_end_transaction+0x52/0x5b
> [17159.105091]  [<ffffffff81232d1f>] btrfs_drop_snapshot+0x36f/0x696
> [17159.105095]  [<ffffffff81287651>] ? btrfs_kill_all_delayed_nodes+0x43/0xbd
> [17159.105098]  [<ffffffff8167ab1a>] ? __schedule+0x472/0x505
> [17159.105100]  [<ffffffff81241f3c>] btrfs_clean_one_deleted_snapshot+0xce/0xdb
> [17159.105103]  [<ffffffff8123b7c1>] cleaner_kthread+0x112/0x146
> [17159.105105]  [<ffffffff8123b6af>] ? atomic_add_unless.constprop.53+0x24/0x24
> [17159.105107]  [<ffffffff8106d296>] kthread+0xae/0xb6
> [17159.105110]  [<ffffffff8106d1e8>] ? __kthread_parkme+0x61/0x61
> [17159.105113]  [<ffffffff8167d818>] ret_from_fork+0x58/0x90
> [17159.105115]  [<ffffffff8106d1e8>] ? __kthread_parkme+0x61/0x61
> [17159.105116] ---[ end trace d372a2208cf37ced ]---
> [17159.105118] BTRFS: error (device dm-0) in __btrfs_free_extent:5987: errno=-2 No such entry
> [17159.105120] BTRFS info (device dm-0): forced readonly
> [17159.105123] BTRFS: error (device dm-0) in btrfs_run_delayed_refs:2792: errno=-2 No such entry
> 
> -- 
> "A mouse is a device used to point at the xterm you want to type in" - A.S.R.
> Microsoft is to operating systems ....
>                                       .... what McDonalds is to gourmet cooking
> Home page: http://marc.merlins.org/  

Re: 3.19.6: __btrfs_free_extent:5987: errno=-2 No such entry, did btrfs check --repair break it?

[Duncan]

Marc MERLIN posted on Tue, 05 May 2015 14:02:09 -0700 as excerpted:

> Please let me know what I can do for you if anything, before I wipe the
> filesystem and recreate it (I'm assuming that running btrfs check
> --repair a 2nd time won't help).

FWIW... I had a second /similar/ case here, but I didn't report it, as I 
thought it related to a relocated-sector count bump I saw on one of the 
ssds at about the same time.  My case may well have indeed been relocated-
sector-triggered, but reading your case, it's similar enough that I 
thought I'd mention it.

Like you I noticed a problem that didn't seem to have too much real-world 
effect, in my case, simply an apparently empty directory that would 
trigger a "directory not empty" error on attempt to delete.  Having read 
about others having such problems, I immediately used their solution, 
simply rename the directory out of the way, for the time being, after 
which it didn't seem to cause any harm, just annoyance as I still 
couldn't delete it.

But, with scrub fixing a couple corruptions and further scrubs finding 
nothing more, yet I still couldn't delete the dir, again like you, I 
decided to try a btrfs check, first read-only, which reported a few 
problems (which I didn't save), then with --repair, which /seemed/ to 
work just fine.

And finally, like you, a few hours later, actually in my case after a 
shutdown and reboot with successful error-free remount... suddenly that 
filesystem went read-only!

That's the end of the similarities, but as you can see, they're striking 
enough to think it might be the same issue (1) relatively minor initial 
problem, (2) btrfs check --repair apparently fixes it, (3) a few hours 
later the filesystem goes read-only, and immediate efforts to fix it fail.

What follows is how I recovered...

Well, I have my system on multiple independent btrfs for a reason, and 
that was /home, with / a totally independent btrfs mounted read-only by 
default, and mounted read-only at the time.

So I quit X and logged out of my normal user, logged in as root at the 
CLI, did a systemctl emergency to stop all normal services, and a
umount -a to unmount all filesystems that could unmount.

Then I went to work trying to fix the bad btrfs, still running from the 
unaffected read-only /, of course.

Long story short, nothing I tried (mounting with recovery, the new 
integrated btrfs rescue zero-log, btrfs check --repair...) seemed to fix 
it.  =:^(  So I got to use the new btrfs-progs v4.0 metadata-restoration 
option in btrfs restore, updating my previous btrfs restore experience 
from nearly a year ago. =:^)

The good news: btrfs restore worked without having to do the manual find-
root thing, and the metadata-restore options are very nice indeed!  I 
seem to have gotten my files back, and didn't have to resort to my backup
(which was out of date, but I'd have used it and been happy if I had to, 
I just didn't have to). =:^)

The bad news: restore's dry-run option doesn't work well with metadata-
restore, giving metadata unavailable errors on the dry-run that aren't 
there on an actual write-it-out restore.  Also, the old too many loops on 
the same dir warning and abort, no longer affects a write-out run, but 
still affects a dry-run.  So the dry-run provides /some/ idea what you'll 
get, but it's not really an accurate literal dry-run, going thru all the 
motions but effectively /dev/nulling the output.  =:^(

Also, I guess the symlink restore patch didn't make it in time for progs 
4.0.  Hopefully for 4.0.1 or at least 4.1...  So my symlinks weren't 
restored and I still had to recreate them manually.  But the experience 
was markedly better than last year, since I didn't have to: (1) rerun 
restore repeatedly until I no longer got the looping errors, (2) manually 
fix ownership/perms.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: 3.19.6: __btrfs_free_extent:5987: errno=-2 No such entry, did btrfs check --repair break it?

[Chris Murphy]

On Wed, May 6, 2015 at 5:04 AM, Duncan <1i5t5.duncan@cox.net> wrote:

> Long story short, nothing I tried (mounting with recovery, the new
> integrated btrfs rescue zero-log, btrfs check --repair...)

What is integrated btrfs rescue zero-log btrfs check? Where is it? I
don't see a check or repair option in btrfs rescue, I do see zero-log
(and chunk-recover and super-recover).

> So I got to use the new btrfs-progs v4.0 metadata-restoration
> option in btrfs restore,

In man btrfs-restore I only see a -m option, is that what you're referring to?

-- 
Chris Murphy

Re: 3.19.6: __btrfs_free_extent:5987: errno=-2 No such entry, did btrfs check --repair break it?

[Marc MERLIN]

On Wed, May 06, 2015 at 11:04:41AM +0000, Duncan wrote:
> Long story short, nothing I tried (mounting with recovery, the new 
> integrated btrfs rescue zero-log, btrfs check --repair...) seemed to fix 
> it.  =:^(  So I got to use the new btrfs-progs v4.0 metadata-restoration 
> option in btrfs restore, updating my previous btrfs restore experience 
> from nearly a year ago. =:^)

My filesystem is still mounting ok. I just did recovery,ro, but I'm pretty
sure it would work read/write for a while.

I'll probably destroy and re-create it tonight unless I hear from a dev who
wants me to try something first.

Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: 3.19.6: __btrfs_free_extent:5987: errno=-2 No such entry, did btrfs check --repair break it?

[Duncan]

Chris Murphy posted on Wed, 06 May 2015 11:25:51 -0600 as excerpted:

> On Wed, May 6, 2015 at 5:04 AM, Duncan <1i5t5.duncan@cox.net> wrote:
> 
>> Long story short, nothing I tried (mounting with recovery, the new
>> integrated btrfs rescue zero-log, btrfs check --repair...)
> 
> What is integrated btrfs rescue zero-log btrfs check? Where is it? I
> don't see a check or repair option in btrfs rescue, I do see zero-log
> (and chunk-recover and super-recover).

I think you misread me.  Either that or I'm misreading you and am totally 
confused about what your question is...

Btrfs-zero-log was previously its own command.  In 4.0 it has moved under 
the btrfs rescue command as btrfs rescue zero-log.  That's what I was 
referring to as the new integrated btrfs rescue zero-log.

Then there's a comma, and specifically stated btrfs check --repair 
(starting with btrfs, thus suggesting an entirely new command as that's 
what btrfs commands start with), a /separate/ command I ran.

I'm not sure where you got btrfs rescue zero log btrfs check, apparently 
interpreting that as a single command, despite the comma and repeated 
btrfs, and the hint in "nothing I tried", which suggests several things 
were tried...

So someone's obviously confused, but I'm not sure if it's you or me or 
both! =:^)

>> So I got to use the new btrfs-progs v4.0 metadata-restoration option in
>> btrfs restore,
> 
> In man btrfs-restore I only see a -m option, is that what you're
> referring to?

Yes.  That too is new in 4.0, the patch in fact being new enough I wasn't 
actually expecting to see it in a release until 4.1 or at least 4.0.1.

But it works well (except with dry-run, -D, which apparently is falling a 
bit behind as it still has the too many loops error from older code that 
no longer affects a normal write-out run, too). =:^)

Since I suggested restoring ownership/perms to the person who came up 
with the patch (his first one restored dates but not ownership/perms, as 
the date issue was his itch and he scratched it, so I suggested ownership/
perms too, my itch, which not being a coder I can't scratch directly), 
I'm happy as a kid at Christmas! =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman