[PATCH 00/18] Patch series related to my btrfs recovery

[PATCH 00/18] Patch series related to my btrfs recovery

[mwilck]

From: Martin Wilck <mwilck@arcor.de>

This patch series contains all changes I made to the btrfs tools
in the course of analyzing and repairing the corruption I described
in my other mail to linux-btrfs titled "A story of btrfs corruption
and recovery".

The bottom line of this patch set is: 1) have the tools continue with
error messages instead of aborting in certain error cases; and 
2) look for meta data outside the current trees. Both is useful if
the tree is internally corrupted in the way I described. I have also
added support for extracting inode meta data (times, permissions) 
in "btrfs restore"; this was also useful for my recovery case.

Please review and apply what you find useful.

Martin Wilck (18):
  btrfs-progs: btrfs-debug-tree: add option -f for "block only"
  btrfs-progs: btrfs-debug-tree: add option -B (backup root)
  btrfs-progs: btrfs-debug-tree: fix usage message
  btrfs-progs: btrfs-debug-tree: handle corruption more gracefully
  btrfs-progs: ctree.h: fix btrfs_inode_[amc]time
  btrfs restore: set uid/gid/mode/times
  btrfs restore: better output readability
  btrfs restore: track number of bytes restored
  btrfs restore: more graceful error handling in copy_file
  btrfs restore: hide "offset is X" messages
  btrfs restore: print progress marks for big files
  btrfs restore: check progress of file restoration
  btrfs restore: improve user-asking logic for files with many extents
  btrfs restore: report mismatch in file size
  btrfs-progs: NEW: btrfs-raw
  btrfs-progs: NEW: brtfs-search-metadata
  btrfs-progs: ctree.c: make bin_search non-static
  btrfs-progs: documentation for btrfs-raw and btrfs-search-metadata

 Documentation/Makefile                  |    2 +
 Documentation/btrfs-debug-tree.txt      |   10 ++
 Documentation/btrfs-raw.txt             |   54 ++++++++
 Documentation/btrfs-search-metadata.txt |   57 ++++++++
 Makefile                                |    4 +-
 btrfs-debug-tree.c                      |   78 +++++++++--
 btrfs-raw.c                             |  143 ++++++++++++++++++++
 btrfs-search-metadata.c                 |  224 +++++++++++++++++++++++++++++++
 cmds-restore.c                          |  205 +++++++++++++++++++++++-----
 ctree.c                                 |    4 +-
 ctree.h                                 |   15 ++-
 print-tree.c                            |   22 +++-
 12 files changed, 752 insertions(+), 66 deletions(-)
 create mode 100644 Documentation/btrfs-raw.txt
 create mode 100644 Documentation/btrfs-search-metadata.txt
 create mode 100644 btrfs-raw.c
 create mode 100644 btrfs-search-metadata.c

-- 
1.7.3.4

[PATCH 01/18] btrfs-progs: btrfs-debug-tree: add option -f for "block only"

[mwilck]

From: Martin Wilck <mwilck@arcor.de>

btrfs-debug-tree prints only the given block. It is sometimes
useful to be able to print the subtree under this block.
This patch enables this behavior with the option "-f".

Signed-off-by: Martin Wilck <mwilck@arcor.de>
---
 btrfs-debug-tree.c |   10 ++++++++--
 1 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/btrfs-debug-tree.c b/btrfs-debug-tree.c
index e46500d..e61c71c 100644
--- a/btrfs-debug-tree.c
+++ b/btrfs-debug-tree.c
@@ -41,6 +41,8 @@ static int print_usage(void)
 	fprintf(stderr, "\t-u : print info of uuid tree only\n");
 	fprintf(stderr, "\t-b block_num : print info of the specified block"
                     " only\n");
+	fprintf(stderr, "\t-f : (with -b) follow subtree of the specified"
+		" block\n");
 	fprintf(stderr,
 		"\t-t tree_id : print only the tree with the given id\n");
 	fprintf(stderr, "%s\n", BTRFS_BUILD_VERSION);
@@ -137,6 +139,7 @@ int main(int ac, char **av)
 	int roots_only = 0;
 	int root_backups = 0;
 	u64 block_only = 0;
+	int block_follow = 0;
 	struct btrfs_root *tree_root_scan;
 	u64 tree_id = 0;
 
@@ -144,7 +147,7 @@ int main(int ac, char **av)
 
 	while(1) {
 		int c;
-		c = getopt(ac, av, "deb:rRut:");
+		c = getopt(ac, av, "defb:rRut:");
 		if (c < 0)
 			break;
 		switch(c) {
@@ -167,6 +170,9 @@ int main(int ac, char **av)
 			case 'b':
 				block_only = arg_strtou64(optarg);
 				break;
+			case 'f':
+				block_follow = 1;
+				break;
 			case 't':
 				tree_id = arg_strtou64(optarg);
 				break;
@@ -211,7 +217,7 @@ int main(int ac, char **av)
 				(unsigned long long)block_only);
 			goto close_root;
 		}
-		btrfs_print_tree(root, leaf, 0);
+		btrfs_print_tree(root, leaf, block_follow);
 		goto close_root;
 	}
 
-- 
1.7.3.4

[PATCH 02/18] btrfs-progs: btrfs-debug-tree: add option -B (backup root)

[mwilck]

From: Martin Wilck <mwilck@arcor.de>

Option -B causes btrfs-debug-tree to dump the tree rooted at
the backup root number given instead of the real root.

Signed-off-by: Martin Wilck <mwilck@arcor.de>
---
 btrfs-debug-tree.c |   39 ++++++++++++++++++++++++++++++++++++++-
 1 files changed, 38 insertions(+), 1 deletions(-)

diff --git a/btrfs-debug-tree.c b/btrfs-debug-tree.c
index e61c71c..7cdc368 100644
--- a/btrfs-debug-tree.c
+++ b/btrfs-debug-tree.c
@@ -45,6 +45,8 @@ static int print_usage(void)
 		" block\n");
 	fprintf(stderr,
 		"\t-t tree_id : print only the tree with the given id\n");
+	fprintf(stderr,
+		"\t-B nr: use root backup <nr> instead of real root\n");
 	fprintf(stderr, "%s\n", BTRFS_BUILD_VERSION);
 	exit(1);
 }
@@ -140,6 +142,7 @@ int main(int ac, char **av)
 	int root_backups = 0;
 	u64 block_only = 0;
 	int block_follow = 0;
+	int use_backup = -1;
 	struct btrfs_root *tree_root_scan;
 	u64 tree_id = 0;
 
@@ -147,7 +150,7 @@ int main(int ac, char **av)
 
 	while(1) {
 		int c;
-		c = getopt(ac, av, "defb:rRut:");
+		c = getopt(ac, av, "defb:rRut:B:");
 		if (c < 0)
 			break;
 		switch(c) {
@@ -176,6 +179,9 @@ int main(int ac, char **av)
 			case 't':
 				tree_id = arg_strtou64(optarg);
 				break;
+			case 'B':
+				use_backup = arg_strtou64(optarg);
+				break;
 			default:
 				print_usage();
 		}
@@ -221,6 +227,37 @@ int main(int ac, char **av)
 		goto close_root;
 	}
 
+	if (use_backup >= BTRFS_NUM_BACKUP_ROOTS) {
+		fprintf(stderr, "Invalid backup root number %d\n",
+			use_backup);
+		exit(1);
+	} else if (use_backup >= 0) {
+		u64 bytenr, generation;
+		u32 blocksize;
+		struct btrfs_super_block *sb = info->super_copy;
+		struct btrfs_root_backup *backup = sb->super_roots + use_backup;
+		struct extent_buffer *eb;
+		bytenr = btrfs_backup_tree_root(backup);
+		generation = btrfs_backup_tree_root_gen(backup);
+		blocksize = btrfs_level_size(info->tree_root,
+					     btrfs_super_root_level(sb));
+		eb = info->tree_root->node;
+		info->tree_root->node = read_tree_block(root, bytenr,
+							blocksize, generation);
+		free_extent_buffer(eb);
+		bytenr = btrfs_backup_chunk_root(backup);
+		generation = btrfs_backup_chunk_root_gen(backup);
+		eb =  info->chunk_root->node;
+		info->chunk_root->node = read_tree_block(root, bytenr,
+							blocksize, generation);
+		free_extent_buffer(eb);
+		if (!extent_buffer_uptodate(info->tree_root->node) ||
+		    !extent_buffer_uptodate(info->tree_root->node)) {
+			fprintf(stderr, "Couldn't backup root\n");
+			return 1;
+		}
+	}
+
 	if (!(extent_only || uuid_tree_only || tree_id)) {
 		if (roots_only) {
 			printf("root tree: %llu level %d\n",
-- 
1.7.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-

[PATCH 03/18] btrfs-progs: btrfs-debug-tree: fix usage message

[mwilck]

From: Martin Wilck <mwilck@arcor.de>

Adapt usage message to the additional options introduced.

Signed-off-by: Martin Wilck <mwilck@arcor.de>
---
 btrfs-debug-tree.c |   13 +++++++------
 1 files changed, 7 insertions(+), 6 deletions(-)

diff --git a/btrfs-debug-tree.c b/btrfs-debug-tree.c
index 7cdc368..4c1e835 100644
--- a/btrfs-debug-tree.c
+++ b/btrfs-debug-tree.c
@@ -31,22 +31,23 @@
 
 static int print_usage(void)
 {
-	fprintf(stderr, "usage: btrfs-debug-tree [-e] [-d] [-r] [-R] [-u]\n");
-	fprintf(stderr, "                        [-b block_num ] device\n");
+	fprintf(stderr, "usage: btrfs-debug-tree [-e] [-d] [-r] [-R] [-u] [-B]\n");
+	fprintf(stderr, "                        [-t tree_id] device\n");
+	fprintf(stderr, "       btrfs-debug-tree [-b block_num [-f]] device\n");
 	fprintf(stderr, "\t-e : print detailed extents info\n");
 	fprintf(stderr, "\t-d : print info of btrfs device and root tree dirs"
                     " only\n");
 	fprintf(stderr, "\t-r : print info of roots only\n");
 	fprintf(stderr, "\t-R : print info of roots and root backups\n");
 	fprintf(stderr, "\t-u : print info of uuid tree only\n");
-	fprintf(stderr, "\t-b block_num : print info of the specified block"
-                    " only\n");
-	fprintf(stderr, "\t-f : (with -b) follow subtree of the specified"
-		" block\n");
 	fprintf(stderr,
 		"\t-t tree_id : print only the tree with the given id\n");
 	fprintf(stderr,
 		"\t-B nr: use root backup <nr> instead of real root\n");
+	fprintf(stderr, "\t-b block_num : print info of the specified block"
+                    " only\n");
+	fprintf(stderr, "\t-f : (with -b) follow subtree of the specified"
+		" block\n");
 	fprintf(stderr, "%s\n", BTRFS_BUILD_VERSION);
 	exit(1);
 }
-- 
1.7.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-

[PATCH 04/18] btrfs-progs: btrfs-debug-tree: handle corruption more gracefully

[mwilck]

From: Martin Wilck <mwilck@arcor.de>

This patch fixes the same thing in two different places. First,
the first of the two BUG() tests is just a special case of the
second one and can therefore be omitted. Second, instead of bailing
out with BUG(), just print a reasonable error message and check the
next child.

Signed-off-by: Martin Wilck <mwilck@arcor.de>
---
 btrfs-debug-tree.c |   22 ++++++++++++++++------
 print-tree.c       |   22 ++++++++++++++++------
 2 files changed, 32 insertions(+), 12 deletions(-)

diff --git a/btrfs-debug-tree.c b/btrfs-debug-tree.c
index 4c1e835..d7c1155 100644
--- a/btrfs-debug-tree.c
+++ b/btrfs-debug-tree.c
@@ -73,13 +73,23 @@ static void print_extents(struct btrfs_root *root, struct extent_buffer *eb)
 					     btrfs_node_blockptr(eb, i),
 					     size,
 					     btrfs_node_ptr_generation(eb, i));
-		if (btrfs_is_leaf(next) &&
-		    btrfs_header_level(eb) != 1)
-			BUG();
 		if (btrfs_header_level(next) !=
-			btrfs_header_level(eb) - 1)
-			BUG();
-		print_extents(root, next);
+		    btrfs_header_level(eb) - 1) {
+			fprintf(stderr, "EXTENT TREE CORRUPTION detected at %llu, "
+				"slot %d pointing at %llu.\n"
+				"\tExpected child level: %d, found %d\n"
+				"\tExpected tree/transid: %llu/%llu,"
+				" found %llu/%llu\n",
+				eb->start, i, next->start,
+				btrfs_header_level(eb) - 1,
+				btrfs_header_level(next),
+				(unsigned long long)btrfs_header_owner(eb),
+				(unsigned long long)btrfs_header_generation(eb),
+				(unsigned long long)btrfs_header_owner(next),
+				(unsigned long long)
+				btrfs_header_generation(next));
+		} else
+			print_extents(root, next);
 		free_extent_buffer(next);
 	}
 }
diff --git a/print-tree.c b/print-tree.c
index 70a7acc..6769e20 100644
--- a/print-tree.c
+++ b/print-tree.c
@@ -1066,13 +1066,23 @@ void btrfs_print_tree(struct btrfs_root *root, struct extent_buffer *eb, int fol
 				(unsigned long long)btrfs_header_owner(eb));
 			continue;
 		}
-		if (btrfs_is_leaf(next) &&
-		    btrfs_header_level(eb) != 1)
-			BUG();
 		if (btrfs_header_level(next) !=
-			btrfs_header_level(eb) - 1)
-			BUG();
-		btrfs_print_tree(root, next, 1);
+		    btrfs_header_level(eb) - 1) {
+			fprintf(stderr, "TREE CORRUPTION detected at %llu, "
+				"slot %d pointing at %llu.\n"
+				"\tExpected child level: %d, found %d\n"
+				"\tExpected tree/transid: %llu/%llu,"
+				" found %llu/%llu\n",
+				eb->start, i, next->start,
+				btrfs_header_level(eb) - 1,
+				btrfs_header_level(next),
+				(unsigned long long)btrfs_header_owner(eb),
+				(unsigned long long)btrfs_header_generation(eb),
+				(unsigned long long)btrfs_header_owner(next),
+				(unsigned long long)
+				btrfs_header_generation(next));
+		} else
+			btrfs_print_tree(root, next, 1);
 		free_extent_buffer(next);
 	}
 }
-- 
1.7.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-

[PATCH 05/18] btrfs-progs: ctree.h: fix btrfs_inode_[amc]time

[mwilck]

From: Martin Wilck <mwilck@arcor.de>

make btrfs_inode_[amc]time work like the other btrfs_inode_xxx
functions. The current definition appears broken to me; it never
returns valid pointer unless an extent buffer address is added.

Signed-off-by: Martin Wilck <mwilck@arcor.de>
---
 ctree.h |   15 +++++++++------
 1 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/ctree.h b/ctree.h
index 89036de..1d5a5fc 100644
--- a/ctree.h
+++ b/ctree.h
@@ -1414,27 +1414,30 @@ BTRFS_SETGET_STACK_FUNCS(stack_inode_flags,
 			 struct btrfs_inode_item, flags, 64);
 
 static inline struct btrfs_timespec *
-btrfs_inode_atime(struct btrfs_inode_item *inode_item)
+btrfs_inode_atime(struct extent_buffer *eb,
+		  struct btrfs_inode_item *inode_item)
 {
 	unsigned long ptr = (unsigned long)inode_item;
 	ptr += offsetof(struct btrfs_inode_item, atime);
-	return (struct btrfs_timespec *)ptr;
+	return (struct btrfs_timespec *)(ptr + eb->data);
 }
 
 static inline struct btrfs_timespec *
-btrfs_inode_mtime(struct btrfs_inode_item *inode_item)
+btrfs_inode_mtime(struct extent_buffer *eb,
+		  struct btrfs_inode_item *inode_item)
 {
 	unsigned long ptr = (unsigned long)inode_item;
 	ptr += offsetof(struct btrfs_inode_item, mtime);
-	return (struct btrfs_timespec *)ptr;
+	return (struct btrfs_timespec *)(ptr + eb->data);
 }
 
 static inline struct btrfs_timespec *
-btrfs_inode_ctime(struct btrfs_inode_item *inode_item)
+btrfs_inode_ctime(struct extent_buffer *eb,
+		  struct btrfs_inode_item *inode_item)
 {
 	unsigned long ptr = (unsigned long)inode_item;
 	ptr += offsetof(struct btrfs_inode_item, ctime);
-	return (struct btrfs_timespec *)ptr;
+	return (struct btrfs_timespec *)(ptr + eb->data);
 }
 
 static inline struct btrfs_timespec *
-- 
1.7.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-

[PATCH 06/18] btrfs restore: set uid/gid/mode/times

[mwilck]

From: Martin Wilck <mwilck@arcor.de>

current btrfs restore will discard file attributes. This patch
sets them regular files and directories, as found in the
meta data.

Signed-off-by: Martin Wilck <mwilck@arcor.de>
---
 cmds-restore.c |  116 ++++++++++++++++++++++++++++++++++++++++++++++++-------
 1 files changed, 101 insertions(+), 15 deletions(-)

diff --git a/cmds-restore.c b/cmds-restore.c
index 2f9b72d..5aa2167 100644
--- a/cmds-restore.c
+++ b/cmds-restore.c
@@ -33,6 +33,7 @@
 #include <zlib.h>
 #include <regex.h>
 #include <getopt.h>
+#include <sys/time.h>
 #include <sys/types.h>
 #include <sys/xattr.h>
 
@@ -549,6 +550,95 @@ out:
 	return ret;
 }
 
+#define _INVALID_SIZE ((off_t)~0ULL)
+static int stat_from_inode(struct stat *st, struct btrfs_root *root,
+			   struct btrfs_key *key)
+{
+	static struct btrfs_path *path;
+	struct btrfs_inode_item *inode_item;
+	struct btrfs_timespec *ts;
+	struct extent_buffer *eb;
+
+	if (!path)
+		path = btrfs_alloc_path();
+
+	memset(st, 0, sizeof(*st));
+	st->st_size = _INVALID_SIZE;
+
+	if (!path) {
+		fprintf(stderr, "Ran out of memory\n");
+		return -ENOMEM;
+	}
+
+	if (btrfs_lookup_inode(NULL, root, path, key, 0)) {
+		btrfs_release_path(path);
+		return -ENOENT;
+	}
+
+	inode_item = btrfs_item_ptr(path->nodes[0], path->slots[0],
+				    struct btrfs_inode_item);
+	eb = path->nodes[0];
+
+	st->st_size = btrfs_inode_size(eb, inode_item);
+	st->st_uid = btrfs_inode_uid(eb, inode_item);
+	st->st_gid = btrfs_inode_gid(eb, inode_item);
+	st->st_mode = btrfs_inode_mode(eb, inode_item);
+
+	ts = btrfs_inode_atime(eb, inode_item);
+	st->st_atim.tv_sec = ts->sec;
+	st->st_atim.tv_nsec = ts->nsec;
+
+	ts = btrfs_inode_mtime(eb, inode_item);
+	st->st_mtim.tv_sec = ts->sec;
+	st->st_mtim.tv_nsec = ts->nsec;
+
+	ts = btrfs_inode_ctime(eb, inode_item);
+	st->st_ctim.tv_sec = ts->sec;
+	st->st_ctim.tv_nsec = ts->nsec;
+
+	btrfs_release_path(path);
+	return 0;
+}
+
+static void set_fd_attrs(int fd, const struct stat *st, const char *file)
+{
+	struct timeval tv[2];
+	if (st->st_size == _INVALID_SIZE)
+		return;
+
+	tv[0].tv_sec = st->st_atim.tv_sec;
+	tv[0].tv_usec = st->st_atim.tv_nsec/1000;
+	tv[1].tv_sec = st->st_mtim.tv_sec;
+	tv[1].tv_usec = st->st_mtim.tv_nsec/1000;
+	if (S_ISREG(st->st_mode) && ftruncate(fd, st->st_size) == -1)
+		fprintf(stderr, "failed to set file size on %s\n",
+			file);
+	if (fchown(fd, st->st_uid, st->st_gid) == -1)
+		fprintf(stderr, "failed to set uid/gid on %s\n",
+			file);
+	if (fchmod(fd, st->st_mode) == -1)
+		fprintf(stderr, "failed to set permissions on %s\n",
+			file);
+	if (futimes(fd, tv) == -1)
+		fprintf(stderr, "failed to set file times on %s\n",
+			file);
+}
+
+static int set_file_attrs(const char *output_rootdir, const char *file,
+			  const struct stat *st)
+{
+	int fd;
+	static char path[4096];
+	snprintf(path, sizeof(path), "%s%s", output_rootdir, file);
+	fd = open(path, O_RDONLY|O_NOATIME);
+	if (fd == -1) {
+		fprintf(stderr, "failed to open %s\n", path_name);
+		return -1;
+	}
+	set_fd_attrs(fd, st, path);
+	close(fd);
+	return 0;
+}
 
 static int copy_file(struct btrfs_root *root, int fd, struct btrfs_key *key,
 		     const char *file)
@@ -556,13 +646,12 @@ static int copy_file(struct btrfs_root *root, int fd, struct btrfs_key *key,
 	struct extent_buffer *leaf;
 	struct btrfs_path *path;
 	struct btrfs_file_extent_item *fi;
-	struct btrfs_inode_item *inode_item;
 	struct btrfs_key found_key;
 	int ret;
 	int extent_type;
 	int compression;
 	int loops = 0;
-	u64 found_size = 0;
+	struct stat st;
 
 	path = btrfs_alloc_path();
 	if (!path) {
@@ -570,13 +659,7 @@ static int copy_file(struct btrfs_root *root, int fd, struct btrfs_key *key,
 		return -ENOMEM;
 	}
 
-	ret = btrfs_lookup_inode(NULL, root, path, key, 0);
-	if (ret == 0) {
-		inode_item = btrfs_item_ptr(path->nodes[0], path->slots[0],
-				    struct btrfs_inode_item);
-		found_size = btrfs_inode_size(path->nodes[0], inode_item);
-	}
-	btrfs_release_path(path);
+	stat_from_inode(&st, root, key);
 
 	key->offset = 0;
 	key->type = BTRFS_EXTENT_DATA_KEY;
@@ -672,16 +755,14 @@ next:
 
 	btrfs_free_path(path);
 set_size:
-	if (found_size) {
-		ret = ftruncate(fd, (loff_t)found_size);
-		if (ret)
-			return ret;
-	}
 	if (get_xattrs) {
 		ret = set_file_xattrs(root, key->objectid, fd, file);
 		if (ret)
-			return ret;
+			fprintf(stderr, "failed to set xattrs on %s\n",
+				file);
 	}
+
+	set_fd_attrs(fd, &st, file);
 	return 0;
 }
 
@@ -693,6 +774,7 @@ static int search_dir(struct btrfs_root *root, struct btrfs_key *key,
 	struct extent_buffer *leaf;
 	struct btrfs_dir_item *dir_item;
 	struct btrfs_key found_key, location;
+	struct stat dirst;
 	char filename[BTRFS_NAME_LEN + 1];
 	unsigned long name_ptr;
 	int name_len;
@@ -707,6 +789,8 @@ static int search_dir(struct btrfs_root *root, struct btrfs_key *key,
 		return -ENOMEM;
 	}
 
+	stat_from_inode(&dirst, root, key);
+
 	key->offset = 0;
 	key->type = BTRFS_DIR_INDEX_KEY;
 
@@ -931,6 +1015,8 @@ next:
 		path->slots[0]++;
 	}
 
+	set_file_attrs(output_rootdir, in_dir, &dirst);
+
 	if (verbose)
 		printf("Done searching %s\n", in_dir);
 	btrfs_free_path(path);
-- 
1.7.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo

[PATCH 07/18] btrfs restore: better output readability

[mwilck]

From: Martin Wilck <mwilck@arcor.de>

Don't print whole path for files, which will mangle output
for long path names. Rather distinguish between directories
and files.

Signed-off-by: Martin Wilck <mwilck@arcor.de>
---
 cmds-restore.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/cmds-restore.c b/cmds-restore.c
index 5aa2167..f9dab7e 100644
--- a/cmds-restore.c
+++ b/cmds-restore.c
@@ -908,7 +908,7 @@ static int search_dir(struct btrfs_root *root, struct btrfs_key *key,
 				ret = 0;
 			}
 			if (verbose)
-				printf("Restoring %s\n", path_name);
+				printf("Restoring %s\n", filename);
 			if (dry_run)
 				goto next;
 			fd = open(path_name, O_CREAT|O_WRONLY, 0644);
@@ -982,7 +982,7 @@ static int search_dir(struct btrfs_root *root, struct btrfs_key *key,
 			}
 
 			if (verbose)
-				printf("Restoring %s\n", path_name);
+				printf("Searching directory %s\n", path_name);
 
 			errno = 0;
 			if (dry_run)
-- 
1.7.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-

[PATCH 08/18] btrfs restore: track number of bytes restored

[mwilck]

From: Martin Wilck <mwilck@arcor.de>

Track the number of bytes read from extents and restored.
This is useful for detecting errors and corruptions.

Signed-off-by: Martin Wilck <mwilck@arcor.de>
---
 cmds-restore.c |   16 ++++++++++++----
 1 files changed, 12 insertions(+), 4 deletions(-)

diff --git a/cmds-restore.c b/cmds-restore.c
index f9dab7e..8ecd896 100644
--- a/cmds-restore.c
+++ b/cmds-restore.c
@@ -222,7 +222,8 @@ again:
 	return 0;
 }
 
-static int copy_one_inline(int fd, struct btrfs_path *path, u64 pos)
+static int copy_one_inline(int fd, struct btrfs_path *path, u64 pos,
+			   u64 *bytes_written)
 {
 	struct extent_buffer *leaf = path->nodes[0];
 	struct btrfs_file_extent_item *fi;
@@ -246,6 +247,7 @@ static int copy_one_inline(int fd, struct btrfs_path *path, u64 pos)
 	compress = btrfs_file_extent_compression(leaf, fi);
 	if (compress == BTRFS_COMPRESS_NONE) {
 		done = pwrite(fd, buf, len, pos);
+		*bytes_written += done;
 		if (done < len) {
 			fprintf(stderr, "Short inline write, wanted %d, did "
 				"%zd: %d\n", len, done, errno);
@@ -269,6 +271,7 @@ static int copy_one_inline(int fd, struct btrfs_path *path, u64 pos)
 
 	done = pwrite(fd, outbuf, ram_size, pos);
 	free(outbuf);
+	*bytes_written += done;
 	if (done < ram_size) {
 		fprintf(stderr, "Short compressed inline write, wanted %Lu, "
 			"did %zd: %d\n", ram_size, done, errno);
@@ -280,7 +283,8 @@ static int copy_one_inline(int fd, struct btrfs_path *path, u64 pos)
 
 static int copy_one_extent(struct btrfs_root *root, int fd,
 			   struct extent_buffer *leaf,
-			   struct btrfs_file_extent_item *fi, u64 pos)
+			   struct btrfs_file_extent_item *fi, u64 pos,
+			   u64 *bytes_written)
 {
 	struct btrfs_multi_bio *multi = NULL;
 	struct btrfs_device *device;
@@ -410,6 +414,7 @@ again:
 		total += done;
 	}
 out:
+	*bytes_written += total;
 	free(inbuf);
 	free(outbuf);
 	return ret;
@@ -651,6 +656,8 @@ static int copy_file(struct btrfs_root *root, int fd, struct btrfs_key *key,
 	int extent_type;
 	int compression;
 	int loops = 0;
+	u64 bytes_written, next_pos = 0ULL;
+	u64 total_written = 0ULL;
 	struct stat st;
 
 	path = btrfs_alloc_path();
@@ -734,14 +741,15 @@ static int copy_file(struct btrfs_root *root, int fd, struct btrfs_key *key,
 		if (extent_type == BTRFS_FILE_EXTENT_PREALLOC)
 			goto next;
 		if (extent_type == BTRFS_FILE_EXTENT_INLINE) {
-			ret = copy_one_inline(fd, path, found_key.offset);
+			ret = copy_one_inline(fd, path, found_key.offset,
+					      &bytes_written);
 			if (ret) {
 				btrfs_free_path(path);
 				return -1;
 			}
 		} else if (extent_type == BTRFS_FILE_EXTENT_REG) {
 			ret = copy_one_extent(root, fd, leaf, fi,
-					      found_key.offset);
+					      found_key.offset, &bytes_written);
 			if (ret) {
 				btrfs_free_path(path);
 				return ret;
-- 
1.7.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-

[PATCH 09/18] btrfs restore: more graceful error handling in copy_file

[mwilck]

From: Martin Wilck <mwilck@arcor.de>

Setting size and attributes of a file makes sense even if some
errors have occured during revovery.

Also, do something useful with the number of bytes written.

Signed-off-by: Martin Wilck <mwilck@arcor.de>
---
 cmds-restore.c |   27 ++++++++++++++-------------
 1 files changed, 14 insertions(+), 13 deletions(-)

diff --git a/cmds-restore.c b/cmds-restore.c
index 8ecd896..10bb8be 100644
--- a/cmds-restore.c
+++ b/cmds-restore.c
@@ -715,7 +715,7 @@ static int copy_file(struct btrfs_root *root, int fd, struct btrfs_key *key,
 					return ret;
 				} else if (ret) {
 					/* No more leaves to search */
-					btrfs_free_path(path);
+					ret = 0;
 					goto set_size;
 				}
 				leaf = path->nodes[0];
@@ -734,35 +734,36 @@ static int copy_file(struct btrfs_root *root, int fd, struct btrfs_key *key,
 		if (compression >= BTRFS_COMPRESS_LAST) {
 			fprintf(stderr, "Don't support compression yet %d\n",
 				compression);
-			btrfs_free_path(path);
-			return -1;
+			ret = -1;
+			goto set_size;
 		}
 
+		bytes_written = 0ULL;
 		if (extent_type == BTRFS_FILE_EXTENT_PREALLOC)
 			goto next;
 		if (extent_type == BTRFS_FILE_EXTENT_INLINE) {
 			ret = copy_one_inline(fd, path, found_key.offset,
 					      &bytes_written);
-			if (ret) {
-				btrfs_free_path(path);
-				return -1;
-			}
 		} else if (extent_type == BTRFS_FILE_EXTENT_REG) {
 			ret = copy_one_extent(root, fd, leaf, fi,
 					      found_key.offset, &bytes_written);
-			if (ret) {
-				btrfs_free_path(path);
-				return ret;
-			}
 		} else {
 			printf("Weird extent type %d\n", extent_type);
 		}
+		total_written += bytes_written;
+		next_pos = found_key.offset + bytes_written;
+		if (ret) {
+			fprintf(stderr, "ERROR after writing %llu bytes\n",
+				total_written);
+			ret = -1;
+			goto set_size;
+		}
 next:
 		path->slots[0]++;
 	}
 
-	btrfs_free_path(path);
 set_size:
+	btrfs_free_path(path);
 	if (get_xattrs) {
 		ret = set_file_xattrs(root, key->objectid, fd, file);
 		if (ret)
@@ -771,7 +772,7 @@ set_size:
 	}
 
 	set_fd_attrs(fd, &st, file);
-	return 0;
+	return ret;
 }
 
 static int search_dir(struct btrfs_root *root, struct btrfs_key *key,
-- 
1.7.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-

[PATCH 10/18] btrfs restore: hide "offset is X" messages

[mwilck]

From: Martin Wilck <mwilck@arcor.de>

Almost everyone who cares about her data will run btrfs restore
with -v. The "offset is" messages displayed will irritate users
because they reveal only btrfs internals. Users will think that
"offset" refers to a file offset and suspect severe corruption.

Limit these messages to verbose > 1.

Signed-off-by: Martin Wilck <mwilck@arcor.de>
---
 cmds-restore.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/cmds-restore.c b/cmds-restore.c
index 10bb8be..f1c63ed 100644
--- a/cmds-restore.c
+++ b/cmds-restore.c
@@ -315,7 +315,7 @@ static int copy_one_extent(struct btrfs_root *root, int fd,
 	if (compress == BTRFS_COMPRESS_NONE)
 		bytenr += offset;
 
-	if (verbose && offset)
+	if (verbose > 1 && offset)
 		printf("offset is %Lu\n", offset);
 	/* we found a hole */
 	if (disk_size == 0)
-- 
1.7.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-

[PATCH 11/18] btrfs restore: print progress marks for big files

[mwilck]

From: Martin Wilck <mwilck@arcor.de>

print a '+' for every 64k restored. This gives people more confidence
in long-running restore processes.

Signed-off-by: Martin Wilck <mwilck@arcor.de>
---
 cmds-restore.c |    8 ++++++++
 1 files changed, 8 insertions(+), 0 deletions(-)

diff --git a/cmds-restore.c b/cmds-restore.c
index f1c63ed..004c82e 100644
--- a/cmds-restore.c
+++ b/cmds-restore.c
@@ -658,6 +658,8 @@ static int copy_file(struct btrfs_root *root, int fd, struct btrfs_key *key,
 	int loops = 0;
 	u64 bytes_written, next_pos = 0ULL;
 	u64 total_written = 0ULL;
+#define MAYBE_NL (verbose && (next_pos >> display_shift) ? "\n" : "")
+	const u64 display_shift = 16;
 	struct stat st;
 
 	path = btrfs_alloc_path();
@@ -751,6 +753,10 @@ static int copy_file(struct btrfs_root *root, int fd, struct btrfs_key *key,
 			printf("Weird extent type %d\n", extent_type);
 		}
 		total_written += bytes_written;
+		if (verbose && 
+		    ((next_pos +  bytes_written) >> display_shift) > 
+		    (next_pos >> display_shift))
+			fprintf(stderr, "+");
 		next_pos = found_key.offset + bytes_written;
 		if (ret) {
 			fprintf(stderr, "ERROR after writing %llu bytes\n",
@@ -764,6 +770,8 @@ next:
 
 set_size:
 	btrfs_free_path(path);
+
+	printf(MAYBE_NL);
 	if (get_xattrs) {
 		ret = set_file_xattrs(root, key->objectid, fd, file);
 		if (ret)
-- 
1.7.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-

[PATCH 12/18] btrfs restore: check progress of file restoration

[mwilck]

From: Martin Wilck <mwilck@arcor.de>

extents should be ordered by file offset. Expect no overlaps,
and report holes.

Signed-off-by: Martin Wilck <mwilck@arcor.de>
---
 cmds-restore.c |    8 ++++++++
 1 files changed, 8 insertions(+), 0 deletions(-)

diff --git a/cmds-restore.c b/cmds-restore.c
index 004c82e..80081b8 100644
--- a/cmds-restore.c
+++ b/cmds-restore.c
@@ -739,6 +739,14 @@ static int copy_file(struct btrfs_root *root, int fd, struct btrfs_key *key,
 			ret = -1;
 			goto set_size;
 		}
+		if (found_key.offset < next_pos) {
+			fprintf(stderr, "extent overlap, %llu < %llu\n",
+				found_key.offset, next_pos);
+			ret = -1;
+			goto set_size;
+		} else if (found_key.offset > next_pos)
+			fprintf(stderr, "hole at %llu (%llu bytes)\n",
+				next_pos, found_key.offset - next_pos);
 
 		bytes_written = 0ULL;
 		if (extent_type == BTRFS_FILE_EXTENT_PREALLOC)
-- 
1.7.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-

[PATCH 13/18] btrfs restore: improve user-asking logic for files with many extents

[mwilck]

From: Martin Wilck <mwilck@arcor.de>

The logic to ask after 1024 extents is broken. It unnecessarily
confuses users if big files are being restored, making them think
somthing is going wrong.

Change it to two cases: 1) no or little progress restoring,
2) writing beyond the file size.

Signed-off-by: Martin Wilck <mwilck@arcor.de>
---
 cmds-restore.c |   18 +++++++++++++++---
 1 files changed, 15 insertions(+), 3 deletions(-)

diff --git a/cmds-restore.c b/cmds-restore.c
index 80081b8..8ae3337 100644
--- a/cmds-restore.c
+++ b/cmds-restore.c
@@ -661,7 +661,7 @@ static int copy_file(struct btrfs_root *root, int fd, struct btrfs_key *key,
 #define MAYBE_NL (verbose && (next_pos >> display_shift) ? "\n" : "")
 	const u64 display_shift = 16;
 	struct stat st;
-
+	int dont_ask = 0;
 	path = btrfs_alloc_path();
 	if (!path) {
 		fprintf(stderr, "Ran out of memory\n");
@@ -697,9 +697,21 @@ static int copy_file(struct btrfs_root *root, int fd, struct btrfs_key *key,
 	}
 
 	while (1) {
-		if (loops >= 0 && loops++ >= 1024) {
+		int problem = 0;
+		if (st.st_size == _INVALID_SIZE && next_pos > st.st_size) {
+			fprintf(stderr, "%swriting at offset %llu beyond size "
+				"of file (%llu)\n",
+				MAYBE_NL, next_pos, st.st_size);
+			problem = 1;
+		}
+		if ((++loops % 1024) == 0 && (next_pos / loops < 4096)) {
+			fprintf(stderr, "%smany loops (%d) and little progress "
+				"(%llu bytes)\n", 
+				MAYBE_NL, loops, next_pos);
+			problem = 1;
+		}
+		if (problem && !dont_ask && loops++) {
 			enum loop_response resp;
-
 			resp = ask_to_continue(file);
 			if (resp == LOOP_STOP)
 				break;
-- 
1.7.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-

[PATCH 14/18] btrfs restore: report mismatch in file size

[mwilck]

From: Martin Wilck <mwilck@arcor.de>

A mismatch between the file size stored in the inode and the
number of bytes restored may indicate a problem.
restore reads data in 4k chunks, so it's normal that files are
truncated. Only emit the warning in unusual cases.

Signed-off-by: Martin Wilck <mwilck@arcor.de>
---
 cmds-restore.c |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/cmds-restore.c b/cmds-restore.c
index 8ae3337..3c4dc7a 100644
--- a/cmds-restore.c
+++ b/cmds-restore.c
@@ -799,6 +799,12 @@ set_size:
 				file);
 	}
 
+	if (st.st_size != _INVALID_SIZE && (st.st_size > next_pos || 
+					    (st.st_size < next_pos &&
+					     (st.st_size >> 12) !=
+					     (next_pos >> 12) - 1)))
+		fprintf(stderr, "size mismatch: extpected %llu, got %llu (written %llu)\n",
+			st.st_size, next_pos, total_written);	
 	set_fd_attrs(fd, &st, file);
 	return ret;
 }
-- 
1.7.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-

[PATCH 15/18] btrfs-progs: NEW: btrfs-raw

[mwilck]

From: Martin Wilck <mwilck@arcor.de>

This program can be used to dump a meta data block, fix it e.g.
using a hex editor, and write it back to disk, adapting the check
sum.

Signed-off-by: Martin Wilck <mwilck@arcor.de>
---
 Makefile    |    2 +-
 btrfs-raw.c |  143 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 144 insertions(+), 1 deletions(-)
 create mode 100644 btrfs-raw.c

diff --git a/Makefile b/Makefile
index 4cae30c..fe65867 100644
--- a/Makefile
+++ b/Makefile
@@ -46,7 +46,7 @@ endif
 
 MAKEOPTS = --no-print-directory Q=$(Q)
 
-progs = mkfs.btrfs btrfs-debug-tree btrfsck \
+progs = mkfs.btrfs btrfs-debug-tree btrfs-raw btrfsck \
 	btrfs btrfs-map-logical btrfs-image btrfs-zero-log btrfs-convert \
 	btrfs-find-root btrfstune btrfs-show-super
 
diff --git a/btrfs-raw.c b/btrfs-raw.c
new file mode 100644
index 0000000..1dfeed9
--- /dev/null
+++ b/btrfs-raw.c
@@ -0,0 +1,143 @@
+/*
+ * Copyright (C) 2007 Oracle.  All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include "kerncompat.h"
+#include "radix-tree.h"
+#include "ctree.h"
+#include "utils.h"
+#include "disk-io.h"
+
+static int print_usage(void)
+{
+	fprintf(stderr, "usage: btrfs-raw [ -r block|-w block] device\n");
+	exit(1);
+}
+
+static int read_block(struct btrfs_root *root, u64 block_nr,
+		      struct extent_buffer **eb)
+{
+	struct extent_buffer *leaf;
+	leaf = read_tree_block(root,
+			       block_nr,
+			       root->leafsize, 0);
+	
+	if (leaf && btrfs_header_level(leaf) != 0) {
+		free_extent_buffer(leaf);
+		leaf = NULL;
+	}
+	
+	if (!leaf) {
+		leaf = read_tree_block(root,
+				       block_nr,
+				       root->nodesize, 0);
+	}
+	if (!leaf) {
+		fprintf(stderr, "failed to read %llu\n",
+			(unsigned long long)block_nr);
+		return -1;
+	}
+
+	*eb = leaf;
+	return btrfs_is_leaf(leaf) ? root->leafsize : root->nodesize;
+}
+
+int main(int ac, char **av)
+{
+	struct btrfs_root *root;
+	struct btrfs_fs_info *info;
+	struct extent_buffer *eb = NULL;
+	struct btrfs_trans_handle *trans = NULL;
+	u64 block = ~0ULL;
+	int len;
+	enum btrfs_open_ctree_flags flags = OPEN_CTREE_PARTIAL;
+	radix_tree_init();
+
+	while(1) {
+		int c;
+		c = getopt(ac, av, "r:w:");
+		if (c < 0)
+			break;
+		switch(c) {
+			case 'r':
+				block = arg_strtou64(optarg);
+				break;
+			case 'w':
+				flags |= OPEN_CTREE_WRITES;
+				block = arg_strtou64(optarg);
+				break;
+			default:
+				print_usage();
+		}
+	}
+	set_argv0(av);
+	ac = ac - optind;
+	if (check_argc_exact(ac, 1) || block == ~0ULL)
+		print_usage();
+
+	info = open_ctree_fs_info(av[optind], 0, 0, flags);
+	if (!info) {
+		fprintf(stderr, "unable to open %s\n", av[optind]);
+		exit(1);
+	}
+
+	root = info->fs_root;
+	if (!root) {
+		fprintf(stderr, "unable to open %s\n", av[optind]);
+		exit(1);
+	}
+
+	len = read_block(root, block, &eb);
+	if (eb->len != len) {
+		fprintf(stderr, "length mismatch: %u %d\n", eb->len, len);
+		return 1;
+	}
+
+	if (flags & OPEN_CTREE_WRITES) {
+		char buf[4];
+		int ret;
+		fprintf(stderr, "*** THIS MAY CORRUPT YOUR FILE SYSTEM ***\n");
+		fprintf(stderr, "*** Do you want to write logical block %llu "
+			"on device %s ?\n", block, av[optind]);
+		fprintf(stderr, "*** Type upper case \"yes\" to continue: ");
+		memset(buf, 0, 4);
+		ret = read(fileno(stderr), buf, 3);
+		if (strcmp(buf, "YES")) {
+			fprintf(stderr, "*** Aborted.\n");
+			goto out;
+		}
+		fprintf(stderr, "*** Writing block ... ");
+		if (fread(&eb->data, len, 1, stdin) < 1) {
+			fprintf(stderr, "failed to read %d bytes\n", len);
+			return 1;
+		}
+		btrfs_set_header_flag(eb, BTRFS_HEADER_FLAG_WRITTEN);
+		csum_tree_block(root, eb, 0);
+		if (write_and_map_eb(trans, root, eb))
+			fprintf(stderr, "error writing block %llu\n", block);
+		else
+			fprintf(stderr, " done.\n");
+	} else if (fwrite(&eb->data, len, 1, stdout) < 1)
+		fprintf(stderr, "failed to write %d bytes\n", len);
+
+out:
+	free_extent_buffer(eb);
+	return close_ctree(root);
+}
-- 
1.7.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo

[PATCH 16/18] btrfs-progs: NEW: brtfs-search-metadata

[mwilck]

From: Martin Wilck <mwilck@arcor.de>

A new tool for dumping all meta data (also unlinked nodes and leaves)
and searching nodes or leaves with certain properties.

Signed-off-by: Martin Wilck <mwilck@arcor.de>
---
 Makefile                |    2 +-
 btrfs-search-metadata.c |  224 +++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 225 insertions(+), 1 deletions(-)
 create mode 100644 btrfs-search-metadata.c

diff --git a/Makefile b/Makefile
index fe65867..c670f67 100644
--- a/Makefile
+++ b/Makefile
@@ -48,7 +48,7 @@ MAKEOPTS = --no-print-directory Q=$(Q)
 
 progs = mkfs.btrfs btrfs-debug-tree btrfs-raw btrfsck \
 	btrfs btrfs-map-logical btrfs-image btrfs-zero-log btrfs-convert \
-	btrfs-find-root btrfstune btrfs-show-super
+	btrfs-find-root btrfstune btrfs-show-super btrfs-search-metadata
 
 progs_extra = btrfs-corrupt-block btrfs-fragments btrfs-calc-size \
 	      btrfs-select-super
diff --git a/btrfs-search-metadata.c b/btrfs-search-metadata.c
new file mode 100644
index 0000000..80dc326
--- /dev/null
+++ b/btrfs-search-metadata.c
@@ -0,0 +1,224 @@
+/*
+ * Copyright (C) 2007 Oracle.  All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <uuid/uuid.h>
+#include "kerncompat.h"
+#include "radix-tree.h"
+#include "ctree.h"
+#include "disk-io.h"
+#include "print-tree.h"
+#include "version.h"
+#include "utils.h"
+#include "volumes.h"
+
+static int print_usage(void)
+{
+	fprintf(stderr, "usage: btrfs-search-metadata [options] device\n");
+	fprintf(stderr, "\t-k <objid>/<type>/<offset>: search for given key\n");
+	fprintf(stderr, "\t-g <generation>: search for given generation (transid)\n");
+	fprintf(stderr, "\t-t <tree-id>: search for given tree\n");
+	fprintf(stderr, "\t-l <level>: search for node level (0=leaf)\n");
+	fprintf(stderr, "\t-L: print full listing of matching leaf/node contents\n");
+	fprintf(stderr, "%s\n", BTRFS_BUILD_VERSION);
+	exit(1);
+}
+
+int bin_search(struct extent_buffer *eb, struct btrfs_key *key,
+	       int level, int *slot);
+
+static int do_one_block(struct btrfs_root *root, u64 block_nr, u64 tree_id,
+			u64 gen_id, int level, struct btrfs_key *key, int brief)
+{
+	struct extent_buffer *leaf;
+	int ret;
+	int slot = -1;
+	struct btrfs_disk_key disk_key;
+
+	leaf = read_tree_block(root,
+			       block_nr,
+			       root->leafsize, 0);
+
+	if (leaf && btrfs_header_level(leaf) != 0) {
+		free_extent_buffer(leaf);
+		leaf = NULL;
+	}
+
+	if (!leaf) {
+		leaf = read_tree_block(root,
+				       block_nr,
+				       root->nodesize, 0);
+	}
+	if (!leaf) {
+		fprintf(stderr, "failed to read %llu\n",
+			(unsigned long long)block_nr);
+		return -1;
+	}
+
+	ret = btrfs_is_leaf(leaf);
+	if (tree_id != 0 && tree_id != btrfs_header_owner(leaf))
+		goto out;
+	if (gen_id != 0 && gen_id != btrfs_header_generation(leaf))
+		goto out;
+	if (level != -1 && level != (int)btrfs_header_level(leaf))
+		goto out;
+
+	if (key && key->type != 0ULL) {
+		if (bin_search(leaf, key, btrfs_header_level(leaf), &slot))
+			goto out;
+	}
+
+	if (brief)
+		printf("%s %llu level %u items %d free %lu generation %llu owner %llu\n",
+		       (ret ? "leaf" : "node"),
+		       (unsigned long long)btrfs_header_bytenr(leaf),
+		       btrfs_header_level(leaf),
+		       btrfs_header_nritems(leaf),
+		       (ret ? btrfs_leaf_free_space(root, leaf) :
+			(unsigned long)BTRFS_NODEPTRS_PER_BLOCK(root) -
+			btrfs_header_nritems(leaf)),
+		       (u64)btrfs_header_generation(leaf),
+		       (u64)btrfs_header_owner(leaf));
+	else
+		btrfs_print_tree(root, leaf, 0);
+
+	if (key->objectid != 0ULL) {
+		btrfs_cpu_key_to_disk(&disk_key, key);
+		printf("\t");
+		btrfs_print_key(&disk_key);
+		printf(" found @ slot %d in %s %llu\n", slot,
+		       (ret ? "leaf" : "node"),
+		       (unsigned long long)btrfs_header_bytenr(leaf));
+	}
+out:
+	free_extent_buffer(leaf);
+	return ret;
+}
+
+static int walk_metadata(struct btrfs_fs_info *info, u64 tree_id, u64 gen_id,
+			 int level, struct btrfs_key *key, int brief)
+{
+	struct cache_extent *ce;
+	struct map_lookup *map;
+	struct btrfs_root *root = info->tree_root;
+	int ret = 0;
+	u64 ofs;
+	for (ce = first_cache_extent(&info->mapping_tree.cache_tree);
+	     ce;
+	     ce = next_cache_extent(ce)) {
+		map = container_of(ce, struct map_lookup, ce);
+		if (!(map->type & BTRFS_BLOCK_GROUP_METADATA))
+			continue;
+		for (ofs = 0; ofs < map->ce.size; ) {
+			int rv = do_one_block(root, map->ce.start + ofs,
+					      tree_id, gen_id, level,
+					      key, brief);
+			if (rv == 1)
+				ofs += root->leafsize;
+			else if (rv == 0)
+				ofs += root->nodesize;
+			else
+				ofs += (root->leafsize < root->nodesize ?
+					root->leafsize : root->nodesize);
+		}
+	}
+	return ret;
+}
+
+static int parse_key(char *arg, struct btrfs_key *bk)
+{
+	char *p1, *p2;
+	p1 = strchr(arg, '/');
+	if (!p1)
+		goto bad;
+	p2 = strchr(p1+1, '/');
+	if (!p2)
+		goto bad;
+	*p1 = *p2 = '\0';
+	bk->objectid = arg_strtou64(arg);
+	bk->type = arg_strtou64(p1 + 1);
+	bk->offset = arg_strtou64(p2 + 1);
+	return 0;
+
+bad:
+	fprintf(stderr, "Invalid format for key: %s, should be <objid>/<key>/<ofs>\n",
+		arg);
+	return 1;
+}
+
+int main(int ac, char **av)
+{
+	struct btrfs_root *root;
+	struct btrfs_fs_info *info;
+	u64 tree_id = 0ULL;
+	u64 gen_id = 0ULL;
+	int brief = 1;
+	int level = -1;
+	struct btrfs_key search_key = { 0ULL, 0, 0ULL };
+
+	radix_tree_init();
+
+	while(1) {
+		int c;
+		c = getopt(ac, av, "Lk:t:g:l:");
+		if (c < 0)
+			break;
+		switch(c) {
+			case 'k':
+				if (parse_key(optarg, &search_key))
+					return 1;
+				break;
+			case 't':
+				tree_id = arg_strtou64(optarg);
+				break;
+			case 'g':
+				gen_id = arg_strtou64(optarg);
+				break;
+			case 'l':
+				level = arg_strtou64(optarg);
+				break;
+			case 'L':
+				brief = 0;
+				break;
+			default:
+				print_usage();
+		}
+	}
+	set_argv0(av);
+	ac = ac - optind;
+	if (check_argc_exact(ac, 1))
+		print_usage();
+
+	info = open_ctree_fs_info(av[optind], 0, 0, OPEN_CTREE_PARTIAL);
+	if (!info) {
+		fprintf(stderr, "unable to open %s\n", av[optind]);
+		exit(1);
+	}
+
+	root = info->fs_root;
+	if (!root) {
+		fprintf(stderr, "unable to open %s\n", av[optind]);
+		exit(1);
+	}
+
+	walk_metadata(info, tree_id, gen_id, level, &search_key, brief);
+
+	return close_ctree(root);
+}
-- 
1.7.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo

[PATCH 17/18] btrfs-progs: ctree.c: make bin_search non-static

[mwilck]

From: Martin Wilck <mwilck@arcor.de>

I need it in btrfs-search-metadata

Signed-off-by: Martin Wilck <mwilck@arcor.de>
---
 ctree.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/ctree.c b/ctree.c
index 23399e2..1137312 100644
--- a/ctree.c
+++ b/ctree.c
@@ -602,8 +602,8 @@ static int generic_bin_search(struct extent_buffer *eb, unsigned long p,
  * simple bin_search frontend that does the right thing for
  * leaves vs nodes
  */
-static int bin_search(struct extent_buffer *eb, struct btrfs_key *key,
-		      int level, int *slot)
+int bin_search(struct extent_buffer *eb, struct btrfs_key *key,
+	       int level, int *slot)
 {
 	if (level == 0)
 		return generic_bin_search(eb,
-- 
1.7.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-

[PATCH 18/18] btrfs-progs: documentation for btrfs-raw and btrfs-search-metadata

[mwilck]

From: Martin Wilck <mwilck@arcor.de>

Update documentation for btrfs-debug-tree, and add pages for
btrfs-search-metadata and btrfs-raw.

Signed-off-by: Martin Wilck <mwilck@arcor.de>
---
 Documentation/Makefile                  |    2 +
 Documentation/btrfs-debug-tree.txt      |   10 +++++
 Documentation/btrfs-raw.txt             |   54 +++++++++++++++++++++++++++++
 Documentation/btrfs-search-metadata.txt |   57 +++++++++++++++++++++++++++++++
 4 files changed, 123 insertions(+), 0 deletions(-)
 create mode 100644 Documentation/btrfs-raw.txt
 create mode 100644 Documentation/btrfs-search-metadata.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index ef4f1bd..354c412 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -5,6 +5,8 @@ MAN8_TXT =
 MAN8_TXT += btrfs.txt
 MAN8_TXT += btrfs-convert.txt
 MAN8_TXT += btrfs-debug-tree.txt
+MAN8_TXT += btrfs-raw.txt
+MAN8_TXT += btrfs-search-metadata.txt
 MAN8_TXT += btrfs-find-root.txt
 MAN8_TXT += btrfs-image.txt
 MAN8_TXT += btrfs-map-logical.txt
diff --git a/Documentation/btrfs-debug-tree.txt b/Documentation/btrfs-debug-tree.txt
index 23fc115..69a547d 100644
--- a/Documentation/btrfs-debug-tree.txt
+++ b/Documentation/btrfs-debug-tree.txt
@@ -25,8 +25,18 @@ Print detailed extents info.
 Print info of btrfs device and root tree dirs only.
 -r::
 Print info of roots only.
+-u::
+Print info of UUID tree only.
+-R::
+Print info of roots and root backups.
+-t <object id>::
+Only print the subvolume tree with given object ID.
+-B <backup_num>::
+Start at backup root from superblock rather than current root.
 -b <block_num>::
 Print info of the specified block only.
+-f::
+Follow (descend) the (sub)tree rooted at the block given with -b.
 
 EXIT STATUS
 -----------
diff --git a/Documentation/btrfs-raw.txt b/Documentation/btrfs-raw.txt
new file mode 100644
index 0000000..ae7bd2d
--- /dev/null
+++ b/Documentation/btrfs-raw.txt
@@ -0,0 +1,54 @@
+btrfs-raw(8)
+============
+
+NAME
+----
+btrfs-raw - low-level manipulation of btrfs meta data blocks.
+
+SYNOPSIS
+--------
+*btrfs-raw* [[-r|-w] <bytenr>] <device>
+
+DESCRIPTION
+-----------
+*btrfs-raw* is used to dump the raw contents of the given logical block
+of a btrfs device to stdout, or write raw data read from stdin to the
+given logical block.
+
+*THIS TOOL IS DANGEROUS; IT MAY CORRUPT YOUR FILESYSTEM BEYOND REPAIR!!*
+
+Please exert extreme caution when writing modified blocks to disk. You
+should make a backup copy of the entire file system before doing so, even
+if the file system is already corrupted. Make sure you have a thorough
+understanding of the btrfs disk data structures before making any changes.
+
+*YOU USE THIS TOOL AT YOUR OWN RISK!*
+
+OPTIONS
+-------
+-r <bytenr>::
+Read the logical block starting at <bytenr> and write the raw contents
+to stdout (caution, binary data).
+-w <bytenr>::
+Write the logical block starting at <bytenr> to disk, reading data from
+stdin. The tool will adjust the header checksum before writing to disk.
+
+EXIT STATUS
+-----------
+*btrfs-raw* will return 0 if no error happened.
+If any problems happened, 1 will be returned.
+
+EXAMPLE
+-------
+
+`btrfs-raw -r 874991616 /dev/sda >/tmp/blob`
+
+`btrfs-raw -w 874991616 /dev/sda </tmp/blob`
+
+Dump the data of the given block to a file and write it back.
+Between these operations, a hex editor could be used to manually fix
+inconsistencies in the meta data.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8), `btrfs-debug-tree`(8)
diff --git a/Documentation/btrfs-search-metadata.txt b/Documentation/btrfs-search-metadata.txt
new file mode 100644
index 0000000..d97295b
--- /dev/null
+++ b/Documentation/btrfs-search-metadata.txt
@@ -0,0 +1,57 @@
+btrfs-search-metadata(8)
+========================
+
+NAME
+----
+btrfs-search-metadata - search for certain nodes or leaves in btrfs metadata
+
+SYNOPSIS
+--------
+*btrfs-search-metadata* [options] <device>
+
+DESCRIPTION
+-----------
+*btrfs-search-metadata* is used to dump the meta data of a device, or to 
+selectively dump nodes or leaves matching certain conditions.
+
+Unlike `btrfs-dump-tree`, this tool will also find tree "branches" that
+are disconnected from the root tree, and previous meta data copies. If a
+corruption occurs, this may be useful for finding old, still healthy copies.
+
+This is maybe useful for analyzing filesystem state or inconsistence and has
+a positive educational effect on understanding the internal structure.
+<device> is the device file where the filesystem is stored.
+
+OPTIONS
+-------
+-k <object id>/<key type>/<offset>::
+Search for leaves and nodes containing the given key.
+-g <generation>::
+Search for leaves and nodes with the given generation (transid).
+-l <level>::
+Search for nodes with the given level, or leaves (level 0).
+-t <object id>::
+Search for leaves and nodes with the given owner object ID.
+-L::
+dump full content of found nodes or leaves, like btrfs-debug-tree.
+
+EXIT STATUS
+-----------
+*btrfs-search-metadata* will return 0 if no error happened.
+If any problems happened, 1 will be returned.
+
+EXAMPLE
+-------
+
+`btrfs-search-metadata -t 260 -l 0 -k 256/1/0 /dev/sda`
+
+Search the btrfs file system on `/dev/sda` for leaves belonging to
+subvolume 260 and containing the first inode item (type 1: inode item,
+object ID 256: first available object ID).
+
+See `ctree.h` in the btrfs source code and the btrfs Wiki for 
+assigned object IDs and key types.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8), `btrfs-debug-tree`(8)
-- 
1.7.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo

Re: [PATCH 01/18] btrfs-progs: btrfs-debug-tree: add option -f for "block only"

[Qu Wenruo]

-------- Original Message --------
Subject: [PATCH 01/18] btrfs-progs: btrfs-debug-tree: add option -f for 
"block only"
From: <mwilck@arcor.de>
To: <linux-btrfs@vger.kernel.org>
Date: 2014年12月11日 04:51
> From: Martin Wilck <mwilck@arcor.de>
>
> btrfs-debug-tree prints only the given block. It is sometimes
> useful to be able to print the subtree under this block.
> This patch enables this behavior with the option "-f".
>
> Signed-off-by: Martin Wilck <mwilck@arcor.de>
> ---
>   btrfs-debug-tree.c |   10 ++++++++--
>   1 files changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/btrfs-debug-tree.c b/btrfs-debug-tree.c
> index e46500d..e61c71c 100644
> --- a/btrfs-debug-tree.c
> +++ b/btrfs-debug-tree.c
> @@ -41,6 +41,8 @@ static int print_usage(void)
>   	fprintf(stderr, "\t-u : print info of uuid tree only\n");
>   	fprintf(stderr, "\t-b block_num : print info of the specified block"
>                       " only\n");
> +	fprintf(stderr, "\t-f : (with -b) follow subtree of the specified"
> +		" block\n");
>   	fprintf(stderr,
>   		"\t-t tree_id : print only the tree with the given id\n");
>   	fprintf(stderr, "%s\n", BTRFS_BUILD_VERSION);
> @@ -137,6 +139,7 @@ int main(int ac, char **av)
>   	int roots_only = 0;
>   	int root_backups = 0;
>   	u64 block_only = 0;
> +	int block_follow = 0;
>   	struct btrfs_root *tree_root_scan;
>   	u64 tree_id = 0;
>   
> @@ -144,7 +147,7 @@ int main(int ac, char **av)
>   
>   	while(1) {
>   		int c;
> -		c = getopt(ac, av, "deb:rRut:");
> +		c = getopt(ac, av, "defb:rRut:");
>   		if (c < 0)
>   			break;
>   		switch(c) {
> @@ -167,6 +170,9 @@ int main(int ac, char **av)
>   			case 'b':
>   				block_only = arg_strtou64(optarg);
>   				break;
> +			case 'f':
> +				block_follow = 1;
> +				break;
>   			case 't':
>   				tree_id = arg_strtou64(optarg);
>   				break;
> @@ -211,7 +217,7 @@ int main(int ac, char **av)
>   				(unsigned long long)block_only);
>   			goto close_root;
>   		}
> -		btrfs_print_tree(root, leaf, 0);
> +		btrfs_print_tree(root, leaf, block_follow);
Although not a bug of your patch, but would you please fix the extent 
buffer leak by
adding a free_extent_buffer(buf)?

Thanks,
Qu
>   		goto close_root;
>   	}
>   

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-

Re: [PATCH 05/18] btrfs-progs: ctree.h: fix btrfs_inode_[amc]time

[Qu Wenruo]

-------- Original Message --------
Subject: [PATCH 05/18] btrfs-progs: ctree.h: fix btrfs_inode_[amc]time
From: <mwilck@arcor.de>
To: <linux-btrfs@vger.kernel.org>
Date: 2014年12月11日 04:51
> From: Martin Wilck <mwilck@arcor.de>
>
> make btrfs_inode_[amc]time work like the other btrfs_inode_xxx
> functions. The current definition appears broken to me; it never
> returns valid pointer unless an extent buffer address is added.
The original method returns the ptr in the way just like 
btrfs_item_ptr(), which you should call
read_extent_buffer() to read them into a struct of btrfs_timespec.
That's the same in kernel codes.

If just want to read the timespec, you could try 
btrfs_stack_timespec_[n]spec() marco.

Thanks,
Qu
> Signed-off-by: Martin Wilck <mwilck@arcor.de>
> ---
>   ctree.h |   15 +++++++++------
>   1 files changed, 9 insertions(+), 6 deletions(-)
>
> diff --git a/ctree.h b/ctree.h
> index 89036de..1d5a5fc 100644
> --- a/ctree.h
> +++ b/ctree.h
> @@ -1414,27 +1414,30 @@ BTRFS_SETGET_STACK_FUNCS(stack_inode_flags,
>   			 struct btrfs_inode_item, flags, 64);
>   
>   static inline struct btrfs_timespec *
> -btrfs_inode_atime(struct btrfs_inode_item *inode_item)
> +btrfs_inode_atime(struct extent_buffer *eb,
> +		  struct btrfs_inode_item *inode_item)
>   {
>   	unsigned long ptr = (unsigned long)inode_item;
>   	ptr += offsetof(struct btrfs_inode_item, atime);
> -	return (struct btrfs_timespec *)ptr;
> +	return (struct btrfs_timespec *)(ptr + eb->data);
>   }
>   
>   static inline struct btrfs_timespec *
> -btrfs_inode_mtime(struct btrfs_inode_item *inode_item)
> +btrfs_inode_mtime(struct extent_buffer *eb,
> +		  struct btrfs_inode_item *inode_item)
>   {
>   	unsigned long ptr = (unsigned long)inode_item;
>   	ptr += offsetof(struct btrfs_inode_item, mtime);
> -	return (struct btrfs_timespec *)ptr;
> +	return (struct btrfs_timespec *)(ptr + eb->data);
>   }
>   
>   static inline struct btrfs_timespec *
> -btrfs_inode_ctime(struct btrfs_inode_item *inode_item)
> +btrfs_inode_ctime(struct extent_buffer *eb,
> +		  struct btrfs_inode_item *inode_item)
>   {
>   	unsigned long ptr = (unsigned long)inode_item;
>   	ptr += offsetof(struct btrfs_inode_item, ctime);
> -	return (struct btrfs_timespec *)ptr;
> +	return (struct btrfs_timespec *)(ptr + eb->data);
>   }
>   
>   static inline struct btrfs_timespec *

Re: [PATCH 05/18] btrfs-progs: ctree.h: fix btrfs_inode_[amc]time

[Qu Wenruo]

-------- Original Message --------
Subject: Re: [PATCH 05/18] btrfs-progs: ctree.h: fix btrfs_inode_[amc]time
From: Qu Wenruo <quwenruo@cn.fujitsu.com>
To: mwilck@arcor.de, linux-btrfs@vger.kernel.org
Date: 2014年12月11日 15:59
>
> -------- Original Message --------
> Subject: [PATCH 05/18] btrfs-progs: ctree.h: fix btrfs_inode_[amc]time
> From: <mwilck@arcor.de>
> To: <linux-btrfs@vger.kernel.org>
> Date: 2014年12月11日 04:51
>> From: Martin Wilck <mwilck@arcor.de>
>>
>> make btrfs_inode_[amc]time work like the other btrfs_inode_xxx
>> functions. The current definition appears broken to me; it never
>> returns valid pointer unless an extent buffer address is added.
> The original method returns the ptr in the way just like 
> btrfs_item_ptr(), which you should call
> read_extent_buffer() to read them into a struct of btrfs_timespec.
> That's the same in kernel codes.
>
> If just want to read the timespec, you could try 
> btrfs_stack_timespec_[n]spec() marco.
Oh, sorry, for your case, it is btrfs_timespec_[n]spec() marcro, not the 
stack ones.
And that one uses the timespec returned by btrfs_inode_[amc]time() marcor.

Thanks,
Qu
>
> Thanks,
> Qu
>> Signed-off-by: Martin Wilck <mwilck@arcor.de>
>> ---
>>   ctree.h |   15 +++++++++------
>>   1 files changed, 9 insertions(+), 6 deletions(-)
>>
>> diff --git a/ctree.h b/ctree.h
>> index 89036de..1d5a5fc 100644
>> --- a/ctree.h
>> +++ b/ctree.h
>> @@ -1414,27 +1414,30 @@ BTRFS_SETGET_STACK_FUNCS(stack_inode_flags,
>>                struct btrfs_inode_item, flags, 64);
>>     static inline struct btrfs_timespec *
>> -btrfs_inode_atime(struct btrfs_inode_item *inode_item)
>> +btrfs_inode_atime(struct extent_buffer *eb,
>> +          struct btrfs_inode_item *inode_item)
>>   {
>>       unsigned long ptr = (unsigned long)inode_item;
>>       ptr += offsetof(struct btrfs_inode_item, atime);
>> -    return (struct btrfs_timespec *)ptr;
>> +    return (struct btrfs_timespec *)(ptr + eb->data);
>>   }
>>     static inline struct btrfs_timespec *
>> -btrfs_inode_mtime(struct btrfs_inode_item *inode_item)
>> +btrfs_inode_mtime(struct extent_buffer *eb,
>> +          struct btrfs_inode_item *inode_item)
>>   {
>>       unsigned long ptr = (unsigned long)inode_item;
>>       ptr += offsetof(struct btrfs_inode_item, mtime);
>> -    return (struct btrfs_timespec *)ptr;
>> +    return (struct btrfs_timespec *)(ptr + eb->data);
>>   }
>>     static inline struct btrfs_timespec *
>> -btrfs_inode_ctime(struct btrfs_inode_item *inode_item)
>> +btrfs_inode_ctime(struct extent_buffer *eb,
>> +          struct btrfs_inode_item *inode_item)
>>   {
>>       unsigned long ptr = (unsigned long)inode_item;
>>       ptr += offsetof(struct btrfs_inode_item, ctime);
>> -    return (struct btrfs_timespec *)ptr;
>> +    return (struct btrfs_timespec *)(ptr + eb->data);
>>   }
>>     static inline struct btrfs_timespec *
>

Re: [PATCH 09/18] btrfs restore: more graceful error handling in copy_file

[Qu Wenruo]

-------- Original Message --------
Subject: [PATCH 09/18] btrfs restore: more graceful error handling in 
copy_file
From: <mwilck@arcor.de>
To: <linux-btrfs@vger.kernel.org>
Date: 2014年12月11日 04:51
> From: Martin Wilck <mwilck@arcor.de>
>
> Setting size and attributes of a file makes sense even if some
> errors have occured during revovery.
>
> Also, do something useful with the number of bytes written.
>
> Signed-off-by: Martin Wilck <mwilck@arcor.de>
> ---
>   cmds-restore.c |   27 ++++++++++++++-------------
>   1 files changed, 14 insertions(+), 13 deletions(-)
>
> diff --git a/cmds-restore.c b/cmds-restore.c
> index 8ecd896..10bb8be 100644
> --- a/cmds-restore.c
> +++ b/cmds-restore.c
> @@ -715,7 +715,7 @@ static int copy_file(struct btrfs_root *root, int fd, struct btrfs_key *key,
>   					return ret;
>   				} else if (ret) {
>   					/* No more leaves to search */
> -					btrfs_free_path(path);
> +					ret = 0;
>   					goto set_size;
>   				}
>   				leaf = path->nodes[0];
> @@ -734,35 +734,36 @@ static int copy_file(struct btrfs_root *root, int fd, struct btrfs_key *key,
>   		if (compression >= BTRFS_COMPRESS_LAST) {
>   			fprintf(stderr, "Don't support compression yet %d\n",
>   				compression);
> -			btrfs_free_path(path);
> -			return -1;
> +			ret = -1;
Although the original one uses -1, IMHO it would be more meaningful 
using -ENOTTY.
If someone later adds stderr() for the ret and print it, the author will 
be graceful.
> +			goto set_size;
>   		}
>   
> +		bytes_written = 0ULL;
>   		if (extent_type == BTRFS_FILE_EXTENT_PREALLOC)
>   			goto next;
>   		if (extent_type == BTRFS_FILE_EXTENT_INLINE) {
>   			ret = copy_one_inline(fd, path, found_key.offset,
>   					      &bytes_written);
> -			if (ret) {
> -				btrfs_free_path(path);
> -				return -1;
> -			}
>   		} else if (extent_type == BTRFS_FILE_EXTENT_REG) {
>   			ret = copy_one_extent(root, fd, leaf, fi,
>   					      found_key.offset, &bytes_written);
> -			if (ret) {
> -				btrfs_free_path(path);
> -				return ret;
> -			}
>   		} else {
>   			printf("Weird extent type %d\n", extent_type);
>   		}
> +		total_written += bytes_written;
> +		next_pos = found_key.offset + bytes_written;
> +		if (ret) {
> +			fprintf(stderr, "ERROR after writing %llu bytes\n",
> +				total_written);
Just a advice, what about add stderr(-ret) for the info?
> +			ret = -1;
Why change ret to -1? At least some of the ret above can be meaningful 
like -EIO or -ENOENT.
> +			goto set_size;
> +		}
>   next:
>   		path->slots[0]++;
>   	}
>   
> -	btrfs_free_path(path);
>   set_size:
> +	btrfs_free_path(path);
>   	if (get_xattrs) {
>   		ret = set_file_xattrs(root, key->objectid, fd, file);
>   		if (ret)
> @@ -771,7 +772,7 @@ set_size:
>   	}
>   
>   	set_fd_attrs(fd, &st, file);
> -	return 0;
> +	return ret;
>   }
>   
>   static int search_dir(struct btrfs_root *root, struct btrfs_key *key,
Other part looks good for me.

Thanks,
Qu

Re: [PATCH 12/18] btrfs restore: check progress of file restoration

[Qu Wenruo]

-------- Original Message --------
Subject: [PATCH 12/18] btrfs restore: check progress of file restoration
From: <mwilck@arcor.de>
To: <linux-btrfs@vger.kernel.org>
Date: 2014年12月11日 04:51
> From: Martin Wilck <mwilck@arcor.de>
>
> extents should be ordered by file offset. Expect no overlaps,
> and report holes.
>
> Signed-off-by: Martin Wilck <mwilck@arcor.de>
> ---
>   cmds-restore.c |    8 ++++++++
>   1 files changed, 8 insertions(+), 0 deletions(-)
>
> diff --git a/cmds-restore.c b/cmds-restore.c
> index 004c82e..80081b8 100644
> --- a/cmds-restore.c
> +++ b/cmds-restore.c
> @@ -739,6 +739,14 @@ static int copy_file(struct btrfs_root *root, int fd, struct btrfs_key *key,
>   			ret = -1;
>   			goto set_size;
>   		}
> +		if (found_key.offset < next_pos) {
> +			fprintf(stderr, "extent overlap, %llu < %llu\n",
> +				found_key.offset, next_pos);
> +			ret = -1;
> +			goto set_size;
Would it be better to continue recovery?
Even overlap, later extents may still be OK. What about just skip to 
next extent?

Thanks,
Qu
> +		} else if (found_key.offset > next_pos)
> +			fprintf(stderr, "hole at %llu (%llu bytes)\n",
> +				next_pos, found_key.offset - next_pos);
>   
>   		bytes_written = 0ULL;
>   		if (extent_type == BTRFS_FILE_EXTENT_PREALLOC)