[PATCH v2 00/13] Introduce offline fsid/chunk tree uuid change for btrfstune.

[PATCH v2 00/13] Introduce offline fsid/chunk tree uuid change for btrfstune.

[Qu Wenruo]

This patchset will introduce offline fsid/chunk tree uuid change
function to 'btrfs property' command, with "fsid" and "chunk_tree_uuid"
respectively.

The implement is in-place btrfs_header modification, without transaction
protection. So the uuid change can be done on any valid image even it is
already full.

Since we don't use transaction to protect the convert, it's possible one
can stop the running convert progress, causing the fs in a inconsistent
status and unable to be mounted.
To avoid such problem, we introduce new btrfs super flags:
BTRFS_SUPER_FLAG_CHANGING_FSID and BTRFS_SUPER_FLAG_CHANGING_CHUNK_TREE_ID

Btrfs-progs and kernel will abort open_ctree if these super flags is found.
Btrfs-progs has the exception if OPEN_CTREE_IGNORE_FSID/CHUNK_TREE_ID is
given, allowing user to finished unfinished uuid changing process.

Along with new super_flags, also add human-readable flags output for
btrfs-show-super command.

Qu Wenruo (13):
  btrfs-progs: Add CHANGIND_FSID and CHANGING_CHUNK_TREE_ID super flags.
  btrfs-progs: Use unified function to implement print_readable_*_flag()
        function.
  btrfs-progs: Make btrfs-show-super print human readable flag for super
        flags.
  btrfs-progs: Add open_ctree check for uuid changing.
  btrfs-progs: Export write_tree_block().
  btrfs-progs: Introduce change_header_uuid() function.
  btrfs-progs: Introduce change_extents_uuid() function.
  btrfs-progs: Introduce change_device_uuid() function.
  btrfs-progs: Introduce change_devices_uuid() function.
  btrfs-progs: Introduce change_id_prepare() and change_id_done()    
    functions.
  btrfs-progs: Introduce change_uuid() function.
  btrfs-progs: Add offline type for btrfs property.
  btrfs-progs: Update Doc for btrfs-property

 Documentation/btrfs-property.asciidoc |  17 +-
 btrfs-show-super.c                    |  48 ++++-
 cmds-property.c                       |   5 +-
 ctree.h                               |   6 +
 disk-io.c                             |  23 +-
 disk-io.h                             |   6 +
 props.c                               | 386 ++++++++++++++++++++++++++++++++++
 props.h                               |   1 +
 8 files changed, 483 insertions(+), 9 deletions(-)

-- 
2.4.0

[PATCH v2 01/13] btrfs-progs: Add CHANGIND_FSID and CHANGING_CHUNK_TREE_ID super flags.

[Qu Wenruo]

Add these super flags to info kernel not to mount such unfinished
fsid/chunk tree id changing.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
v2:
  Newly introduced.
---
 ctree.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/ctree.h b/ctree.h
index 10dc838..8dc70a5 100644
--- a/ctree.h
+++ b/ctree.h
@@ -307,6 +307,8 @@ static inline unsigned long btrfs_chunk_item_size(int num_stripes)
 
 #define BTRFS_HEADER_FLAG_WRITTEN		(1ULL << 0)
 #define BTRFS_HEADER_FLAG_RELOC			(1ULL << 1)
+#define BTRFS_SUPER_FLAG_CHANGING_FSID		(1ULL << 30)
+#define BTRFS_SUPER_FLAG_CHANGING_CHUNK_TREE_ID	(1ULL << 31)
 #define BTRFS_SUPER_FLAG_SEEDING		(1ULL << 32)
 #define BTRFS_SUPER_FLAG_METADUMP		(1ULL << 33)
 #define BTRFS_SUPER_FLAG_METADUMP_V2		(1ULL << 34)
-- 
2.4.0

Re: [PATCH v2 01/13] btrfs-progs: Add CHANGIND_FSID and CHANGING_CHUNK_TREE_ID super flags.

[David Sterba]

On Mon, May 11, 2015 at 04:08:42PM +0800, Qu Wenruo wrote:
> @@ -307,6 +307,8 @@ static inline unsigned long btrfs_chunk_item_size(int num_stripes)
>  
>  #define BTRFS_HEADER_FLAG_WRITTEN		(1ULL << 0)
>  #define BTRFS_HEADER_FLAG_RELOC			(1ULL << 1)
> +#define BTRFS_SUPER_FLAG_CHANGING_FSID		(1ULL << 30)
> +#define BTRFS_SUPER_FLAG_CHANGING_CHUNK_TREE_ID	(1ULL << 31)

Seems that the values should continue from the end of the sequence, ie.
1 << 35, I'll change that.

>  #define BTRFS_SUPER_FLAG_SEEDING		(1ULL << 32)
>  #define BTRFS_SUPER_FLAG_METADUMP		(1ULL << 33)
>  #define BTRFS_SUPER_FLAG_METADUMP_V2		(1ULL << 34)

[PATCH v2 02/13] btrfs-progs: Use unified function to implement print_readable_*_flag() function.

[Qu Wenruo]

Now add a new unified __print_readable_flag() function to implement
print_readable_incompat_flag().

This makes later extension for human readable flags easier.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
v2:
  Newly introduced.
---
 btrfs-show-super.c | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/btrfs-show-super.c b/btrfs-show-super.c
index 1c7a086..430ddfe 100644
--- a/btrfs-show-super.c
+++ b/btrfs-show-super.c
@@ -305,7 +305,8 @@ struct readable_flag_entry incompat_flags_array[] = {
 static const int incompat_flags_num = sizeof(incompat_flags_array) /
 				      sizeof(struct readable_flag_entry);
 
-static void print_readable_incompat_flag(u64 flag)
+static void __print_readable_flag(u64 flag, struct readable_flag_entry *array,
+				  int array_size, u64 supported_flags)
 {
 	int i;
 	int first = 1;
@@ -313,9 +314,10 @@ static void print_readable_incompat_flag(u64 flag)
 
 	if (!flag)
 		return;
+
 	printf("\t\t\t( ");
-	for (i = 0; i < incompat_flags_num; i++) {
-		entry = incompat_flags_array + i;
+	for (i = 0; i < array_size; i++) {
+		entry = array + i;
 		if (flag & entry->bit) {
 			if (first)
 				printf("%s ", entry->output);
@@ -324,7 +326,7 @@ static void print_readable_incompat_flag(u64 flag)
 			first = 0;
 		}
 	}
-	flag &= ~BTRFS_FEATURE_INCOMPAT_SUPP;
+	flag &= ~supported_flags;
 	if (flag) {
 		if (first)
 			printf("unknown flag: 0x%llx ", flag);
@@ -334,6 +336,13 @@ static void print_readable_incompat_flag(u64 flag)
 	printf(")\n");
 }
 
+static void print_readable_incompat_flag(u64 flag)
+{
+	return __print_readable_flag(flag, incompat_flags_array,
+				     incompat_flags_num,
+				     BTRFS_FEATURE_INCOMPAT_SUPP);
+}
+
 static void dump_superblock(struct btrfs_super_block *sb, int full)
 {
 	int i;
-- 
2.4.0

[PATCH v2 03/13] btrfs-progs: Make btrfs-show-super print human readable flag for super flags.

[Qu Wenruo]

Use the new __print_readable_flag() to implement
print_readable_super_flag().

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
v2:
  Newly introduced.
---
 btrfs-show-super.c | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/btrfs-show-super.c b/btrfs-show-super.c
index 430ddfe..6da80d8 100644
--- a/btrfs-show-super.c
+++ b/btrfs-show-super.c
@@ -305,6 +305,30 @@ struct readable_flag_entry incompat_flags_array[] = {
 static const int incompat_flags_num = sizeof(incompat_flags_array) /
 				      sizeof(struct readable_flag_entry);
 
+#define DEF_HEADER_FLAG_ENTRY(bit_name)			\
+	{BTRFS_HEADER_FLAG_##bit_name, #bit_name}
+#define DEF_SUPER_FLAG_ENTRY(bit_name)			\
+	{BTRFS_SUPER_FLAG_##bit_name, #bit_name}
+
+struct readable_flag_entry super_flags_array[] = {
+	DEF_HEADER_FLAG_ENTRY(WRITTEN),
+	DEF_HEADER_FLAG_ENTRY(RELOC),
+	DEF_SUPER_FLAG_ENTRY(CHANGING_FSID),
+	DEF_SUPER_FLAG_ENTRY(CHANGING_CHUNK_TREE_ID),
+	DEF_SUPER_FLAG_ENTRY(SEEDING),
+	DEF_SUPER_FLAG_ENTRY(METADUMP),
+	DEF_SUPER_FLAG_ENTRY(METADUMP_V2)
+};
+static const int super_flags_num = ARRAY_SIZE(super_flags_array);
+
+#define BTRFS_SUPER_FLAG_SUPP	(BTRFS_HEADER_FLAG_WRITTEN |\
+				 BTRFS_HEADER_FLAG_RELOC |\
+				 BTRFS_SUPER_FLAG_CHANGING_FSID |\
+				 BTRFS_SUPER_FLAG_CHANGING_CHUNK_TREE_ID |\
+				 BTRFS_SUPER_FLAG_SEEDING |\
+				 BTRFS_SUPER_FLAG_METADUMP |\
+				 BTRFS_SUPER_FLAG_METADUMP_V2)
+
 static void __print_readable_flag(u64 flag, struct readable_flag_entry *array,
 				  int array_size, u64 supported_flags)
 {
@@ -343,6 +367,12 @@ static void print_readable_incompat_flag(u64 flag)
 				     BTRFS_FEATURE_INCOMPAT_SUPP);
 }
 
+static void print_readable_super_flag(u64 flag)
+{
+	return __print_readable_flag(flag, super_flags_array,
+				     super_flags_num, BTRFS_SUPER_FLAG_SUPP);
+}
+
 static void dump_superblock(struct btrfs_super_block *sb, int full)
 {
 	int i;
@@ -362,6 +392,7 @@ static void dump_superblock(struct btrfs_super_block *sb, int full)
 		(unsigned long long)btrfs_super_bytenr(sb));
 	printf("flags\t\t\t0x%llx\n",
 		(unsigned long long)btrfs_super_flags(sb));
+	print_readable_super_flag(btrfs_super_flags(sb));
 
 	printf("magic\t\t\t");
 	s = (char *) &sb->magic;
-- 
2.4.0

[PATCH v2 04/13] btrfs-progs: Add open_ctree check for uuid changing.

[Qu Wenruo]

Now open_ctree will exit if it found the superblock is marked
CHANGING_FSID/CHUNK_TREE_UUID, except given IGNORE_FSID/CHUNK_TREE_ID
open ctree flags.

Kernel will do the same thing later.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
v2:
  Abort open_ctree if uuid change is not done.
---
 ctree.h   |  2 ++
 disk-io.c | 19 ++++++++++++++++++-
 disk-io.h |  3 +++
 3 files changed, 23 insertions(+), 1 deletion(-)

diff --git a/ctree.h b/ctree.h
index 8dc70a5..1608645 100644
--- a/ctree.h
+++ b/ctree.h
@@ -1012,6 +1012,8 @@ struct btrfs_fs_info {
 	unsigned int is_chunk_recover:1;
 	unsigned int quota_enabled:1;
 	unsigned int suppress_check_block_errors:1;
+	unsigned int ignore_fsid:1;
+	unsigned int ignore_chunk_tree_id:1;
 
 	int (*free_extent_hook)(struct btrfs_trans_handle *trans,
 				struct btrfs_root *root,
diff --git a/disk-io.c b/disk-io.c
index c1cf146..0edebfa 100644
--- a/disk-io.c
+++ b/disk-io.c
@@ -49,7 +49,8 @@ static int check_tree_block(struct btrfs_root *root, struct extent_buffer *buf)
 
 	fs_devices = root->fs_info->fs_devices;
 	while (fs_devices) {
-		if (!memcmp_extent_buffer(buf, fs_devices->fsid,
+		if (root->fs_info->ignore_fsid ||
+		    !memcmp_extent_buffer(buf, fs_devices->fsid,
 					  btrfs_header_fsid(),
 					  BTRFS_FSID_SIZE)) {
 			ret = 0;
@@ -1149,6 +1150,10 @@ static struct btrfs_fs_info *__open_ctree_fd(int fp, const char *path,
 		fs_info->on_restoring = 1;
 	if (flags & OPEN_CTREE_SUPPRESS_CHECK_BLOCK_ERRORS)
 		fs_info->suppress_check_block_errors = 1;
+	if (flags & OPEN_CTREE_IGNORE_FSID)
+		fs_info->ignore_fsid = 1;
+	if (flags & OPEN_CTREE_IGNORE_CHUNK_TREE_ID)
+		fs_info->ignore_chunk_tree_id = 1;
 
 	ret = btrfs_scan_fs_devices(fp, path, &fs_devices, sb_bytenr,
 				    (flags & OPEN_CTREE_RECOVER_SUPER),
@@ -1180,6 +1185,18 @@ static struct btrfs_fs_info *__open_ctree_fd(int fp, const char *path,
 		goto out_devices;
 	}
 
+	if (btrfs_super_flags(disk_super) & BTRFS_SUPER_FLAG_CHANGING_FSID &&
+	    !fs_info->ignore_fsid) {
+		fprintf(stderr, "Under uuid changing, fsid inconsistent\n");
+		goto out_devices;
+	}
+	if (btrfs_super_flags(disk_super) &
+			BTRFS_SUPER_FLAG_CHANGING_CHUNK_TREE_ID &&
+	    !fs_info->ignore_chunk_tree_id) {
+		fprintf(stderr, "Under uuid changing, chunk tree uuid inconsistent\n");
+		goto out_devices;
+	}
+
 	memcpy(fs_info->fsid, &disk_super->fsid, BTRFS_FSID_SIZE);
 
 	ret = btrfs_check_fs_compatibility(fs_info->super_copy,
diff --git a/disk-io.h b/disk-io.h
index 4caebeb..e1c2030 100644
--- a/disk-io.h
+++ b/disk-io.h
@@ -49,6 +49,9 @@ enum btrfs_open_ctree_flags {
 	 * tree bits.
 	 * Like split PARTIAL into SKIP_CSUM/SKIP_EXTENT
 	 */
+
+	OPEN_CTREE_IGNORE_FSID		= (1 << 10),
+	OPEN_CTREE_IGNORE_CHUNK_TREE_ID	= (1 << 11)
 };
 
 static inline u64 btrfs_sb_offset(int mirror)
-- 
2.4.0

[PATCH v2 05/13] btrfs-progs: Export write_tree_block().

[Qu Wenruo]

Export write_tree_block() function and allow it write extent without
transaction.

This provides the basis for later uuid change function.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
v2:
  None
---
 disk-io.c | 4 ++--
 disk-io.h | 3 +++
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/disk-io.c b/disk-io.c
index 0edebfa..2ab2d2e 100644
--- a/disk-io.c
+++ b/disk-io.c
@@ -372,7 +372,7 @@ int write_and_map_eb(struct btrfs_trans_handle *trans,
 	return 0;
 }
 
-static int write_tree_block(struct btrfs_trans_handle *trans,
+int write_tree_block(struct btrfs_trans_handle *trans,
 		     struct btrfs_root *root,
 		     struct extent_buffer *eb)
 {
@@ -381,7 +381,7 @@ static int write_tree_block(struct btrfs_trans_handle *trans,
 		BUG();
 	}
 
-	if (!btrfs_buffer_uptodate(eb, trans->transid))
+	if (trans && !btrfs_buffer_uptodate(eb, trans->transid))
 		BUG();
 
 	btrfs_set_header_flag(eb, BTRFS_HEADER_FLAG_WRITTEN);
diff --git a/disk-io.h b/disk-io.h
index e1c2030..c1ab875 100644
--- a/disk-io.h
+++ b/disk-io.h
@@ -127,6 +127,9 @@ int csum_tree_block_size(struct extent_buffer *buf, u16 csum_sectorsize,
 			 int verify);
 int verify_tree_block_csum_silent(struct extent_buffer *buf, u16 csum_size);
 int btrfs_read_buffer(struct extent_buffer *buf, u64 parent_transid);
+int write_tree_block(struct btrfs_trans_handle *trans,
+		     struct btrfs_root *root,
+		     struct extent_buffer *eb);
 int write_and_map_eb(struct btrfs_trans_handle *trans, struct btrfs_root *root,
 		     struct extent_buffer *eb);
 
-- 
2.4.0

[PATCH v2 06/13] btrfs-progs: Introduce change_header_uuid() function.

[Qu Wenruo]

This function is used to change fsid/chunk_tree_uuid of a node/leaf.
The function does it without transaction protect.

This is the basis of offline uuid change.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
v2:
  Move to props.c
---
 ctree.h |  2 ++
 props.c | 31 +++++++++++++++++++++++++++++++
 2 files changed, 33 insertions(+)

diff --git a/ctree.h b/ctree.h
index 1608645..0fc876a 100644
--- a/ctree.h
+++ b/ctree.h
@@ -956,7 +956,9 @@ struct btrfs_device;
 struct btrfs_fs_devices;
 struct btrfs_fs_info {
 	u8 fsid[BTRFS_FSID_SIZE];
+	u8 *new_fsid;
 	u8 chunk_tree_uuid[BTRFS_UUID_SIZE];
+	u8 *new_chunk_tree_uuid;
 	struct btrfs_root *fs_root;
 	struct btrfs_root *extent_root;
 	struct btrfs_root *tree_root;
diff --git a/props.c b/props.c
index c7c6752..abc3ae0 100644
--- a/props.c
+++ b/props.c
@@ -25,6 +25,7 @@
 #include "commands.h"
 #include "utils.h"
 #include "props.h"
+#include "disk-io.h"
 
 #define XATTR_BTRFS_PREFIX     "btrfs."
 #define XATTR_BTRFS_PREFIX_LEN (sizeof(XATTR_BTRFS_PREFIX) - 1)
@@ -187,6 +188,36 @@ out:
 	return ret;
 }
 
+static int change_header_uuid(struct btrfs_root *root, struct extent_buffer *eb)
+{
+	struct btrfs_fs_info *fs_info = root->fs_info;
+	int same_fsid = 1;
+	int same_chunk_tree_uuid = 1;
+	int ret = 0;
+
+	/* Check for whether we need to change fs/chunk id */
+	if (!fs_info->new_fsid && !fs_info->new_chunk_tree_uuid)
+		return 0;
+	if (fs_info->new_fsid)
+		same_fsid = !memcmp_extent_buffer(eb, fs_info->new_fsid,
+					  btrfs_header_fsid(), BTRFS_FSID_SIZE);
+	if (fs_info->new_chunk_tree_uuid)
+		same_chunk_tree_uuid =
+			!memcmp_extent_buffer(eb, fs_info->new_chunk_tree_uuid,
+					      btrfs_header_chunk_tree_uuid(eb),
+					      BTRFS_UUID_SIZE);
+	if (same_fsid && same_chunk_tree_uuid)
+		return 0;
+	if (!same_fsid)
+		write_extent_buffer(eb, fs_info->new_fsid, btrfs_header_fsid(),
+				    BTRFS_FSID_SIZE);
+	if (!same_chunk_tree_uuid)
+		write_extent_buffer(eb, fs_info->new_chunk_tree_uuid,
+				    btrfs_header_chunk_tree_uuid(eb),
+				    BTRFS_UUID_SIZE);
+	ret = write_tree_block(NULL, root, eb);
+	return ret;
+}
 
 const struct prop_handler prop_handlers[] = {
 	{"ro", "Set/get read-only flag of subvolume.", 0, prop_object_subvol,
-- 
2.4.0

[PATCH v2 07/13] btrfs-progs: Introduce change_extents_uuid() function.

[Qu Wenruo]

This is the function which iterates all metadata extent and change their
fsid.

This function also does it without transaction.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
v2:
  Move to props.c
---
 props.c | 68 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 68 insertions(+)

diff --git a/props.c b/props.c
index abc3ae0..33fc39b 100644
--- a/props.c
+++ b/props.c
@@ -219,6 +219,74 @@ static int change_header_uuid(struct btrfs_root *root, struct extent_buffer *eb)
 	return ret;
 }
 
+static int change_extents_uuid(struct btrfs_fs_info *fs_info)
+{
+	struct btrfs_root *root = fs_info->extent_root;
+	struct btrfs_path *path;
+	struct btrfs_key key = {0, 0, 0};
+	int ret = 0;
+
+	if (!fs_info->new_fsid && !fs_info->new_chunk_tree_uuid)
+		return 0;
+
+	path = btrfs_alloc_path();
+	if (!path)
+		return -ENOMEM;
+
+	/*
+	 * Here we don't use transaction as it will takes a lot of reserve
+	 * space, and that will make a near-full btrfs unable to change uuid
+	 */
+	ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
+	if (ret < 0)
+		goto out;
+
+	while (1) {
+		struct btrfs_extent_item *ei;
+		struct extent_buffer *eb;
+		u64 flags;
+		u64 bytenr;
+
+		btrfs_item_key_to_cpu(path->nodes[0], &key, path->slots[0]);
+		if (key.type != BTRFS_EXTENT_ITEM_KEY &&
+		    key.type != BTRFS_METADATA_ITEM_KEY)
+			goto next;
+		ei = btrfs_item_ptr(path->nodes[0], path->slots[0],
+				    struct btrfs_extent_item);
+		flags = btrfs_extent_flags(path->nodes[0], ei);
+		if (!(flags & BTRFS_EXTENT_FLAG_TREE_BLOCK))
+			goto next;
+
+		bytenr = key.objectid;
+		eb = read_tree_block(root, bytenr, root->nodesize, 0);
+		if (IS_ERR(eb)) {
+			fprintf(stderr, "Failed to read tree block: %llu\n",
+				bytenr);
+			ret = PTR_ERR(eb);
+			goto out;
+		}
+		ret = change_header_uuid(root, eb);
+		free_extent_buffer(eb);
+		if (ret < 0) {
+			fprintf(stderr, "Failed to change uuid of tree block: %llu\n",
+				bytenr);
+			goto out;
+		}
+next:
+		ret = btrfs_next_item(root, path);
+		if (ret < 0)
+			goto out;
+		if (ret > 0) {
+			ret = 0;
+			goto out;
+		}
+	}
+
+out:
+	btrfs_free_path(path);
+	return ret;
+}
+
 const struct prop_handler prop_handlers[] = {
 	{"ro", "Set/get read-only flag of subvolume.", 0, prop_object_subvol,
 	 prop_read_only},
-- 
2.4.0

[PATCH v2 08/13] btrfs-progs: Introduce change_device_uuid() function.

[Qu Wenruo]

This function just change a device item's uuid.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
v2:
  Move to props.c
---
 props.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/props.c b/props.c
index 33fc39b..6cd9b4d 100644
--- a/props.c
+++ b/props.c
@@ -287,6 +287,27 @@ out:
 	return ret;
 }
 
+static int change_device_uuid(struct btrfs_root *root, struct extent_buffer *eb,
+			      int slot)
+{
+	struct btrfs_fs_info *fs_info = root->fs_info;
+	struct btrfs_dev_item *di;
+	int ret = 0;
+
+	di = btrfs_item_ptr(eb, slot, struct btrfs_dev_item);
+	if (fs_info->new_fsid) {
+		if (!memcmp_extent_buffer(eb, fs_info->new_fsid,
+					  (unsigned long)btrfs_device_fsid(di),
+					  BTRFS_FSID_SIZE))
+			return ret;
+		write_extent_buffer(eb, fs_info->new_fsid,
+				    (unsigned long)btrfs_device_fsid(di),
+				    BTRFS_FSID_SIZE);
+		ret = write_tree_block(NULL, root, eb);
+	}
+	return ret;
+}
+
 const struct prop_handler prop_handlers[] = {
 	{"ro", "Set/get read-only flag of subvolume.", 0, prop_object_subvol,
 	 prop_read_only},
-- 
2.4.0

[PATCH v2 09/13] btrfs-progs: Introduce change_devices_uuid() function.

[Qu Wenruo]

This function will change all dev items' fsid.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
v2:
  Move to props.c
---
 props.c | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/props.c b/props.c
index 6cd9b4d..59e09b3 100644
--- a/props.c
+++ b/props.c
@@ -308,6 +308,49 @@ static int change_device_uuid(struct btrfs_root *root, struct extent_buffer *eb,
 	return ret;
 }
 
+static int change_devices_uuid(struct btrfs_fs_info *fs_info)
+{
+	struct btrfs_root *root = fs_info->chunk_root;
+	struct btrfs_path *path;
+	struct btrfs_key key = {0, 0, 0};
+	int ret = 0;
+
+	/*
+	 * Unlike change_extents_uuid, we only need to change fsid in dev_item
+	 */
+	if (!fs_info->new_fsid)
+		return 0;
+
+	path = btrfs_alloc_path();
+	if (!path)
+		return -ENOMEM;
+	/* No transaction again */
+	ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
+	if (ret < 0)
+		goto out;
+
+	while (1) {
+		btrfs_item_key_to_cpu(path->nodes[0], &key, path->slots[0]);
+		if (key.type != BTRFS_DEV_ITEM_KEY ||
+		    key.objectid != BTRFS_DEV_ITEMS_OBJECTID)
+			goto next;
+		ret = change_device_uuid(root, path->nodes[0], path->slots[0]);
+		if (ret < 0)
+			goto out;
+next:
+		ret = btrfs_next_item(root, path);
+		if (ret < 0)
+			goto out;
+		if (ret > 0) {
+			ret = 0;
+			goto out;
+		}
+	}
+out:
+	btrfs_free_path(path);
+	return ret;
+}
+
 const struct prop_handler prop_handlers[] = {
 	{"ro", "Set/get read-only flag of subvolume.", 0, prop_object_subvol,
 	 prop_read_only},
-- 
2.4.0

[PATCH v2 10/13] btrfs-progs: Introduce change_id_prepare() and change_id_done() functions.

[Qu Wenruo]

These two functions will write flags to all supers before and after
fsid/chunk tree id change, informing kernel not to mount a inconsistent
fs.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
v2:
  Newly introduced to inform kernel and progs not to open fs with
  unfinished uuid change.
---
 props.c | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/props.c b/props.c
index 59e09b3..0a1bc1e 100644
--- a/props.c
+++ b/props.c
@@ -351,6 +351,38 @@ out:
 	return ret;
 }
 
+static int change_id_prepare(struct btrfs_fs_info *fs_info)
+{
+	u64 flags = btrfs_super_flags(fs_info->super_copy);
+
+	if (!fs_info->new_fsid && !fs_info->new_chunk_tree_uuid)
+		return 0;
+
+	if (fs_info->new_fsid)
+		flags |= BTRFS_SUPER_FLAG_CHANGING_FSID;
+	if (fs_info->new_chunk_tree_uuid)
+		flags |= BTRFS_SUPER_FLAG_CHANGING_CHUNK_TREE_ID;
+	btrfs_set_super_flags(fs_info->super_copy, flags);
+
+	return write_all_supers(fs_info->tree_root);
+}
+
+static int change_id_done(struct btrfs_fs_info *fs_info)
+{
+	u64 flags = btrfs_super_flags(fs_info->super_copy);
+
+	if (!fs_info->new_fsid && !fs_info->new_chunk_tree_uuid)
+		return 0;
+
+	if (fs_info->new_fsid)
+		flags &= ~BTRFS_SUPER_FLAG_CHANGING_FSID;
+	if (fs_info->new_chunk_tree_uuid)
+		flags &= ~BTRFS_SUPER_FLAG_CHANGING_CHUNK_TREE_ID;
+	btrfs_set_super_flags(fs_info->super_copy, flags);
+
+	return write_all_supers(fs_info->tree_root);
+}
+
 const struct prop_handler prop_handlers[] = {
 	{"ro", "Set/get read-only flag of subvolume.", 0, prop_object_subvol,
 	 prop_read_only},
-- 
2.4.0

[PATCH v2 11/13] btrfs-progs: Introduce change_uuid() function.

[Qu Wenruo]

This function does all the needed things for changing uuid.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
v2:
   Move to props.c
   Call change_id_prepare() and change_id_done().
---
 props.c | 68 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 68 insertions(+)

diff --git a/props.c b/props.c
index 0a1bc1e..9b2117c 100644
--- a/props.c
+++ b/props.c
@@ -20,12 +20,14 @@
 #include <sys/xattr.h>
 #include <fcntl.h>
 #include <unistd.h>
+#include <uuid/uuid.h>
 
 #include "ctree.h"
 #include "commands.h"
 #include "utils.h"
 #include "props.h"
 #include "disk-io.h"
+#include "volumes.h"
 
 #define XATTR_BTRFS_PREFIX     "btrfs."
 #define XATTR_BTRFS_PREFIX_LEN (sizeof(XATTR_BTRFS_PREFIX) - 1)
@@ -383,6 +385,72 @@ static int change_id_done(struct btrfs_fs_info *fs_info)
 	return write_all_supers(fs_info->tree_root);
 }
 
+static int change_uuid(struct btrfs_fs_info *fs_info, const char *new_fsid,
+		       const char *new_chunk_uuid)
+{
+	int ret = 0;
+
+	/* caller should do extra check on passed uuid */
+	if (new_fsid) {
+		/* allocated mem will be freed at close_ctree() */
+		fs_info->new_fsid = malloc(BTRFS_FSID_SIZE);
+		if (!fs_info->new_fsid) {
+			ret = -ENOMEM;
+			goto out;
+		}
+		ret = uuid_parse(new_fsid, fs_info->new_fsid);
+		if (ret < 0)
+			goto out;
+	}
+
+	if (new_chunk_uuid) {
+		/* allocated mem will be freed at close_ctree() */
+		fs_info->new_chunk_tree_uuid = malloc(BTRFS_UUID_SIZE);
+		if (!fs_info->new_chunk_tree_uuid) {
+			ret = -ENOMEM;
+			goto out;
+		}
+		ret = uuid_parse(new_chunk_uuid, fs_info->new_chunk_tree_uuid);
+		if (ret < 0)
+			goto out;
+	}
+
+	/* Now we can begin id change */
+	ret = change_id_prepare(fs_info);
+	if (ret < 0)
+		goto out;
+
+	/* Change extents first */
+	ret = change_extents_uuid(fs_info);
+	if (ret < 0) {
+		fprintf(stderr, "Failed to change UUID of metadata\n");
+		goto out;
+	}
+
+	/* Then devices */
+	ret = change_devices_uuid(fs_info);
+	if (ret < 0) {
+		fprintf(stderr, "Failed to change UUID of devices\n");
+		goto out;
+	}
+
+	/* Last, change fsid in super, only fsid change needs this */
+	if (new_fsid) {
+		memcpy(fs_info->fs_devices->fsid, fs_info->new_fsid,
+		       BTRFS_FSID_SIZE);
+		memcpy(fs_info->super_copy->fsid, fs_info->new_fsid,
+		       BTRFS_FSID_SIZE);
+		ret = write_all_supers(fs_info->tree_root);
+		if (ret < 0)
+			goto out;
+	}
+
+	/* Now id change is done */
+	ret = change_id_done(fs_info);
+out:
+	return ret;
+}
+
 const struct prop_handler prop_handlers[] = {
 	{"ro", "Set/get read-only flag of subvolume.", 0, prop_object_subvol,
 	 prop_read_only},
-- 
2.4.0

[PATCH v2 12/13] btrfs-progs: Add offline type for btrfs property.

[Qu Wenruo]

Add new offline type for btrfs property, it has two members:
1) fsid: Set/get fsid of an *OFFLINE* btrfs.
2) chunk_tree_uuid: Set/get chunk tree uuid of an *OFFLINE* btrfs

The new type is added to distinguish these dangerous offline operation
from the normal online operations.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
v2:
  Newly introduced.
---
 cmds-property.c |   5 ++-
 props.c         | 123 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 props.h         |   1 +
 3 files changed, 128 insertions(+), 1 deletion(-)

diff --git a/cmds-property.c b/cmds-property.c
index 6501338..6c699db 100644
--- a/cmds-property.c
+++ b/cmds-property.c
@@ -41,7 +41,7 @@ static const char * const cmd_get_usage[] = {
 	"an inode or a device. The '-t <type>' option can be used to explicitly",
 	"specify what type of object you meant. This is only needed when a",
 	"property could be set for more then one object type. Possible types",
-	"are s[ubvol], f[ilesystem], i[node] and d[evice].",
+	"are s[ubvol], f[ilesystem], i[node], d[evice] and o[ffline].",
 	NULL
 };
 
@@ -360,6 +360,9 @@ static void parse_args(int argc, char **argv,
 		} else if (!strcmp(type_str, "d") ||
 			   !strcmp(type_str, "device")) {
 			*types = prop_object_dev;
+		} else if (!strcmp(type_str, "o") ||
+			   !strcmp(type_str, "offline")) {
+			*types = prop_object_offline;
 		} else {
 			fprintf(stderr, "ERROR: invalid object type.\n");
 			usage(usage_str);
diff --git a/props.c b/props.c
index 9b2117c..73537bd 100644
--- a/props.c
+++ b/props.c
@@ -451,6 +451,125 @@ out:
 	return ret;
 }
 
+/* Will check if it is a block device/softlink/regular file, or mounted */
+static int check_offline(const char *device)
+{
+	struct stat statbuf;
+	int ret = 0;
+
+	ret = stat(device, &statbuf);
+	if (ret < 0) {
+		fprintf(stderr, "Failed to check status for %s: %s\n",
+			device, strerror(errno));
+		ret = -errno;
+		goto out;
+	}
+	if (!(S_ISBLK(statbuf.st_mode) || S_ISREG(statbuf.st_mode) ||
+	      S_ISLNK(statbuf.st_mode))) {
+		fprintf(stderr,
+			"%s is not a regular/device file containing btrfs\n",
+			device);
+		ret = -EINVAL;
+		goto out;
+	}
+	ret = check_mounted(device);
+	if (ret < 0) {
+		fprintf(stderr, "Could not check mount status: %s\n",
+			strerror(-ret));
+	} else if (ret > 0) {
+		fprintf(stderr, "%s is mounted\n", device);
+		ret = -EINVAL;
+	}
+out:
+	return ret;
+}
+
+static int prop_offline_uuid(enum prop_object_type type,
+			     const char *object,
+			     const char *name,
+			     const char *value)
+{
+	struct btrfs_fs_info *fs_info;
+	enum btrfs_open_ctree_flags ctree_flags = 0;
+	char *uuid_name = NULL;
+	int is_fsid;
+	int ret = 0;
+
+	ret = check_offline(object);
+	if (ret < 0)
+		goto out;
+
+	if (!strcmp(name, "fsid")) {
+		is_fsid = 1;
+		uuid_name = "fsid";
+	} else {
+		is_fsid = 0;
+		uuid_name = "chunk_tree_uuid";
+	}
+
+	if (value) {
+		uuid_t tmp;
+		char fsid[BTRFS_UUID_UNPARSED_SIZE];
+
+		ctree_flags |= OPEN_CTREE_WRITES;
+		if (is_fsid)
+			ctree_flags |= OPEN_CTREE_IGNORE_FSID;
+		else
+			ctree_flags |= OPEN_CTREE_IGNORE_CHUNK_TREE_ID;
+
+		ret = uuid_parse(value, tmp);
+		if (ret < 0) {
+			fprintf(stderr, "could not parse UUID: %s\n", value);
+			goto out;
+		}
+
+		/*
+		 * Copy value to fsid, to avoid warning about discarding
+		 * 'const' qualifier.
+		 * The root cause is from blkid library used by
+		 * test_uuid_unique, no good fix in btrfs.
+		 */
+		strcpy(fsid, value);
+
+		if (!test_uuid_unique(fsid)) {
+			fprintf(stderr, "non-unique UUID: %s\n", fsid);
+			ret = -EINVAL;
+			goto out;
+		}
+	}
+
+	fs_info = open_ctree_fs_info(object, 0, 0, ctree_flags);
+	if (!fs_info) {
+		ret = -EIO;
+		goto out;
+	}
+
+	/* For get, we just use fsid from fs_info */
+	if (!value) {
+		char buf[BTRFS_UUID_UNPARSED_SIZE] = {'\0'};
+
+		uuid_unparse(fs_info->fsid, buf);
+		fprintf(stdout, "%s=%s\n", uuid_name, buf);
+		goto close_ctree;
+	}
+
+	/* Here, we are going to change uuid*/
+	if (is_fsid)
+		ret = change_uuid(fs_info, value, NULL);
+	else
+		ret = change_uuid(fs_info, NULL, value);
+	if (ret < 0)
+		fprintf(stderr, "Failed to change %s: %s\n",
+			uuid_name, strerror(-ret));
+	else
+		printf("%s changed to %s\n", uuid_name, value);
+
+close_ctree:
+	close_ctree(fs_info->tree_root);
+out:
+	return ret;
+}
+
 const struct prop_handler prop_handlers[] = {
 	{"ro", "Set/get read-only flag of subvolume.", 0, prop_object_subvol,
 	 prop_read_only},
@@ -458,5 +577,9 @@ const struct prop_handler prop_handlers[] = {
 	 prop_object_dev | prop_object_root, prop_label},
 	{"compression", "Set/get compression for a file or directory", 0,
 	 prop_object_inode, prop_compression},
+	{"fsid", "Set/get fsid of a offline filesystem", 0,
+	 prop_object_offline, prop_offline_uuid},
+	{"chunk_tree_uuid", "Set/get fsid of a offline filesystem", 0,
+	 prop_object_offline, prop_offline_uuid},
 	{0, 0, 0, 0, 0}
 };
diff --git a/props.h b/props.h
index a43cb25..05c7761 100644
--- a/props.h
+++ b/props.h
@@ -22,6 +22,7 @@ enum prop_object_type {
 	prop_object_root	= (1 << 1),
 	prop_object_subvol	= (1 << 2),
 	prop_object_inode	= (1 << 3),
+	prop_object_offline	= (1 << 4),
 	__prop_object_max,
 };
 
-- 
2.4.0

[PATCH v2 13/13] btrfs-progs: Update Doc for btrfs-property

[Qu Wenruo]

Update offline related new function for btrfs-property.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
v2:
  Seperated from previous patch.
---
 Documentation/btrfs-property.asciidoc | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/Documentation/btrfs-property.asciidoc b/Documentation/btrfs-property.asciidoc
index 8b9b7f0..c7239ef 100644
--- a/Documentation/btrfs-property.asciidoc
+++ b/Documentation/btrfs-property.asciidoc
@@ -32,7 +32,8 @@ The '-t <type>' option can be used to explicitly
 specify what type of object you meant. This is only needed when a
 property could be set for more then one object type.
 +
-Possible types are 's[ubvol]', 'f[ilesystem]', 'i[node]' and 'd[evice]'.
+Possible types are 's[ubvol]', 'f[ilesystem]', 'i[node]', 'd[evice]' and
+'o[ffline]'.
 +
 Set the name of property by '<name>'. If no '<name>' is specified,
 all properties for the given object are printed. '<name>' is one of
@@ -44,6 +45,20 @@ label::::
 label of device
 compression::::
 compression setting for an inode: lzo, zlib, or "" (empty string)
+fsid::::
+fsid for an offline btrfs: 36 bytes UUID string in `printf`(3) format:
+"%08x-%04x-%04x-%04x-%012x"
+chunk_tree_uuid::::
+chunk tree uuid for an offline btrfs: 36 bytes UUID, same as 'fsid'.
+
+
+NOTE: To use 'fsid' and 'chunk_tree_uuid' in 'offline' type, '-t offline' must
+be specified explicitly, as most offline operation are dangerous and user
+should be aware of the danger.
+
+WARNING: If process of changing 'fsid' or 'chunk_tree_uuid' is canceled, the
+filesystem will not be mountable as it is in a inconsistent status. To make it
+mountable again, the uuid change needs to be reran and ensure it completes.
 
 *list* [-t <type>] <object>::
 Lists available properties with their descriptions for the given object.
-- 
2.4.0

Re: [PATCH v2 00/13] Introduce offline fsid/chunk tree uuid change for btrfstune.

[David Sterba]

On Mon, May 11, 2015 at 04:08:41PM +0800, Qu Wenruo wrote:
> This patchset will introduce offline fsid/chunk tree uuid change
> function to 'btrfs property' command, with "fsid" and "chunk_tree_uuid"
> respectively.

One more thing: the goal is to change the filesystem uuid, the chunk
tree uuid should be changed as well. But I think both uuids have to be
changed at the same time and the chunk tree uuid should be generated
randomly as it's entirely internal value.

> To avoid such problem, we introduce new btrfs super flags:
> BTRFS_SUPER_FLAG_CHANGING_FSID and BTRFS_SUPER_FLAG_CHANGING_CHUNK_TREE_ID

Based on that, the separate BTRFS_SUPER_FLAG_CHANGING_CHUNK_TREE_ID flag
does not make sense and could be dropped.

Re: [PATCH v2 00/13] Introduce offline fsid/chunk tree uuid change for btrfstune.

[David Sterba]

On Mon, May 11, 2015 at 04:08:41PM +0800, Qu Wenruo wrote:
> Qu Wenruo (13):
>   btrfs-progs: Add CHANGIND_FSID and CHANGING_CHUNK_TREE_ID super flags.
>   btrfs-progs: Use unified function to implement print_readable_*_flag()
>         function.
>   btrfs-progs: Make btrfs-show-super print human readable flag for super
>         flags.
>   btrfs-progs: Add open_ctree check for uuid changing.
>   btrfs-progs: Export write_tree_block().
>   btrfs-progs: Introduce change_header_uuid() function.
>   btrfs-progs: Introduce change_extents_uuid() function.
>   btrfs-progs: Introduce change_device_uuid() function.
>   btrfs-progs: Introduce change_devices_uuid() function.
>   btrfs-progs: Introduce change_id_prepare() and change_id_done()    
>     functions.
>   btrfs-progs: Introduce change_uuid() function.

All of the above merged (with some minor modifications), this covers
patches 1-7 from the first patchset. The btrfstune options are not
there, it needs some updates after I dropped the extra option to set the
chunk tree uuid.

The expected use:

* change the fsid to something that user specifies
* change the fsid but generate it randomly
* if the operation is interrupted, pick up the new fsid and continue
  automatically, ie. the user does not have to remember uuid (this is
  for convenience, the new fsid is stored in the superblock already and
  can be obtained by show-super)
* the new chunk tree uuid has to be stored in some block, eg. the root
  node of the tree_root; then picked from there when the uuid change
  process is restarted

I'll get to that after doing the 4.0.1 release.

Re: [PATCH v2 00/13] Introduce offline fsid/chunk tree uuid change for btrfstune.

[Qu Wenruo]

That's very kind of you to modify them by yourself.

I have already prepared branches for both property and btrfstune, but 
you did it even before I send the v3 patchset.

Would you mind me to do all the modification you mentioned and send the 
v3 patchset?
This should save some time for you.

Thanks,
Qu

-------- Original Message  --------
Subject: Re: [PATCH v2 00/13]  Introduce offline fsid/chunk tree uuid 
change for btrfstune.
From: David Sterba <dsterba@suse.cz>
To: Qu Wenruo <quwenruo@cn.fujitsu.com>
Date: 2015年05月14日 00:03

> On Mon, May 11, 2015 at 04:08:41PM +0800, Qu Wenruo wrote:
>> Qu Wenruo (13):
>>    btrfs-progs: Add CHANGIND_FSID and CHANGING_CHUNK_TREE_ID super flags.
>>    btrfs-progs: Use unified function to implement print_readable_*_flag()
>>          function.
>>    btrfs-progs: Make btrfs-show-super print human readable flag for super
>>          flags.
>>    btrfs-progs: Add open_ctree check for uuid changing.
>>    btrfs-progs: Export write_tree_block().
>>    btrfs-progs: Introduce change_header_uuid() function.
>>    btrfs-progs: Introduce change_extents_uuid() function.
>>    btrfs-progs: Introduce change_device_uuid() function.
>>    btrfs-progs: Introduce change_devices_uuid() function.
>>    btrfs-progs: Introduce change_id_prepare() and change_id_done()
>>      functions.
>>    btrfs-progs: Introduce change_uuid() function.
>
> All of the above merged (with some minor modifications), this covers
> patches 1-7 from the first patchset. The btrfstune options are not
> there, it needs some updates after I dropped the extra option to set the
> chunk tree uuid.
>
> The expected use:
>
> * change the fsid to something that user specifies
> * change the fsid but generate it randomly
> * if the operation is interrupted, pick up the new fsid and continue
>    automatically, ie. the user does not have to remember uuid (this is
>    for convenience, the new fsid is stored in the superblock already and
>    can be obtained by show-super)
> * the new chunk tree uuid has to be stored in some block, eg. the root
>    node of the tree_root; then picked from there when the uuid change
>    process is restarted
>
> I'll get to that after doing the 4.0.1 release.
>

Re: [PATCH v2 00/13] Introduce offline fsid/chunk tree uuid change for btrfstune.

[David Sterba]

On Thu, May 14, 2015 at 08:33:19AM +0800, Qu Wenruo wrote:
> That's very kind of you to modify them by yourself.
> 
> I have already prepared branches for both property and btrfstune, but 
> you did it even before I send the v3 patchset.

I'd like to do a release soon so I merge what seems ready even if it
does not take the usual route of resending.

> Would you mind me to do all the modification you mentioned and send the 
> v3 patchset?
> This should save some time for you.

The work is done already, please send new patches, based on top of the
current deveolopment branch.