[PATCH 0/6] btrfs-progs: btrfs-image related fixes

[PATCH 0/6] btrfs-progs: btrfs-image related fixes

[Qu Wenruo]

This branch can be fetched from github:
https://github.com/adam900710/btrfs-progs/tree/image_fixes

Since there are two binary files updates, and one big code move, it's
recommended to fetch github repo, in case some patches didn't reach mail
list.

This is inspried by one log tree replay dead loop bug, where the kind
reporter, Pierre Abbat <phma@bezitopo.org>, gave the btrfs-image to
reproduce it.

Then the image fails to pass check due to the existing log tree
conflicting with device/chunk fixup.
As log tree blocks are not recorded in extent tree, later COW can use
log tree blocks and cause transid mismatch.

To address the problem, this patchset will:
- Don't do any fixup if the source dump is single device
  Since the dump has the full super block contents, we can easily check
  if the source fs is single deivce.

  The chunk/device fixup is mostly for older btrfs-image behavior, which
  always restores the fs into SINGLE profile.
  However since commit 9088ab6a1067 ("btrfs-progs: make btrfs-image
  restore to support dup"), btrfs-image can restore into DUP profile,
  allowing us to do exact replay for single device fs.
  This is patch 5.

- Pin down all log tree blocks for fixup
  For cases we still need to fixup chunk/device items, at least pin down
  all log tree blocks to avoid transid mimsatch.
  This is patch 6.

After above fixes, fsck/012 and fsck/035 fails, due to bad original
images.
The old btrfs-image can fixup those bad device total_bytes and
bytes_used, but that just hides the problem.
We still need to update those images to make them correct, so here comes
patch 3 and 4.

During the debugging of btrfs-image restore, I found dump_superblock()
would help a lot to expose bad values in images, so is
btrfs_print_leaf().

Enahance them to be more handy for usage inside gdb, and here comes
patch 1 and 2.


Qu Wenruo (6):
  btrfs-progs: Allow btrfs_print_leaf() to be called on dummy eb whose
    fs_info is NULL
  btrfs-progs: print-tree: Export dump_superblock()
  btrfs-progs: fsck-tests: Update the image in 012
  btrfs-progs: fsck-tests: Update the image of test case 035
  btrfs-progs: image: Don't modify the chunk and device tree if  the
    source dump is single device
  btrfs-progs: image: Pin down log tree blocks before fixup

 cmds/inspect-dump-super.c                     | 454 +-----------------
 image/main.c                                  |  86 +++-
 print-tree.c                                  | 449 ++++++++++++++++-
 print-tree.h                                  |   1 +
 .../012-leaf-corruption/good.img.xz           | Bin 0 -> 186392 bytes
 .../012-leaf-corruption/no_data_extent.tar.xz | Bin 130260 -> 0 bytes
 tests/fsck-tests/012-leaf-corruption/test.sh  |  17 +-
 .../offset_by_one.img                         | Bin 3072 -> 3072 bytes
 8 files changed, 540 insertions(+), 467 deletions(-)
 create mode 100644 tests/fsck-tests/012-leaf-corruption/good.img.xz
 delete mode 100644 tests/fsck-tests/012-leaf-corruption/no_data_extent.tar.xz

-- 
2.26.2

[PATCH 1/6] btrfs-progs: Allow btrfs_print_leaf() to be called on dummy eb whose fs_info is NULL

[Qu Wenruo]

In certain call sites, like btrfs-image, we want full extent_buffer
accessors, but don't have a btrfs_fs_info.

In that case, if an extent buffer whose fs_info is NULL, then
btrfs_print_leaf() is called on such eb, we will trigger a NULL pointer
dereference, at BTRFS_LEAF_DATA_SIZE(fs_info).

Fix this by using __BTRFS_LEAF_DATA_SIZE(), which only needs nodesize,
and nodesize can be extracted from extent_buffer::len.

This allows us to call btrfs_print_leaf() on any extent buffer, which is
super handy in gdb.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 print-tree.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/print-tree.c b/print-tree.c
index 27acadb22205..38cf8d34ac2e 100644
--- a/print-tree.c
+++ b/print-tree.c
@@ -1201,7 +1201,7 @@ void btrfs_print_leaf(struct extent_buffer *eb)
 	struct btrfs_item *item;
 	struct btrfs_disk_key disk_key;
 	char flags_str[128];
-	u32 leaf_data_size = BTRFS_LEAF_DATA_SIZE(fs_info);
+	u32 leaf_data_size = __BTRFS_LEAF_DATA_SIZE(eb->len);
 	u32 i;
 	u32 nr;
 	u64 flags;
-- 
2.26.2

[PATCH 2/6] btrfs-progs: print-tree: Export dump_superblock()

[Qu Wenruo]

dump_superblock() is useful to debug btrfs-image error, like fsck/012-*
test case, where the superblock itself has something wrong from the
original image.

Export it so that we can call it in gdb.

Since we're exporting dump_superblock(), rename it to
btrfs_print_superblock() to following the existing naming schema.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 cmds/inspect-dump-super.c | 454 +-------------------------------------
 print-tree.c              | 447 +++++++++++++++++++++++++++++++++++++
 print-tree.h              |   1 +
 3 files changed, 449 insertions(+), 453 deletions(-)

diff --git a/cmds/inspect-dump-super.c b/cmds/inspect-dump-super.c
index f22633b99390..729af84cc93d 100644
--- a/cmds/inspect-dump-super.c
+++ b/cmds/inspect-dump-super.c
@@ -16,471 +16,19 @@
 
 #include "kerncompat.h"
 #include <stdio.h>
-#include <stdlib.h>
 #include <unistd.h>
 #include <fcntl.h>
 #include <sys/stat.h>
-#include <ctype.h>
-#include <uuid/uuid.h>
 #include <errno.h>
 #include <getopt.h>
 
 #include "ctree.h"
 #include "disk-io.h"
 #include "print-tree.h"
-#include "transaction.h"
-#include "kernel-lib/list.h"
 #include "common/utils.h"
 #include "cmds/commands.h"
-#include "crypto/crc32c.h"
 #include "common/help.h"
 
-static int check_csum_sblock(void *sb, int csum_size, u16 csum_type)
-{
-	u8 result[BTRFS_CSUM_SIZE];
-
-	btrfs_csum_data(csum_type, (u8 *)sb + BTRFS_CSUM_SIZE,
-			result, BTRFS_SUPER_INFO_SIZE - BTRFS_CSUM_SIZE);
-
-	return !memcmp(sb, result, csum_size);
-}
-
-static void print_sys_chunk_array(struct btrfs_super_block *sb)
-{
-	struct extent_buffer *buf;
-	struct btrfs_disk_key *disk_key;
-	struct btrfs_chunk *chunk;
-	u8 *array_ptr;
-	unsigned long sb_array_offset;
-	u32 num_stripes;
-	u32 array_size;
-	u32 len = 0;
-	u32 cur_offset;
-	struct btrfs_key key;
-	int item;
-
-	buf = malloc(sizeof(*buf) + sizeof(*sb));
-	if (!buf) {
-		error("not enough memory");
-		return;
-	}
-	write_extent_buffer(buf, sb, 0, sizeof(*sb));
-	buf->len = sizeof(*sb);
-	array_size = btrfs_super_sys_array_size(sb);
-
-	array_ptr = sb->sys_chunk_array;
-	sb_array_offset = offsetof(struct btrfs_super_block, sys_chunk_array);
-
-	if (array_size > BTRFS_SYSTEM_CHUNK_ARRAY_SIZE) {
-		error("sys_array_size %u shouldn't exceed %u bytes",
-				array_size, BTRFS_SYSTEM_CHUNK_ARRAY_SIZE);
-		goto out;
-	}
-
-	cur_offset = 0;
-	item = 0;
-
-	while (cur_offset < array_size) {
-		disk_key = (struct btrfs_disk_key *)array_ptr;
-		len = sizeof(*disk_key);
-		if (cur_offset + len > array_size)
-			goto out_short_read;
-
-		btrfs_disk_key_to_cpu(&key, disk_key);
-
-		array_ptr += len;
-		sb_array_offset += len;
-		cur_offset += len;
-
-		printf("\titem %d ", item);
-		btrfs_print_key(disk_key);
-		putchar('\n');
-
-		if (key.type == BTRFS_CHUNK_ITEM_KEY) {
-			chunk = (struct btrfs_chunk *)sb_array_offset;
-			/*
-			 * At least one btrfs_chunk with one stripe must be
-			 * present, exact stripe count check comes afterwards
-			 */
-			len = btrfs_chunk_item_size(1);
-			if (cur_offset + len > array_size)
-				goto out_short_read;
-
-			num_stripes = btrfs_chunk_num_stripes(buf, chunk);
-			if (!num_stripes) {
-				error(
-			"invalid number of stripes %u in sys_array at offset %u",
-					num_stripes, cur_offset);
-				break;
-			}
-			len = btrfs_chunk_item_size(num_stripes);
-			if (cur_offset + len > array_size)
-				goto out_short_read;
-			print_chunk_item(buf, chunk);
-		} else {
-			error("unexpected item type %u in sys_array at offset %u",
-				(u32)key.type, cur_offset);
-			break;
-		}
-		array_ptr += len;
-		sb_array_offset += len;
-		cur_offset += len;
-
-		item++;
-	}
-
-out:
-	free(buf);
-	return;
-
-out_short_read:
-	error("sys_array too short to read %u bytes at offset %u",
-			len, cur_offset);
-	free(buf);
-}
-
-static int empty_backup(struct btrfs_root_backup *backup)
-{
-	if (backup == NULL ||
-		(backup->tree_root == 0 &&
-		 backup->tree_root_gen == 0))
-		return 1;
-	return 0;
-}
-
-static void print_root_backup(struct btrfs_root_backup *backup)
-{
-	printf("\t\tbackup_tree_root:\t%llu\tgen: %llu\tlevel: %d\n",
-			btrfs_backup_tree_root(backup),
-			btrfs_backup_tree_root_gen(backup),
-			btrfs_backup_tree_root_level(backup));
-	printf("\t\tbackup_chunk_root:\t%llu\tgen: %llu\tlevel: %d\n",
-			btrfs_backup_chunk_root(backup),
-			btrfs_backup_chunk_root_gen(backup),
-			btrfs_backup_chunk_root_level(backup));
-	printf("\t\tbackup_extent_root:\t%llu\tgen: %llu\tlevel: %d\n",
-			btrfs_backup_extent_root(backup),
-			btrfs_backup_extent_root_gen(backup),
-			btrfs_backup_extent_root_level(backup));
-	printf("\t\tbackup_fs_root:\t\t%llu\tgen: %llu\tlevel: %d\n",
-			btrfs_backup_fs_root(backup),
-			btrfs_backup_fs_root_gen(backup),
-			btrfs_backup_fs_root_level(backup));
-	printf("\t\tbackup_dev_root:\t%llu\tgen: %llu\tlevel: %d\n",
-			btrfs_backup_dev_root(backup),
-			btrfs_backup_dev_root_gen(backup),
-			btrfs_backup_dev_root_level(backup));
-	printf("\t\tbackup_csum_root:\t%llu\tgen: %llu\tlevel: %d\n",
-			btrfs_backup_csum_root(backup),
-			btrfs_backup_csum_root_gen(backup),
-			btrfs_backup_csum_root_level(backup));
-
-	printf("\t\tbackup_total_bytes:\t%llu\n",
-					btrfs_backup_total_bytes(backup));
-	printf("\t\tbackup_bytes_used:\t%llu\n",
-					btrfs_backup_bytes_used(backup));
-	printf("\t\tbackup_num_devices:\t%llu\n",
-					btrfs_backup_num_devices(backup));
-	putchar('\n');
-}
-
-static void print_backup_roots(struct btrfs_super_block *sb)
-{
-	struct btrfs_root_backup *backup;
-	int i;
-
-	for (i = 0; i < BTRFS_NUM_BACKUP_ROOTS; i++) {
-		backup = sb->super_roots + i;
-		if (!empty_backup(backup)) {
-			printf("\tbackup %d:\n", i);
-			print_root_backup(backup);
-		}
-	}
-}
-
-struct readable_flag_entry {
-	u64 bit;
-	char *output;
-};
-
-#define DEF_COMPAT_RO_FLAG_ENTRY(bit_name)		\
-	{BTRFS_FEATURE_COMPAT_RO_##bit_name, #bit_name}
-
-static struct readable_flag_entry compat_ro_flags_array[] = {
-	DEF_COMPAT_RO_FLAG_ENTRY(FREE_SPACE_TREE),
-	DEF_COMPAT_RO_FLAG_ENTRY(FREE_SPACE_TREE_VALID),
-};
-static const int compat_ro_flags_num = sizeof(compat_ro_flags_array) /
-				       sizeof(struct readable_flag_entry);
-
-#define DEF_INCOMPAT_FLAG_ENTRY(bit_name)		\
-	{BTRFS_FEATURE_INCOMPAT_##bit_name, #bit_name}
-
-static struct readable_flag_entry incompat_flags_array[] = {
-	DEF_INCOMPAT_FLAG_ENTRY(MIXED_BACKREF),
-	DEF_INCOMPAT_FLAG_ENTRY(DEFAULT_SUBVOL),
-	DEF_INCOMPAT_FLAG_ENTRY(MIXED_GROUPS),
-	DEF_INCOMPAT_FLAG_ENTRY(COMPRESS_LZO),
-	DEF_INCOMPAT_FLAG_ENTRY(COMPRESS_ZSTD),
-	DEF_INCOMPAT_FLAG_ENTRY(BIG_METADATA),
-	DEF_INCOMPAT_FLAG_ENTRY(EXTENDED_IREF),
-	DEF_INCOMPAT_FLAG_ENTRY(RAID56),
-	DEF_INCOMPAT_FLAG_ENTRY(SKINNY_METADATA),
-	DEF_INCOMPAT_FLAG_ENTRY(NO_HOLES),
-	DEF_INCOMPAT_FLAG_ENTRY(METADATA_UUID),
-	DEF_INCOMPAT_FLAG_ENTRY(RAID1C34),
-};
-static const int incompat_flags_num = sizeof(incompat_flags_array) /
-				      sizeof(struct readable_flag_entry);
-
-#define DEF_HEADER_FLAG_ENTRY(bit_name)			\
-	{BTRFS_HEADER_FLAG_##bit_name, #bit_name}
-#define DEF_SUPER_FLAG_ENTRY(bit_name)			\
-	{BTRFS_SUPER_FLAG_##bit_name, #bit_name}
-
-static struct readable_flag_entry super_flags_array[] = {
-	DEF_HEADER_FLAG_ENTRY(WRITTEN),
-	DEF_HEADER_FLAG_ENTRY(RELOC),
-	DEF_SUPER_FLAG_ENTRY(CHANGING_FSID),
-	DEF_SUPER_FLAG_ENTRY(SEEDING),
-	DEF_SUPER_FLAG_ENTRY(METADUMP),
-	DEF_SUPER_FLAG_ENTRY(METADUMP_V2)
-};
-static const int super_flags_num = ARRAY_SIZE(super_flags_array);
-
-#define BTRFS_SUPER_FLAG_SUPP	(BTRFS_HEADER_FLAG_WRITTEN |\
-				 BTRFS_HEADER_FLAG_RELOC |\
-				 BTRFS_SUPER_FLAG_CHANGING_FSID |\
-				 BTRFS_SUPER_FLAG_SEEDING |\
-				 BTRFS_SUPER_FLAG_METADUMP |\
-				 BTRFS_SUPER_FLAG_METADUMP_V2)
-
-static void __print_readable_flag(u64 flag, struct readable_flag_entry *array,
-				  int array_size, u64 supported_flags)
-{
-	int i;
-	int first = 1;
-	struct readable_flag_entry *entry;
-
-	if (!flag)
-		return;
-
-	printf("\t\t\t( ");
-	for (i = 0; i < array_size; i++) {
-		entry = array + i;
-		if (flag & entry->bit) {
-			if (first)
-				printf("%s ", entry->output);
-			else
-				printf("|\n\t\t\t  %s ", entry->output);
-			first = 0;
-		}
-	}
-	flag &= ~supported_flags;
-	if (flag) {
-		if (first)
-			printf("unknown flag: 0x%llx ", flag);
-		else
-			printf("|\n\t\t\t  unknown flag: 0x%llx ", flag);
-	}
-	printf(")\n");
-}
-
-static void print_readable_compat_ro_flag(u64 flag)
-{
-	/*
-	 * We know about the FREE_SPACE_TREE{,_VALID} bits, but we don't
-	 * actually support them yet.
-	 */
-	return __print_readable_flag(flag, compat_ro_flags_array,
-				     compat_ro_flags_num,
-				     BTRFS_FEATURE_COMPAT_RO_SUPP |
-				     BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE |
-				     BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE_VALID);
-}
-
-static void print_readable_incompat_flag(u64 flag)
-{
-	return __print_readable_flag(flag, incompat_flags_array,
-				     incompat_flags_num,
-				     BTRFS_FEATURE_INCOMPAT_SUPP);
-}
-
-static void print_readable_super_flag(u64 flag)
-{
-	return __print_readable_flag(flag, super_flags_array,
-				     super_flags_num, BTRFS_SUPER_FLAG_SUPP);
-}
-
-static bool is_valid_csum_type(u16 csum_type)
-{
-	switch (csum_type) {
-	case BTRFS_CSUM_TYPE_CRC32:
-	case BTRFS_CSUM_TYPE_XXHASH:
-	case BTRFS_CSUM_TYPE_SHA256:
-	case BTRFS_CSUM_TYPE_BLAKE2:
-		return true;
-	default:
-		return false;
-	}
-}
-
-static void dump_superblock(struct btrfs_super_block *sb, int full)
-{
-	int i;
-	char *s, buf[BTRFS_UUID_UNPARSED_SIZE];
-	u8 *p;
-	u32 csum_size;
-	u16 csum_type;
-	bool metadata_uuid_present = (btrfs_super_incompat_flags(sb) &
-		BTRFS_FEATURE_INCOMPAT_METADATA_UUID);
-	int cmp_res = 0;
-
-
-	csum_type = btrfs_super_csum_type(sb);
-	csum_size = BTRFS_CSUM_SIZE;
-	printf("csum_type\t\t%hu (", csum_type);
-	if (!is_valid_csum_type(csum_type)) {
-		printf("INVALID");
-	} else {
-		printf("%s", btrfs_super_csum_name(csum_type));
-		csum_size = btrfs_super_csum_size(sb);
-	}
-	printf(")\n");
-	printf("csum_size\t\t%llu\n", (unsigned long long)csum_size);
-
-	printf("csum\t\t\t0x");
-	for (i = 0, p = sb->csum; i < csum_size; i++)
-		printf("%02x", p[i]);
-	if (!is_valid_csum_type(csum_type))
-		printf(" [UNKNOWN CSUM TYPE OR SIZE]");
-	else if (check_csum_sblock(sb, csum_size, csum_type))
-		printf(" [match]");
-	else
-		printf(" [DON'T MATCH]");
-	putchar('\n');
-
-	printf("bytenr\t\t\t%llu\n",
-		(unsigned long long)btrfs_super_bytenr(sb));
-	printf("flags\t\t\t0x%llx\n",
-		(unsigned long long)btrfs_super_flags(sb));
-	print_readable_super_flag(btrfs_super_flags(sb));
-
-	printf("magic\t\t\t");
-	s = (char *) &sb->magic;
-	for (i = 0; i < 8; i++)
-		putchar(isprint(s[i]) ? s[i] : '.');
-	if (btrfs_super_magic(sb) == BTRFS_MAGIC)
-		printf(" [match]\n");
-	else
-		printf(" [DON'T MATCH]\n");
-
-	uuid_unparse(sb->fsid, buf);
-	printf("fsid\t\t\t%s\n", buf);
-	if (metadata_uuid_present) {
-		uuid_unparse(sb->metadata_uuid, buf);
-		printf("metadata_uuid\t\t%s\n", buf);
-	} else {
-		printf("metadata_uuid\t\t%s\n", buf);
-	}
-
-	printf("label\t\t\t");
-	s = sb->label;
-	for (i = 0; i < BTRFS_LABEL_SIZE && s[i]; i++)
-		putchar(isprint(s[i]) ? s[i] : '.');
-	putchar('\n');
-
-	printf("generation\t\t%llu\n",
-	       (unsigned long long)btrfs_super_generation(sb));
-	printf("root\t\t\t%llu\n", (unsigned long long)btrfs_super_root(sb));
-	printf("sys_array_size\t\t%llu\n",
-	       (unsigned long long)btrfs_super_sys_array_size(sb));
-	printf("chunk_root_generation\t%llu\n",
-	       (unsigned long long)btrfs_super_chunk_root_generation(sb));
-	printf("root_level\t\t%llu\n",
-	       (unsigned long long)btrfs_super_root_level(sb));
-	printf("chunk_root\t\t%llu\n",
-	       (unsigned long long)btrfs_super_chunk_root(sb));
-	printf("chunk_root_level\t%llu\n",
-	       (unsigned long long)btrfs_super_chunk_root_level(sb));
-	printf("log_root\t\t%llu\n",
-	       (unsigned long long)btrfs_super_log_root(sb));
-	printf("log_root_transid\t%llu\n",
-	       (unsigned long long)btrfs_super_log_root_transid(sb));
-	printf("log_root_level\t\t%llu\n",
-	       (unsigned long long)btrfs_super_log_root_level(sb));
-	printf("total_bytes\t\t%llu\n",
-	       (unsigned long long)btrfs_super_total_bytes(sb));
-	printf("bytes_used\t\t%llu\n",
-	       (unsigned long long)btrfs_super_bytes_used(sb));
-	printf("sectorsize\t\t%llu\n",
-	       (unsigned long long)btrfs_super_sectorsize(sb));
-	printf("nodesize\t\t%llu\n",
-	       (unsigned long long)btrfs_super_nodesize(sb));
-	printf("leafsize (deprecated)\t%u\n",
-	       le32_to_cpu(sb->__unused_leafsize));
-	printf("stripesize\t\t%llu\n",
-	       (unsigned long long)btrfs_super_stripesize(sb));
-	printf("root_dir\t\t%llu\n",
-	       (unsigned long long)btrfs_super_root_dir(sb));
-	printf("num_devices\t\t%llu\n",
-	       (unsigned long long)btrfs_super_num_devices(sb));
-	printf("compat_flags\t\t0x%llx\n",
-	       (unsigned long long)btrfs_super_compat_flags(sb));
-	printf("compat_ro_flags\t\t0x%llx\n",
-	       (unsigned long long)btrfs_super_compat_ro_flags(sb));
-	print_readable_compat_ro_flag(btrfs_super_compat_ro_flags(sb));
-	printf("incompat_flags\t\t0x%llx\n",
-	       (unsigned long long)btrfs_super_incompat_flags(sb));
-	print_readable_incompat_flag(btrfs_super_incompat_flags(sb));
-	printf("cache_generation\t%llu\n",
-	       (unsigned long long)btrfs_super_cache_generation(sb));
-	printf("uuid_tree_generation\t%llu\n",
-	       (unsigned long long)btrfs_super_uuid_tree_generation(sb));
-
-	uuid_unparse(sb->dev_item.uuid, buf);
-	printf("dev_item.uuid\t\t%s\n", buf);
-
-	uuid_unparse(sb->dev_item.fsid, buf);
-	if (metadata_uuid_present) {
-		cmp_res = !memcmp(sb->dev_item.fsid, sb->metadata_uuid,
-				 BTRFS_FSID_SIZE);
-	} else {
-		cmp_res = !memcmp(sb->dev_item.fsid, sb->fsid, BTRFS_FSID_SIZE);
-	}
-	printf("dev_item.fsid\t\t%s %s\n", buf,
-	       cmp_res ? "[match]" : "[DON'T MATCH]");
-
-	printf("dev_item.type\t\t%llu\n", (unsigned long long)
-	       btrfs_stack_device_type(&sb->dev_item));
-	printf("dev_item.total_bytes\t%llu\n", (unsigned long long)
-	       btrfs_stack_device_total_bytes(&sb->dev_item));
-	printf("dev_item.bytes_used\t%llu\n", (unsigned long long)
-	       btrfs_stack_device_bytes_used(&sb->dev_item));
-	printf("dev_item.io_align\t%u\n", (unsigned int)
-	       btrfs_stack_device_io_align(&sb->dev_item));
-	printf("dev_item.io_width\t%u\n", (unsigned int)
-	       btrfs_stack_device_io_width(&sb->dev_item));
-	printf("dev_item.sector_size\t%u\n", (unsigned int)
-	       btrfs_stack_device_sector_size(&sb->dev_item));
-	printf("dev_item.devid\t\t%llu\n",
-	       btrfs_stack_device_id(&sb->dev_item));
-	printf("dev_item.dev_group\t%u\n", (unsigned int)
-	       btrfs_stack_device_group(&sb->dev_item));
-	printf("dev_item.seek_speed\t%u\n", (unsigned int)
-	       btrfs_stack_device_seek_speed(&sb->dev_item));
-	printf("dev_item.bandwidth\t%u\n", (unsigned int)
-	       btrfs_stack_device_bandwidth(&sb->dev_item));
-	printf("dev_item.generation\t%llu\n", (unsigned long long)
-	       btrfs_stack_device_generation(&sb->dev_item));
-	if (full) {
-		printf("sys_chunk_array[%d]:\n", BTRFS_SYSTEM_CHUNK_ARRAY_SIZE);
-		print_sys_chunk_array(sb);
-		printf("backup_roots[%d]:\n", BTRFS_NUM_BACKUP_ROOTS);
-		print_backup_roots(sb);
-	}
-}
-
 static int load_and_dump_sb(char *filename, int fd, u64 sb_bytenr, int full,
 		int force)
 {
@@ -507,7 +55,7 @@ static int load_and_dump_sb(char *filename, int fd, u64 sb_bytenr, int full,
 		error("bad magic on superblock on %s at %llu",
 				filename, (unsigned long long)sb_bytenr);
 	} else {
-		dump_superblock(sb, full);
+		btrfs_print_superblock(sb, full);
 	}
 	return 0;
 }
diff --git a/print-tree.c b/print-tree.c
index 38cf8d34ac2e..6fa3440b813c 100644
--- a/print-tree.c
+++ b/print-tree.c
@@ -19,6 +19,7 @@
 #include <stdio.h>
 #include <stdlib.h>
 #include <uuid/uuid.h>
+#include <ctype.h>
 #include "kerncompat.h"
 #include "kernel-lib/radix-tree.h"
 #include "ctree.h"
@@ -1550,3 +1551,449 @@ void btrfs_print_tree(struct extent_buffer *eb, bool follow, int traverse)
 		bfs_print_children(eb);
 	return;
 }
+
+static bool is_valid_csum_type(u16 csum_type)
+{
+	switch (csum_type) {
+	case BTRFS_CSUM_TYPE_CRC32:
+	case BTRFS_CSUM_TYPE_XXHASH:
+	case BTRFS_CSUM_TYPE_SHA256:
+	case BTRFS_CSUM_TYPE_BLAKE2:
+		return true;
+	default:
+		return false;
+	}
+}
+
+static int check_csum_sblock(void *sb, int csum_size, u16 csum_type)
+{
+	u8 result[BTRFS_CSUM_SIZE];
+
+	btrfs_csum_data(csum_type, (u8 *)sb + BTRFS_CSUM_SIZE,
+			result, BTRFS_SUPER_INFO_SIZE - BTRFS_CSUM_SIZE);
+
+	return !memcmp(sb, result, csum_size);
+}
+
+struct readable_flag_entry {
+	u64 bit;
+	char *output;
+};
+
+#define DEF_COMPAT_RO_FLAG_ENTRY(bit_name)		\
+	{BTRFS_FEATURE_COMPAT_RO_##bit_name, #bit_name}
+
+static struct readable_flag_entry compat_ro_flags_array[] = {
+	DEF_COMPAT_RO_FLAG_ENTRY(FREE_SPACE_TREE),
+	DEF_COMPAT_RO_FLAG_ENTRY(FREE_SPACE_TREE_VALID),
+};
+static const int compat_ro_flags_num = sizeof(compat_ro_flags_array) /
+				       sizeof(struct readable_flag_entry);
+
+#define DEF_INCOMPAT_FLAG_ENTRY(bit_name)		\
+	{BTRFS_FEATURE_INCOMPAT_##bit_name, #bit_name}
+
+static struct readable_flag_entry incompat_flags_array[] = {
+	DEF_INCOMPAT_FLAG_ENTRY(MIXED_BACKREF),
+	DEF_INCOMPAT_FLAG_ENTRY(DEFAULT_SUBVOL),
+	DEF_INCOMPAT_FLAG_ENTRY(MIXED_GROUPS),
+	DEF_INCOMPAT_FLAG_ENTRY(COMPRESS_LZO),
+	DEF_INCOMPAT_FLAG_ENTRY(COMPRESS_ZSTD),
+	DEF_INCOMPAT_FLAG_ENTRY(BIG_METADATA),
+	DEF_INCOMPAT_FLAG_ENTRY(EXTENDED_IREF),
+	DEF_INCOMPAT_FLAG_ENTRY(RAID56),
+	DEF_INCOMPAT_FLAG_ENTRY(SKINNY_METADATA),
+	DEF_INCOMPAT_FLAG_ENTRY(NO_HOLES),
+	DEF_INCOMPAT_FLAG_ENTRY(METADATA_UUID),
+	DEF_INCOMPAT_FLAG_ENTRY(RAID1C34),
+};
+static const int incompat_flags_num = sizeof(incompat_flags_array) /
+				      sizeof(struct readable_flag_entry);
+
+#define DEF_HEADER_FLAG_ENTRY(bit_name)			\
+	{BTRFS_HEADER_FLAG_##bit_name, #bit_name}
+#define DEF_SUPER_FLAG_ENTRY(bit_name)			\
+	{BTRFS_SUPER_FLAG_##bit_name, #bit_name}
+
+static struct readable_flag_entry super_flags_array[] = {
+	DEF_HEADER_FLAG_ENTRY(WRITTEN),
+	DEF_HEADER_FLAG_ENTRY(RELOC),
+	DEF_SUPER_FLAG_ENTRY(CHANGING_FSID),
+	DEF_SUPER_FLAG_ENTRY(SEEDING),
+	DEF_SUPER_FLAG_ENTRY(METADUMP),
+	DEF_SUPER_FLAG_ENTRY(METADUMP_V2)
+};
+static const int super_flags_num = ARRAY_SIZE(super_flags_array);
+
+#define BTRFS_SUPER_FLAG_SUPP	(BTRFS_HEADER_FLAG_WRITTEN |\
+				 BTRFS_HEADER_FLAG_RELOC |\
+				 BTRFS_SUPER_FLAG_CHANGING_FSID |\
+				 BTRFS_SUPER_FLAG_SEEDING |\
+				 BTRFS_SUPER_FLAG_METADUMP |\
+				 BTRFS_SUPER_FLAG_METADUMP_V2)
+
+static void __print_readable_flag(u64 flag, struct readable_flag_entry *array,
+				  int array_size, u64 supported_flags)
+{
+	int i;
+	int first = 1;
+	struct readable_flag_entry *entry;
+
+	if (!flag)
+		return;
+
+	printf("\t\t\t( ");
+	for (i = 0; i < array_size; i++) {
+		entry = array + i;
+		if (flag & entry->bit) {
+			if (first)
+				printf("%s ", entry->output);
+			else
+				printf("|\n\t\t\t  %s ", entry->output);
+			first = 0;
+		}
+	}
+	flag &= ~supported_flags;
+	if (flag) {
+		if (first)
+			printf("unknown flag: 0x%llx ", flag);
+		else
+			printf("|\n\t\t\t  unknown flag: 0x%llx ", flag);
+	}
+	printf(")\n");
+}
+
+static void print_readable_compat_ro_flag(u64 flag)
+{
+	/*
+	 * We know about the FREE_SPACE_TREE{,_VALID} bits, but we don't
+	 * actually support them yet.
+	 */
+	return __print_readable_flag(flag, compat_ro_flags_array,
+				     compat_ro_flags_num,
+				     BTRFS_FEATURE_COMPAT_RO_SUPP |
+				     BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE |
+				     BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE_VALID);
+}
+
+static void print_readable_incompat_flag(u64 flag)
+{
+	return __print_readable_flag(flag, incompat_flags_array,
+				     incompat_flags_num,
+				     BTRFS_FEATURE_INCOMPAT_SUPP);
+}
+
+static void print_readable_super_flag(u64 flag)
+{
+	return __print_readable_flag(flag, super_flags_array,
+				     super_flags_num, BTRFS_SUPER_FLAG_SUPP);
+}
+
+static void print_sys_chunk_array(struct btrfs_super_block *sb)
+{
+	struct extent_buffer *buf;
+	struct btrfs_disk_key *disk_key;
+	struct btrfs_chunk *chunk;
+	u8 *array_ptr;
+	unsigned long sb_array_offset;
+	u32 num_stripes;
+	u32 array_size;
+	u32 len = 0;
+	u32 cur_offset;
+	struct btrfs_key key;
+	int item;
+
+	buf = malloc(sizeof(*buf) + sizeof(*sb));
+	if (!buf) {
+		error("not enough memory");
+		return;
+	}
+	write_extent_buffer(buf, sb, 0, sizeof(*sb));
+	buf->len = sizeof(*sb);
+	array_size = btrfs_super_sys_array_size(sb);
+
+	array_ptr = sb->sys_chunk_array;
+	sb_array_offset = offsetof(struct btrfs_super_block, sys_chunk_array);
+
+	if (array_size > BTRFS_SYSTEM_CHUNK_ARRAY_SIZE) {
+		error("sys_array_size %u shouldn't exceed %u bytes",
+				array_size, BTRFS_SYSTEM_CHUNK_ARRAY_SIZE);
+		goto out;
+	}
+
+	cur_offset = 0;
+	item = 0;
+
+	while (cur_offset < array_size) {
+		disk_key = (struct btrfs_disk_key *)array_ptr;
+		len = sizeof(*disk_key);
+		if (cur_offset + len > array_size)
+			goto out_short_read;
+
+		btrfs_disk_key_to_cpu(&key, disk_key);
+
+		array_ptr += len;
+		sb_array_offset += len;
+		cur_offset += len;
+
+		printf("\titem %d ", item);
+		btrfs_print_key(disk_key);
+		putchar('\n');
+
+		if (key.type == BTRFS_CHUNK_ITEM_KEY) {
+			chunk = (struct btrfs_chunk *)sb_array_offset;
+			/*
+			 * At least one btrfs_chunk with one stripe must be
+			 * present, exact stripe count check comes afterwards
+			 */
+			len = btrfs_chunk_item_size(1);
+			if (cur_offset + len > array_size)
+				goto out_short_read;
+
+			num_stripes = btrfs_chunk_num_stripes(buf, chunk);
+			if (!num_stripes) {
+				error(
+			"invalid number of stripes %u in sys_array at offset %u",
+					num_stripes, cur_offset);
+				break;
+			}
+			len = btrfs_chunk_item_size(num_stripes);
+			if (cur_offset + len > array_size)
+				goto out_short_read;
+			print_chunk_item(buf, chunk);
+		} else {
+			error("unexpected item type %u in sys_array at offset %u",
+				(u32)key.type, cur_offset);
+			break;
+		}
+		array_ptr += len;
+		sb_array_offset += len;
+		cur_offset += len;
+
+		item++;
+	}
+
+out:
+	free(buf);
+	return;
+
+out_short_read:
+	error("sys_array too short to read %u bytes at offset %u",
+			len, cur_offset);
+	free(buf);
+}
+
+static int empty_backup(struct btrfs_root_backup *backup)
+{
+	if (backup == NULL ||
+		(backup->tree_root == 0 &&
+		 backup->tree_root_gen == 0))
+		return 1;
+	return 0;
+}
+
+static void print_root_backup(struct btrfs_root_backup *backup)
+{
+	printf("\t\tbackup_tree_root:\t%llu\tgen: %llu\tlevel: %d\n",
+			btrfs_backup_tree_root(backup),
+			btrfs_backup_tree_root_gen(backup),
+			btrfs_backup_tree_root_level(backup));
+	printf("\t\tbackup_chunk_root:\t%llu\tgen: %llu\tlevel: %d\n",
+			btrfs_backup_chunk_root(backup),
+			btrfs_backup_chunk_root_gen(backup),
+			btrfs_backup_chunk_root_level(backup));
+	printf("\t\tbackup_extent_root:\t%llu\tgen: %llu\tlevel: %d\n",
+			btrfs_backup_extent_root(backup),
+			btrfs_backup_extent_root_gen(backup),
+			btrfs_backup_extent_root_level(backup));
+	printf("\t\tbackup_fs_root:\t\t%llu\tgen: %llu\tlevel: %d\n",
+			btrfs_backup_fs_root(backup),
+			btrfs_backup_fs_root_gen(backup),
+			btrfs_backup_fs_root_level(backup));
+	printf("\t\tbackup_dev_root:\t%llu\tgen: %llu\tlevel: %d\n",
+			btrfs_backup_dev_root(backup),
+			btrfs_backup_dev_root_gen(backup),
+			btrfs_backup_dev_root_level(backup));
+	printf("\t\tbackup_csum_root:\t%llu\tgen: %llu\tlevel: %d\n",
+			btrfs_backup_csum_root(backup),
+			btrfs_backup_csum_root_gen(backup),
+			btrfs_backup_csum_root_level(backup));
+
+	printf("\t\tbackup_total_bytes:\t%llu\n",
+					btrfs_backup_total_bytes(backup));
+	printf("\t\tbackup_bytes_used:\t%llu\n",
+					btrfs_backup_bytes_used(backup));
+	printf("\t\tbackup_num_devices:\t%llu\n",
+					btrfs_backup_num_devices(backup));
+	putchar('\n');
+}
+
+static void print_backup_roots(struct btrfs_super_block *sb)
+{
+	struct btrfs_root_backup *backup;
+	int i;
+
+	for (i = 0; i < BTRFS_NUM_BACKUP_ROOTS; i++) {
+		backup = sb->super_roots + i;
+		if (!empty_backup(backup)) {
+			printf("\tbackup %d:\n", i);
+			print_root_backup(backup);
+		}
+	}
+}
+
+void btrfs_print_superblock(struct btrfs_super_block *sb, int full)
+{
+	int i;
+	char *s, buf[BTRFS_UUID_UNPARSED_SIZE];
+	u8 *p;
+	u32 csum_size;
+	u16 csum_type;
+	bool metadata_uuid_present = (btrfs_super_incompat_flags(sb) &
+		BTRFS_FEATURE_INCOMPAT_METADATA_UUID);
+	int cmp_res = 0;
+
+
+	csum_type = btrfs_super_csum_type(sb);
+	csum_size = BTRFS_CSUM_SIZE;
+	printf("csum_type\t\t%hu (", csum_type);
+	if (!is_valid_csum_type(csum_type)) {
+		printf("INVALID");
+	} else {
+		printf("%s", btrfs_super_csum_name(csum_type));
+		csum_size = btrfs_super_csum_size(sb);
+	}
+	printf(")\n");
+	printf("csum_size\t\t%llu\n", (unsigned long long)csum_size);
+
+	printf("csum\t\t\t0x");
+	for (i = 0, p = sb->csum; i < csum_size; i++)
+		printf("%02x", p[i]);
+	if (!is_valid_csum_type(csum_type))
+		printf(" [UNKNOWN CSUM TYPE OR SIZE]");
+	else if (check_csum_sblock(sb, csum_size, csum_type))
+		printf(" [match]");
+	else
+		printf(" [DON'T MATCH]");
+	putchar('\n');
+
+	printf("bytenr\t\t\t%llu\n",
+		(unsigned long long)btrfs_super_bytenr(sb));
+	printf("flags\t\t\t0x%llx\n",
+		(unsigned long long)btrfs_super_flags(sb));
+	print_readable_super_flag(btrfs_super_flags(sb));
+
+	printf("magic\t\t\t");
+	s = (char *) &sb->magic;
+	for (i = 0; i < 8; i++)
+		putchar(isprint(s[i]) ? s[i] : '.');
+	if (btrfs_super_magic(sb) == BTRFS_MAGIC)
+		printf(" [match]\n");
+	else
+		printf(" [DON'T MATCH]\n");
+
+	uuid_unparse(sb->fsid, buf);
+	printf("fsid\t\t\t%s\n", buf);
+	if (metadata_uuid_present) {
+		uuid_unparse(sb->metadata_uuid, buf);
+		printf("metadata_uuid\t\t%s\n", buf);
+	} else {
+		printf("metadata_uuid\t\t%s\n", buf);
+	}
+
+	printf("label\t\t\t");
+	s = sb->label;
+	for (i = 0; i < BTRFS_LABEL_SIZE && s[i]; i++)
+		putchar(isprint(s[i]) ? s[i] : '.');
+	putchar('\n');
+
+	printf("generation\t\t%llu\n",
+	       (unsigned long long)btrfs_super_generation(sb));
+	printf("root\t\t\t%llu\n", (unsigned long long)btrfs_super_root(sb));
+	printf("sys_array_size\t\t%llu\n",
+	       (unsigned long long)btrfs_super_sys_array_size(sb));
+	printf("chunk_root_generation\t%llu\n",
+	       (unsigned long long)btrfs_super_chunk_root_generation(sb));
+	printf("root_level\t\t%llu\n",
+	       (unsigned long long)btrfs_super_root_level(sb));
+	printf("chunk_root\t\t%llu\n",
+	       (unsigned long long)btrfs_super_chunk_root(sb));
+	printf("chunk_root_level\t%llu\n",
+	       (unsigned long long)btrfs_super_chunk_root_level(sb));
+	printf("log_root\t\t%llu\n",
+	       (unsigned long long)btrfs_super_log_root(sb));
+	printf("log_root_transid\t%llu\n",
+	       (unsigned long long)btrfs_super_log_root_transid(sb));
+	printf("log_root_level\t\t%llu\n",
+	       (unsigned long long)btrfs_super_log_root_level(sb));
+	printf("total_bytes\t\t%llu\n",
+	       (unsigned long long)btrfs_super_total_bytes(sb));
+	printf("bytes_used\t\t%llu\n",
+	       (unsigned long long)btrfs_super_bytes_used(sb));
+	printf("sectorsize\t\t%llu\n",
+	       (unsigned long long)btrfs_super_sectorsize(sb));
+	printf("nodesize\t\t%llu\n",
+	       (unsigned long long)btrfs_super_nodesize(sb));
+	printf("leafsize (deprecated)\t%u\n",
+	       le32_to_cpu(sb->__unused_leafsize));
+	printf("stripesize\t\t%llu\n",
+	       (unsigned long long)btrfs_super_stripesize(sb));
+	printf("root_dir\t\t%llu\n",
+	       (unsigned long long)btrfs_super_root_dir(sb));
+	printf("num_devices\t\t%llu\n",
+	       (unsigned long long)btrfs_super_num_devices(sb));
+	printf("compat_flags\t\t0x%llx\n",
+	       (unsigned long long)btrfs_super_compat_flags(sb));
+	printf("compat_ro_flags\t\t0x%llx\n",
+	       (unsigned long long)btrfs_super_compat_ro_flags(sb));
+	print_readable_compat_ro_flag(btrfs_super_compat_ro_flags(sb));
+	printf("incompat_flags\t\t0x%llx\n",
+	       (unsigned long long)btrfs_super_incompat_flags(sb));
+	print_readable_incompat_flag(btrfs_super_incompat_flags(sb));
+	printf("cache_generation\t%llu\n",
+	       (unsigned long long)btrfs_super_cache_generation(sb));
+	printf("uuid_tree_generation\t%llu\n",
+	       (unsigned long long)btrfs_super_uuid_tree_generation(sb));
+
+	uuid_unparse(sb->dev_item.uuid, buf);
+	printf("dev_item.uuid\t\t%s\n", buf);
+
+	uuid_unparse(sb->dev_item.fsid, buf);
+	if (metadata_uuid_present) {
+		cmp_res = !memcmp(sb->dev_item.fsid, sb->metadata_uuid,
+				 BTRFS_FSID_SIZE);
+	} else {
+		cmp_res = !memcmp(sb->dev_item.fsid, sb->fsid, BTRFS_FSID_SIZE);
+	}
+	printf("dev_item.fsid\t\t%s %s\n", buf,
+	       cmp_res ? "[match]" : "[DON'T MATCH]");
+
+	printf("dev_item.type\t\t%llu\n", (unsigned long long)
+	       btrfs_stack_device_type(&sb->dev_item));
+	printf("dev_item.total_bytes\t%llu\n", (unsigned long long)
+	       btrfs_stack_device_total_bytes(&sb->dev_item));
+	printf("dev_item.bytes_used\t%llu\n", (unsigned long long)
+	       btrfs_stack_device_bytes_used(&sb->dev_item));
+	printf("dev_item.io_align\t%u\n", (unsigned int)
+	       btrfs_stack_device_io_align(&sb->dev_item));
+	printf("dev_item.io_width\t%u\n", (unsigned int)
+	       btrfs_stack_device_io_width(&sb->dev_item));
+	printf("dev_item.sector_size\t%u\n", (unsigned int)
+	       btrfs_stack_device_sector_size(&sb->dev_item));
+	printf("dev_item.devid\t\t%llu\n",
+	       btrfs_stack_device_id(&sb->dev_item));
+	printf("dev_item.dev_group\t%u\n", (unsigned int)
+	       btrfs_stack_device_group(&sb->dev_item));
+	printf("dev_item.seek_speed\t%u\n", (unsigned int)
+	       btrfs_stack_device_seek_speed(&sb->dev_item));
+	printf("dev_item.bandwidth\t%u\n", (unsigned int)
+	       btrfs_stack_device_bandwidth(&sb->dev_item));
+	printf("dev_item.generation\t%llu\n", (unsigned long long)
+	       btrfs_stack_device_generation(&sb->dev_item));
+	if (full) {
+		printf("sys_chunk_array[%d]:\n", BTRFS_SYSTEM_CHUNK_ARRAY_SIZE);
+		print_sys_chunk_array(sb);
+		printf("backup_roots[%d]:\n", BTRFS_NUM_BACKUP_ROOTS);
+		print_backup_roots(sb);
+	}
+}
diff --git a/print-tree.h b/print-tree.h
index 92ed5fb7c270..0ce8d92fa305 100644
--- a/print-tree.h
+++ b/print-tree.h
@@ -39,4 +39,5 @@ void print_chunk_item(struct extent_buffer *eb, struct btrfs_chunk *chunk);
 void print_extent_item(struct extent_buffer *eb, int slot, int metadata);
 void print_objectid(FILE *stream, u64 objectid, u8 type);
 void print_key_type(FILE *stream, u64 objectid, u8 type);
+void btrfs_print_superblock(struct btrfs_super_block *sb, int full);
 #endif
-- 
2.26.2

[PATCH 4/6] btrfs-progs: fsck-tests: Update the image of test case 035

[Qu Wenruo]

The image used in test case 035 has one problem:
- Bad total_bytes of device item in chunk tree
        item 0 key (DEV_ITEMS DEV_ITEM 1) itemoff 16185 itemsize 98
                devid 1 total_bytes 72564736 bytes_used 92274688
  The total_bytes is even smaller than bytes_used.
  While the dev_item in super block contains the correct value:
    dev_item.total_bytes    134217728
    dev_item.bytes_used     92274688

This patch will update the image, using the correct value in super block
to replace the wrong one in chunk tree.

So that later btrfs-image restore without device/chunk fixup can still
pass fsck-tests.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 .../offset_by_one.img                         | Bin 3072 -> 3072 bytes
 1 file changed, 0 insertions(+), 0 deletions(-)

diff --git a/tests/fsck-tests/035-inline-bad-ram-bytes/offset_by_one.img b/tests/fsck-tests/035-inline-bad-ram-bytes/offset_by_one.img
index 2f58208eed71d5e0ee053d825d2780fe6842a3db..b3cef56ec9d73a6ea397c2084c219999552b7145 100644
GIT binary patch
delta 1642
zcmYjReK->c7^jtL+|Ae4?($hm7NLYIF(e9^#W<g#nr{<#vlH`C-9pMFjz%>ux>z{*
zW;lj?bdK13oFtQNW*cTUc2xH~{r-5rp7(j*_xC=}`zAq?p!d~{`7&F8BV@M~U|Yi9
ze6oAD1JuuUGGN>J=Tm&N(v<?hZ@F;{){mHxJXsK8dJU~i=9r^ttxYw+M}mRlMn)-}
z);jWvw9~kqCA(<i>m4R)b$eW=6{&;{hi~_E?mSF?oPz1xq1B8~xMH?LFH%kJ(1CF(
zA;21NbOH|TlJ_+8*}%rO<StoMud+DC5MZ94COt|0u12&>{$GVgMtJJb&j93wbEOEI
zcPCP^HB>Gr-<r2~!R!&qdw%#B&~^m6i+ou%BgNGSh-gc8%;<JTd8*`skKtwATV$Qj
zbX+yDEH3LyA&2xiRGe1VT7CKcIvS8Kf!C`r+^5gQ=$d)OwTr${TzRzoGjpp}2Utet
ztRAW$j=>qP3~h(2;b}Z+nQTe=n|%)H1(`_p(8ACYyKa~{p>am1u70m&>R20{QJMOt
z#}%X35E~P{q@+-Tg`FoT3Aww8jT78st-9u{_l??0?BW8-HOExL+8<tB>Zn&$Uumsy
zdch$32Aob1bC`>;nq2<u9OhUeRHd-L>eNMzBD;mcD+X1{Z?gNd3GYEp{%FlVcjzvg
z{G^cOx)1Mhhvb)EKS1IoaEtz=lP|I|Edvua>#BvL<L_*v=zl=MT?+TSwrfy+i^bQw
z*a;1WPbR>MT|{sDSA>EDqKWP3kS#-S7Px5~S0n}5O}G=c7!Op+%LYJcJSJhWH!(X9
zk>(_qy$F$)GbOcMG0Fc1qOd7LJnmQt)n^CyT4|9#@^|~(MA&Q3anZP<e+3rY@o{f3
zmT&EF=Fn^7YAdUeF71eykRY;;xHsa(us!WTb9dx84^u|$aghk)85_%K@nYWdY{+*F
zr;_cX9sE-rFh6kS|9<o#7G0ibO)*`~O@?iAy}^v{(KxTu#C>>|Sy~r5E|G$=evjNh
z7RhRp4(;8Vd(t^r&0S4q$!z>ecAl2%gE&NZdQJ+%woG%cX{4dX3!7Vh#Tu|Yct~mP
z0&XP5O^-KSNtm`iJB$Ruj#Q-4hK36*+zd6EhC!;o+&8&)LSR}QHQC=)9Js*6b~oYa
zQgI=j?YCK7WH`{Q{L7f>jR@{PLbjElr{+C3YCleJBe)OUKU!_^sCj9%-PXhMG)6QM
zUu*T#?BBYUY6cAVw4i9qR!OarO+x(=|Dg87E1y$&HM=r9;Rc|l{Y>5%@^bu?dwABp
z%4Z{iEzO;8$DU^GOeCfG`i6si2+4Ra<uxt2h4a&~3MNEqcvyxbCmB$_eWqW5I)|Di
z{Tdpbfty3`CIUkatkd<&jEE?x6HK8PTybo!0R{Dn_%>$*rKAh~{zvcrMidL@815NH
zsVBdd>&i^Did%f>C!Wlb`s?&#UrX=BAD>0-`qW_i1AInOi}QfG_dJ-}Ev2mFN;Qje
z?pr<9!p+0iVqiCGFHkZ%+YK-EtNfq-MUTOAUd9U_Mz(p0Lr^e(x^$2>fx9kQ^6K^H
zuH&ja@aiJz%9KcY(5|s<#@x+s@_W8CU0{oEN@Vh`y^|pEm+kd1#mLwdKc(&=<8$f8
zRk0bj60|3LfPGyMt4YoF@{UFCuX|3*xfx{srUR2%!>8Y200QRb*|8@@&lPWH2T6dg
zuu*?D`y?x7`ObmSjS_5P9{bru*14(t`Jo+Eb3hcOvWlN0k%l((c?nk}zMJ?~Jj;N&
ziuDevA1IS=jo^&<+i&8@*iC8h@qY_?YpFeBtb1acHS@4YPvtu9I}1fkO(Teqvswa=
zd72uM{;{XY?&M@?)M5TsQO3wK710T~8PSud*7cd#>ZLB^Wi(+>|H$2FQQM7-EE9|n
zfSX<jtN_PFMrnNE^kx6bAydvQGtaO|RyotmV>DEY1m^^bAB1g`ALR^OZLD~%o*R*P
zCbxW{YNL_+3c`JbxKON*+_%re#`X!Hz1(+VaIla-3BT=MHI*cOIn@H%AErZsC-@2j
z*o&ElgUzNbEM7;9Ch3%5cFn1oD-HB@0(-A-VCjcFaq*PeCy0rdg1Ml_ABgvorJFQr
a@VZTqZ_yOltoy21eufUFR_g<)vi|{B@h0B@

delta 1601
zcmYk5dpOez7{_B_5jsuGEp^l)g&mdC$j}8Pv(j9eOL9wMEO+@WB?)6FA(iXa8Ah1U
z+9sDyBqWKpTp}i$>@?2|J8}N#eE;}7@AF=s_j%v<c?nPgG)osgx?xSuGHV2{vv-}<
zvTL5cb^G2r->vgQEt&=beuiR(-$h#GbrTc}aLZ<?eNhkUCcL>Gay_wunouZvd`Gt2
z?TI4Gr2eFdYfU~{TE^CDrW;MiO|wC^@aqnTC_AvV8@5<j!?QTxvS&DG?(Gne{w+D>
zN*w3{)WnB_`|-I??A0pl9;P*Rx*7PyUGoD)9f)g_3ECQ;WDkre+Z$>Y+ipsJ3wDZg
zG;mNW(L2K4`NhQNE;&W6)JH8n32mx)f^A?er(g{K-P~PM=W)Y6?5gZv37~?wTU|lR
zX+Bm;#0t266aOk<=q0hMNI7T$G}*IW#gg5?P;n}V8J-)db!xTN>HIErC<H@NjgG=Y
z5nGeeSmOYJon@MD3e{tol4{0|jhjJr9&AeATup>|ZqYcTA-|p75$9S!e{%5KXH*0O
z5CGB4J<n!S`cwRHDvXnYdU5?cpX`*lVBC*p&c3dHKRVe$zsB%DKD-+iYBEx1i!;{f
z#`x7G@b3tlUwM`ZcaQUY@nZ^Y+OV)`^QBbC#^!E;7748m5Ewc)_1;*1wMt!)lm>|?
zI7Vsakg$AF^a)Ym(n$@cO&wdpczFBmx)j?f{rcparR!V{`OJb#2*(nTn$h|i9N|xY
zxTL2${;i!O5Ca=pDd$>$&dosvLC`v^@1J+il&88kLlu%Sl?K7&63hWF4TZ}#?Z<hK
zeJ!@NPypd|PnSC?q3XZswAt)OP<FT}5q*Pga1`UDvuGR+OozoTdI`L`9~EHI6Ocy*
zoE5Hs)75e+Y{^s6`~@>l3o)V*kD8l)lt7PJwF%^jv`Np;BKVXV{0#psL&-#bipKwj
zh+XO(Exnqhp+uh@fO)T4>>tn5&QrY;V$gVmcu%33hKi;`rxoQKJ1*pqG6O=)HDim4
zow&tjmPQljjbF`IvCiveeGD2cR*Av*v~Kp7JMqRD-^U_sT3ev?WhGuuyE47GCt>e}
zNc)ev!<8zVT3%-EE)PyAJDB7-Yc)AgSoSd2(Sr2wJTIeCd;EBOXmuWx_LP7a8|?m$
zY8Y%OS^Z4)lnOk$^rRez@+?Rb1d-jBL2K$KzN&D|r1>}MYVO4X5&jEqQEk#Qo9pos
z(AsaUiR7*VX!Qonb(WF$cvaPDa3XEzl2?(BjxNST1?`?=w=#Y-bWkbM#mgV0<f^wh
z7^UL$Su|2_ODu06)0e$5YR*-Lk*m34bv>iq+Ab*LbjQ5a-LYXMaWjf>k=2kA$0c*&
zj~7wT3u6zXgFo$)Z%!L2yG9fr;prG!)I>HaPP@w6-DkPc6iH|ZP%{*1ULoH_<<jyC
zsCQ2eLFNNW&x*7pSP>3OKRAiO(i6<`-b_AS;e-{k8&BIN=i1DS_eU5Q*5Y642O1N-
zdhI-lvm>|hk0P5@s7C%MRv6D};N^_>>p^5LXCA2C8^O*_Tpsmn?P?j!-uel@UkEjw
zirKs@$-I8Godi@o<jzRHJYA86q1*D;-Vc6R%?2tzns3&Gb*|jH4<cvde8#ZthE84w
z46J^r<hT~d*Zz`j`)in3#zt0MeJ2>B9yMsVEVHxfbiH#;R@&67kU#jbc^Pi`UoQm&
zFpKgfqj#=wY0+F_vJ6Z+{(~O2$TPmNvHtMkr3j;&KxVDPa(nJXj%(2jXAwVQQtkQN
z9GDdq7T1UE`tM%9c<!L<_GtO{@e9(qV6jwyR2{HgjSA;wV!*Q>bMWn7PElKaC9hiq
zfTCVe(esZH9q{f5d0`=0jP^=Q99wmm(yLAgRYA~zJyNFSpHgIjNSPeIOTgbKme#+J
zT#593V)1d<1p8W1*l=~=#NM_A-N1LcwF!2Cu$w~n3$;VvXHI}8l%?nY-)|J@g1V?y
zB1?Ez1rAnCPPR)^oh?Tt_Znoh?>TnJJ8?KZxH3laCN~lg#gX0pxe$HiGOSpl+!m_P
q$qXM5qgR<;p>-VQBjlu%Dk=<=NPgUB3YDlg@Md+^iYytDkJ`V}ng4?T

-- 
2.26.2

[PATCH 5/6] btrfs-progs: image: Don't modify the chunk and device tree if the source dump is single device

[Qu Wenruo]

Btrfs-image -r doesn't always create exactly the same fs of the original
fs, this is because btrfs-image can create dump from multi-device, and
restore it to single device.

Thus we need some special modification to chunk and device tree. This
behavior is mostly to handle the old behavior where we always restore
the metadata into SINGLE profile no matter what.

However this is not needed if the source fs only has one device, in that
case, we can use all the metadata as is, no need to modify the fs at
all, resulting an exact copy of metadata.

This patch will do extra check when doing restore, to avoid modify the
restored fs if the source is also single device.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 image/main.c | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/image/main.c b/image/main.c
index 10f6182e0eac..48d8fcd5078c 100644
--- a/image/main.c
+++ b/image/main.c
@@ -103,6 +103,7 @@ struct mdrestore_struct {
 	struct rb_root physical_tree;
 	struct list_head list;
 	struct list_head overlapping_chunks;
+	struct btrfs_super_block *original_super;
 	size_t num_items;
 	u32 nodesize;
 	u64 devid;
@@ -1085,6 +1086,12 @@ static int update_super(struct mdrestore_struct *mdres, u8 *buffer)
 	u8 *ptr, *write_ptr;
 	int old_num_stripes;
 
+	/* No need to fix, use all data as is */
+	if (btrfs_super_num_devices(mdres->original_super) == 1) {
+		new_array_size = btrfs_super_sys_array_size(super);
+		goto finish;
+	}
+
 	write_ptr = ptr = super->sys_chunk_array;
 	array_size = btrfs_super_sys_array_size(super);
 
@@ -1135,6 +1142,7 @@ static int update_super(struct mdrestore_struct *mdres, u8 *buffer)
 		cur += btrfs_chunk_item_size(old_num_stripes);
 	}
 
+finish:
 	if (mdres->clear_space_cache)
 		btrfs_set_super_cache_generation(super, 0);
 
@@ -1203,6 +1211,8 @@ static int fixup_chunk_tree_block(struct mdrestore_struct *mdres,
 	u64 bytenr = async->start;
 	int i;
 
+	if (btrfs_super_num_devices(mdres->original_super) == 1)
+		return 0;
 	if (size_left % mdres->nodesize)
 		return 0;
 
@@ -1373,6 +1383,8 @@ static void *restore_worker(void *data)
 
 		if (!mdres->multi_devices) {
 			if (async->start == BTRFS_SUPER_INFO_OFFSET) {
+				memcpy(mdres->original_super, outbuf,
+				       BTRFS_SUPER_INFO_SIZE);
 				if (mdres->old_restore) {
 					update_super_old(outbuf);
 				} else {
@@ -1474,6 +1486,7 @@ static void mdrestore_destroy(struct mdrestore_struct *mdres, int num_threads)
 
 	pthread_cond_destroy(&mdres->cond);
 	pthread_mutex_destroy(&mdres->mutex);
+	free(mdres->original_super);
 }
 
 static int mdrestore_init(struct mdrestore_struct *mdres,
@@ -1500,6 +1513,10 @@ static int mdrestore_init(struct mdrestore_struct *mdres,
 	mdres->last_physical_offset = 0;
 	mdres->alloced_chunks = 0;
 
+	mdres->original_super = malloc(BTRFS_SUPER_INFO_SIZE);
+	if (!mdres->original_super)
+		return -ENOMEM;
+
 	if (!num_threads)
 		return 0;
 
@@ -2606,7 +2623,8 @@ static int restore_metadump(const char *input, FILE *out, int old_restore,
 	}
 	ret = wait_for_worker(&mdrestore);
 
-	if (!ret && !multi_devices && !old_restore) {
+	if (!ret && !multi_devices && !old_restore &&
+	    btrfs_super_num_devices(mdrestore.original_super) != 1) {
 		struct btrfs_root *root;
 		struct stat st;
 
-- 
2.26.2

[PATCH 6/6] btrfs-progs: image: Pin down log tree blocks before fixup

[Qu Wenruo]

Although btrfs-image will dump log tree, we will modify the restored
image if it's not a multi-device restore.

In that case, since log tree blocks are not recorded in extent tree,
extent allocator will try to re-use the tree blocks belonging to log
trees, this would lead to transid mismatch if btrfs-restore chooses to
do device/chunk fixup.

This patch will fix such problem by pinning down all the log trees
blocks before fixing up the device and chunk trees.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 image/main.c | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 66 insertions(+)

diff --git a/image/main.c b/image/main.c
index 48d8fcd5078c..71ea131c8c29 100644
--- a/image/main.c
+++ b/image/main.c
@@ -2500,6 +2500,66 @@ out:
 	return ret;
 }
 
+static int iter_tree_blocks(struct btrfs_fs_info *fs_info,
+			    struct extent_buffer *eb, bool pin)
+{
+	void (*func)(struct btrfs_fs_info *fs_info, u64 bytenr, u64 num_bytes);
+	int nritems;
+	int level;
+	int i;
+	int ret;
+
+	if (pin)
+		func = btrfs_pin_extent;
+	else
+		func = btrfs_unpin_extent;
+
+	func(fs_info, eb->start, eb->len);
+
+	level = btrfs_header_level(eb);
+	nritems = btrfs_header_nritems(eb);
+	if (level == 0)
+		return 0;
+
+	for (i = 0; i < nritems; i++) {
+		u64 bytenr;
+		struct extent_buffer *tmp;
+
+		if (level == 0) {
+			struct btrfs_root_item *ri;
+			struct btrfs_key key;
+
+			btrfs_item_key_to_cpu(eb, &key, i);
+			if (key.type != BTRFS_ROOT_ITEM_KEY)
+				continue;
+			ri = btrfs_item_ptr(eb, i, struct btrfs_root_item);
+			bytenr = btrfs_disk_root_bytenr(eb, ri);
+			tmp = read_tree_block(fs_info, bytenr, 0);
+			if (!extent_buffer_uptodate(tmp)) {
+				error("unable to read log root block");
+				return -EIO;
+			}
+			ret = iter_tree_blocks(fs_info, tmp, pin);
+			free_extent_buffer(tmp);
+			if (ret)
+				return ret;
+		} else {
+			bytenr = btrfs_node_blockptr(eb, i);
+			tmp = read_tree_block(fs_info, bytenr, 0);
+			if (!extent_buffer_uptodate(tmp)) {
+				error("unable to read log root block");
+				return -EIO;
+			}
+			ret = iter_tree_blocks(fs_info, tmp, pin);
+			free_extent_buffer(tmp);
+			if (ret)
+				return ret;
+		}
+	}
+
+	return 0;
+}
+
 static int fixup_chunks_and_devices(struct btrfs_fs_info *fs_info,
 			 struct mdrestore_struct *mdres, int out_fd)
 {
@@ -2516,6 +2576,9 @@ static int fixup_chunks_and_devices(struct btrfs_fs_info *fs_info,
 		return PTR_ERR(trans);
 	}
 
+	if (btrfs_super_log_root(fs_info->super_copy) &&
+	    fs_info->log_root_tree)
+		iter_tree_blocks(fs_info, fs_info->log_root_tree->node, true);
 	fixup_block_groups(trans);
 	ret = fixup_dev_extents(trans);
 	if (ret < 0)
@@ -2530,6 +2593,9 @@ static int fixup_chunks_and_devices(struct btrfs_fs_info *fs_info,
 		error("unable to commit transaction: %d", ret);
 		return ret;
 	}
+	if (btrfs_super_log_root(fs_info->super_copy) &&
+	    fs_info->log_root_tree)
+		iter_tree_blocks(fs_info, fs_info->log_root_tree->node, false);
 	return 0;
 error:
 	errno = -ret;
-- 
2.26.2

Re: [PATCH 0/6] btrfs-progs: btrfs-image related fixes

[Qu Wenruo]

[-- Attachment #1.1: Type: text/plain, Size: 3410 bytes --]



On 2020/5/27 下午6:28, Qu Wenruo wrote:
> This branch can be fetched from github:
> https://github.com/adam900710/btrfs-progs/tree/image_fixes
> 
> Since there are two binary files updates, and one big code move, it's
> recommended to fetch github repo, in case some patches didn't reach mail
> list.
> 
> This is inspried by one log tree replay dead loop bug, where the kind
> reporter, Pierre Abbat <phma@bezitopo.org>, gave the btrfs-image to
> reproduce it.
> 
> Then the image fails to pass check due to the existing log tree
> conflicting with device/chunk fixup.
> As log tree blocks are not recorded in extent tree, later COW can use
> log tree blocks and cause transid mismatch.
> 
> To address the problem, this patchset will:
> - Don't do any fixup if the source dump is single device
>   Since the dump has the full super block contents, we can easily check
>   if the source fs is single deivce.
> 
>   The chunk/device fixup is mostly for older btrfs-image behavior, which
>   always restores the fs into SINGLE profile.
>   However since commit 9088ab6a1067 ("btrfs-progs: make btrfs-image
>   restore to support dup"), btrfs-image can restore into DUP profile,
>   allowing us to do exact replay for single device fs.
>   This is patch 5.

As expected, patch 3 can't survive the mail list filter.
It's 402K, so I guess one needs to grab it from github anyway.

Thanks,
Qu
> 
> - Pin down all log tree blocks for fixup
>   For cases we still need to fixup chunk/device items, at least pin down
>   all log tree blocks to avoid transid mimsatch.
>   This is patch 6.
> 
> After above fixes, fsck/012 and fsck/035 fails, due to bad original
> images.
> The old btrfs-image can fixup those bad device total_bytes and
> bytes_used, but that just hides the problem.
> We still need to update those images to make them correct, so here comes
> patch 3 and 4.
> 
> During the debugging of btrfs-image restore, I found dump_superblock()
> would help a lot to expose bad values in images, so is
> btrfs_print_leaf().
> 
> Enahance them to be more handy for usage inside gdb, and here comes
> patch 1 and 2.
> 
> 
> Qu Wenruo (6):
>   btrfs-progs: Allow btrfs_print_leaf() to be called on dummy eb whose
>     fs_info is NULL
>   btrfs-progs: print-tree: Export dump_superblock()
>   btrfs-progs: fsck-tests: Update the image in 012
>   btrfs-progs: fsck-tests: Update the image of test case 035
>   btrfs-progs: image: Don't modify the chunk and device tree if  the
>     source dump is single device
>   btrfs-progs: image: Pin down log tree blocks before fixup
> 
>  cmds/inspect-dump-super.c                     | 454 +-----------------
>  image/main.c                                  |  86 +++-
>  print-tree.c                                  | 449 ++++++++++++++++-
>  print-tree.h                                  |   1 +
>  .../012-leaf-corruption/good.img.xz           | Bin 0 -> 186392 bytes
>  .../012-leaf-corruption/no_data_extent.tar.xz | Bin 130260 -> 0 bytes
>  tests/fsck-tests/012-leaf-corruption/test.sh  |  17 +-
>  .../offset_by_one.img                         | Bin 3072 -> 3072 bytes
>  8 files changed, 540 insertions(+), 467 deletions(-)
>  create mode 100644 tests/fsck-tests/012-leaf-corruption/good.img.xz
>  delete mode 100644 tests/fsck-tests/012-leaf-corruption/no_data_extent.tar.xz
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 0/6] btrfs-progs: btrfs-image related fixes

[David Sterba]

On Wed, May 27, 2020 at 06:46:53PM +0800, Qu Wenruo wrote:
> On 2020/5/27 下午6:28, Qu Wenruo wrote:
> > This branch can be fetched from github:
> > https://github.com/adam900710/btrfs-progs/tree/image_fixes
> > 
> > Since there are two binary files updates, and one big code move, it's
> > recommended to fetch github repo, in case some patches didn't reach mail
> > list.
> > 
> > This is inspried by one log tree replay dead loop bug, where the kind
> > reporter, Pierre Abbat <phma@bezitopo.org>, gave the btrfs-image to
> > reproduce it.
> > 
> > Then the image fails to pass check due to the existing log tree
> > conflicting with device/chunk fixup.
> > As log tree blocks are not recorded in extent tree, later COW can use
> > log tree blocks and cause transid mismatch.
> > 
> > To address the problem, this patchset will:
> > - Don't do any fixup if the source dump is single device
> >   Since the dump has the full super block contents, we can easily check
> >   if the source fs is single deivce.
> > 
> >   The chunk/device fixup is mostly for older btrfs-image behavior, which
> >   always restores the fs into SINGLE profile.
> >   However since commit 9088ab6a1067 ("btrfs-progs: make btrfs-image
> >   restore to support dup"), btrfs-image can restore into DUP profile,
> >   allowing us to do exact replay for single device fs.
> >   This is patch 5.
> 
> As expected, patch 3 can't survive the mail list filter.
> It's 402K, so I guess one needs to grab it from github anyway.

Thanks, patch picked from git and the whole patchset is now in devel.