[PATCH 00/11] btrfs-progs: metadata_uuid feature fixes and portation

[PATCH 00/11] btrfs-progs: metadata_uuid feature fixes and portation

[damenly.su]

From: Su Yue <Damenly_Su@gmx.com>

The series are inspired by easy failing misc-tests/034.
Those patches fix misc-tests/034 and add new test images.

After portation of kernel find fs_devices code, progs is able to
work on devices with FSID_CHANGING_V2 flag, not sure whether the
functionality is necessary. If not, I will remove it in next version.

Patch[1] adds a trivial module reload operation.
Patch[2] fixes failed mount by mounting another device.
Patch[2-10] port code for finding fs_devices. 
Patch[11] adds test images. 

Su Yue (11):
  btrfs-progs: misc-tests/034: reload btrfs module before running
    failure_recovery
  btrfs-progs: misc-tests/034: mount the second device if first device
    mount failed
  btrfs-progs: metadata_uuid: add new member
    btrfs_fs_devices::fsid_change
  btrfs-progs: handle split-brain scenario for scanned changing device
    without INCOMPAT_METADATA_UUID
  btrfs-progs: handle split-brain scenario for scanned changing device
    with INCOMPAT_METADATA_UUID
  btrfs-progs: handle split-brain scenario for scanned changed/unchanged
    device with INCOMPAT_METADATA_UUID
  btrfs-progs: handle split-brain scenario for scanned changed/unchanged
    device without INCOMPAT_METADATA_UUID
  btrfs-progs: metadata_uuid: remove old logic to find fs_devices
  btrfs-progs: metadata_uuid: rewrite fs_devices fsid and metadata_uuid
    if it's changing
  btrfs-progs: metadata_uuid: clear FSID_CHANGING_V2 while open_ctree()
  btrfs-progs: misc-tests/034: add new test images and modify the script

 disk-io.c                                     |  10 +-
 .../misc-tests/034-metadata-uuid/disk7.raw.xz | Bin 0 -> 48388 bytes
 .../misc-tests/034-metadata-uuid/disk8.raw.xz | Bin 0 -> 47084 bytes
 tests/misc-tests/034-metadata-uuid/test.sh    |  46 ++++-
 volumes.c                                     | 187 +++++++++++++++++-
 volumes.h                                     |   1 +
 6 files changed, 227 insertions(+), 17 deletions(-)
 create mode 100644 tests/misc-tests/034-metadata-uuid/disk7.raw.xz
 create mode 100644 tests/misc-tests/034-metadata-uuid/disk8.raw.xz

-- 
2.21.0 (Apple Git-122.2)

[PATCH 01/11] btrfs-progs: misc-tests/034: reload btrfs module before running failure_recovery

[damenly.su]

From: Su Yue <Damenly_Su@gmx.com>

One reload_btrfs is lost, add it.

Fixes: 0de2e22ad226 ("btrfs-progs: tests: Add tests for changing fsid feature")
Signed-off-by: Su Yue <Damenly_Su@gmx.com>
---
 tests/misc-tests/034-metadata-uuid/test.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/misc-tests/034-metadata-uuid/test.sh b/tests/misc-tests/034-metadata-uuid/test.sh
index 6ac55b1cacfa..ff51bf22fadf 100755
--- a/tests/misc-tests/034-metadata-uuid/test.sh
+++ b/tests/misc-tests/034-metadata-uuid/test.sh
@@ -211,11 +211,10 @@ failure_recovery "./disk1.raw.xz" "./disk2.raw.xz" check_inprogress_flag
 reload_btrfs
 failure_recovery "./disk2.raw.xz" "./disk1.raw.xz" check_inprogress_flag
 
-reload_btrfs
-
 # disk4 contains an image in with the in-progress flag set and disk 3 is part
 # of the same filesystem but has both METADATA_UUID incompat and a new
 # metadata uuid set. So disk 3 must always take precedence
+reload_btrfs
 failure_recovery "./disk3.raw.xz" "./disk4.raw.xz" check_completed
 reload_btrfs
 failure_recovery "./disk4.raw.xz" "./disk3.raw.xz" check_completed
@@ -224,6 +223,7 @@ failure_recovery "./disk4.raw.xz" "./disk3.raw.xz" check_completed
 # than once, disk6 on the other hand is member of the same filesystem but
 # hasn't completed its last change. Thus it has both the FSID_CHANGING flag set
 # and METADATA_UUID flag set.
+reload_btrfs
 failure_recovery "./disk5.raw.xz" "./disk6.raw.xz" check_multi_fsid_change
 reload_btrfs
 failure_recovery "./disk6.raw.xz" "./disk5.raw.xz" check_multi_fsid_change
-- 
2.21.0 (Apple Git-122.2)

[PATCH 02/11] btrfs-progs: misc-tests/034: mount the second device if first device mount failed

[damenly.su]

From: Su Yue <Damenly_Su@gmx.com>

The 034 test may fail to mount, and dmesg says open_ctree() failed due
to device missing.

The partly work flow is
step1 loop1 = losetup image1
step2 loop2 = losetup image2
setp3 mount loop1

The dmesg says the loop2 device is missing.
It's possible and known that while step3 is in open_ctree() and
fs_devices->opened is nonzero, loop2 device has not been added into the
fs_devces. Then read_one_chunk() reports that loop2 is missing.

The solution for this test is try to mount loop2 if loop mount failed.

Fixes: 0de2e22ad226 ("btrfs-progs: tests: Add tests for changing fsid feature")
Signed-off-by: Su Yue <Damenly_Su@gmx.com>
---
 tests/misc-tests/034-metadata-uuid/test.sh | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/tests/misc-tests/034-metadata-uuid/test.sh b/tests/misc-tests/034-metadata-uuid/test.sh
index ff51bf22fadf..5fe553705fcf 100755
--- a/tests/misc-tests/034-metadata-uuid/test.sh
+++ b/tests/misc-tests/034-metadata-uuid/test.sh
@@ -173,7 +173,9 @@ failure_recovery() {
 	loop2=$(run_check_stdout $SUDO_HELPER losetup --find --show "$image2")
 
 	# Mount and unmount, on trans commit all disks should be consistent
-	run_check $SUDO_HELPER mount "$loop1" "$TEST_MNT"
+	run_mayfail $SUDO_HELPER mount "$loop1" "$TEST_MNT"
+	[ $? -ne 0 ] && run_check $SUDO_HELPER mount "$loop2" "$TEST_MNT"
+
 	run_check $SUDO_HELPER umount "$TEST_MNT"
 
 	# perform any specific check
-- 
2.21.0 (Apple Git-122.2)

[PATCH 03/11] btrfs-progs: metadata_uuid: add new member btrfs_fs_devices::fsid_change

[damenly.su]

From: Su Yue <Damenly_Su@gmx.com>

Like what is done in kernel, we need extra information for split-brain
cases.
The new member btrfs_fs_devices::fsid_change records whether the
devices are with FSID_CHANGING_V2 flag.

The existing btrfs_fs_devices::last_transid can be used like the
last_generation field in kernel. It records the highest generation of
the fs_devices.

Signed-off-by: Su Yue <Damenly_Su@gmx.com>
---
 volumes.c | 4 +++-
 volumes.h | 1 +
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/volumes.c b/volumes.c
index 143164f02ac0..88e926e98d5b 100644
--- a/volumes.c
+++ b/volumes.c
@@ -214,7 +214,8 @@ static int device_list_add(const char *path,
 	u64 found_transid = btrfs_super_generation(disk_super);
 	bool metadata_uuid = (btrfs_super_incompat_flags(disk_super) &
 		BTRFS_FEATURE_INCOMPAT_METADATA_UUID);
-
+	bool fsid_change_in_progress = (btrfs_super_flags(disk_super) &
+					BTRFS_SUPER_FLAG_CHANGING_FSID_V2);
 	if (metadata_uuid)
 		fs_devices = find_fsid(disk_super->fsid,
 				       disk_super->metadata_uuid);
@@ -238,6 +239,7 @@ static int device_list_add(const char *path,
 		fs_devices->latest_devid = devid;
 		fs_devices->latest_trans = found_transid;
 		fs_devices->lowest_devid = (u64)-1;
+		fs_devices->fsid_change = fsid_change_in_progress;
 		device = NULL;
 	} else {
 		device = find_device(fs_devices, devid,
diff --git a/volumes.h b/volumes.h
index 41574f21dd23..1c734b515346 100644
--- a/volumes.h
+++ b/volumes.h
@@ -72,6 +72,7 @@ struct btrfs_device {
 struct btrfs_fs_devices {
 	u8 fsid[BTRFS_FSID_SIZE]; /* FS specific uuid */
 	u8 metadata_uuid[BTRFS_FSID_SIZE]; /* FS specific uuid */
+	bool fsid_change;
 
 	/* the device with this id has the most recent copy of the super */
 	u64 latest_devid;
-- 
2.21.0 (Apple Git-122.2)

[PATCH 04/11] btrfs-progs: handle split-brain scenario for scanned changing device without INCOMPAT_METADATA_UUID

[damenly.su]

From: Su Yue <Damenly_Su@gmx.com>

For a scanned changing device without INCOMPAT_METADATA_UUID, the
situation may be
a) The scanned failed to be pull into disk, and there are successful
synced devices which is already in fs_devices->devices. So their
fsid and metadata_uuid must differ. Also since the scanned device is
without INCOMPAT_METADATA_UUID, so fs_device->metadata_uuid must equals
disk_super->fs_id.

b) The scanned device is in the newest state. There are some old
devices failed to be with CHANGING_FSID_V2 or some devices just like
the device.

Signed-off-by: Su Yue <Damenly_Su@gmx.com>
---
 volumes.c | 32 ++++++++++++++++++++++++++++++--
 1 file changed, 30 insertions(+), 2 deletions(-)

diff --git a/volumes.c b/volumes.c
index 88e926e98d5b..bc1726975ce4 100644
--- a/volumes.c
+++ b/volumes.c
@@ -205,18 +205,46 @@ static struct btrfs_fs_devices *find_fsid(u8 *fsid, u8 *metadata_uuid)
 	return NULL;
 }
 
+/*
+ * Handle scanned device which has CHANGING_FSID_V2 set, it might belong to
+ * either a filesystem which has disks with completed fsid change or it might
+ * belong to fs with no UUID changes in effect, handle both.
+ */
+static struct btrfs_fs_devices *find_fsid_inprogress(
+					struct btrfs_super_block *disk_super)
+{
+	struct btrfs_fs_devices *fs_devices;
+
+	list_for_each_entry(fs_devices, &fs_uuids, list) {
+		if (memcmp(fs_devices->metadata_uuid, fs_devices->fsid,
+			   BTRFS_FSID_SIZE) != 0 &&
+		    memcmp(fs_devices->metadata_uuid, disk_super->fsid,
+			   BTRFS_FSID_SIZE) == 0 && !fs_devices->fsid_change) {
+			return fs_devices;
+		}
+	}
+
+	return find_fsid(disk_super->fsid, NULL);
+}
+
 static int device_list_add(const char *path,
 			   struct btrfs_super_block *disk_super,
 			   u64 devid, struct btrfs_fs_devices **fs_devices_ret)
 {
 	struct btrfs_device *device;
-	struct btrfs_fs_devices *fs_devices;
+	struct btrfs_fs_devices *fs_devices = NULL;
 	u64 found_transid = btrfs_super_generation(disk_super);
 	bool metadata_uuid = (btrfs_super_incompat_flags(disk_super) &
 		BTRFS_FEATURE_INCOMPAT_METADATA_UUID);
 	bool fsid_change_in_progress = (btrfs_super_flags(disk_super) &
 					BTRFS_SUPER_FLAG_CHANGING_FSID_V2);
-	if (metadata_uuid)
+
+	if (fsid_change_in_progress) {
+		if (!metadata_uuid)
+			fs_devices = find_fsid_inprogress(disk_super);
+	}
+
+	if (metadata_uuid && !fs_devices)
 		fs_devices = find_fsid(disk_super->fsid,
 				       disk_super->metadata_uuid);
 	else
-- 
2.21.0 (Apple Git-122.2)

[PATCH 05/11] btrfs-progs: handle split-brain scenario for scanned changing device with INCOMPAT_METADATA_UUID

[damenly.su]

From: Su Yue <Damenly_Su@gmx.com>

For a scanned changing device with INCOMPAT_METADATA_UUID, which means
its fsid and metadata_uuid differs, then situation can be

a) The final changed devices are still own the INCOMPAT_METADATA_UUID
feature. So their metadata_uuid is same as the device's, but fsids must
differ.

b) The new fsid to be changing is same as the scanned device's
metadata_uuid. So the successful synced devices are like some never
changed. fs_devices's fsid is same with their metadata_uuid in memory.
In disk, those synced device's fsid differ from their metadata_uuid,
because the later member is full zero bytes.

c) The device is the newest device, there are old devices with
INCOMPAT_METADATA_UUID or scanned changing devices.

Signed-off-by: Su Yue <Damenly_Su@gmx.com>
---
 volumes.c | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 50 insertions(+)

diff --git a/volumes.c b/volumes.c
index bc1726975ce4..0fd41186c54a 100644
--- a/volumes.c
+++ b/volumes.c
@@ -227,6 +227,54 @@ static struct btrfs_fs_devices *find_fsid_inprogress(
 	return find_fsid(disk_super->fsid, NULL);
 }
 
+static struct btrfs_fs_devices *find_fsid_changed(
+					struct btrfs_super_block *disk_super)
+{
+	struct btrfs_fs_devices *fs_devices;
+
+	/*
+	 * Handles the case where scanned device is part of an fs that had
+	 * multiple successful changes of FSID but currently device didn't
+	 * observe it.
+	 *
+	 * Case 1: the devices already changed still owns the feature, their
+	 * fsid must differ from the disk_super->fsid.
+	 */
+	list_for_each_entry(fs_devices, &fs_uuids, list) {
+		if (fs_devices->fsid_change)
+			continue;
+		if (memcmp(fs_devices->metadata_uuid, fs_devices->fsid,
+			   BTRFS_FSID_SIZE) != 0 &&
+		    memcmp(fs_devices->metadata_uuid, disk_super->metadata_uuid,
+			   BTRFS_FSID_SIZE) == 0 &&
+		    memcmp(fs_devices->fsid, disk_super->fsid,
+			   BTRFS_FSID_SIZE) != 0) {
+			return fs_devices;
+		}
+	}
+
+	/*
+	 * Case 2: the synced devices doesn't have the metadata_uuid feature.
+	 * NOTE: the fs_devices has same metadata_uuid and fsid in memory, but
+	 * they differs in disk, because fs_id is copied to
+	 * fs_devices->metadata_id while alloc_fs_devices if no metadata
+	 * feature.
+	 */
+	list_for_each_entry(fs_devices, &fs_uuids, list) {
+		if (memcmp(fs_devices->metadata_uuid, fs_devices->fsid,
+			   BTRFS_FSID_SIZE) == 0 &&
+		    memcmp(fs_devices->fsid, disk_super->metadata_uuid,
+			   BTRFS_FSID_SIZE) == 0 && !fs_devices->fsid_change)
+			return fs_devices;
+	}
+
+	/*
+	 * Okay, can't found any fs_devices already synced, back to
+	 * search devices unchanged or changing like the device.
+	 */
+	return find_fsid(disk_super->fsid, disk_super->metadata_uuid);
+}
+
 static int device_list_add(const char *path,
 			   struct btrfs_super_block *disk_super,
 			   u64 devid, struct btrfs_fs_devices **fs_devices_ret)
@@ -242,6 +290,8 @@ static int device_list_add(const char *path,
 	if (fsid_change_in_progress) {
 		if (!metadata_uuid)
 			fs_devices = find_fsid_inprogress(disk_super);
+		else
+			fs_devices = find_fsid_changed(disk_super);
 	}
 
 	if (metadata_uuid && !fs_devices)
-- 
2.21.0 (Apple Git-122.2)

[PATCH 06/11] btrfs-progs: handle split-brain scenario for scanned changed/unchanged device with INCOMPAT_METADATA_UUID

[damenly.su]

From: Su Yue <Damenly_Su@gmx.com>

For the unchanged/changed device, just need to fs_devices in changing
state, or devices with same status:

a) The scanned device succeeded to sync into disk. The fs_devices
can be with INCOMPAT_METADATA_UUID. So their fsid and metadata_uuid
differs, and metadata_uuid is same as its metada_uuid.

b) The scanned device succeeded to sync into disk. The fs_devices
can be without INCOMPAT_METADATA_UUID. So their fsid and metadata_uuid
be same, and fsid is same as its metadata_uuid.

c) The scanned device failed to be into changing state. There
are some devices whose fsids and metadata_uuids are same as the
device's.

d) The above cases all are missed, only unchanged devices are same
as the device.

Case c and d can be merged into one that both fsid and metadata
requirements are meet.

Signed-off-by: Su Yue <Damenly_Su@gmx.com>
---
 volumes.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/volumes.c b/volumes.c
index 0fd41186c54a..d094f85999d4 100644
--- a/volumes.c
+++ b/volumes.c
@@ -275,6 +275,50 @@ static struct btrfs_fs_devices *find_fsid_changed(
 	return find_fsid(disk_super->fsid, disk_super->metadata_uuid);
 }
 
+static struct btrfs_fs_devices *find_fsid_changing_metadata_uuid(
+					struct btrfs_super_block *disk_super)
+{
+	struct btrfs_fs_devices *fs_devices;
+
+	/*
+	 * Handle scanned device having completed its fsid change but
+	 * belonging to a fs_devices that was created by first scanning
+	 * a device which didn't have its fsid/metadata_uuid changed
+	 * at all and the CHANGING_FSID_V2 flag set.
+	 */
+	list_for_each_entry(fs_devices, &fs_uuids, list) {
+		if (fs_devices->fsid_change &&
+		    memcmp(disk_super->metadata_uuid, fs_devices->fsid,
+			   BTRFS_FSID_SIZE) == 0 &&
+		    memcmp(fs_devices->fsid, fs_devices->metadata_uuid,
+			   BTRFS_FSID_SIZE) == 0) {
+			return fs_devices;
+		}
+	}
+	/*
+	 * Handle scanned device having completed its fsid change but
+	 * belonging to a fs_devices that was created by a device that
+	 * has an outdated pair of fsid/metadata_uuid and
+	 * CHANGING_FSID_V2 flag set.
+	 */
+	list_for_each_entry(fs_devices, &fs_uuids, list) {
+		if (fs_devices->fsid_change &&
+		    memcmp(fs_devices->metadata_uuid,
+			   fs_devices->fsid, BTRFS_FSID_SIZE) != 0 &&
+		    memcmp(disk_super->metadata_uuid,
+			   fs_devices->metadata_uuid, BTRFS_FSID_SIZE) == 0) {
+			return fs_devices;
+		}
+	}
+
+	/*
+	 * The scanned device is unchanged. Try to find devices which are
+	 * successful in changing stage. Or old devices failed to be
+	 * changeing liked current device.
+	 */
+	return find_fsid(disk_super->fsid, disk_super->metadata_uuid);
+}
+
 static int device_list_add(const char *path,
 			   struct btrfs_super_block *disk_super,
 			   u64 devid, struct btrfs_fs_devices **fs_devices_ret)
@@ -292,6 +336,8 @@ static int device_list_add(const char *path,
 			fs_devices = find_fsid_inprogress(disk_super);
 		else
 			fs_devices = find_fsid_changed(disk_super);
+	} else if (metadata_uuid) {
+		fs_devices = find_fsid_changing_metadata_uuid(disk_super);
 	}
 
 	if (metadata_uuid && !fs_devices)
-- 
2.21.0 (Apple Git-122.2)

[PATCH 07/11] btrfs-progs: handle split-brain scenario for scanned changed/unchanged device without INCOMPAT_METADATA_UUID

[damenly.su]

From: Su Yue <Damenly_Su@gmx.com>

For the unchanged/changed device, just need to fs_devices in changing
state, or devices with same status:

a) The scanned device succeeded to sync into disk. The fs_devices
must be with INCOMPAT_METADATA_UUID. So their fsid and metadata_uuid
differs, and metadata_uuid is same as its fsid.

b) The scanned device failed to be into changing state. There
are some devices whose fsids are same as the device's.

c) The above cases all are missed, only unchanged devices are same
as the device.

Case b and c can be merged into one that only same fsid requirement is
meet.

Signed-off-by: Su Yue <Damenly_Su@gmx.com>
---
 volumes.c | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/volumes.c b/volumes.c
index d094f85999d4..94940dd82d0f 100644
--- a/volumes.c
+++ b/volumes.c
@@ -319,6 +319,35 @@ static struct btrfs_fs_devices *find_fsid_changing_metadata_uuid(
 	return find_fsid(disk_super->fsid, disk_super->metadata_uuid);
 }
 
+static struct btrfs_fs_devices *find_fsid_changing(
+					struct btrfs_super_block *disk_super)
+{
+	struct btrfs_fs_devices *fs_devices;
+
+	/*
+	 * Handles the case where scanned device is part of an fs that had
+	 * multiple successful changes of FSID but currently device didn't
+	 * observe it.
+	 * Since the scanned devices does not own the metadata_uuid feature,
+	 * fsid and metadata_uuid of the changing devices must differ, and
+	 * their metadata_uuid must be same as disk_super->fsid.
+	 */
+	list_for_each_entry(fs_devices, &fs_uuids, list) {
+		if (!fs_devices->fsid_change)
+			continue;
+		if (memcmp(fs_devices->fsid, fs_devices->metadata_uuid,
+			   BTRFS_FSID_SIZE) != 0 &&
+		    memcmp(fs_devices->metadata_uuid, disk_super->fsid,
+			   BTRFS_FSID_SIZE) == 0)
+			return fs_devices;
+	}
+
+	/*
+	 * Back to find newer fs_devices is changeing or some in same stage.
+	 */
+	return find_fsid(disk_super->fsid, NULL);
+}
+
 static int device_list_add(const char *path,
 			   struct btrfs_super_block *disk_super,
 			   u64 devid, struct btrfs_fs_devices **fs_devices_ret)
@@ -338,6 +367,8 @@ static int device_list_add(const char *path,
 			fs_devices = find_fsid_changed(disk_super);
 	} else if (metadata_uuid) {
 		fs_devices = find_fsid_changing_metadata_uuid(disk_super);
+	} else {
+		fs_devices = find_fsid_changing(disk_super);
 	}
 
 	if (metadata_uuid && !fs_devices)
-- 
2.21.0 (Apple Git-122.2)

[PATCH 08/11] btrfs-progs: metadata_uuid: remove old logic to find fs_devices

[damenly.su]

From: Su Yue <Damenly_Su@gmx.com>

Now, the new code for finding fs_devices should work, remove
those old logic.

Signed-off-by: Su Yue <Damenly_Su@gmx.com>
---
 volumes.c | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/volumes.c b/volumes.c
index 94940dd82d0f..9d76f9a102b2 100644
--- a/volumes.c
+++ b/volumes.c
@@ -371,12 +371,6 @@ static int device_list_add(const char *path,
 		fs_devices = find_fsid_changing(disk_super);
 	}
 
-	if (metadata_uuid && !fs_devices)
-		fs_devices = find_fsid(disk_super->fsid,
-				       disk_super->metadata_uuid);
-	else
-		fs_devices = find_fsid(disk_super->fsid, NULL);
-
 	if (!fs_devices) {
 		fs_devices = kzalloc(sizeof(*fs_devices), GFP_NOFS);
 		if (!fs_devices)
-- 
2.21.0 (Apple Git-122.2)

[PATCH 09/11] btrfs-progs: metadata_uuid: rewrite fs_devices fsid and metadata_uuid if it's changing

[damenly.su]

From: Su Yue <Damenly_Su@gmx.com>

As what kernel does, changing fs_devices's fsid and metadata uuid
should be changed to the successful changed device's.

Signed-off-by: Su Yue <Damenly_Su@gmx.com>
---
 volumes.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/volumes.c b/volumes.c
index 9d76f9a102b2..6decc699ba2c 100644
--- a/volumes.c
+++ b/volumes.c
@@ -359,6 +359,8 @@ static int device_list_add(const char *path,
 		BTRFS_FEATURE_INCOMPAT_METADATA_UUID);
 	bool fsid_change_in_progress = (btrfs_super_flags(disk_super) &
 					BTRFS_SUPER_FLAG_CHANGING_FSID_V2);
+	bool new_device_added = false;
+	bool fs_devices_found = false;
 
 	if (fsid_change_in_progress) {
 		if (!metadata_uuid)
@@ -391,9 +393,11 @@ static int device_list_add(const char *path,
 		fs_devices->fsid_change = fsid_change_in_progress;
 		device = NULL;
 	} else {
+		fs_devices_found = true;
 		device = find_device(fs_devices, devid,
 				       disk_super->dev_item.uuid);
 	}
+
 	if (!device) {
 		device = kzalloc(sizeof(*device), GFP_NOFS);
 		if (!device) {
@@ -424,6 +428,7 @@ static int device_list_add(const char *path,
 			btrfs_stack_device_bytes_used(&disk_super->dev_item);
 		list_add(&device->dev_list, &fs_devices->devices);
 		device->fs_devices = fs_devices;
+		new_device_added = true;
 	} else if (!device->name || strcmp(device->name, path)) {
 		char *name;
 
@@ -446,6 +451,23 @@ static int device_list_add(const char *path,
                 device->name = name;
         }
 
+	/*
+	 * If this disk has been pulled into an fs devices created by
+	 * a device which had the CHANGING_FSID_V2 flag then replace the
+	 * metadata_uuid/fsid values of the fs_devices.
+	 */
+	if (new_device_added && fs_devices_found && fs_devices->fsid_change &&
+	    found_transid > fs_devices->latest_trans) {
+		memcpy(fs_devices->fsid, disk_super->fsid,
+		       BTRFS_FSID_SIZE);
+		if (metadata_uuid)
+			memcpy(fs_devices->metadata_uuid,
+			       disk_super->metadata_uuid, BTRFS_FSID_SIZE);
+		else
+			memcpy(fs_devices->metadata_uuid,
+			       disk_super->fsid, BTRFS_FSID_SIZE);
+		fs_devices->fsid_change = false;
+	}
 
 	if (found_transid > fs_devices->latest_trans) {
 		fs_devices->latest_devid = devid;
-- 
2.21.0 (Apple Git-122.2)

[PATCH 10/11] btrfs-progs: metadata_uuid: clear FSID_CHANGING_V2 while open_ctree()

[damenly.su]

From: Su Yue <Damenly_Su@gmx.com>

Progs now is able to handle the metadata_uuid feature without kernel
help, clear the FSID_CHANGING_V2 flag if latest device is marked as
changing.

Signed-off-by: Su Yue <Damenly_Su@gmx.com>
---
 disk-io.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/disk-io.c b/disk-io.c
index 659f8b93a7ca..09dacbc83e06 100644
--- a/disk-io.c
+++ b/disk-io.c
@@ -1177,6 +1177,7 @@ static struct btrfs_fs_info *__open_ctree_fd(int fp, const char *path,
 	int ret;
 	int oflags;
 	unsigned sbflags = SBREAD_DEFAULT;
+	u64 sflags; /* records btrfs_super_flags */
 
 	if (sb_bytenr == 0)
 		sb_bytenr = BTRFS_SUPER_INFO_OFFSET;
@@ -1242,12 +1243,19 @@ static struct btrfs_fs_info *__open_ctree_fd(int fp, const char *path,
 		goto out_devices;
 	}
 
-	if (btrfs_super_flags(disk_super) & BTRFS_SUPER_FLAG_CHANGING_FSID &&
+	sflags = btrfs_super_flags(disk_super);
+	if (sflags & BTRFS_SUPER_FLAG_CHANGING_FSID &&
 	    !fs_info->ignore_fsid_mismatch) {
 		fprintf(stderr, "ERROR: Filesystem UUID change in progress\n");
 		goto out_devices;
 	}
 
+	if (sflags & BTRFS_SUPER_FLAG_CHANGING_FSID_V2) {
+		sflags &= ~BTRFS_SUPER_FLAG_CHANGING_FSID_V2;
+		btrfs_set_super_flags(disk_super, sflags);
+		printf("Found metadata UUID change in progress flag, clearing\n");
+	}
+
 	ASSERT(!memcmp(disk_super->fsid, fs_devices->fsid, BTRFS_FSID_SIZE));
 	ASSERT(!memcmp(disk_super->fsid, fs_devices->fsid, BTRFS_FSID_SIZE));
 	if (btrfs_fs_incompat(fs_info, METADATA_UUID))
-- 
2.21.0 (Apple Git-122.2)

[PATCH 11/11] btrfs-progs: misc-tests/034: add new test images and modify the script

[damenly.su]

From: Su Yue <Damenly_Su@gmx.com>

Disk7 contains an image which has undergone a successful fsid change
more than once with FSID_CHANGING_v2 flag. disk8 on the other hand is
member of the same filesystem but has completed its last change without
METADATA_UUID flag set.

The expected recovery result is both image don't have FSID_CHANGING_v2
and INCOMPAT_METADATA_UUID. Change the test script to test it.

NOTE: It needs kernel fixes for metadata_uuid feature, otherwise
will fail.

Signed-off-by: Su Yue <Damenly_Su@gmx.com>
---
 .../misc-tests/034-metadata-uuid/disk7.raw.xz | Bin 0 -> 48388 bytes
 .../misc-tests/034-metadata-uuid/disk8.raw.xz | Bin 0 -> 47084 bytes
 tests/misc-tests/034-metadata-uuid/test.sh    |  38 +++++++++++++++---
 3 files changed, 32 insertions(+), 6 deletions(-)
 create mode 100644 tests/misc-tests/034-metadata-uuid/disk7.raw.xz
 create mode 100644 tests/misc-tests/034-metadata-uuid/disk8.raw.xz

diff --git a/tests/misc-tests/034-metadata-uuid/disk7.raw.xz b/tests/misc-tests/034-metadata-uuid/disk7.raw.xz
new file mode 100644
index 0000000000000000000000000000000000000000..157815fd1f87039f81d0f7fbfe6888b8bac96b85
GIT binary patch
literal 48388
zcmeHQc{tSl`kt{9F&L7_QiyCJ*<M+)RLDM-LWohwT10kDw#*P?&t7S=R9eYe%9^Z`
zb&y>uX@qfludZ|buJdy`Z*$(}T>kj@dtBeS?zx})d7k@uKA-QpVj#jA0-<@|b6<xF
zA_(0Dfk4>W5iJ`VduWg+ArQ~?jg9<=8@hk!!M&9nj2~gd-seZtuPlu7hrg9>R91x^
zE5H0ICdNQIpQS&fXe>?`W;$ZEqtP#haDiBv9K!5YGvur9WSd8S$X6#8m85UU@O*oI
z2aEBC`33l_bFPH~f0)lxp9$##s#aH-8NCWEtfE<1s2CKL{C;{h#zrIk!_mdAvb4f)
zBo5U>HRwcqSc*^w!T+}OAlsXO#<?sTFHsKW`dirIc1tJ!Lm?`G&qEQ)2dHb1VXmXk
zP;WlIh0x)bgJ{3<tlV9Gz!Vv^U>K^Y#TNN~WL?Bg^to=yiRCJ;+!uv(3bZ5H=?;U=
z_D>-6BU01yqi2mXd-BgyyYw|g9#jo`RvQJ$Nb6R9Ct8fVImuCtU#<&~$qHXIwW^5q
z<jspSV1MiHEBPt2@Is%d=!8Q3+O}AP^18NXmSV?fuGy0835PmHVO;tlWDKopMh(Gy
zjlE}L*kHWMElts_5reG=69_tpPfA_xf;ZPS%RhBHP=47<$BRK2>ds*)Z^WNdeK$yU
zXU@ZTS&#GC#lm)TV$D%+u47{^i=#h@`hDf8p~D(r*ZWGl6ImPfni9XD56aROmMwVJ
zo;T2CeuIYH#@X{dX)71qVQUpV?PT~cCeRwMHDzFP>l@m@FtzsbyZGU!{l(^wlvh>X
zXwB#Dp=V5C9?w#r9V9ZdO%`hOTif|JnX+Ktu9Vti^uI)O*N?@(U^>Uet)AF-F)BC1
zv5L2R9iO~ID>9ZTmke~lO+zYtT9J*+4xO~9GTD2V#`i@i*%};{(77|<?%9hzOY9_!
z+^G2!HrcA=-?4^aFDf2zz9Tb$UTD(0RFd6CD(Q4cFRD`}F;!fY%~0UqlR4ZQ*F`|Q
zLVVoz+_APA6>Y@lZ(Qg>G%1wtA!rtSoz857Us1jI6dT1?$rf1SwVjGP#@`EFm*I_w
z;?-mf)Eee<wiiFnn0l(^YIp_0yiPO6jds|UYe?g2Vr<cQfr$!>yUkhenlG_j^7Ji$
ze_$AW!&q*f<k;ww?5y9mSZx>icK7R?=~1K4hNm;aYh>Ul3^u~}mH>^osJR(CIw@<U
zpMqX_Fl%|w%=s}EfsEp#*d!_|rc~x8D<)stsiM|-rq+vYxISG+mHGy_pq|QIKdPSR
z><vfPFPL7@(7*X+LW_0CyE76_N*5Svy1TL8bwm5!bWrU(tW#T~A1vii!FH3lGfJw`
zOY<qCFU(!Xx)2}y;m=CByQLHr+H-ba{jk00kV)0&F7$?+sGnmEoo?j4*cjKzDGlm!
zYY$qT-hsQBPKhMBZJu=yyZGwv$&tqb`B=Q?$D=l(O|sHisy#Egflwh-B#G*b&M(sF
zj0vK8+W(H#_=F8D!^%rMZf(^>!nc**MnH3A&{U1M&xuhbv|0Sz@B@{_hAJt$+pK>a
zJENJ%v3-LriRg9Finy(H<GPp*>#bFcat|bDgAFPHne#C%UjLn3|7~A#Eof*pT}H~H
z=33O}h_D*xzYa4BKH8X7y&`9$g_?dV0@gBDU;D}4;%03r*RW@t11>_||M00vwQ8=m
z8?d~h7yf&m4_x+f;ax0%X0YFo9{j5Q^geQ_ofw;&UFjIAaxs8j)Aeijt0}q65sAP|
zzm1Umb{w%>NbAWyjr0P?q*qKYsSEWD>8mbc^AhYaulV{o{0h8`V5WKZWzNNl98u&B
z*1bP^^!AKJ)F3SGG~Qh+n)NA8d#{Mp6h=w|3OnLleL0sfBZy%LU-+2kbme_s2O?Bs
zcbCL2gAY3ePMy_$#feB@E56Dw<B&}_dRBRNE-%DvmZ<@uZ8Yn2Ji=`-LLVQ29m`3j
z-pj#xB4keKY|h)M>W<7r@moc0*567_$o5g4TM|*XoV^h^hh{I}vD9dBlG=}JDtE>9
z7)?#?tFp>)63WNHXhno{Gp!S!=!-nn`~Dw~ymC$msb?T#`%6y0t}I^0eApZ97uH^O
z9BCdPDO2+KQ9J3(Va$%QD}R_n@0;~9f5B&{M(w)=8x+YYjxeZll%Iszrj^&oUyGVj
zpHDkTWzhEQVyA^#r2pebD%6Y}CUhza?R5Ivx^%JDHsVT^L6!+|#|k@hA2d$hVn888
z1gv=z&)n9(8+4zytyGHk6t`?ykJLeu@oG+&m|h>6rX;_}Y?WUR-~Q=R#LoT)*fhP%
zyK-*z_XS89z~@=%vImEi*RqZTb)TG#*5pugxuH!|l(Zk5^{yD#SvFfl`R$i^S9ih;
zEtE<}ZTITD3TMp$MQkA)S~!)@C_Z<4!&>ByH!~NexLWDjo>4M0oSw3BJW!9Hs_SOq
z`k@{`Zhw-F0AOV1ZvYqo?B6yT6t94Qh5-%xJw3?JH0%YGuoY|U)=KfL!&<f5e8d}f
zNu3lo--9nWn)cC*9d~;nKEq#PXp|JgV(%*ud>1x)Z^c$~U)#ZuZe$Cw7;msmmlUQe
zWFg2l6nEQNIQXQz^mYu=*S9)^iU99&BrZL?SG^R1W|xHZdRbLxXTML_-+rpc(VdlD
z#q8w;G&A1!t~ht~M_W$%qanFm&!&t6p|Gj|64JjzK(Z-h<k^rV>BAYHJ>8f57r2e7
zA6waKUr{u|1*LG=8q%De_pOF5^a(JQ?6{q!(ll40yFtRyI<%@IWNy4~wl(1Gvh;B;
z8f_v42tK$eckOg!u(k1NjIX|JgL?&L@<vk|J<USSP@!fsO5GN6t;4WXL@e2Bw;K5s
z<G*+!>{0u-gYprSkH5>!_?eP`Tt!~4A`z+8wxsm-zINIkxybfN(sIe_VXx?G6<KP>
zYqO|4p}go7^rAx+EI4X+_(ks<fxFa8>1r#y8KHQtved{|7YAv~_m0T&V)7gC^SE7#
zGxdUm1Vr##OKB{R>WbxhvOJ;wS?qM!{*6<m69lEhsSl^%nzC~Z-^yOZzK(aedH8yn
zQqsUDnx2wq@3j_Uo17ccE>f`XzN4Y8u-Q8E>H?u5Cnbw$!6x`Yjb-twT!4v7E`4C9
zDJv0m<+>-wn3b&yCvW+(im1k-US_=*v$F+@b0NAqA)R{o<vWj7#`V)lB?@JC;ziue
z;*~XEn)_n=YZ=U;kMo0`A2_glqDms-R9r@HV&tj|0uGUIy|<X3D9g<-jp9J&*x>~)
zS$Z}s6VFZTuyZLgBh@br`f`iVPGIHI>4+J)@I8*>q@@GjzRnTWSU6oA;;8D}FOw|A
zdg~0VDlvVZ`FVHdbm~7Au(=iqUDbHjwbL_g$f1aYk@3h4gwO=d(z+tAy}gwUOFkev
z#9DKjeXV&igYR-lq>+kn2>0jWJpL|w&T&4Xgurh3!6BK%S@&e=hjF`jmrI2><o@7(
z1JA085bO+CKHfd@d=c+vvT$J7Bj!|+Aa~M`*WQ8lCKs-o);E{WX1;2A<ATt>e^|0Y
z(k3S|S#Rm=SQ7Oj{}C>ud}giY<7X-up#@dJ9?1q3<$eb1_xkwFT>IGsSeVYsiHfGB
z&fkH)L+(_qnVAi2Xb<}AQCgtndE|vFhvQCIbnOqPbF()5l~@1Yme9~_A_`<Rkkvp|
z+hV3Yih65KITkDm(s;_@rkw8Gh7t^Ms4jY;z0waYC6Kmaci>OvRKBIJIF_GBB6pT(
zoYg^}JlXi^sJlMq`U@!)NhUq_2M9ueqAPovjM_Z2H*R_$)Jrv%w!77OSFqemrcPI_
z&imKeW2%y;%a*X>V}mL_l9NZBb`o`JIdr8@D|#GDyIOQ^3>W+QOK7<5K<_tpb_PYh
z;V;J1Jp9PB^s~DfbZLgB?9FTbNT}sGIQB`}8P^fovO2MQ-B<ir$DDb~3D`nPvreL0
zRm_y_cwv83;n`zGE7~P5%O0pE-?2M0&e~EYp9N>}F*kGF8-HRSkCKH)Qj`DC4h6r5
zuEMAtPvFs6sNM$q#i<Mn+HlAzQsZcSWTEF?g-RRx(lE!+UOfR4ve%jL?U>iT?mFVY
z4PRFt1WkT<z2z0u5pLY{=c{2(cE#GdEP^Sb<`WJ}VcCSiYZpq~IQ_hPZ`O?sWkr0>
zqn!&jxWL;AZED6%8vC`gxvq^IuE=?FXf>f{3160&{534WTugF#q56nj!&B!6oa2n=
z9w9C~ZavwLM<>jnTf^Dny(5-fydg_;I(g!1P@WwTW>ouw`d$hknd-d{oZvHD$z?)S
zZjakhsmB}dd1}`cD@ieEZ!2v}x?ED<bB#r0c-L;(EW2tm_0TC-BP3)Lvb^w81d}O>
zN_A@6R~KCHh)PyKa`~7v-m$XRs&W45!OTV}f>+cI<`S>!4}wDn?SIGUnXGiCMV+z&
zau~>AAct+S9L5ISzZE;PHi!vNgH}LRc6SU%muD3QZ}F<7s(&g5mnyj-HJko_3ovA<
z(0|%o_=S2teo*rg5HTQPK*YA3h;7Y6{D18fY+A?KENWn3;6LuK0l>ig8_d5q?{#jD
z3re}2K*Rp-4({gP>WAn@LRBEM*}__n#PXarY}pC~Jv`r<acEITGriuRGpRHfMe=#B
z9z(I6`^{6?ADilbRj4au-}kjy`3*zhUwhmyZ#4gjX4>e(t5-nfPj=;xh01Nk=CYL(
zUeW%5MVK~;aL1JT`WJcY$R|<-^Df?pDOcJ5s$2@%F^mAFk<BzPktc5=4^C+O>;%it
zvL|2|U>Ifh69K~j!vMp!)UiAek{~2ONP>_g%Pj!I5i$;pZ!WigwNw7vXVo`9`k%v*
z?;~+vB=>M_y;0N`9!=``q}*<IOjiQ_%&6gvTADtW#Pt>3Z-$d51Pq-8Pqub8p_B4Q
zWB=u#0|h*;|Bkhz5jkDJt^jrg<&3j`#w3{V{@V*TelB7FFaX%^xY7kM42;CVNF0pB
zDNt{3fl?o2NsuK$mfS)&p#jSkSgycw1(qwYT>sxyuYa{b2LJ<r{k_YC0mI0=eHj1-
z0Q-J>FtDh>nVRo+u~SeD0K)*o0K+y<B7lKNW?TTk0AQ3A62LIPFu*YK-{eEV>Jfw_
z2uTo<AS8dh>j?CcWU2uG3;;%1`2-9D3<C@U_Rim4GzFDDnL+{p1AtLhc>%)!!vMoT
z^9`EsUwD1=<`NLJMr6t-01N;|*?a>S1{ekywmDUSN}tTQ0Du9&C@Um@VSr(PVW9a2
z%{OSipR|=z{Mszg8j+cA0AK(x$|h*QFu*XtFyK~z-HQM9C0?NUCR2F<U;r@6rYgWN
zz%alt;D-S}4ESNd4+F1fmhf%mw-ErX5onEMJ<ewr3xoT^HX9|tKqRwf27m#;DBENJ
zh5?2FhHXw&z}_J<E&yNvFv<!EU>IN+U>I;*f#V7sSKzn;#}zoPvb2R|3!cDn1&-^_
z)ZkBhC-kZ)gb6a|V_Lix0ke%AfqegKvN=w7SiPMV0_|U+3JncSfkGfI=F!8P+$u8E
Ke|~_*%K9H3vMksD

literal 0
HcmV?d00001

diff --git a/tests/misc-tests/034-metadata-uuid/disk8.raw.xz b/tests/misc-tests/034-metadata-uuid/disk8.raw.xz
new file mode 100644
index 0000000000000000000000000000000000000000..ec1fa6709348b713368d3afb6ba8b741e3eec582
GIT binary patch
literal 47084
zcmeI5_g7PE7KTqKSE`{Z(o|5Xp@<D5h^P^`SO7($RB7TMAVn@xEvN($P?4qq<;GA2
zMWiSY1VIG?iXgoxz3U}P3z=DWX05qvO_&h}gdcYPfvmO9{@(qZcki=5`qpzM#}I_A
zyY2~&1zC#{M-YUs%A|t9;9$j>A&4`b!AQtv?9<c{c2&1B$hjfgop5>C#BitNg&O6T
z+jpRTedPC=K-jOGz}w*e;L|l}exr80$V(5xN6+!6xBPidQCi(~ZA|0+uVj6SJE$kZ
z=!tzQN%e_;m#~-SF;;AFh@YfEq*|j(mNhsO#$t4}Q(DaLrD`{BcJg-DciSDNXe>dn
z`81fXB;$u^9gtFJU?=w}ObcQ41qJs$ptAAiXt(IgB_)XxEKaPq;^61&Pio^@`^6^Y
za&Lm&l@KC!O4rQm6vNnub@Zxnfl$D2wK1af)v9ZnJC;`}p1f9Dl_bU{cqt+<d#pnK
zKyuLtH@VUO+xAOmJ1z{2d>d_jjJb5fT&#ARA@-Cq%O@$4JS}?v7gfGe-JF_%AUW3^
z4td^Pg(vUSIXj!8L<WK!NO5Y7hAPSh)-jK~Q-hwwd4}_<YhSp9s#@_9uZ4tWtvYr@
zy(9m=rm?M`&RDg(@x?oAP8InLy@a$#3`1!kMk0^;NJhjIf4R@j(4?U6Y%0F9>o_~d
zL|J3n^pv53d!^*@HQEy`MjGRCHk{aia(TzLEbM4WzLG^UTJu*s?J&XR48EJ=XMV$v
zb5}AVWO3+3MzfihveL)*UZ%Zo(>0Pz1}_a|`Z!b>h7F}k*6D5cBI?^}CYp)WS3X#s
z@yh6UKyPuKWd_ZYW#)JGGn}P@V%@{}JRDQGvi4=x{6^aLrwsbtS><+{92xS~QlsgW
z6xe@09o?2?zk!w#Q=BJ<%T!Vy3K1|g%aCqL8jA96{%j-?m$9xqIm@ItQlGKE>~)ux
zql3%@{}nZw6m2R#U%v9y>eZwJVP`JrN(}3T7R*fT_#5@y#YvvX`e%X_gAW3$ZzYlF
z+%vf~wu3n<&XYgrMjzC&c289nKt>Bnt}64V#J20uiHgzNh;g^ECs*a@-shBPI^=RI
zGdAo)k2G(HDb6*z5z$QBKv|pzkdT(@702L1DIkMxMr3-yLmySqxa<&BCl?}F+G0?)
zJn(Hq6v0nE{ELjow^fv1qV`A8>(gt)&?W1Q#%bYORM|-x!_Gx6`}gs@4d+izvz-4N
zNt*U)or+OWGNyzE{&Dd<#dcBu+5o-tF$8S3^UUN~LBRf-0L~h)9M<+Fc_j514~8pu
zmfcU!#J=yFjz@gk-g<jJ@7<)mllU69clQatXRT<<_-<@^cE7|(!Uk-s(xW7|+I6D6
z9OsNS?-a<|Tpe08dYiqeczJNHe{7RKemqdGrdQ2HLm~T@+INMW4VM`^wq;J#8*1w%
zIbkqGC;hFiVO}LG3g*r@@f>n*nQA^NvG1_c`Nq_jvgY^1gGe5J^{!*F=JL9lcapH0
zEpGB^;g?s6nNozC`PQ4CYq0ev^q<t?(#ghaQ9N7+kEGd+&lKh!y(_gM%1hc(Ijwx1
zjRCILV_ZT}qN*J=O#6dE$*eo*<|6P^c6id6<ass0No>fgCm7Fgh>e*^5%9jghGwov
zzLGDj*W!=ouAUs}$f>fJTzd~EaQyeN3QB48knpa2XSq^`UHum&r(}FFYdNQOwU`}d
z&nT%^x1~Ds6>hz+LXAy)zu`r$s*m<NZjoN6eY<E)IeMQyij&(m4mgcZ6El-1pGfZE
zk7W}$CgXf<;<Kf}%WZa6Xb;!ZE#gLo;X2%pdnS`bsdUxyL_3@G^v&h@is6-dqgEN;
zT+gC2&|RWC?%q9;{fut1y{UHGEY;6*V{yXn^}$=H2vrVaxAi8<S6nXHdu8{vH^GrA
z8aHigzc_Y{m(XVpIegod=u3+ckIQ>|Z_GSy<RNdFw*YI^!x21z+~wGWZ}RSPYkYO?
z#Ke=w1F>h-B88IM$)reoa>PV>pHkteBFE3BGLKc7UtE&qJs}&s$L4_ML*vlG0I59j
zS6{CWE&u4aR@f-GVT-(LeRbbalQN=e*sZ{z<$aD)yqUPYM7bN!-S=Gg;Ji;SuL@ER
z{8HGdZ~0ZlsIIh%??q>+g08Hy<}&ugiKzYxl#X=|!E+zKV6VK5b@p>4P$ih>%@dP1
zhUv??IL2^DRFMU@$R>0Tm+I=Ycw2)A%HhG+#qD)9xoDQ?4~?5n3pVZ9;+w0Lg}%=D
zlpz$sCD7$HRD+qZiu{Gu;<XD_h_Z2O6iFCG>P_I3A53N(+mPzYv*HH1Ks|I_$Gnur
z?KQbjM`q>1S!EGOg*iw=z~=bo0Re-6{q*-1VBsHPVGw-~{duvQK=eWMLG=IAhGlXt
z7SY{<DhX9`9(xQ-k{dD%G7K^d`YLm4Rx%M1sFHIO5(pRsY+;obG7K^dG7QEw{y8ZE
zq(4VBfPg{37FIqX!+t2ka#32yK-tDl+v^R8c}+@<wC{N>sw$zajjF*O)6#K4vQqb=
z(p`Gp`hQPX{_GbPx2Ky}s;HcvaVs-_(iFuUx&q?F9wivugTcLdnp|dbwm{f^NZ6pe
z3*Fs0==kg(6Tp##?k;q9p}YHkf0e{UNWfK`V{M0kLBJN)M}Q2241){<qaTd^|NWgc
z6QTjPVvh0&0fT@oY`uXDgA9WVgRw*yON50Yuz}^z1wjC=^XKU+E`^06S~+1D?yuXv
zYdaK^hyQa-AMK2;NSGCXSpk?8SaN%}mV}6Xcf<bY$Q2I#SEU@|k;X0Ko>9o^7BK`7
z0}%reTXG@>+ZLHE1O2Or{cF)9IAY+4fg=Wv7&v0!h=C&pj@Xav7zeDFKMN7S6~J{K
z*8anhgd+))`Y@>vllpL(23XW!QG-Pd7WMaE8pVVj7o*Px7WH>r@Wtd=fu|3iK6v`z
z>4T>ao<4Z`;OT>>zc`m@0*3*I0fzyH0fzyH0fzyH0fzyHp_lZR4dA$f<0_FyeIz4d
z3g8Oh3g8Oh3g8Oh3g8Oh3gBw8P&>3?wqlMxF$4?(wy^IfkYSKvkYTX;7FOTF>Ra%`
zW_tsMDhX8*sw7lNsFF}6p-MuPoF|_cumZ3GumZ3GumZ3GumZ3GuwrrA&%j~8VZdR)
zVZdR)VZdR)VZdR)VLqy&aoHg-6g-<6z&GD>WM&B14+JcLKtZ_vyliZG$f=4niXdD;
l5h)Ck{U!ueJi-zX5O5oXAo>2)Dncu<Dy+yqer)(-e+NJhvxxuz

literal 0
HcmV?d00001

diff --git a/tests/misc-tests/034-metadata-uuid/test.sh b/tests/misc-tests/034-metadata-uuid/test.sh
index 5fe553705fcf..9c6fe101cf7a 100755
--- a/tests/misc-tests/034-metadata-uuid/test.sh
+++ b/tests/misc-tests/034-metadata-uuid/test.sh
@@ -145,19 +145,34 @@ check_inprogress_flag() {
 }
 
 check_completed() {
+	local metadata_uuid="$3"
+
 	# check that metadata uuid is indeed completed
 	run_check_stdout $SUDO_HELPER "$TOP/btrfs" inspect-internal dump-super \
 		"$1" | grep -q METADATA_UUID
-	[ $? -eq 0 ] || _fail "metadata_uuid not set on $1"
+	if [ $? -ne 0 -a "$metadata_uuid" = 'true' ] ; then
+		_fail "metadata_uuid not set on $1"
+	elif [  $? -eq 0 -a "$metadata_uuid" = 'false' ] ; then
+		_fail "metadata_uuid set on $1"
+	fi
 
 	run_check_stdout $SUDO_HELPER "$TOP/btrfs" inspect-internal dump-super \
-		"$2" | grep -q METADATA_UUID
-	[ $? -eq 0 ] || _fail "metadata_uuid not set on $2"
+		 "$2" | grep -q METADATA_UUID
+	if [ $? -ne 0 -a "$metadata_uuid" = 'true' ] ; then
+		_fail "metadata_uuid not set on $2"
+	elif [  $? -eq 0 -a "$metadata_uuid" = 'false' ] ; then
+		_fail "metadata_uuid set on $2"
+	fi
 }
 
 check_multi_fsid_change() {
 	check_inprogress_flag "$1" "$2"
-	check_completed "$1" "$2"
+	check_completed "$1" "$2" true
+}
+
+check_multi_fsid_change_without_metadata_uuid() {
+	check_inprogress_flag "$1" "$2"
+	check_completed "$1" "$2" false
 }
 
 failure_recovery() {
@@ -217,9 +232,9 @@ failure_recovery "./disk2.raw.xz" "./disk1.raw.xz" check_inprogress_flag
 # of the same filesystem but has both METADATA_UUID incompat and a new
 # metadata uuid set. So disk 3 must always take precedence
 reload_btrfs
-failure_recovery "./disk3.raw.xz" "./disk4.raw.xz" check_completed
+failure_recovery "./disk3.raw.xz" "./disk4.raw.xz" check_completed true
 reload_btrfs
-failure_recovery "./disk4.raw.xz" "./disk3.raw.xz" check_completed
+failure_recovery "./disk4.raw.xz" "./disk3.raw.xz" check_completed true
 
 # disk5 contains an image which has undergone a successful fsid change more
 # than once, disk6 on the other hand is member of the same filesystem but
@@ -229,3 +244,14 @@ reload_btrfs
 failure_recovery "./disk5.raw.xz" "./disk6.raw.xz" check_multi_fsid_change
 reload_btrfs
 failure_recovery "./disk6.raw.xz" "./disk5.raw.xz" check_multi_fsid_change
+
+# disk7 contains an image which has undergone a successful fsid change more
+# than once with FSID_CHANGING flag. disk8 on the other hand is member
+# of the same filesystem but has completed its last change without
+# METADATA_UUID flag set.
+reload_btrfs
+failure_recovery "./disk7.raw.xz" "./disk8.raw.xz" \
+		 check_multi_fsid_change_without_metadata_uuid
+reload_btrfs
+failure_recovery "./disk8.raw.xz" "./disk7.raw.xz" \
+		 check_multi_fsid_change_without_metadata_uuid
-- 
2.21.0 (Apple Git-122.2)

Re: [PATCH 11/11] btrfs-progs: misc-tests/034: add new test images and modify the script

[Nikolay Borisov]

On 12.12.19 г. 13:02 ч., damenly.su@gmail.com wrote:
> From: Su Yue <Damenly_Su@gmx.com>
> 
> Disk7 contains an image which has undergone a successful fsid change
> more than once with FSID_CHANGING_v2 flag. disk8 on the other hand is
> member of the same filesystem but has completed its last change without
> METADATA_UUID flag set.
> 
> The expected recovery result is both image don't have FSID_CHANGING_v2
> and INCOMPAT_METADATA_UUID. Change the test script to test it.
> 
> NOTE: It needs kernel fixes for metadata_uuid feature, otherwise
> will fail.
> 
> Signed-off-by: Su Yue <Damenly_Su@gmx.com>


This test has been superseded by my btrfs-progs test so can be dropped.

Re: [PATCH 01/11] btrfs-progs: misc-tests/034: reload btrfs module before running failure_recovery

[Nikolay Borisov]

On 12.12.19 г. 13:01 ч., damenly.su@gmail.com wrote:
> From: Su Yue <Damenly_Su@gmx.com>
> 
> One reload_btrfs is lost, add it.
> 
> Fixes: 0de2e22ad226 ("btrfs-progs: tests: Add tests for changing fsid feature")
> Signed-off-by: Su Yue <Damenly_Su@gmx.com>

Reviewed-by: Nikolay Borisov <nborisov@suse.com>

Re: [PATCH 02/11] btrfs-progs: misc-tests/034: mount the second device if first device mount failed

[Nikolay Borisov]

On 12.12.19 г. 13:01 ч., damenly.su@gmail.com wrote:
> From: Su Yue <Damenly_Su@gmx.com>
> 
> The 034 test may fail to mount, and dmesg says open_ctree() failed due
> to device missing.
> 
> The partly work flow is
> step1 loop1 = losetup image1
> step2 loop2 = losetup image2
> setp3 mount loop1
> 
> The dmesg says the loop2 device is missing.
> It's possible and known that while step3 is in open_ctree() and
> fs_devices->opened is nonzero, loop2 device has not been added into the


Care to give more details how this can happen? I haven't observed such a
failure, meaning it's likely due to some race condition. More details
are needed though. In your change log you say "it's known" but
apparently only to you in this case.

Re: [PATCH 00/11] btrfs-progs: metadata_uuid feature fixes and portation

[Nikolay Borisov]

On 12.12.19 г. 13:01 ч., damenly.su@gmail.com wrote:
> From: Su Yue <Damenly_Su@gmx.com>
> 
> The series are inspired by easy failing misc-tests/034.
> Those patches fix misc-tests/034 and add new test images.
> 
> After portation of kernel find fs_devices code, progs is able to
> work on devices with FSID_CHANGING_V2 flag, not sure whether the
> functionality is necessary. If not, I will remove it in next version.

For now I think it's best if this is not added. Kernel is supposed to
handle split-brain scenarios upon device scan which is triggered
automatically by udev. If the need arises in the future then we can
think about integrating this code in btrfs-progs.

Re: [PATCH 02/11] btrfs-progs: misc-tests/034: mount the second device if first device mount failed

[Su Yue]

On 2020/1/31 4:03 PM, Nikolay Borisov wrote:
>
>
> On 12.12.19 г. 13:01 ч., damenly.su@gmail.com wrote:
>> From: Su Yue <Damenly_Su@gmx.com>
>>
>> The 034 test may fail to mount, and dmesg says open_ctree() failed due
>> to device missing.
>>
>> The partly work flow is
>> step1 loop1 = losetup image1
>> step2 loop2 = losetup image2
>> setp3 mount loop1
>>
>> The dmesg says the loop2 device is missing.
>> It's possible and known that while step3 is in open_ctree() and
>> fs_devices->opened is nonzero, loop2 device has not been added into the
>
>
> Care to give more details how this can happen? I haven't observed such a
> failure, meaning it's likely due to some race condition. More details
> are needed though. In your change log you say "it's known" but
> apparently only to you in this case.
>

Sure. There's a device missing situation[1] if two
devices(raid 1/0) were caught by udev. Yes, it's
not related to the metadata fsid feature. It just
makes the mount operation due to the missing device then
the test fails.

In this script, $loop1 *may* be failed to be mounted because
$loop2 is "missing". Mounting $loop2 device can verify the
metadata fsid functionality but without the degraded option.


[1]: https://www.spinics.net/lists/linux-btrfs/msg96312.html

Re: [PATCH 00/11] btrfs-progs: metadata_uuid feature fixes and portation

[Su Yue]

On 2020/1/31 4:05 PM, Nikolay Borisov wrote:
>
>
> On 12.12.19 г. 13:01 ч., damenly.su@gmail.com wrote:
>> From: Su Yue <Damenly_Su@gmx.com>
>>
>> The series are inspired by easy failing misc-tests/034.
>> Those patches fix misc-tests/034 and add new test images.
>>
>> After portation of kernel find fs_devices code, progs is able to
>> work on devices with FSID_CHANGING_V2 flag, not sure whether the
>> functionality is necessary. If not, I will remove it in next version.
>
> For now I think it's best if this is not added. Kernel is supposed to
> handle split-brain scenarios upon device scan which is triggered
> automatically by udev. If the need arises in the future then we can
> think about integrating this code in btrfs-progs.
>

Okay, so drop patch[3-11].
>

Re: [PATCH 02/11] btrfs-progs: misc-tests/034: mount the second device if first device mount failed

[Nikolay Borisov]

On 31.01.20 г. 12:01 ч., Su Yue wrote:
> On 2020/1/31 4:03 PM, Nikolay Borisov wrote:
>>
>>
>> On 12.12.19 г. 13:01 ч., damenly.su@gmail.com wrote:
>>> From: Su Yue <Damenly_Su@gmx.com>
>>>
>>> The 034 test may fail to mount, and dmesg says open_ctree() failed due
>>> to device missing.
>>>
>>> The partly work flow is
>>> step1 loop1 = losetup image1
>>> step2 loop2 = losetup image2
>>> setp3 mount loop1
>>>
>>> The dmesg says the loop2 device is missing.
>>> It's possible and known that while step3 is in open_ctree() and
>>> fs_devices->opened is nonzero, loop2 device has not been added into the
>>
>>
>> Care to give more details how this can happen? I haven't observed such a
>> failure, meaning it's likely due to some race condition. More details
>> are needed though. In your change log you say "it's known" but
>> apparently only to you in this case.
>>
> 
> Sure. There's a device missing situation[1] if two
> devices(raid 1/0) were caught by udev. Yes, it's
> not related to the metadata fsid feature. It just
> makes the mount operation due to the missing device then
> the test fails.

Ok but in those mail posts it says the problem occurs if we have a
multi-device btrfs volume, in this case raid1, and one of the devices is
missing. The pertinent question is why would any of the testing devices
be missing? Did you actually experience such failure ? loop1 is acquired
after running losetup --find --show, implying that after the command is
finished the given loopback device is fully present to the system?



> 
> In this script, $loop1 *may* be failed to be mounted because
> $loop2 is "missing". Mounting $loop2 device can verify the
> metadata fsid functionality but without the degraded option.
> 
> 
> [1]: https://www.spinics.net/lists/linux-btrfs/msg96312.html

Re: [PATCH 02/11] btrfs-progs: misc-tests/034: mount the second device if first device mount failed

[Su Yue]

On 2020/1/31 8:47 PM, Nikolay Borisov wrote:
>
>
> On 31.01.20 г. 12:01 ч., Su Yue wrote:
>> On 2020/1/31 4:03 PM, Nikolay Borisov wrote:
>>>
>>>
>>> On 12.12.19 г. 13:01 ч., damenly.su@gmail.com wrote:
>>>> From: Su Yue <Damenly_Su@gmx.com>
>>>>
>>>> The 034 test may fail to mount, and dmesg says open_ctree() failed due
>>>> to device missing.
>>>>
>>>> The partly work flow is
>>>> step1 loop1 = losetup image1
>>>> step2 loop2 = losetup image2
>>>> setp3 mount loop1
>>>>
>>>> The dmesg says the loop2 device is missing.
>>>> It's possible and known that while step3 is in open_ctree() and
>>>> fs_devices->opened is nonzero, loop2 device has not been added into the
>>>
>>>
>>> Care to give more details how this can happen? I haven't observed such a
>>> failure, meaning it's likely due to some race condition. More details
>>> are needed though. In your change log you say "it's known" but
>>> apparently only to you in this case.
>>>
>>
>> Sure. There's a device missing situation[1] if two
>> devices(raid 1/0) were caught by udev. Yes, it's
>> not related to the metadata fsid feature. It just
>> makes the mount operation due to the missing device then
>> the test fails.
>
> Ok but in those mail posts it says the problem occurs if we have a
> multi-device btrfs volume, in this case raid1, and one of the devices is
> missing. The pertinent question is why would any of the testing devices
> be missing? Did you actually experience such failure ? loop1 is acquired
> after running losetup --find --show, implying that after the command is
> finished the given loopback device is fully present to the system?
>
>
Yes, I did experience such failures. Although I'm not familiar with
udevd, found something for your questions.
My superficial answers blow after looking through loop device
and udevd codes. If found something wrong please correct me.

After "losetup --find --show", the loopback devices are full
present to the system. And userspace received uevents from
kernel. Losetup only handles the loopback device things but
not such fs things on the loopback device. It's udevd' work to
handle the uevent for the device by rules.

The issue is that udevd may be handling the uevent of one device while
doing mount the another device. For btrfs, udevd calls ioctls on
/dev/btrfs-control.


Thread *mounting device1*            Thread *scanning device2*


btrfs_mount_root                     btrfs_control_ioctl

   mutex_lock(&uuid_mutex);

     btrfs_read_disk_super
     btrfs_scan_one_device
     --> there is only device1
     in the fs_devices

     btrfs_open_devices
       fs_devices->opened = 1
     mutex_unlock(&uuid_mutex);

                                       mutex_lock(&uuid_mutex);
                                       btrfs_scan_one_device
                                         btrfs_read_disk_super

                                         device_list_add
                                           found fs_devices
                                             device = btrfs_find_device

                                           if (!device)
                                              if(fs_devices->opened)
                                                 return -EBUSY
                                              --> the device2 adding
                                                  aborts since
						 fs_devices was opened
                                       mutex_unlock(&uuid_mutex);
   btrfs_fill_super
     open_ctree
       btrfs_read_sys_array
         read_one_chunk
	--> error due to the
	    device2 missing


Then mount failed because of the missing device.


>
>>
>> In this script, $loop1 *may* be failed to be mounted because
>> $loop2 is "missing". Mounting $loop2 device can verify the
>> metadata fsid functionality but without the degraded option.
>>
>>
>> [1]: https://www.spinics.net/lists/linux-btrfs/msg96312.html

Re: [PATCH 00/11] btrfs-progs: metadata_uuid feature fixes and portation

[David Sterba]

On Fri, Jan 31, 2020 at 06:04:42PM +0800, Su Yue wrote:
> On 2020/1/31 4:05 PM, Nikolay Borisov wrote:
> >
> >
> > On 12.12.19 г. 13:01 ч., damenly.su@gmail.com wrote:
> >> From: Su Yue <Damenly_Su@gmx.com>
> >>
> >> The series are inspired by easy failing misc-tests/034.
> >> Those patches fix misc-tests/034 and add new test images.
> >>
> >> After portation of kernel find fs_devices code, progs is able to
> >> work on devices with FSID_CHANGING_V2 flag, not sure whether the
> >> functionality is necessary. If not, I will remove it in next version.
> >
> > For now I think it's best if this is not added. Kernel is supposed to
> > handle split-brain scenarios upon device scan which is triggered
> > automatically by udev. If the need arises in the future then we can
> > think about integrating this code in btrfs-progs.
> >
> 
> Okay, so drop patch[3-11].

So patches 1 and 2 have been applied. Thanks.

Re: [PATCH 00/11] btrfs-progs: metadata_uuid feature fixes and portation

[Su Yue]

On 2020/3/4 10:14 PM, David Sterba wrote:
> On Fri, Jan 31, 2020 at 06:04:42PM +0800, Su Yue wrote:
>> On 2020/1/31 4:05 PM, Nikolay Borisov wrote:
>>>
>>>
>>> On 12.12.19 г. 13:01 ч., damenly.su@gmail.com wrote:
>>>> From: Su Yue <Damenly_Su@gmx.com>
>>>>
>>>> The series are inspired by easy failing misc-tests/034.
>>>> Those patches fix misc-tests/034 and add new test images.
>>>>
>>>> After portation of kernel find fs_devices code, progs is able to
>>>> work on devices with FSID_CHANGING_V2 flag, not sure whether the
>>>> functionality is necessary. If not, I will remove it in next version.
>>>
>>> For now I think it's best if this is not added. Kernel is supposed to
>>> handle split-brain scenarios upon device scan which is triggered
>>> automatically by udev. If the need arises in the future then we can
>>> think about integrating this code in btrfs-progs.
>>>
>>
>> Okay, so drop patch[3-11].
>
> So patches 1 and 2 have been applied. Thanks.
>
Sorry. David, please remove the patch 2 if applied.
There is another better solution applied by Marcos
https://www.spinics.net/lists/linux-btrfs/msg98370.html.

Try to update status about dropping things on time next time:).

Thanks

Re: [PATCH 00/11] btrfs-progs: metadata_uuid feature fixes and portation

[David Sterba]

On Thu, Mar 05, 2020 at 09:18:26AM +0800, Su Yue wrote:
> On 2020/3/4 10:14 PM, David Sterba wrote:
> > On Fri, Jan 31, 2020 at 06:04:42PM +0800, Su Yue wrote:
> >> On 2020/1/31 4:05 PM, Nikolay Borisov wrote:
> >>>
> >>>
> >>> On 12.12.19 г. 13:01 ч., damenly.su@gmail.com wrote:
> >>>> From: Su Yue <Damenly_Su@gmx.com>
> >>>>
> >>>> The series are inspired by easy failing misc-tests/034.
> >>>> Those patches fix misc-tests/034 and add new test images.
> >>>>
> >>>> After portation of kernel find fs_devices code, progs is able to
> >>>> work on devices with FSID_CHANGING_V2 flag, not sure whether the
> >>>> functionality is necessary. If not, I will remove it in next version.
> >>>
> >>> For now I think it's best if this is not added. Kernel is supposed to
> >>> handle split-brain scenarios upon device scan which is triggered
> >>> automatically by udev. If the need arises in the future then we can
> >>> think about integrating this code in btrfs-progs.
> >>>
> >>
> >> Okay, so drop patch[3-11].
> >
> > So patches 1 and 2 have been applied. Thanks.
> >
> Sorry. David, please remove the patch 2 if applied.
> There is another better solution applied by Marcos
> https://www.spinics.net/lists/linux-btrfs/msg98370.html.
> 
> Try to update status about dropping things on time next time:).

No problem, this could happen when there's long delay between posting
and merging. Thanks for letting me know.