[PATCH] fs/btrfs: fix a bug that U-boot fs btrfs implementation doesn't handle NO_HOLE feature correctly

[PATCH] fs/btrfs: fix a bug that U-boot fs btrfs implementation doesn't handle NO_HOLE feature correctly

[Qu Wenruo]

[BUG]
When passing a btrfs with NO_HOLE feature to U-boot, and if one file
contains holes, then the hash of the file is not correct in U-boot:

 # mkfs.btrfs -f test.img	# Since v5.15, mkfs defaults to NO_HOLES
 # mount test.img /mnt/btrfs
 # xfs_io -f -c "pwrite 0 4k" -c "pwrite 8k 4k" /mnt/btrfs/file
 # md5sum /mnt/btrfs/file
 277f3840b275c74d01e979ea9d75ac19  /mnt/btrfs/file
 # umount /mnt/btrfs
 # ./u-boot
 => host bind 0 /home/adam/test.img
 => ls host 0
 <   >      12288  Mon Dec 27 05:35:23 2021  file
 => load host 0 0x1000000 file
 12288 bytes read in 0 ms
 => md5sum 0x1000000 0x3000
 md5 for 01000000 ... 01002fff ==> 855ffdbe4d0ccc5acab92e1b5330e4c1

The md5sum doesn't match at all.

[CAUSE]
In U-boot btrfs implementation, the function btrfs_read_file() has the
following iteration for file extent iteration:

	/* Read the aligned part */
	while (cur < aligned_end) {
		ret = lookup_data_extent(root, &path, ino, cur, &next_offset);
		if (ret < 0)
			goto out;
		if (ret > 0) {
			/* No next, direct exit */
			if (!next_offset) {
				ret = 0;
				goto out;
			}
		}
		/* Read file extent */

But for NO_HOLES features, hole extents will not have any extent item
for it.
Thus if @cur is at a hole, lookup_data_extent() will just return >0, and
update @next_offset.

But we still believe there is some data to read for @cur for ret > 0
case, causing we read extent data from the next file extent.

This means, what we do for above NO_HOLES btrfs is:
- Read 4K data from disk to file offset [0, 4K)
  So far the data is still correct

- Read 4K data from disk to file offset [4K, 8K)
  We didn't skip the 4K hole, but read the data at file offset [8K, 12K)
  into file offset [4K, 8K).

  This causes the checksum mismatch.

[FIX]
Add extra check to skip to the next non-hole range after
lookup_data_extent().

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
This bug exposed another missing link, that we don't have good test
coverage in U-boot btrfs.

This is partially caused by the fact that, btrfs-progs code is not
designed to read file contents, but just to check the cross-reference
(aka, btrfs-check).

If we really only want read-only support in U-boot, and don't ever plan
to add write support, then I'd say the btrfs-fuse project
(https://github.com/adam900710/btrfs-fuse/) is more suitable for U-boot.

As that project already has full fs content verification selftest along
with extra multi-device recovery tests.
And shares the same code style between btrfs-progs/kernel.
---
 fs/btrfs/inode.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 2c2379303d74..d00b5153336d 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -717,6 +717,14 @@ int btrfs_file_read(struct btrfs_root *root, u64 ino, u64 file_offset, u64 len,
 				ret = 0;
 				goto out;
 			}
+			/*
+			 * Find a extent gap, mostly caused by NO_HOLE feature.
+			 * Just to next offset directly.
+			 */
+			if (next_offset > cur) {
+				cur = next_offset;
+				continue;
+			}
 		}
 		fi = btrfs_item_ptr(path.nodes[0], path.slots[0],
 				    struct btrfs_file_extent_item);
-- 
2.34.1

Re: [PATCH] fs/btrfs: fix a bug that U-boot fs btrfs implementation doesn't handle NO_HOLE feature correctly

[Qu Wenruo]

On 2021/12/27 14:11, Qu Wenruo wrote:
> [BUG]
> When passing a btrfs with NO_HOLE feature to U-boot, and if one file
> contains holes, then the hash of the file is not correct in U-boot:
> 
>   # mkfs.btrfs -f test.img	# Since v5.15, mkfs defaults to NO_HOLES
>   # mount test.img /mnt/btrfs
>   # xfs_io -f -c "pwrite 0 4k" -c "pwrite 8k 4k" /mnt/btrfs/file
>   # md5sum /mnt/btrfs/file
>   277f3840b275c74d01e979ea9d75ac19  /mnt/btrfs/file
>   # umount /mnt/btrfs
>   # ./u-boot
>   => host bind 0 /home/adam/test.img
>   => ls host 0
>   <   >      12288  Mon Dec 27 05:35:23 2021  file
>   => load host 0 0x1000000 file
>   12288 bytes read in 0 ms
>   => md5sum 0x1000000 0x3000
>   md5 for 01000000 ... 01002fff ==> 855ffdbe4d0ccc5acab92e1b5330e4c1
> 
> The md5sum doesn't match at all.
> 
> [CAUSE]
> In U-boot btrfs implementation, the function btrfs_read_file() has the
> following iteration for file extent iteration:
> 
> 	/* Read the aligned part */
> 	while (cur < aligned_end) {
> 		ret = lookup_data_extent(root, &path, ino, cur, &next_offset);
> 		if (ret < 0)
> 			goto out;
> 		if (ret > 0) {
> 			/* No next, direct exit */
> 			if (!next_offset) {
> 				ret = 0;
> 				goto out;
> 			}
> 		}
> 		/* Read file extent */
> 
> But for NO_HOLES features, hole extents will not have any extent item
> for it.
> Thus if @cur is at a hole, lookup_data_extent() will just return >0, and
> update @next_offset.
> 
> But we still believe there is some data to read for @cur for ret > 0
> case, causing we read extent data from the next file extent.
> 
> This means, what we do for above NO_HOLES btrfs is:
> - Read 4K data from disk to file offset [0, 4K)
>    So far the data is still correct
> 
> - Read 4K data from disk to file offset [4K, 8K)
>    We didn't skip the 4K hole, but read the data at file offset [8K, 12K)
>    into file offset [4K, 8K).
> 
>    This causes the checksum mismatch.
> 
> [FIX]
> Add extra check to skip to the next non-hole range after
> lookup_data_extent().
> 
> Signed-off-by: Qu Wenruo <wqu@suse.com>
> ---
> This bug exposed another missing link, that we don't have good test
> coverage in U-boot btrfs.
> 
> This is partially caused by the fact that, btrfs-progs code is not
> designed to read file contents, but just to check the cross-reference
> (aka, btrfs-check).
> 
> If we really only want read-only support in U-boot, and don't ever plan
> to add write support, then I'd say the btrfs-fuse project
> (https://github.com/adam900710/btrfs-fuse/) is more suitable for U-boot.
> 
> As that project already has full fs content verification selftest along
> with extra multi-device recovery tests.
> And shares the same code style between btrfs-progs/kernel.

OK, things are not that bad.

In fact, the btrfs_read_file() implementation in btrfs-fuse has the same 
naming, same lookup_file_extent() (just a little naming different than 
lookup_data_extent()), same parameter list.

Just without the unaligned sector handling (handled by FUSE, and it may 
also be unnecessary for U-boot too), and already have the correct 
handling for lookup_file_extent(), thanks to the selftest.

So this already means, it can be pretty easy for U-boot to take code 
from btrfs-fuse part by part, without huge refactor again.

Thanks,
Qu

> ---
>   fs/btrfs/inode.c | 8 ++++++++
>   1 file changed, 8 insertions(+)
> 
> diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
> index 2c2379303d74..d00b5153336d 100644
> --- a/fs/btrfs/inode.c
> +++ b/fs/btrfs/inode.c
> @@ -717,6 +717,14 @@ int btrfs_file_read(struct btrfs_root *root, u64 ino, u64 file_offset, u64 len,
>   				ret = 0;
>   				goto out;
>   			}
> +			/*
> +			 * Find a extent gap, mostly caused by NO_HOLE feature.
> +			 * Just to next offset directly.
> +			 */
> +			if (next_offset > cur) {
> +				cur = next_offset;
> +				continue;
> +			}
>   		}
>   		fi = btrfs_item_ptr(path.nodes[0], path.slots[0],
>   				    struct btrfs_file_extent_item);

Re: [PATCH] fs/btrfs: fix a bug that U-boot fs btrfs implementation doesn't handle NO_HOLE feature correctly

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 2165 bytes --]

On Mon, Dec 27, 2021 at 02:11:14PM +0800, Qu Wenruo wrote:

> [BUG]
> When passing a btrfs with NO_HOLE feature to U-boot, and if one file
> contains holes, then the hash of the file is not correct in U-boot:
> 
>  # mkfs.btrfs -f test.img	# Since v5.15, mkfs defaults to NO_HOLES
>  # mount test.img /mnt/btrfs
>  # xfs_io -f -c "pwrite 0 4k" -c "pwrite 8k 4k" /mnt/btrfs/file
>  # md5sum /mnt/btrfs/file
>  277f3840b275c74d01e979ea9d75ac19  /mnt/btrfs/file
>  # umount /mnt/btrfs
>  # ./u-boot
>  => host bind 0 /home/adam/test.img
>  => ls host 0
>  <   >      12288  Mon Dec 27 05:35:23 2021  file
>  => load host 0 0x1000000 file
>  12288 bytes read in 0 ms
>  => md5sum 0x1000000 0x3000
>  md5 for 01000000 ... 01002fff ==> 855ffdbe4d0ccc5acab92e1b5330e4c1
> 
> The md5sum doesn't match at all.
> 
> [CAUSE]
> In U-boot btrfs implementation, the function btrfs_read_file() has the
> following iteration for file extent iteration:
> 
> 	/* Read the aligned part */
> 	while (cur < aligned_end) {
> 		ret = lookup_data_extent(root, &path, ino, cur, &next_offset);
> 		if (ret < 0)
> 			goto out;
> 		if (ret > 0) {
> 			/* No next, direct exit */
> 			if (!next_offset) {
> 				ret = 0;
> 				goto out;
> 			}
> 		}
> 		/* Read file extent */
> 
> But for NO_HOLES features, hole extents will not have any extent item
> for it.
> Thus if @cur is at a hole, lookup_data_extent() will just return >0, and
> update @next_offset.
> 
> But we still believe there is some data to read for @cur for ret > 0
> case, causing we read extent data from the next file extent.
> 
> This means, what we do for above NO_HOLES btrfs is:
> - Read 4K data from disk to file offset [0, 4K)
>   So far the data is still correct
> 
> - Read 4K data from disk to file offset [4K, 8K)
>   We didn't skip the 4K hole, but read the data at file offset [8K, 12K)
>   into file offset [4K, 8K).
> 
>   This causes the checksum mismatch.
> 
> [FIX]
> Add extra check to skip to the next non-hole range after
> lookup_data_extent().
> 
> Signed-off-by: Qu Wenruo <wqu@suse.com>

Applied to u-boot/master, thanks!

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]