linux-btrfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 0/3] btrfs-progs: convert: better ENOSPC handling
@ 2020-07-29  8:40 Qu Wenruo
  2020-07-29  8:40 ` [PATCH 1/3] btrfs-progs: convert: handle errors better in ext2_copy_inodes() Qu Wenruo
                   ` (3 more replies)
  0 siblings, 4 replies; 9+ messages in thread
From: Qu Wenruo @ 2020-07-29  8:40 UTC (permalink / raw)
  To: linux-btrfs

This patchset is to address a bug report [1], where even with the bit
overflow bug fixed, the user is still unable to convert an ext4 fs to
btrfs.

The error is -ENOSPC, which triggers BUG_ON() and brings the end to the
convertion.

We're still waiting for the image dump to determine what's the real
cause is, but considering the user is still reporting around 40% free
space, I guess it's something wrong with the extent allocator.

But still, we can enhance btrfs-convert to make it handle errors more
gracefully, with better error message, and even some debugging info like
the available space / total space ratio.

Qu Wenruo (3):
  btrfs-progs: convert: handle errors better in ext2_copy_inodes()
  btrfs-progs: convert: update error message to reflect original fs
    unmodified cases
  btrfs-progs: convert: report available space before convertion happens

 convert/common.h      |  9 +++++++++
 convert/main.c        | 34 +++++++++++++++++++++++++++++++---
 convert/source-ext2.c | 42 +++++++++++++++++++++++++++++++-----------
 convert/source-fs.c   |  1 +
 4 files changed, 72 insertions(+), 14 deletions(-)

-- 
2.27.0


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH 1/3] btrfs-progs: convert: handle errors better in ext2_copy_inodes()
  2020-07-29  8:40 [PATCH 0/3] btrfs-progs: convert: better ENOSPC handling Qu Wenruo
@ 2020-07-29  8:40 ` Qu Wenruo
  2020-07-29  8:40 ` [PATCH 2/3] btrfs-progs: convert: update error message to reflect original fs unmodified cases Qu Wenruo
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 9+ messages in thread
From: Qu Wenruo @ 2020-07-29  8:40 UTC (permalink / raw)
  To: linux-btrfs

This patch will enhance the error handling of ext2_copy_inodes by:
- Return more meaningful error number
  Instead of -1 (-EPERM), now return -EIO for ext2 calls error, and
  proper error number from btrfs calls.

- Commit transaction \if ext2fs_open_inode_scan() failed

- Call ext2fs_close_inode_scan() on error

- Hunt down the BUG_ON()s

- Add error messages for transaction related calls

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 convert/source-ext2.c | 42 +++++++++++++++++++++++++++++++-----------
 1 file changed, 31 insertions(+), 11 deletions(-)

diff --git a/convert/source-ext2.c b/convert/source-ext2.c
index d73684ef19e7..26514a09c9f1 100644
--- a/convert/source-ext2.c
+++ b/convert/source-ext2.c
@@ -797,7 +797,7 @@ static int ext2_copy_inodes(struct btrfs_convert_context *cctx,
 			    u32 convert_flags, struct task_ctx *p)
 {
 	ext2_filsys ext2_fs = cctx->fs_data;
-	int ret;
+	int ret = 0;
 	errcode_t err;
 	ext2_inode_scan ext2_scan;
 	struct ext2_inode ext2_inode;
@@ -810,8 +810,9 @@ static int ext2_copy_inodes(struct btrfs_convert_context *cctx,
 		return PTR_ERR(trans);
 	err = ext2fs_open_inode_scan(ext2_fs, 0, &ext2_scan);
 	if (err) {
-		fprintf(stderr, "ext2fs_open_inode_scan: %s\n", error_message(err));
-		return -1;
+		error("ext2fs_open_inode_scan failed: %s", error_message(err));
+		btrfs_commit_transaction(trans, root);
+		return -EIO;
 	}
 	while (!(err = ext2fs_get_next_inode(ext2_scan, &ext2_ino,
 					     &ext2_inode))) {
@@ -827,8 +828,11 @@ static int ext2_copy_inodes(struct btrfs_convert_context *cctx,
 		pthread_mutex_lock(&p->mutex);
 		p->cur_copy_inodes++;
 		pthread_mutex_unlock(&p->mutex);
-		if (ret)
-			return ret;
+		if (ret) {
+			error("failed to copy ext2 inode %u: %d", ext2_ino,
+			      ret);
+			goto out;
+		}
 		/*
 		 * blocks_used is the number of new tree blocks allocated in
 		 * current transaction.
@@ -842,17 +846,33 @@ static int ext2_copy_inodes(struct btrfs_convert_context *cctx,
 		 */
 		if (trans->blocks_used >= SZ_2M / root->fs_info->nodesize) {
 			ret = btrfs_commit_transaction(trans, root);
-			BUG_ON(ret);
+			if (ret < 0) {
+				error("failed to commit transaction: %d", ret);
+				goto out;
+			}
 			trans = btrfs_start_transaction(root, 1);
-			BUG_ON(IS_ERR(trans));
+			if (IS_ERR(trans)) {
+				ret = PTR_ERR(trans);
+				error("failed to start transaction: %d", ret);
+				trans = NULL;
+				goto out;
+			}
 		}
 	}
 	if (err) {
-		fprintf(stderr, "ext2fs_get_next_inode: %s\n", error_message(err));
-		return -1;
+		error("ext2fs_get_next_inode failed: %s", error_message(err));
+		ret = -EIO;
+		goto out;
+	}
+out:
+	if (ret < 0) {
+		if (trans)
+			btrfs_abort_transaction(trans, ret);
+	} else {
+		ret = btrfs_commit_transaction(trans, root);
+		if (ret < 0)
+			error("failed to commit transaction: %d", ret);
 	}
-	ret = btrfs_commit_transaction(trans, root);
-	BUG_ON(ret);
 	ext2fs_close_inode_scan(ext2_scan);
 
 	return ret;
-- 
2.27.0


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH 2/3] btrfs-progs: convert: update error message to reflect original fs unmodified cases
  2020-07-29  8:40 [PATCH 0/3] btrfs-progs: convert: better ENOSPC handling Qu Wenruo
  2020-07-29  8:40 ` [PATCH 1/3] btrfs-progs: convert: handle errors better in ext2_copy_inodes() Qu Wenruo
@ 2020-07-29  8:40 ` Qu Wenruo
  2020-07-29  8:40 ` [PATCH 3/3] btrfs-progs: convert: report available space before convertion happens Qu Wenruo
  2020-07-31 16:16 ` [PATCH 0/3] btrfs-progs: convert: better ENOSPC handling David Sterba
  3 siblings, 0 replies; 9+ messages in thread
From: Qu Wenruo @ 2020-07-29  8:40 UTC (permalink / raw)
  To: linux-btrfs

The original fs is not touched until we migrate the super blocks.

Under most error cases, we fail before that thus the original fs is
still safe.

So change the error message according the stages we failed to reflect
that.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 convert/main.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/convert/main.c b/convert/main.c
index df6a2ae32722..451e4f158689 100644
--- a/convert/main.c
+++ b/convert/main.c
@@ -1123,6 +1123,7 @@ static int do_convert(const char *devname, u32 convert_flags, u32 nodesize,
 	struct task_ctx ctx;
 	char features_buf[64];
 	struct btrfs_mkfs_config mkfs_cfg;
+	bool btrfs_sb_commited = false;
 
 	init_convert_context(&cctx);
 	ret = convert_open_fs(devname, &cctx);
@@ -1270,6 +1271,7 @@ static int do_convert(const char *devname, u32 convert_flags, u32 nodesize,
 		error("unable to migrate super block: %d", ret);
 		goto fail;
 	}
+	btrfs_sb_commited = true;
 
 	root = open_ctree_fd(fd, devname, 0,
 			     OPEN_CTREE_WRITES | OPEN_CTREE_TEMPORARY_SUPER);
@@ -1287,8 +1289,12 @@ fail:
 	clean_convert_context(&cctx);
 	if (fd != -1)
 		close(fd);
-	warning(
+	if (btrfs_sb_commited)
+		warning(
 "an error occurred during conversion, filesystem is partially created but not finalized and not mountable");
+	else
+		warning(
+"an error occurred during conversion, the original filesystem is not modified");
 	return -1;
 }
 
-- 
2.27.0


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH 3/3] btrfs-progs: convert: report available space before convertion happens
  2020-07-29  8:40 [PATCH 0/3] btrfs-progs: convert: better ENOSPC handling Qu Wenruo
  2020-07-29  8:40 ` [PATCH 1/3] btrfs-progs: convert: handle errors better in ext2_copy_inodes() Qu Wenruo
  2020-07-29  8:40 ` [PATCH 2/3] btrfs-progs: convert: update error message to reflect original fs unmodified cases Qu Wenruo
@ 2020-07-29  8:40 ` Qu Wenruo
  2020-07-29  9:05   ` Su Yue
  2020-07-31 16:16 ` [PATCH 0/3] btrfs-progs: convert: better ENOSPC handling David Sterba
  3 siblings, 1 reply; 9+ messages in thread
From: Qu Wenruo @ 2020-07-29  8:40 UTC (permalink / raw)
  To: linux-btrfs

Now if an ENOSPC error happened, the free space report would help user
to determine if it's a real ENOSPC or a bug in btrfs.

The reported free space is the calculated free space, which doesn't
include super block space, nor merged data chunks.

The free space is always smaller than the reported available space of
the original fs, as we need extra padding space for used space to avoid
too fragmented data chunks.

The output exact would be:

$ ./btrfs-convert  /dev/nvme/btrfs
create btrfs filesystem:
        blocksize: 4096
        nodesize:  16384
        features:  extref, skinny-metadata (default)
        checksum:  crc32c
free space report: free / total = 0 / 10737418240 (0%) <<<
ERROR: unable to create initial ctree: No space left on device
WARNING: an error occurred during conversion, the original filesystem is not modified

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 convert/common.h    |  9 +++++++++
 convert/main.c      | 26 ++++++++++++++++++++++++--
 convert/source-fs.c |  1 +
 3 files changed, 34 insertions(+), 2 deletions(-)

diff --git a/convert/common.h b/convert/common.h
index 7ec26cd996d3..2c7799433294 100644
--- a/convert/common.h
+++ b/convert/common.h
@@ -35,6 +35,7 @@ struct btrfs_convert_context {
 	u64 inodes_count;
 	u64 free_inodes_count;
 	u64 total_bytes;
+	u64 free_bytes_initial;
 	char *volume_name;
 	const struct btrfs_convert_operations *convert_ops;
 
@@ -47,6 +48,14 @@ struct btrfs_convert_context {
 	/* Free space which is not covered by data_chunks */
 	struct cache_tree free_space;
 
+	/*
+	 * Free space reserved for ENOSPC report, it's just a copy
+	 * free_space.
+	 * But after initial calculation, free_space_initial is no longer
+	 * updated, so we have a good idea on how many free space we really
+	 * have for btrfs.
+	 */
+	struct cache_tree free_space_initial;
 	void *fs_data;
 };
 
diff --git a/convert/main.c b/convert/main.c
index 451e4f158689..49c95e092cbb 100644
--- a/convert/main.c
+++ b/convert/main.c
@@ -727,6 +727,23 @@ out:
 	return ret;
 }
 
+static int copy_free_space_tree(struct btrfs_convert_context *cctx)
+{
+	struct cache_tree *src = &cctx->free_space;
+	struct cache_tree *dst = &cctx->free_space_initial;
+	struct cache_extent *cache;
+	int ret = 0;
+
+	for (cache = search_cache_extent(src, 0); cache;
+	     cache = next_cache_extent(cache)) {
+		ret = add_merge_cache_extent(dst, cache->start, cache->size);
+		if (ret < 0)
+			return ret;
+		cctx->free_bytes_initial += cache->size;
+	}
+	return ret;
+}
+
 /*
  * Read used space, and since we have the used space,
  * calculate data_chunks and free for later mkfs
@@ -740,7 +757,10 @@ static int convert_read_used_space(struct btrfs_convert_context *cctx)
 		return ret;
 
 	ret = calculate_available_space(cctx);
-	return ret;
+	if (ret < 0)
+		return ret;
+
+	return copy_free_space_tree(cctx);
 }
 
 /*
@@ -1165,7 +1185,9 @@ static int do_convert(const char *devname, u32 convert_flags, u32 nodesize,
 	printf("\tnodesize:  %u\n", nodesize);
 	printf("\tfeatures:  %s\n", features_buf);
 	printf("\tchecksum:  %s\n", btrfs_super_csum_name(csum_type));
-
+	printf("free space report: free / total = %llu / %llu (%lld%%)\n",
+		cctx.free_bytes_initial, cctx.total_bytes,
+		cctx.free_bytes_initial * 100 / cctx.total_bytes);
 	memset(&mkfs_cfg, 0, sizeof(mkfs_cfg));
 	mkfs_cfg.csum_type = csum_type;
 	mkfs_cfg.label = cctx.volume_name;
diff --git a/convert/source-fs.c b/convert/source-fs.c
index f7fd3d6055b7..d2f7a825238d 100644
--- a/convert/source-fs.c
+++ b/convert/source-fs.c
@@ -74,6 +74,7 @@ void init_convert_context(struct btrfs_convert_context *cctx)
 	cache_tree_init(&cctx->used_space);
 	cache_tree_init(&cctx->data_chunks);
 	cache_tree_init(&cctx->free_space);
+	cache_tree_init(&cctx->free_space_initial);
 }
 
 void clean_convert_context(struct btrfs_convert_context *cctx)
-- 
2.27.0


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* Re: [PATCH 3/3] btrfs-progs: convert: report available space before convertion happens
  2020-07-29  8:40 ` [PATCH 3/3] btrfs-progs: convert: report available space before convertion happens Qu Wenruo
@ 2020-07-29  9:05   ` Su Yue
  2020-07-29  9:13     ` Qu Wenruo
  0 siblings, 1 reply; 9+ messages in thread
From: Su Yue @ 2020-07-29  9:05 UTC (permalink / raw)
  To: Qu Wenruo; +Cc: linux-btrfs


On Wed 29 Jul 2020 at 16:40, Qu Wenruo <wqu@suse.com> wrote:

> Now if an ENOSPC error happened, the free space report would
> help user to determine if it's a real ENOSPC or a bug in btrfs.
>
> The reported free space is the calculated free space, which
> doesn't include super block space, nor merged data chunks.
>
> The free space is always smaller than the reported available
> space of the original fs, as we need extra padding space for
> used space to avoid too fragmented data chunks.
>
> The output exact would be:
>
> $ ./btrfs-convert  /dev/nvme/btrfs create btrfs filesystem:
>         blocksize: 4096 nodesize:  16384 features:  extref,
>         skinny-metadata (default) checksum:  crc32c
> free space report: free / total = 0 / 10737418240 (0%) <<<
> ERROR: unable to create initial ctree: No space left on device
> WARNING: an error occurred during conversion, the original
> filesystem is not modified
>
> Signed-off-by: Qu Wenruo <wqu@suse.com> ---
>  convert/common.h    |  9 +++++++++ convert/main.c      | 26
>  ++++++++++++++++++++++++-- convert/source-fs.c |  1 + 3 files
>  changed, 34 insertions(+), 2 deletions(-)
>
> diff --git a/convert/common.h b/convert/common.h index
> 7ec26cd996d3..2c7799433294 100644 --- a/convert/common.h +++
> b/convert/common.h @@ -35,6 +35,7 @@ struct
> btrfs_convert_context {
>  	u64 inodes_count; u64 free_inodes_count; u64 total_bytes;
> +	u64 free_bytes_initial;
>  	char *volume_name; const struct btrfs_convert_operations
>  *convert_ops;
> @@ -47,6 +48,14 @@ struct btrfs_convert_context {
>  	/* Free space which is not covered by data_chunks */ struct
>  cache_tree free_space;
> +	/* +	 * Free space reserved for ENOSPC report, it's just a
> copy +	 * free_space.  +	 * But after initial calculation,
> free_space_initial is no longer +	 * updated, so we have a good
> idea on how many free space we really +	 * have for btrfs.  +
> */ +	struct cache_tree free_space_initial;
>  	void *fs_data; };
> diff --git a/convert/main.c b/convert/main.c index
> 451e4f158689..49c95e092cbb 100644 --- a/convert/main.c +++
> b/convert/main.c @@ -727,6 +727,23 @@ out:
>  	return ret; }
> +static int copy_free_space_tree(struct btrfs_convert_context
> *cctx) +{ +	struct cache_tree *src = &cctx->free_space; +
> struct cache_tree *dst = &cctx->free_space_initial; +	struct
> cache_extent *cache; +	int ret = 0; + +	for (cache =
> search_cache_extent(src, 0); cache; +	     cache =
> next_cache_extent(cache)) { +		ret =
> add_merge_cache_extent(dst, cache->start, cache->size); +		if
> (ret < 0) +			return ret; +
> cctx->free_bytes_initial += cache->size; +	} +	return ret; +}
> +
>  /*
>   * Read used space, and since we have the used space, *
>   calculate data_chunks and free for later mkfs
> @@ -740,7 +757,10 @@ static int convert_read_used_space(struct
> btrfs_convert_context *cctx)
>  		return ret;  ret = calculate_available_space(cctx);
> -	return ret; +	if (ret < 0) +		return ret; + +	return
> copy_free_space_tree(cctx);
>  }  /*
> @@ -1165,7 +1185,9 @@ static int do_convert(const char *devname,
> u32 convert_flags, u32 nodesize,
>  	printf("\tnodesize:  %u\n", nodesize); printf("\tfeatures:
>  %s\n", features_buf); printf("\tchecksum:  %s\n",
>  btrfs_super_csum_name(csum_type));
> - +	printf("free space report: free / total = %llu / %llu
> (%lld%%)\n", +		cctx.free_bytes_initial, cctx.total_bytes,
> +		cctx.free_bytes_initial * 100 / cctx.total_bytes);
>  	memset(&mkfs_cfg, 0, sizeof(mkfs_cfg)); mkfs_cfg.csum_type =
>  csum_type; mkfs_cfg.label = cctx.volume_name;
> diff --git a/convert/source-fs.c b/convert/source-fs.c index
> f7fd3d6055b7..d2f7a825238d 100644 --- a/convert/source-fs.c +++
> b/convert/source-fs.c @@ -74,6 +74,7 @@ void
> init_convert_context(struct btrfs_convert_context *cctx)
>  	cache_tree_init(&cctx->used_space);
>  cache_tree_init(&cctx->data_chunks);
>  cache_tree_init(&cctx->free_space);
> +	cache_tree_init(&cctx->free_space_initial);
>  }  void clean_convert_context(struct btrfs_convert_context
>  *cctx)

Did you forget the clean path? :)

---
Su


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 3/3] btrfs-progs: convert: report available space before convertion happens
  2020-07-29  9:05   ` Su Yue
@ 2020-07-29  9:13     ` Qu Wenruo
  2020-07-31 15:35       ` David Sterba
  0 siblings, 1 reply; 9+ messages in thread
From: Qu Wenruo @ 2020-07-29  9:13 UTC (permalink / raw)
  To: Su Yue, Qu Wenruo; +Cc: linux-btrfs


[-- Attachment #1.1: Type: text/plain, Size: 4707 bytes --]



On 2020/7/29 下午5:05, Su Yue wrote:
> 
> On Wed 29 Jul 2020 at 16:40, Qu Wenruo <wqu@suse.com> wrote:
> 
>> Now if an ENOSPC error happened, the free space report would
>> help user to determine if it's a real ENOSPC or a bug in btrfs.
>>
>> The reported free space is the calculated free space, which
>> doesn't include super block space, nor merged data chunks.
>>
>> The free space is always smaller than the reported available
>> space of the original fs, as we need extra padding space for
>> used space to avoid too fragmented data chunks.
>>
>> The output exact would be:
>>
>> $ ./btrfs-convert  /dev/nvme/btrfs create btrfs filesystem:
>>         blocksize: 4096 nodesize:  16384 features:  extref,
>>         skinny-metadata (default) checksum:  crc32c
>> free space report: free / total = 0 / 10737418240 (0%) <<<
>> ERROR: unable to create initial ctree: No space left on device
>> WARNING: an error occurred during conversion, the original
>> filesystem is not modified
>>
>> Signed-off-by: Qu Wenruo <wqu@suse.com> ---
>>  convert/common.h    |  9 +++++++++ convert/main.c      | 26
>>  ++++++++++++++++++++++++-- convert/source-fs.c |  1 + 3 files
>>  changed, 34 insertions(+), 2 deletions(-)
>>
>> diff --git a/convert/common.h b/convert/common.h index
>> 7ec26cd996d3..2c7799433294 100644 --- a/convert/common.h +++
>> b/convert/common.h @@ -35,6 +35,7 @@ struct
>> btrfs_convert_context {
>>      u64 inodes_count; u64 free_inodes_count; u64 total_bytes;
>> +    u64 free_bytes_initial;
>>      char *volume_name; const struct btrfs_convert_operations
>>  *convert_ops;
>> @@ -47,6 +48,14 @@ struct btrfs_convert_context {
>>      /* Free space which is not covered by data_chunks */ struct
>>  cache_tree free_space;
>> +    /* +     * Free space reserved for ENOSPC report, it's just a
>> copy +     * free_space.  +     * But after initial calculation,
>> free_space_initial is no longer +     * updated, so we have a good
>> idea on how many free space we really +     * have for btrfs.  +
>> */ +    struct cache_tree free_space_initial;
>>      void *fs_data; };
>> diff --git a/convert/main.c b/convert/main.c index
>> 451e4f158689..49c95e092cbb 100644 --- a/convert/main.c +++
>> b/convert/main.c @@ -727,6 +727,23 @@ out:
>>      return ret; }
>> +static int copy_free_space_tree(struct btrfs_convert_context
>> *cctx) +{ +    struct cache_tree *src = &cctx->free_space; +
>> struct cache_tree *dst = &cctx->free_space_initial; +    struct
>> cache_extent *cache; +    int ret = 0; + +    for (cache =
>> search_cache_extent(src, 0); cache; +         cache =
>> next_cache_extent(cache)) { +        ret =
>> add_merge_cache_extent(dst, cache->start, cache->size); +        if
>> (ret < 0) +            return ret; +
>> cctx->free_bytes_initial += cache->size; +    } +    return ret; +}
>> +
>>  /*
>>   * Read used space, and since we have the used space, *
>>   calculate data_chunks and free for later mkfs
>> @@ -740,7 +757,10 @@ static int convert_read_used_space(struct
>> btrfs_convert_context *cctx)
>>          return ret;  ret = calculate_available_space(cctx);
>> -    return ret; +    if (ret < 0) +        return ret; + +    return
>> copy_free_space_tree(cctx);
>>  }  /*
>> @@ -1165,7 +1185,9 @@ static int do_convert(const char *devname,
>> u32 convert_flags, u32 nodesize,
>>      printf("\tnodesize:  %u\n", nodesize); printf("\tfeatures:
>>  %s\n", features_buf); printf("\tchecksum:  %s\n",
>>  btrfs_super_csum_name(csum_type));
>> - +    printf("free space report: free / total = %llu / %llu
>> (%lld%%)\n", +        cctx.free_bytes_initial, cctx.total_bytes,
>> +        cctx.free_bytes_initial * 100 / cctx.total_bytes);
>>      memset(&mkfs_cfg, 0, sizeof(mkfs_cfg)); mkfs_cfg.csum_type =
>>  csum_type; mkfs_cfg.label = cctx.volume_name;
>> diff --git a/convert/source-fs.c b/convert/source-fs.c index
>> f7fd3d6055b7..d2f7a825238d 100644 --- a/convert/source-fs.c +++
>> b/convert/source-fs.c @@ -74,6 +74,7 @@ void
>> init_convert_context(struct btrfs_convert_context *cctx)
>>      cache_tree_init(&cctx->used_space);
>>  cache_tree_init(&cctx->data_chunks);
>>  cache_tree_init(&cctx->free_space);
>> +    cache_tree_init(&cctx->free_space_initial);
>>  }  void clean_convert_context(struct btrfs_convert_context
>>  *cctx)
> 
> Did you forget the clean path? :)

Oh, thanks for that!
Just forgot that.

Thanks,
Qu
> 
> ---
> Su
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 3/3] btrfs-progs: convert: report available space before convertion happens
  2020-07-29  9:13     ` Qu Wenruo
@ 2020-07-31 15:35       ` David Sterba
  0 siblings, 0 replies; 9+ messages in thread
From: David Sterba @ 2020-07-31 15:35 UTC (permalink / raw)
  To: Qu Wenruo; +Cc: Su Yue, Qu Wenruo, linux-btrfs

On Wed, Jul 29, 2020 at 05:13:14PM +0800, Qu Wenruo wrote:
> >> +        cctx.free_bytes_initial * 100 / cctx.total_bytes);
> >>      memset(&mkfs_cfg, 0, sizeof(mkfs_cfg)); mkfs_cfg.csum_type =
> >>  csum_type; mkfs_cfg.label = cctx.volume_name;
> >> diff --git a/convert/source-fs.c b/convert/source-fs.c index
> >> f7fd3d6055b7..d2f7a825238d 100644 --- a/convert/source-fs.c +++
> >> b/convert/source-fs.c @@ -74,6 +74,7 @@ void
> >> init_convert_context(struct btrfs_convert_context *cctx)
> >>      cache_tree_init(&cctx->used_space);
> >>  cache_tree_init(&cctx->data_chunks);
> >>  cache_tree_init(&cctx->free_space);
> >> +    cache_tree_init(&cctx->free_space_initial);
> >>  }  void clean_convert_context(struct btrfs_convert_context
> >>  *cctx)
> > 
> > Did you forget the clean path? :)
> 
> Oh, thanks for that!
> Just forgot that.

Which I assume means

--- a/convert/source-fs.c
+++ b/convert/source-fs.c
@@ -82,6 +82,7 @@ void clean_convert_context(struct btrfs_convert_context *cctx)
        free_extent_cache_tree(&cctx->used_space);
        free_extent_cache_tree(&cctx->data_chunks);
        free_extent_cache_tree(&cctx->free_space);
+       free_extent_cache_tree(&cctx->free_space_initial);
 }
 
 int block_iterate_proc(u64 disk_block, u64 file_block,

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 0/3] btrfs-progs: convert: better ENOSPC handling
  2020-07-29  8:40 [PATCH 0/3] btrfs-progs: convert: better ENOSPC handling Qu Wenruo
                   ` (2 preceding siblings ...)
  2020-07-29  8:40 ` [PATCH 3/3] btrfs-progs: convert: report available space before convertion happens Qu Wenruo
@ 2020-07-31 16:16 ` David Sterba
  2020-07-31 23:37   ` Qu Wenruo
  3 siblings, 1 reply; 9+ messages in thread
From: David Sterba @ 2020-07-31 16:16 UTC (permalink / raw)
  To: Qu Wenruo; +Cc: linux-btrfs

On Wed, Jul 29, 2020 at 04:40:35PM +0800, Qu Wenruo wrote:
> This patchset is to address a bug report [1], where even with the bit
> overflow bug fixed, the user is still unable to convert an ext4 fs to
> btrfs.
> 
> The error is -ENOSPC, which triggers BUG_ON() and brings the end to the
> convertion.
> 
> We're still waiting for the image dump to determine what's the real
> cause is, but considering the user is still reporting around 40% free
> space, I guess it's something wrong with the extent allocator.
> 
> But still, we can enhance btrfs-convert to make it handle errors more
> gracefully, with better error message, and even some debugging info like
> the available space / total space ratio.
> 
> Qu Wenruo (3):
>   btrfs-progs: convert: handle errors better in ext2_copy_inodes()
>   btrfs-progs: convert: update error message to reflect original fs
>     unmodified cases
>   btrfs-progs: convert: report available space before convertion happens

Added to devel, thanks. With the fixup and I've updated the space report
to look like this:

create btrfs filesystem:
        blocksize: 4096
        nodesize:  16384
        features:  extref, skinny-metadata (default)
        checksum:  crc32c
free space report:
        total:     2147483648
        free:      1610547200 (75.00%)

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 0/3] btrfs-progs: convert: better ENOSPC handling
  2020-07-31 16:16 ` [PATCH 0/3] btrfs-progs: convert: better ENOSPC handling David Sterba
@ 2020-07-31 23:37   ` Qu Wenruo
  0 siblings, 0 replies; 9+ messages in thread
From: Qu Wenruo @ 2020-07-31 23:37 UTC (permalink / raw)
  To: dsterba, Qu Wenruo, linux-btrfs


[-- Attachment #1.1: Type: text/plain, Size: 1506 bytes --]



On 2020/8/1 上午12:16, David Sterba wrote:
> On Wed, Jul 29, 2020 at 04:40:35PM +0800, Qu Wenruo wrote:
>> This patchset is to address a bug report [1], where even with the bit
>> overflow bug fixed, the user is still unable to convert an ext4 fs to
>> btrfs.
>>
>> The error is -ENOSPC, which triggers BUG_ON() and brings the end to the
>> convertion.
>>
>> We're still waiting for the image dump to determine what's the real
>> cause is, but considering the user is still reporting around 40% free
>> space, I guess it's something wrong with the extent allocator.
>>
>> But still, we can enhance btrfs-convert to make it handle errors more
>> gracefully, with better error message, and even some debugging info like
>> the available space / total space ratio.
>>
>> Qu Wenruo (3):
>>   btrfs-progs: convert: handle errors better in ext2_copy_inodes()
>>   btrfs-progs: convert: update error message to reflect original fs
>>     unmodified cases
>>   btrfs-progs: convert: report available space before convertion happens
> 
> Added to devel, thanks. With the fixup and I've updated the space report
> to look like this:
> 
> create btrfs filesystem:
>         blocksize: 4096
>         nodesize:  16384
>         features:  extref, skinny-metadata (default)
>         checksum:  crc32c
> free space report:
>         total:     2147483648
>         free:      1610547200 (75.00%)
> 
Wow, that looks even better!

Thanks for the hot fix and enhanced UI!
Qu


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2020-07-31 23:37 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-07-29  8:40 [PATCH 0/3] btrfs-progs: convert: better ENOSPC handling Qu Wenruo
2020-07-29  8:40 ` [PATCH 1/3] btrfs-progs: convert: handle errors better in ext2_copy_inodes() Qu Wenruo
2020-07-29  8:40 ` [PATCH 2/3] btrfs-progs: convert: update error message to reflect original fs unmodified cases Qu Wenruo
2020-07-29  8:40 ` [PATCH 3/3] btrfs-progs: convert: report available space before convertion happens Qu Wenruo
2020-07-29  9:05   ` Su Yue
2020-07-29  9:13     ` Qu Wenruo
2020-07-31 15:35       ` David Sterba
2020-07-31 16:16 ` [PATCH 0/3] btrfs-progs: convert: better ENOSPC handling David Sterba
2020-07-31 23:37   ` Qu Wenruo

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).