[PATCH 1/5] ext4: Use filemap_write_and_wait_range() correctly in collapse range

[PATCH 1/5] ext4: Use filemap_write_and_wait_range() correctly in collapse range

[Lukas Czerner]

Currently we're passing -1 as lend argumnet for
filemap_write_and_wait_range() which is wrong since lend is signed type
so it would cause some confusion and we might not write_and_wait for the
entire range we're expecting to write.

Fix it by using LLONG_MAX instead.

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
---
 fs/ext4/extents.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
index ff823b7..821c1d4 100644
--- a/fs/ext4/extents.c
+++ b/fs/ext4/extents.c
@@ -5378,7 +5378,7 @@ int ext4_collapse_range(struct inode *inode, loff_t offset, loff_t len)
 	punch_stop = (offset + len) >> EXT4_BLOCK_SIZE_BITS(sb);
 
 	/* Write out all dirty pages */
-	ret = filemap_write_and_wait_range(inode->i_mapping, offset, -1);
+	ret = filemap_write_and_wait_range(inode->i_mapping, offset, LLONG_MAX);
 	if (ret)
 		return ret;
 
-- 
1.8.3.1

[PATCH 2/5] ext4: fix removing status extents in ext4_collapse_range()

[Lukas Czerner]

Currently in ext4_collapse_range() when calling ext4_es_remove_extent() to
remove status extents we're passing (EXT_MAX_BLOCKS - punch_start - 1)
in order to remove all extents from start of the collapse range to the
end of the file. However this is wrong because we might miss the
possible extent covering the last block of the file.

Fix it by removing the -1.

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
---
 fs/ext4/extents.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
index 821c1d4..25ed60f 100644
--- a/fs/ext4/extents.c
+++ b/fs/ext4/extents.c
@@ -5422,7 +5422,7 @@ int ext4_collapse_range(struct inode *inode, loff_t offset, loff_t len)
 	ext4_discard_preallocations(inode);
 
 	ret = ext4_es_remove_extent(inode, punch_start,
-				    EXT_MAX_BLOCKS - punch_start - 1);
+				    EXT_MAX_BLOCKS - punch_start);
 	if (ret) {
 		up_write(&EXT4_I(inode)->i_data_sem);
 		goto out_stop;
-- 
1.8.3.1

[PATCH 3/5] ext4: No need to truncate pagecache twice in collapse range

[Lukas Czerner]

We're already calling truncate_pagecache_range() before we attempt to
do any actual job so there is not need to truncate pagecache once more
using truncate_setsize() after we're finished.

Remove truncate_setsize() and replace it just with i_size_write() note
that we're holding appropriate locks.

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
---
 fs/ext4/extents.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
index 25ed60f..9cd762c 100644
--- a/fs/ext4/extents.c
+++ b/fs/ext4/extents.c
@@ -5442,7 +5442,7 @@ int ext4_collapse_range(struct inode *inode, loff_t offset, loff_t len)
 	}
 
 	new_size = i_size_read(inode) - len;
-	truncate_setsize(inode, new_size);
+	i_size_write(inode, new_size);
 	EXT4_I(inode)->i_disksize = new_size;
 
 	ext4_discard_preallocations(inode);
-- 
1.8.3.1

Re: [PATCH 3/5] ext4: No need to truncate pagecache twice in collapse range

[Theodore Ts'o]

On Wed, Apr 16, 2014 at 08:33:00PM +0200, Lukas Czerner wrote:
> We're already calling truncate_pagecache_range() before we attempt to
> do any actual job so there is not need to truncate pagecache once more
> using truncate_setsize() after we're finished.
> 
> Remove truncate_setsize() and replace it just with i_size_write() note
> that we're holding appropriate locks.
> 
> Signed-off-by: Lukas Czerner <lczerner@redhat.com>

Applied, with truncate_pagecache_range() changed to be
truncate_pagecache() in the commit description.

Thanks!!

					- Ted

[PATCH 4/5] ext4: Discard preallocations after removing space

[Lukas Czerner]

Currently in ext4_collapse_range() and ext4_punch_hole() we're
discarding preallocation twice. Once before we attempt to do any changes
and second time after we're done with the changes.

While the second call to ext4_discard_preallocations() in
ext4_punch_hole() case is not needed, we need to discard preallocation
right after ext4_ext_remove_space() in collapse range case because in
the case we had to restart a transaction in the middle of removing space
we might have new preallocations created.

Remove unneeded ext4_discard_preallocations() ext4_punch_hole() and move
it to the better place in ext4_collapse_range()

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
---
 fs/ext4/extents.c | 2 +-
 fs/ext4/inode.c   | 1 -
 2 files changed, 1 insertion(+), 2 deletions(-)

diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
index 9cd762c..84bb668 100644
--- a/fs/ext4/extents.c
+++ b/fs/ext4/extents.c
@@ -5433,6 +5433,7 @@ int ext4_collapse_range(struct inode *inode, loff_t offset, loff_t len)
 		up_write(&EXT4_I(inode)->i_data_sem);
 		goto out_stop;
 	}
+	ext4_discard_preallocations(inode);
 
 	ret = ext4_ext_shift_extents(inode, handle, punch_stop,
 				     punch_stop - punch_start);
@@ -5445,7 +5446,6 @@ int ext4_collapse_range(struct inode *inode, loff_t offset, loff_t len)
 	i_size_write(inode, new_size);
 	EXT4_I(inode)->i_disksize = new_size;
 
-	ext4_discard_preallocations(inode);
 	up_write(&EXT4_I(inode)->i_data_sem);
 	if (IS_SYNC(inode))
 		ext4_handle_sync(handle);
diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 56f1ff4..8765d49 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -3613,7 +3613,6 @@ int ext4_punch_hole(struct inode *inode, loff_t offset, loff_t length)
 		ret = ext4_free_hole_blocks(handle, inode, first_block,
 					    stop_block);
 
-	ext4_discard_preallocations(inode);
 	up_write(&EXT4_I(inode)->i_data_sem);
 	if (IS_SYNC(inode))
 		ext4_handle_sync(handle);
-- 
1.8.3.1

[PATCH 5/5] ext4: Fix extent merging in ext4_ext_shift_path_extents()

[Lukas Czerner]

There is a bug in ext4_ext_shift_path_extents() where if we actually
manage to merge a extent we would skip shifting the next extent. This
will result in in one extent in the extent tree not being properly
shifted.

This is causing failure in various xfstests tests using fsx or fsstress
with collapse range support. It will also cause file system corruption
which looks something like:

 e2fsck 1.42.9 (4-Feb-2014)
 Pass 1: Checking inodes, blocks, and sizes
 Inode 20 has out of order extents
        (invalid logical block 3, physical block 492938, len 2)
 Clear? yes
 ...

when running e2fsck.

It's also very easily reproducible just by running fsx without any
parameters. I can usually hit the problem within a minute.

Fix it by increasing ex_start only if we're not merging the extent.

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
---
 fs/ext4/extents.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
index 84bb668..5fa31cb 100644
--- a/fs/ext4/extents.c
+++ b/fs/ext4/extents.c
@@ -5230,13 +5230,14 @@ ext4_ext_shift_path_extents(struct ext4_ext_path *path, ext4_lblk_t shift,
 
 			while (ex_start <= ex_last) {
 				ex_start->ee_block -= shift;
-				if (ex_start >
-					EXT_FIRST_EXTENT(path[depth].p_hdr)) {
-					if (ext4_ext_try_to_merge_right(inode,
-						path, ex_start - 1))
-						ex_last--;
-				}
-				ex_start++;
+				/* Try to merge to the left. */
+				if ((ex_start >
+				     EXT_FIRST_EXTENT(path[depth].p_hdr)) &&
+				    ext4_ext_try_to_merge_right(inode,
+							path, ex_start - 1))
+					ex_last--;
+				else
+					ex_start++;
 			}
 			err = ext4_ext_dirty(handle, inode, path + depth);
 			if (err)
-- 
1.8.3.1

Re: [PATCH 1/5] ext4: Use filemap_write_and_wait_range() correctly in collapse range

[Lukáš Czerner]

Ted,

there are still endianness problems with collapse range, I kind of
remember seeing some patches to fix that but I can not find those
anywhere. So let me know if you have those, otherwise I'll send
some.

Also since this patch set (or rather the patch #5) fixes the
collapse range so that it does not fail on fsx and fsstress we can
drop the fallocate mode block patches I think :)

Thanks!
-Lukas

On Wed, 16 Apr 2014, Lukas Czerner wrote:

> Date: Wed, 16 Apr 2014 20:32:58 +0200
> From: Lukas Czerner <lczerner@redhat.com>
> To: linux-ext4@vger.kernel.org
> Cc: linkinjeon@gmail.com, Lukas Czerner <lczerner@redhat.com>
> Subject: [PATCH 1/5] ext4: Use filemap_write_and_wait_range() correctly in
>     collapse range
> 
> Currently we're passing -1 as lend argumnet for
> filemap_write_and_wait_range() which is wrong since lend is signed type
> so it would cause some confusion and we might not write_and_wait for the
> entire range we're expecting to write.
> 
> Fix it by using LLONG_MAX instead.
> 
> Signed-off-by: Lukas Czerner <lczerner@redhat.com>
> ---
>  fs/ext4/extents.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
> index ff823b7..821c1d4 100644
> --- a/fs/ext4/extents.c
> +++ b/fs/ext4/extents.c
> @@ -5378,7 +5378,7 @@ int ext4_collapse_range(struct inode *inode, loff_t offset, loff_t len)
>  	punch_stop = (offset + len) >> EXT4_BLOCK_SIZE_BITS(sb);
>  
>  	/* Write out all dirty pages */
> -	ret = filemap_write_and_wait_range(inode->i_mapping, offset, -1);
> +	ret = filemap_write_and_wait_range(inode->i_mapping, offset, LLONG_MAX);
>  	if (ret)
>  		return ret;
>  
> 

Re: [PATCH 1/5] ext4: Use filemap_write_and_wait_range() correctly in collapse range

[Theodore Ts'o]

On Wed, Apr 16, 2014 at 08:38:01PM +0200, Lukáš Czerner wrote:
> there are still endianness problems with collapse range, I kind of
> remember seeing some patches to fix that but I can not find those
> anywhere. So let me know if you have those, otherwise I'll send
> some.

Yes, those are in the tree already.  I had to adjust the patch #5
slightly so it would apply as a result.

> Also since this patch set (or rather the patch #5) fixes the
> collapse range so that it does not fail on fsx and fsstress we can
> drop the fallocate mode block patches I think :)

I'll move it to the unstable part of the tree as a debugging patch,
pending the xfstests changes to make it easy to block things like the
on-deck INSERT_RANGE patchset.  (BTW, it would be great if the
xfstests changes for INSERT_RANGE had an easy way to disable
INSERT_RANGE with a single environment variable.)

The point is that otherwise, I can't let the INSERT_RANGE patches into
the dev branch until it's completely regression free, or else it
interferes the testing of other patches during the development cycle.
Of course, if there are still problems that aren't solved by the end
of the development cycle, I'll still drop the patches and hold them
for the next merge window.

Thanks,

					- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/5] ext4: Use filemap_write_and_wait_range() correctly in collapse range

[Lukáš Czerner]

So unfortunately this does not fix all the problems. Even though
this fixes all the problems with collapse rage when page size ==
block size it still fails some tests when page size > block size so
I guess we still have some work to do.

Thanks!
-Lukas

On Wed, 16 Apr 2014, Lukas Czerner wrote:

> Date: Wed, 16 Apr 2014 20:32:58 +0200
> From: Lukas Czerner <lczerner@redhat.com>
> To: linux-ext4@vger.kernel.org
> Cc: linkinjeon@gmail.com, Lukas Czerner <lczerner@redhat.com>
> Subject: [PATCH 1/5] ext4: Use filemap_write_and_wait_range() correctly in
>     collapse range
> 
> Currently we're passing -1 as lend argumnet for
> filemap_write_and_wait_range() which is wrong since lend is signed type
> so it would cause some confusion and we might not write_and_wait for the
> entire range we're expecting to write.
> 
> Fix it by using LLONG_MAX instead.
> 
> Signed-off-by: Lukas Czerner <lczerner@redhat.com>
> ---
>  fs/ext4/extents.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
> index ff823b7..821c1d4 100644
> --- a/fs/ext4/extents.c
> +++ b/fs/ext4/extents.c
> @@ -5378,7 +5378,7 @@ int ext4_collapse_range(struct inode *inode, loff_t offset, loff_t len)
>  	punch_stop = (offset + len) >> EXT4_BLOCK_SIZE_BITS(sb);
>  
>  	/* Write out all dirty pages */
> -	ret = filemap_write_and_wait_range(inode->i_mapping, offset, -1);
> +	ret = filemap_write_and_wait_range(inode->i_mapping, offset, LLONG_MAX);
>  	if (ret)
>  		return ret;
>  
> 

Re: [PATCH 1/5] ext4: Use filemap_write_and_wait_range() correctly in collapse range

[Theodore Ts'o]

On Wed, Apr 16, 2014 at 08:32:58PM +0200, Lukas Czerner wrote:
> Currently we're passing -1 as lend argumnet for
> filemap_write_and_wait_range() which is wrong since lend is signed type
> so it would cause some confusion and we might not write_and_wait for the
> entire range we're expecting to write.
> 
> Fix it by using LLONG_MAX instead.
> 
> Signed-off-by: Lukas Czerner <lczerner@redhat.com>

Thanks, applied.

					- Ted

RE: [PATCH 2/5] ext4: fix removing status extents in ext4_collapse_range()

[Namjae Jeon]

> Currently in ext4_collapse_range() when calling ext4_es_remove_extent() to
> remove status extents we're passing (EXT_MAX_BLOCKS - punch_start - 1)
> in order to remove all extents from start of the collapse range to the
> end of the file. However this is wrong because we might miss the
> possible extent covering the last block of the file.
> 
> Fix it by removing the -1.
> 
> Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Looks good to me! 

Reviewed-by: Namjae Jeon <namjae.jeon@samsung.com>

Thanks Lukas.

Re: [PATCH 2/5] ext4: fix removing status extents in ext4_collapse_range()

[Theodore Ts'o]

On Thu, Apr 17, 2014 at 02:54:18PM +0900, Namjae Jeon wrote:
> > Currently in ext4_collapse_range() when calling ext4_es_remove_extent() to
> > remove status extents we're passing (EXT_MAX_BLOCKS - punch_start - 1)
> > in order to remove all extents from start of the collapse range to the
> > end of the file. However this is wrong because we might miss the
> > possible extent covering the last block of the file.
> > 
> > Fix it by removing the -1.
> > 
> > Signed-off-by: Lukas Czerner <lczerner@redhat.com>
> Looks good to me! 
> 
> Reviewed-by: Namjae Jeon <namjae.jeon@samsung.com>

Thanks, applied.

					- Ted