[PATCH 0/2] fs-verity: fix error handling in ->end_enable_verity()

[PATCH 0/2] fs-verity: fix error handling in ->end_enable_verity()

[Eric Biggers]

Fix some bugs in how ext4_end_enable_verity() and
f2fs_end_enable_verity() handle failure to enable verity on the file.

This is intended to supersede the f2fs patch from Yunlei He
(https://lore.kernel.org/r/20210302113850.17011-1-heyunlei@hihonor.com,
 https://lore.kernel.org/r/20210301141506.6410-1-heyunlei@hihonor.com,
 https://lore.kernel.org/r/c1ce1421-2576-5b48-322c-fa682c7510d7@kernel.org).
I fixed the same bugs in ext4 too, reworked the functions to cleanly
separate the success and error paths, and improved the commit message.

These patches can be taken independently via the ext4 and f2fs trees.
I've just sent them out together since they're similar.

Eric Biggers (2):
  ext4: fix error handling in ext4_end_enable_verity()
  f2fs: fix error handling in f2fs_end_enable_verity()

 fs/ext4/verity.c | 90 ++++++++++++++++++++++++++++++------------------
 fs/f2fs/verity.c | 61 +++++++++++++++++++++-----------
 2 files changed, 97 insertions(+), 54 deletions(-)

-- 
2.30.1

[PATCH 1/2] ext4: fix error handling in ext4_end_enable_verity()

[Eric Biggers]

From: Eric Biggers <ebiggers@google.com>

ext4 didn't properly clean up if verity failed to be enabled on a file:

- It left verity metadata (pages past EOF) in the page cache, which
  would be exposed to userspace if the file was later extended.

- It didn't truncate the verity metadata at all (either from cache or
  from disk) if an error occurred while setting the verity bit.

Fix these bugs by adding a call to truncate_inode_pages() and ensuring
that we truncate the verity metadata (both from cache and from disk) in
all error paths.  Also rework the code to cleanly separate the success
path from the error paths, which makes it much easier to understand.

Reported-by: Yunlei He <heyunlei@hihonor.com>
Fixes: c93d8f885809 ("ext4: add basic fs-verity support")
Cc: <stable@vger.kernel.org> # v5.4+
Signed-off-by: Eric Biggers <ebiggers@google.com>
---
 fs/ext4/verity.c | 90 ++++++++++++++++++++++++++++++------------------
 1 file changed, 56 insertions(+), 34 deletions(-)

diff --git a/fs/ext4/verity.c b/fs/ext4/verity.c
index 5b7ba8f711538..acb12441c549b 100644
--- a/fs/ext4/verity.c
+++ b/fs/ext4/verity.c
@@ -201,55 +201,77 @@ static int ext4_end_enable_verity(struct file *filp, const void *desc,
 	struct inode *inode = file_inode(filp);
 	const int credits = 2; /* superblock and inode for ext4_orphan_del() */
 	handle_t *handle;
+	struct ext4_iloc iloc;
 	int err = 0;
-	int err2;
 
-	if (desc != NULL) {
-		/* Succeeded; write the verity descriptor. */
-		err = ext4_write_verity_descriptor(inode, desc, desc_size,
-						   merkle_tree_size);
-
-		/* Write all pages before clearing VERITY_IN_PROGRESS. */
-		if (!err)
-			err = filemap_write_and_wait(inode->i_mapping);
-	}
+	/*
+	 * If an error already occurred (which fs/verity/ signals by passing
+	 * desc == NULL), then only clean-up is needed.
+	 */
+	if (desc == NULL)
+		goto cleanup;
 
-	/* If we failed, truncate anything we wrote past i_size. */
-	if (desc == NULL || err)
-		ext4_truncate(inode);
+	/* Append the verity descriptor. */
+	err = ext4_write_verity_descriptor(inode, desc, desc_size,
+					   merkle_tree_size);
+	if (err)
+		goto cleanup;
 
 	/*
-	 * We must always clean up by clearing EXT4_STATE_VERITY_IN_PROGRESS and
-	 * deleting the inode from the orphan list, even if something failed.
-	 * If everything succeeded, we'll also set the verity bit in the same
-	 * transaction.
+	 * Write all pages (both data and verity metadata).  Note that this must
+	 * happen before clearing EXT4_STATE_VERITY_IN_PROGRESS; otherwise pages
+	 * beyond i_size won't be written properly.  For crash consistency, this
+	 * also must happen before the verity inode flag gets persisted.
 	 */
+	err = filemap_write_and_wait(inode->i_mapping);
+	if (err)
+		goto cleanup;
 
-	ext4_clear_inode_state(inode, EXT4_STATE_VERITY_IN_PROGRESS);
+	/*
+	 * Finally, set the verity inode flag and remove the inode from the
+	 * orphan list (in a single transaction).
+	 */
 
 	handle = ext4_journal_start(inode, EXT4_HT_INODE, credits);
 	if (IS_ERR(handle)) {
-		ext4_orphan_del(NULL, inode);
-		return PTR_ERR(handle);
+		err = PTR_ERR(handle);
+		goto cleanup;
 	}
 
-	err2 = ext4_orphan_del(handle, inode);
-	if (err2)
-		goto out_stop;
+	err = ext4_orphan_del(handle, inode);
+	if (err)
+		goto stop_and_cleanup;
 
-	if (desc != NULL && !err) {
-		struct ext4_iloc iloc;
+	err = ext4_reserve_inode_write(handle, inode, &iloc);
+	if (err)
+		goto stop_and_cleanup;
+
+	ext4_set_inode_flag(inode, EXT4_INODE_VERITY);
+	ext4_set_inode_flags(inode, false);
+	err = ext4_mark_iloc_dirty(handle, inode, &iloc);
+	if (err)
+		goto stop_and_cleanup;
 
-		err = ext4_reserve_inode_write(handle, inode, &iloc);
-		if (err)
-			goto out_stop;
-		ext4_set_inode_flag(inode, EXT4_INODE_VERITY);
-		ext4_set_inode_flags(inode, false);
-		err = ext4_mark_iloc_dirty(handle, inode, &iloc);
-	}
-out_stop:
 	ext4_journal_stop(handle);
-	return err ?: err2;
+
+	ext4_clear_inode_state(inode, EXT4_STATE_VERITY_IN_PROGRESS);
+	return 0;
+
+
+stop_and_cleanup:
+	ext4_journal_stop(handle);
+cleanup:
+	/*
+	 * Verity failed to be enabled, so clean up by truncating any verity
+	 * metadata that was written beyond i_size (both from cache and from
+	 * disk), removing the inode from the orphan list (if it wasn't done
+	 * already), and clearing EXT4_STATE_VERITY_IN_PROGRESS.
+	 */
+	truncate_inode_pages(inode->i_mapping, inode->i_size);
+	ext4_truncate(inode);
+	ext4_orphan_del(NULL, inode);
+	ext4_clear_inode_state(inode, EXT4_STATE_VERITY_IN_PROGRESS);
+	return err;
 }
 
 static int ext4_get_verity_descriptor_location(struct inode *inode,
-- 
2.30.1

Re: [PATCH 1/2] ext4: fix error handling in ext4_end_enable_verity()

[Theodore Ts'o]

On Tue, Mar 02, 2021 at 12:04:19PM -0800, Eric Biggers wrote:
> From: Eric Biggers <ebiggers@google.com>
> 
> ext4 didn't properly clean up if verity failed to be enabled on a file:
> 
> - It left verity metadata (pages past EOF) in the page cache, which
>   would be exposed to userspace if the file was later extended.
> 
> - It didn't truncate the verity metadata at all (either from cache or
>   from disk) if an error occurred while setting the verity bit.
> 
> Fix these bugs by adding a call to truncate_inode_pages() and ensuring
> that we truncate the verity metadata (both from cache and from disk) in
> all error paths.  Also rework the code to cleanly separate the success
> path from the error paths, which makes it much easier to understand.
> 
> Reported-by: Yunlei He <heyunlei@hihonor.com>
> Fixes: c93d8f885809 ("ext4: add basic fs-verity support")
> Cc: <stable@vger.kernel.org> # v5.4+
> Signed-off-by: Eric Biggers <ebiggers@google.com>

Thanks, LGTM.

I've applied this to the ext4 with minor adjustment; I eliminated the
double blank line here:

> +	ext4_clear_inode_state(inode, EXT4_STATE_VERITY_IN_PROGRESS);
> +	return 0;
> +
> +
> +stop_and_cleanup:
> +	ext4_journal_stop(handle);
     ...

						- Ted

[PATCH 2/2] f2fs: fix error handling in f2fs_end_enable_verity()

[Eric Biggers]

From: Eric Biggers <ebiggers@google.com>

f2fs didn't properly clean up if verity failed to be enabled on a file:

- It left verity metadata (pages past EOF) in the page cache, which
  would be exposed to userspace if the file was later extended.

- It didn't truncate the verity metadata at all (either from cache or
  from disk) if an error occurred while setting the verity bit.

Fix these bugs by adding a call to truncate_inode_pages() and ensuring
that we truncate the verity metadata (both from cache and from disk) in
all error paths.  Also rework the code to cleanly separate the success
path from the error paths, which makes it much easier to understand.

Reported-by: Yunlei He <heyunlei@hihonor.com>
Fixes: 95ae251fe828 ("f2fs: add fs-verity support")
Cc: <stable@vger.kernel.org> # v5.4+
Signed-off-by: Eric Biggers <ebiggers@google.com>
---
 fs/f2fs/verity.c | 61 ++++++++++++++++++++++++++++++++----------------
 1 file changed, 41 insertions(+), 20 deletions(-)

diff --git a/fs/f2fs/verity.c b/fs/f2fs/verity.c
index 054ec852b5ea4..2db89967fde37 100644
--- a/fs/f2fs/verity.c
+++ b/fs/f2fs/verity.c
@@ -160,31 +160,52 @@ static int f2fs_end_enable_verity(struct file *filp, const void *desc,
 	};
 	int err = 0;
 
-	if (desc != NULL) {
-		/* Succeeded; write the verity descriptor. */
-		err = pagecache_write(inode, desc, desc_size, desc_pos);
+	/*
+	 * If an error already occurred (which fs/verity/ signals by passing
+	 * desc == NULL), then only clean-up is needed.
+	 */
+	if (desc == NULL)
+		goto cleanup;
 
-		/* Write all pages before clearing FI_VERITY_IN_PROGRESS. */
-		if (!err)
-			err = filemap_write_and_wait(inode->i_mapping);
-	}
+	/* Append the verity descriptor. */
+	err = pagecache_write(inode, desc, desc_size, desc_pos);
+	if (err)
+		goto cleanup;
 
-	/* If we failed, truncate anything we wrote past i_size. */
-	if (desc == NULL || err)
-		f2fs_truncate(inode);
+	/*
+	 * Write all pages (both data and verity metadata).  Note that this must
+	 * happen before clearing FI_VERITY_IN_PROGRESS; otherwise pages beyond
+	 * i_size won't be written properly.  For crash consistency, this also
+	 * must happen before the verity inode flag gets persisted.
+	 */
+	err = filemap_write_and_wait(inode->i_mapping);
+	if (err)
+		goto cleanup;
+
+	/* Set the verity xattr. */
+	err = f2fs_setxattr(inode, F2FS_XATTR_INDEX_VERITY,
+			    F2FS_XATTR_NAME_VERITY, &dloc, sizeof(dloc),
+			    NULL, XATTR_CREATE);
+	if (err)
+		goto cleanup;
+
+	/* Finally, set the verity inode flag. */
+	file_set_verity(inode);
+	f2fs_set_inode_flags(inode);
+	f2fs_mark_inode_dirty_sync(inode, true);
 
 	clear_inode_flag(inode, FI_VERITY_IN_PROGRESS);
+	return 0;
 
-	if (desc != NULL && !err) {
-		err = f2fs_setxattr(inode, F2FS_XATTR_INDEX_VERITY,
-				    F2FS_XATTR_NAME_VERITY, &dloc, sizeof(dloc),
-				    NULL, XATTR_CREATE);
-		if (!err) {
-			file_set_verity(inode);
-			f2fs_set_inode_flags(inode);
-			f2fs_mark_inode_dirty_sync(inode, true);
-		}
-	}
+cleanup:
+	/*
+	 * Verity failed to be enabled, so clean up by truncating any verity
+	 * metadata that was written beyond i_size (both from cache and from
+	 * disk) and clearing FI_VERITY_IN_PROGRESS.
+	 */
+	truncate_inode_pages(inode->i_mapping, inode->i_size);
+	f2fs_truncate(inode);
+	clear_inode_flag(inode, FI_VERITY_IN_PROGRESS);
 	return err;
 }
 
-- 
2.30.1

Re: [f2fs-dev] [PATCH 2/2] f2fs: fix error handling in f2fs_end_enable_verity()

[Chao Yu]

On 2021/3/3 4:04, Eric Biggers wrote:
> From: Eric Biggers <ebiggers@google.com>
> 
> f2fs didn't properly clean up if verity failed to be enabled on a file:
> 
> - It left verity metadata (pages past EOF) in the page cache, which
>    would be exposed to userspace if the file was later extended.
> 
> - It didn't truncate the verity metadata at all (either from cache or
>    from disk) if an error occurred while setting the verity bit.
> 
> Fix these bugs by adding a call to truncate_inode_pages() and ensuring
> that we truncate the verity metadata (both from cache and from disk) in
> all error paths.  Also rework the code to cleanly separate the success
> path from the error paths, which makes it much easier to understand.
> 
> Reported-by: Yunlei He <heyunlei@hihonor.com>
> Fixes: 95ae251fe828 ("f2fs: add fs-verity support")
> Cc: <stable@vger.kernel.org> # v5.4+
> Signed-off-by: Eric Biggers <ebiggers@google.com>
> ---
>   fs/f2fs/verity.c | 61 ++++++++++++++++++++++++++++++++----------------
>   1 file changed, 41 insertions(+), 20 deletions(-)
> 
> diff --git a/fs/f2fs/verity.c b/fs/f2fs/verity.c
> index 054ec852b5ea4..2db89967fde37 100644
> --- a/fs/f2fs/verity.c
> +++ b/fs/f2fs/verity.c
> @@ -160,31 +160,52 @@ static int f2fs_end_enable_verity(struct file *filp, const void *desc,
>   	};
>   	int err = 0;
>   
> -	if (desc != NULL) {
> -		/* Succeeded; write the verity descriptor. */
> -		err = pagecache_write(inode, desc, desc_size, desc_pos);
> +	/*
> +	 * If an error already occurred (which fs/verity/ signals by passing
> +	 * desc == NULL), then only clean-up is needed.
> +	 */
> +	if (desc == NULL)
> +		goto cleanup;
>   
> -		/* Write all pages before clearing FI_VERITY_IN_PROGRESS. */
> -		if (!err)
> -			err = filemap_write_and_wait(inode->i_mapping);
> -	}
> +	/* Append the verity descriptor. */
> +	err = pagecache_write(inode, desc, desc_size, desc_pos);
> +	if (err)
> +		goto cleanup;
>   
> -	/* If we failed, truncate anything we wrote past i_size. */
> -	if (desc == NULL || err)
> -		f2fs_truncate(inode);
> +	/*
> +	 * Write all pages (both data and verity metadata).  Note that this must
> +	 * happen before clearing FI_VERITY_IN_PROGRESS; otherwise pages beyond
> +	 * i_size won't be written properly.  For crash consistency, this also
> +	 * must happen before the verity inode flag gets persisted.
> +	 */
> +	err = filemap_write_and_wait(inode->i_mapping);
> +	if (err)
> +		goto cleanup;
> +
> +	/* Set the verity xattr. */
> +	err = f2fs_setxattr(inode, F2FS_XATTR_INDEX_VERITY,
> +			    F2FS_XATTR_NAME_VERITY, &dloc, sizeof(dloc),
> +			    NULL, XATTR_CREATE);
> +	if (err)
> +		goto cleanup;
> +
> +	/* Finally, set the verity inode flag. */
> +	file_set_verity(inode);
> +	f2fs_set_inode_flags(inode);
> +	f2fs_mark_inode_dirty_sync(inode, true);
>   
>   	clear_inode_flag(inode, FI_VERITY_IN_PROGRESS);
> +	return 0;
>   
> -	if (desc != NULL && !err) {
> -		err = f2fs_setxattr(inode, F2FS_XATTR_INDEX_VERITY,
> -				    F2FS_XATTR_NAME_VERITY, &dloc, sizeof(dloc),
> -				    NULL, XATTR_CREATE);
> -		if (!err) {
> -			file_set_verity(inode);
> -			f2fs_set_inode_flags(inode);
> -			f2fs_mark_inode_dirty_sync(inode, true);
> -		}
> -	}
> +cleanup:
> +	/*
> +	 * Verity failed to be enabled, so clean up by truncating any verity
> +	 * metadata that was written beyond i_size (both from cache and from
> +	 * disk) and clearing FI_VERITY_IN_PROGRESS.
> +	 */
> +	truncate_inode_pages(inode->i_mapping, inode->i_size);
> +	f2fs_truncate(inode);

Eric,

Truncation can fail due to a lot of reasons, if we fail in f2fs_truncate(),
do we need to at least print a message here? or it allows to keep those
meta/data silently.

One other concern is that how do you think of covering truncate_inode_pages &
f2fs_truncate with F2FS_I(inode)->i_gc_rwsem[WRITE] lock to avoid racing with
GC, so that page cache won't be revalidated after truncate_inode_pages().

Thanks,

> +	clear_inode_flag(inode, FI_VERITY_IN_PROGRESS); >   	return err;
>   }
>   
> 

Re: [PATCH 2/2] f2fs: fix error handling in f2fs_end_enable_verity()

[Eric Biggers]

On Fri, Mar 05, 2021 at 09:37:26AM +0800, Chao Yu wrote:
> > +cleanup:
> > +	/*
> > +	 * Verity failed to be enabled, so clean up by truncating any verity
> > +	 * metadata that was written beyond i_size (both from cache and from
> > +	 * disk) and clearing FI_VERITY_IN_PROGRESS.
> > +	 */
> > +	truncate_inode_pages(inode->i_mapping, inode->i_size);
> > +	f2fs_truncate(inode);
> 
> Eric,
> 
> Truncation can fail due to a lot of reasons, if we fail in f2fs_truncate(),
> do we need to at least print a message here? or it allows to keep those
> meta/data silently.

I suppose we might as well, although hopefully there will already be a message
for the underlying failure reason too.  Also, f2fs_file_write_iter() has the
same issue too, right?

> One other concern is that how do you think of covering truncate_inode_pages &
> f2fs_truncate with F2FS_I(inode)->i_gc_rwsem[WRITE] lock to avoid racing with
> GC, so that page cache won't be revalidated after truncate_inode_pages().

Yes, that does seem to be needed, due to the way the f2fs garbage collection
works.

- Eric