All of lore.kernel.org
 help / color / mirror / Atom feed
From: Jan Kara <jack@suse.cz>
To: Al Viro <viro@ZenIV.linux.org.uk>
Cc: <linux-fsdevel@vger.kernel.org>,
	Shijie Luo <luoshijie1@huawei.com>, Jan Kara <jack@suse.cz>
Subject: [PATCH] fs: fix a hungtask problem when freeze/unfreeze fs
Date: Tue,  5 Jan 2021 14:46:00 +0100	[thread overview]
Message-ID: <20210105134600.24022-1-jack@suse.cz> (raw)

We found the following deadlock when running xfstests generic/390 with ext4
filesystem, and simutaneously offlining/onlining the disk we tested. It will
cause a deadlock whose call trace is like this:

fsstress        D    0 11672  11625 0x00000080
Call Trace:
 ? __schedule+0x2fc/0x930
 ? filename_parentat+0x10b/0x1a0
 schedule+0x28/0x70
 rwsem_down_read_failed+0x102/0x1c0
 ? __percpu_down_read+0x93/0xb0
 __percpu_down_read+0x93/0xb0
 __sb_start_write+0x5f/0x70
 mnt_want_write+0x20/0x50
 do_renameat2+0x1f3/0x550
 __x64_sys_rename+0x1c/0x20
 do_syscall_64+0x5b/0x1b0
 entry_SYSCALL_64_after_hwframe+0x65/0xca

The root cause is that when ext4 hits IO error due to disk being
offline, it will switch itself into read-only state. When it is frozen
at that moment, following thaw_super() call will not unlock percpu
freeze semaphores (as the fs is read-only) causing the deadlock.

Fix the problem by tracking whether the superblock was read-only at the
time we were freezing it.

Reported-and-tested-by: Shijie Luo <luoshijie1@huawei.com>
Signed-off-by: Jan Kara <jack@suse.cz>
---
 fs/super.c         | 9 ++++++++-
 include/linux/fs.h | 4 +++-
 2 files changed, 11 insertions(+), 2 deletions(-)

Al, can you pick up this patch please? Thanks!

diff --git a/fs/super.c b/fs/super.c
index 2c6cdea2ab2d..c35a2938ee99 100644
--- a/fs/super.c
+++ b/fs/super.c
@@ -1674,10 +1674,12 @@ int freeze_super(struct super_block *sb)
 	if (sb_rdonly(sb)) {
 		/* Nothing to do really... */
 		sb->s_writers.frozen = SB_FREEZE_COMPLETE;
+		sb->s_writers.frozen_ro = 1;
 		up_write(&sb->s_umount);
 		return 0;
 	}
 
+	sb->s_writers.frozen_ro = 0;
 	sb->s_writers.frozen = SB_FREEZE_WRITE;
 	/* Release s_umount to preserve sb_start_write -> s_umount ordering */
 	up_write(&sb->s_umount);
@@ -1733,7 +1735,12 @@ static int thaw_super_locked(struct super_block *sb)
 		return -EINVAL;
 	}
 
-	if (sb_rdonly(sb)) {
+	/*
+	 * Was the fs frozen in read-only state? Note that we cannot just check
+	 * sb_rdonly(sb) as the filesystem might have switched to read-only
+	 * state due to internal errors or so.
+	 */
+	if (sb->s_writers.frozen_ro) {
 		sb->s_writers.frozen = SB_UNFROZEN;
 		goto out;
 	}
diff --git a/include/linux/fs.h b/include/linux/fs.h
index ad4cf1bae586..346ab981128f 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -1406,7 +1406,9 @@ enum {
 #define SB_FREEZE_LEVELS (SB_FREEZE_COMPLETE - 1)
 
 struct sb_writers {
-	int				frozen;		/* Is sb frozen? */
+	unsigned short			frozen;		/* Is sb frozen? */
+	unsigned short			frozen_ro;	/* Was sb read-only
+							 * when frozen? */
 	wait_queue_head_t		wait_unfrozen;	/* wait for thaw */
 	struct percpu_rw_semaphore	rw_sem[SB_FREEZE_LEVELS];
 };
-- 
2.26.2


             reply	other threads:[~2021-01-05 13:46 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-01-05 13:46 Jan Kara [this message]
2021-01-21  2:00 ` [PATCH] fs: fix a hungtask problem when freeze/unfreeze fs Shijie Luo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210105134600.24022-1-jack@suse.cz \
    --to=jack@suse.cz \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=luoshijie1@huawei.com \
    --cc=viro@ZenIV.linux.org.uk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.