From: "Ondřej Jirman" <megi@xff.cz> To: Jaegeuk Kim <jaegeuk@kernel.org> Cc: linux-kernel@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net Subject: Re: [PATCH 1/6] f2fs: call f2fs_balance_fs outside of locked page Date: Sat, 22 Feb 2020 05:46:17 +0100 [thread overview] Message-ID: <20200222044617.pfrhnz2iavkrtdn6@core.my.home> (raw) In-Reply-To: <20191209222345.1078-1-jaegeuk@kernel.org> Hello, On Mon, Dec 09, 2019 at 02:23:40PM -0800, Jaegeuk Kim wrote: > Otherwise, we can hit deadlock by waiting for the locked page in > move_data_block in GC. I had the task hangs on 5.6 shortly after boot. (f2fs as rootfs). See below for stacktrace. So I went through the changelog in f2fs and noticed this patch as a suspect, and after reverting it the hung task panics went away. I reverted it manually, because the master changed too much for a clean revert: https://megous.com/git/linux/commit/?h=orange-pi-5.6&id=9983bdae4974edc2af6ff547a401ae397388b6b5 regards, o. INFO: task kworker/u16:2:341 blocked for more than 122 seconds. Not tainted 5.6.0-rc2-00254-g9a029a493dc16 #4 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. kworker/u16:2 D 0 341 2 0x00000000 Workqueue: writeback wb_workfn (flush-179:0) Backtrace: [<c0912bd0>] (__schedule) from [<c0913274>] (schedule+0x78/0xf4) r10:ede1a000 r9:00000000 r8:ede1ba60 r7:ec417290 r6:00000002 r5:ede1a000 r4:ee8e8000 [<c09131fc>] (schedule) from [<c017ec74>] (rwsem_down_write_slowpath+0x24c/0x4c0) r5:00000001 r4:ec417280 [<c017ea28>] (rwsem_down_write_slowpath) from [<c0915f6c>] (down_write+0x6c/0x70) r10:ec417280 r9:ede1bd80 r8:ee128000 r7:00000001 r6:00000000 r5:eff0afc4 r4:ec417280 [<c0915f00>] (down_write) from [<c0435b68>] (f2fs_write_single_data_page+0x608/0x7ac) r5:eff0afc4 r4:ec4170e0 [<c0435560>] (f2fs_write_single_data_page) from [<c0435fc0>] (f2fs_write_cache_pages+0x2b4/0x7c4) r10:ede1bc28 r9:ec4171e0 r8:ec4170e0 r7:00000001 r6:ede1bd80 r5:00000001 r4:eff0afc4 [<c0435d0c>] (f2fs_write_cache_pages) from [<c0436814>] (f2fs_write_data_pages+0x344/0x35c) r10:0000012c r9:ee12802c r8:ee128000 r7:00000004 r6:ec4171e0 r5:ec4170e0 r4:ede1bd80 [<c04364d0>] (f2fs_write_data_pages) from [<c0267fa0>] (do_writepages+0x3c/0xd4) r10:0000012c r9:c0e03d00 r8:00001400 r7:c0264e94 r6:ede1bd80 r5:ec4171e0 r4:ec4170e0 [<c0267f64>] (do_writepages) from [<c0310d24>] (__writeback_single_inode+0x44/0x454) r7:ec4171e0 r6:ede1beac r5:ede1bd80 r4:ec4170e0 [<c0310ce0>] (__writeback_single_inode) from [<c0311338>] (writeback_sb_inodes+0x204/0x4b0) r10:0000012c r9:c0e03d00 r8:ec417148 r7:ec4170e0 r6:ede1beac r5:ec417188 r4:eebed848 [<c0311134>] (writeback_sb_inodes) from [<c0311634>] (__writeback_inodes_wb+0x50/0xe4) r10:ee7128e8 r9:c0e03d00 r8:eebed85c r7:ede1beac r6:00000000 r5:eebed848 r4:ee120000 [<c03115e4>] (__writeback_inodes_wb) from [<c031195c>] (wb_writeback+0x294/0x338) r10:00020800 r9:ede1a000 r8:c0e04e64 r7:eebed848 r6:000192d0 r5:ede1beac r4:eebed848 [<c03116c8>] (wb_writeback) from [<c0312e98>] (wb_workfn+0x3e0/0x54c) r10:ee894005 r9:eebed84c r8:eebed948 r7:eebed848 r6:00000000 r5:eebed954 r4:00002b6e [<c0312ab8>] (wb_workfn) from [<c014f2b8>] (process_one_work+0x214/0x544) r10:ee894005 r9:00000200 r8:00000000 r7:ee894000 r6:ef044400 r5:edb1c700 r4:eebed954 [<c014f0a4>] (process_one_work) from [<c014f634>] (worker_thread+0x4c/0x574) r10:ef044400 r9:c0e03d00 r8:ef044418 r7:00000088 r6:ef044400 r5:edb1c714 r4:edb1c700 [<c014f5e8>] (worker_thread) from [<c01564fc>] (kthread+0x144/0x170) r10:ef125e90 r9:ec0f235c r8:edb1c700 r7:ede1a000 r6:00000000 r5:ec0f2300 r4:ec0f2340 [<c01563b8>] (kthread) from [<c01010e8>] (ret_from_fork+0x14/0x2c) Exception stack(0xede1bfb0 to 0xede1bff8) bfa0: 00000000 00000000 00000000 00000000 bfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 bfe0: 00000000 00000000 00000000 00000000 00000013 00000000 r10:00000000 r9:00000000 r8:00000000 r7:00000000 r6:00000000 r5:c01563b8 r4:ec0f2300 NMI backtrace for cpu 2 CPU: 2 PID: 52 Comm: khungtaskd Not tainted 5.6.0-rc2-00254-g9a029a493dc16 #4 Hardware name: Allwinner A83t board Backtrace: [<c010db5c>] (dump_backtrace) from [<c010dee0>] (show_stack+0x20/0x24) r7:00000000 r6:60060013 r5:00000000 r4:c0e9ab10 > Thread A Thread B > - do_page_mkwrite > - f2fs_vm_page_mkwrite > - lock_page > - f2fs_balance_fs > - mutex_lock(gc_mutex) > - f2fs_gc > - do_garbage_collect > - ra_data_block > - grab_cache_page > - f2fs_balance_fs > - mutex_lock(gc_mutex) > > Fixes: 39a8695824510 ("f2fs: refactor ->page_mkwrite() flow") > Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org> > --- > fs/f2fs/file.c | 7 ++++--- > 1 file changed, 4 insertions(+), 3 deletions(-) > > diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c > index e7fcbd8c23f4..6cebc6681487 100644 > --- a/fs/f2fs/file.c > +++ b/fs/f2fs/file.c > @@ -50,7 +50,7 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf) > struct page *page = vmf->page; > struct inode *inode = file_inode(vmf->vma->vm_file); > struct f2fs_sb_info *sbi = F2FS_I_SB(inode); > - struct dnode_of_data dn = { .node_changed = false }; > + struct dnode_of_data dn; > int err; > > if (unlikely(f2fs_cp_error(sbi))) { > @@ -63,6 +63,9 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf) > goto err; > } > > + /* should do out of any locked page */ > + f2fs_balance_fs(sbi, true); > + > sb_start_pagefault(inode->i_sb); > > f2fs_bug_on(sbi, f2fs_has_inline_data(inode)); > @@ -120,8 +123,6 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf) > out_sem: > up_read(&F2FS_I(inode)->i_mmap_sem); > > - f2fs_balance_fs(sbi, dn.node_changed); > - > sb_end_pagefault(inode->i_sb); > err: > return block_page_mkwrite_return(err); > -- > 2.19.0.605.g01d371f741-goog >
WARNING: multiple messages have this Message-ID (diff)
From: "Ondřej Jirman" <megi@xff.cz> To: Jaegeuk Kim <jaegeuk@kernel.org> Cc: linux-kernel@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net Subject: Re: [f2fs-dev] [PATCH 1/6] f2fs: call f2fs_balance_fs outside of locked page Date: Sat, 22 Feb 2020 05:46:17 +0100 [thread overview] Message-ID: <20200222044617.pfrhnz2iavkrtdn6@core.my.home> (raw) In-Reply-To: <20191209222345.1078-1-jaegeuk@kernel.org> Hello, On Mon, Dec 09, 2019 at 02:23:40PM -0800, Jaegeuk Kim wrote: > Otherwise, we can hit deadlock by waiting for the locked page in > move_data_block in GC. I had the task hangs on 5.6 shortly after boot. (f2fs as rootfs). See below for stacktrace. So I went through the changelog in f2fs and noticed this patch as a suspect, and after reverting it the hung task panics went away. I reverted it manually, because the master changed too much for a clean revert: https://megous.com/git/linux/commit/?h=orange-pi-5.6&id=9983bdae4974edc2af6ff547a401ae397388b6b5 regards, o. INFO: task kworker/u16:2:341 blocked for more than 122 seconds. Not tainted 5.6.0-rc2-00254-g9a029a493dc16 #4 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. kworker/u16:2 D 0 341 2 0x00000000 Workqueue: writeback wb_workfn (flush-179:0) Backtrace: [<c0912bd0>] (__schedule) from [<c0913274>] (schedule+0x78/0xf4) r10:ede1a000 r9:00000000 r8:ede1ba60 r7:ec417290 r6:00000002 r5:ede1a000 r4:ee8e8000 [<c09131fc>] (schedule) from [<c017ec74>] (rwsem_down_write_slowpath+0x24c/0x4c0) r5:00000001 r4:ec417280 [<c017ea28>] (rwsem_down_write_slowpath) from [<c0915f6c>] (down_write+0x6c/0x70) r10:ec417280 r9:ede1bd80 r8:ee128000 r7:00000001 r6:00000000 r5:eff0afc4 r4:ec417280 [<c0915f00>] (down_write) from [<c0435b68>] (f2fs_write_single_data_page+0x608/0x7ac) r5:eff0afc4 r4:ec4170e0 [<c0435560>] (f2fs_write_single_data_page) from [<c0435fc0>] (f2fs_write_cache_pages+0x2b4/0x7c4) r10:ede1bc28 r9:ec4171e0 r8:ec4170e0 r7:00000001 r6:ede1bd80 r5:00000001 r4:eff0afc4 [<c0435d0c>] (f2fs_write_cache_pages) from [<c0436814>] (f2fs_write_data_pages+0x344/0x35c) r10:0000012c r9:ee12802c r8:ee128000 r7:00000004 r6:ec4171e0 r5:ec4170e0 r4:ede1bd80 [<c04364d0>] (f2fs_write_data_pages) from [<c0267fa0>] (do_writepages+0x3c/0xd4) r10:0000012c r9:c0e03d00 r8:00001400 r7:c0264e94 r6:ede1bd80 r5:ec4171e0 r4:ec4170e0 [<c0267f64>] (do_writepages) from [<c0310d24>] (__writeback_single_inode+0x44/0x454) r7:ec4171e0 r6:ede1beac r5:ede1bd80 r4:ec4170e0 [<c0310ce0>] (__writeback_single_inode) from [<c0311338>] (writeback_sb_inodes+0x204/0x4b0) r10:0000012c r9:c0e03d00 r8:ec417148 r7:ec4170e0 r6:ede1beac r5:ec417188 r4:eebed848 [<c0311134>] (writeback_sb_inodes) from [<c0311634>] (__writeback_inodes_wb+0x50/0xe4) r10:ee7128e8 r9:c0e03d00 r8:eebed85c r7:ede1beac r6:00000000 r5:eebed848 r4:ee120000 [<c03115e4>] (__writeback_inodes_wb) from [<c031195c>] (wb_writeback+0x294/0x338) r10:00020800 r9:ede1a000 r8:c0e04e64 r7:eebed848 r6:000192d0 r5:ede1beac r4:eebed848 [<c03116c8>] (wb_writeback) from [<c0312e98>] (wb_workfn+0x3e0/0x54c) r10:ee894005 r9:eebed84c r8:eebed948 r7:eebed848 r6:00000000 r5:eebed954 r4:00002b6e [<c0312ab8>] (wb_workfn) from [<c014f2b8>] (process_one_work+0x214/0x544) r10:ee894005 r9:00000200 r8:00000000 r7:ee894000 r6:ef044400 r5:edb1c700 r4:eebed954 [<c014f0a4>] (process_one_work) from [<c014f634>] (worker_thread+0x4c/0x574) r10:ef044400 r9:c0e03d00 r8:ef044418 r7:00000088 r6:ef044400 r5:edb1c714 r4:edb1c700 [<c014f5e8>] (worker_thread) from [<c01564fc>] (kthread+0x144/0x170) r10:ef125e90 r9:ec0f235c r8:edb1c700 r7:ede1a000 r6:00000000 r5:ec0f2300 r4:ec0f2340 [<c01563b8>] (kthread) from [<c01010e8>] (ret_from_fork+0x14/0x2c) Exception stack(0xede1bfb0 to 0xede1bff8) bfa0: 00000000 00000000 00000000 00000000 bfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 bfe0: 00000000 00000000 00000000 00000000 00000013 00000000 r10:00000000 r9:00000000 r8:00000000 r7:00000000 r6:00000000 r5:c01563b8 r4:ec0f2300 NMI backtrace for cpu 2 CPU: 2 PID: 52 Comm: khungtaskd Not tainted 5.6.0-rc2-00254-g9a029a493dc16 #4 Hardware name: Allwinner A83t board Backtrace: [<c010db5c>] (dump_backtrace) from [<c010dee0>] (show_stack+0x20/0x24) r7:00000000 r6:60060013 r5:00000000 r4:c0e9ab10 > Thread A Thread B > - do_page_mkwrite > - f2fs_vm_page_mkwrite > - lock_page > - f2fs_balance_fs > - mutex_lock(gc_mutex) > - f2fs_gc > - do_garbage_collect > - ra_data_block > - grab_cache_page > - f2fs_balance_fs > - mutex_lock(gc_mutex) > > Fixes: 39a8695824510 ("f2fs: refactor ->page_mkwrite() flow") > Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org> > --- > fs/f2fs/file.c | 7 ++++--- > 1 file changed, 4 insertions(+), 3 deletions(-) > > diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c > index e7fcbd8c23f4..6cebc6681487 100644 > --- a/fs/f2fs/file.c > +++ b/fs/f2fs/file.c > @@ -50,7 +50,7 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf) > struct page *page = vmf->page; > struct inode *inode = file_inode(vmf->vma->vm_file); > struct f2fs_sb_info *sbi = F2FS_I_SB(inode); > - struct dnode_of_data dn = { .node_changed = false }; > + struct dnode_of_data dn; > int err; > > if (unlikely(f2fs_cp_error(sbi))) { > @@ -63,6 +63,9 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf) > goto err; > } > > + /* should do out of any locked page */ > + f2fs_balance_fs(sbi, true); > + > sb_start_pagefault(inode->i_sb); > > f2fs_bug_on(sbi, f2fs_has_inline_data(inode)); > @@ -120,8 +123,6 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf) > out_sem: > up_read(&F2FS_I(inode)->i_mmap_sem); > > - f2fs_balance_fs(sbi, dn.node_changed); > - > sb_end_pagefault(inode->i_sb); > err: > return block_page_mkwrite_return(err); > -- > 2.19.0.605.g01d371f741-goog > _______________________________________________ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
next prev parent reply other threads:[~2020-02-22 4:52 UTC|newest] Thread overview: 83+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-12-09 22:23 [PATCH 1/6] f2fs: call f2fs_balance_fs outside of locked page Jaegeuk Kim 2019-12-09 22:23 ` [f2fs-dev] " Jaegeuk Kim 2019-12-09 22:23 ` [PATCH 2/6] f2fs: declare nested quota_sem and remove unnecessary sems Jaegeuk Kim 2019-12-09 22:23 ` [f2fs-dev] " Jaegeuk Kim 2019-12-10 6:20 ` Chao Yu 2019-12-10 6:20 ` Chao Yu 2019-12-09 22:23 ` [PATCH 3/6] f2fs: keep quota data on write_begin failure Jaegeuk Kim 2019-12-09 22:23 ` [f2fs-dev] " Jaegeuk Kim 2019-12-10 6:22 ` Chao Yu 2019-12-10 6:22 ` Chao Yu 2019-12-09 22:23 ` [PATCH 4/6] f2fs: should avoid recursive filesystem ops Jaegeuk Kim 2019-12-09 22:23 ` [f2fs-dev] " Jaegeuk Kim 2019-12-10 6:22 ` Chao Yu 2019-12-10 6:22 ` Chao Yu 2019-12-09 22:23 ` [PATCH 5/6] f2fs: set GFP_NOFS when moving inline dentries Jaegeuk Kim 2019-12-09 22:23 ` [f2fs-dev] " Jaegeuk Kim 2019-12-10 6:22 ` Chao Yu 2019-12-10 6:22 ` Chao Yu 2019-12-09 22:23 ` [PATCH 6/6] f2fs: set I_LINKABLE early to avoid wrong access by vfs Jaegeuk Kim 2019-12-09 22:23 ` [f2fs-dev] " Jaegeuk Kim 2019-12-10 6:37 ` Chao Yu 2019-12-10 6:37 ` Chao Yu 2019-12-11 1:21 ` Jaegeuk Kim 2019-12-11 1:21 ` Jaegeuk Kim 2019-12-11 1:23 ` Chao Yu 2019-12-11 1:23 ` Chao Yu 2019-12-11 1:31 ` Jaegeuk Kim 2019-12-11 1:31 ` Jaegeuk Kim 2019-12-11 1:42 ` Chao Yu 2019-12-11 1:42 ` Chao Yu 2019-12-12 16:55 ` Jaegeuk Kim 2019-12-12 16:55 ` Jaegeuk Kim 2019-12-10 2:09 ` [f2fs-dev] [PATCH 1/6] f2fs: call f2fs_balance_fs outside of locked page Chao Yu 2019-12-10 2:09 ` Chao Yu 2020-02-22 4:46 ` Ondřej Jirman [this message] 2020-02-22 4:46 ` Ondřej Jirman 2020-02-22 18:17 ` Writes stoped working on f2fs after the compression support was added Ondřej Jirman 2020-02-22 18:17 ` [f2fs-dev] " Ondřej Jirman 2020-02-24 10:37 ` Chao Yu 2020-02-24 10:37 ` [f2fs-dev] " Chao Yu 2020-02-24 10:41 ` Chao Yu 2020-02-24 10:41 ` Chao Yu 2020-02-24 13:58 ` Ondřej Jirman 2020-02-24 13:58 ` Ondřej Jirman 2020-02-24 14:03 ` Ondřej Jirman 2020-02-24 14:03 ` Ondřej Jirman 2020-02-24 14:31 ` Ondřej Jirman 2020-02-24 14:31 ` Ondřej Jirman 2020-02-25 11:24 ` Chao Yu 2020-02-25 11:24 ` Chao Yu 2020-02-25 11:32 ` Chao Yu 2020-02-25 11:32 ` Chao Yu 2020-02-25 12:08 ` Ondřej Jirman 2020-02-25 12:08 ` Ondřej Jirman 2020-02-25 12:27 ` Ondřej Jirman 2020-02-25 12:27 ` Ondřej Jirman 2020-02-26 1:58 ` Chao Yu 2020-02-26 1:58 ` Chao Yu 2020-02-26 12:11 ` Ondřej Jirman 2020-02-26 12:11 ` Ondřej Jirman 2020-02-26 18:05 ` Ondřej Jirman 2020-02-26 18:05 ` Ondřej Jirman 2020-02-27 2:01 ` Chao Yu 2020-02-27 2:01 ` Chao Yu 2020-03-06 12:02 ` Ondřej Jirman 2020-03-06 12:02 ` Ondřej Jirman 2020-03-06 12:43 ` Ondřej Jirman 2020-03-06 12:43 ` Ondřej Jirman 2020-03-11 9:02 ` Chao Yu 2020-03-11 9:02 ` Chao Yu 2020-03-11 10:33 ` Ondřej Jirman 2020-03-11 10:33 ` Ondřej Jirman 2020-03-11 10:51 ` Chao Yu 2020-03-11 10:51 ` Chao Yu 2020-03-11 11:01 ` Ondřej Jirman 2020-03-11 11:01 ` Ondřej Jirman 2020-03-11 12:13 ` Chao Yu 2020-03-11 17:01 ` Jaegeuk Kim 2020-03-11 17:01 ` Jaegeuk Kim 2020-03-22 10:15 ` Chao Yu 2020-03-22 10:15 ` Chao Yu 2020-02-24 14:20 ` Ondřej Jirman 2020-02-24 14:20 ` Ondřej Jirman
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200222044617.pfrhnz2iavkrtdn6@core.my.home \ --to=megi@xff.cz \ --cc=jaegeuk@kernel.org \ --cc=linux-f2fs-devel@lists.sourceforge.net \ --cc=linux-kernel@vger.kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.