Linux-ext4 Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH] ext4: add cond_resched() to __ext4_find_entry()
@ 2020-02-15  8:02 Shijie Luo
  2020-02-17 10:31 ` Jan Kara
  2020-02-20  4:55 ` Theodore Y. Ts'o
  0 siblings, 2 replies; 3+ messages in thread
From: Shijie Luo @ 2020-02-15  8:02 UTC (permalink / raw)
  To: linux-ext4; +Cc: tytso, jack, luoshijie1, lutianxiong

We tested a soft lockup problem in linux 4.19 which could also
be found in linux 5.x.

When dir inode takes up a large number of blocks, and if the
directory is growing when we are searching, it's possible the
restart branch could be called many times, and the do while loop
could hold cpu a long time.

Here is the call trace in linux 4.19.

[  473.756186] Call trace:
[  473.756196]  dump_backtrace+0x0/0x198
[  473.756199]  show_stack+0x24/0x30
[  473.756205]  dump_stack+0xa4/0xcc
[  473.756210]  watchdog_timer_fn+0x300/0x3e8
[  473.756215]  __hrtimer_run_queues+0x114/0x358
[  473.756217]  hrtimer_interrupt+0x104/0x2d8
[  473.756222]  arch_timer_handler_virt+0x38/0x58
[  473.756226]  handle_percpu_devid_irq+0x90/0x248
[  473.756231]  generic_handle_irq+0x34/0x50
[  473.756234]  __handle_domain_irq+0x68/0xc0
[  473.756236]  gic_handle_irq+0x6c/0x150
[  473.756238]  el1_irq+0xb8/0x140
[  473.756286]  ext4_es_lookup_extent+0xdc/0x258 [ext4]
[  473.756310]  ext4_map_blocks+0x64/0x5c0 [ext4]
[  473.756333]  ext4_getblk+0x6c/0x1d0 [ext4]
[  473.756356]  ext4_bread_batch+0x7c/0x1f8 [ext4]
[  473.756379]  ext4_find_entry+0x124/0x3f8 [ext4]
[  473.756402]  ext4_lookup+0x8c/0x258 [ext4]
[  473.756407]  __lookup_hash+0x8c/0xe8
[  473.756411]  filename_create+0xa0/0x170
[  473.756413]  do_mkdirat+0x6c/0x140
[  473.756415]  __arm64_sys_mkdirat+0x28/0x38
[  473.756419]  el0_svc_common+0x78/0x130
[  473.756421]  el0_svc_handler+0x38/0x78
[  473.756423]  el0_svc+0x8/0xc
[  485.755156] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [tmp:5149]

Add cond_resched() to avoid soft lockup and to provide a better
system responding.

Signed-off-by: Shijie Luo <luoshijie1@huawei.com>
---
 fs/ext4/namei.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c
index 129d2ebae00d..9a1836632f51 100644
--- a/fs/ext4/namei.c
+++ b/fs/ext4/namei.c
@@ -1511,6 +1511,7 @@ static struct buffer_head *__ext4_find_entry(struct inode *dir,
 		/*
 		 * We deal with the read-ahead logic here.
 		 */
+		cond_resched();
 		if (ra_ptr >= ra_max) {
 			/* Refill the readahead buffer */
 			ra_ptr = 0;
-- 
2.19.1


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [PATCH] ext4: add cond_resched() to __ext4_find_entry()
  2020-02-15  8:02 [PATCH] ext4: add cond_resched() to __ext4_find_entry() Shijie Luo
@ 2020-02-17 10:31 ` Jan Kara
  2020-02-20  4:55 ` Theodore Y. Ts'o
  1 sibling, 0 replies; 3+ messages in thread
From: Jan Kara @ 2020-02-17 10:31 UTC (permalink / raw)
  To: Shijie Luo; +Cc: linux-ext4, tytso, jack, lutianxiong

On Sat 15-02-20 03:02:06, Shijie Luo wrote:
> We tested a soft lockup problem in linux 4.19 which could also
> be found in linux 5.x.
> 
> When dir inode takes up a large number of blocks, and if the
> directory is growing when we are searching, it's possible the
> restart branch could be called many times, and the do while loop
> could hold cpu a long time.
> 
> Here is the call trace in linux 4.19.
> 
> [  473.756186] Call trace:
> [  473.756196]  dump_backtrace+0x0/0x198
> [  473.756199]  show_stack+0x24/0x30
> [  473.756205]  dump_stack+0xa4/0xcc
> [  473.756210]  watchdog_timer_fn+0x300/0x3e8
> [  473.756215]  __hrtimer_run_queues+0x114/0x358
> [  473.756217]  hrtimer_interrupt+0x104/0x2d8
> [  473.756222]  arch_timer_handler_virt+0x38/0x58
> [  473.756226]  handle_percpu_devid_irq+0x90/0x248
> [  473.756231]  generic_handle_irq+0x34/0x50
> [  473.756234]  __handle_domain_irq+0x68/0xc0
> [  473.756236]  gic_handle_irq+0x6c/0x150
> [  473.756238]  el1_irq+0xb8/0x140
> [  473.756286]  ext4_es_lookup_extent+0xdc/0x258 [ext4]
> [  473.756310]  ext4_map_blocks+0x64/0x5c0 [ext4]
> [  473.756333]  ext4_getblk+0x6c/0x1d0 [ext4]
> [  473.756356]  ext4_bread_batch+0x7c/0x1f8 [ext4]
> [  473.756379]  ext4_find_entry+0x124/0x3f8 [ext4]
> [  473.756402]  ext4_lookup+0x8c/0x258 [ext4]
> [  473.756407]  __lookup_hash+0x8c/0xe8
> [  473.756411]  filename_create+0xa0/0x170
> [  473.756413]  do_mkdirat+0x6c/0x140
> [  473.756415]  __arm64_sys_mkdirat+0x28/0x38
> [  473.756419]  el0_svc_common+0x78/0x130
> [  473.756421]  el0_svc_handler+0x38/0x78
> [  473.756423]  el0_svc+0x8/0xc
> [  485.755156] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [tmp:5149]
> 
> Add cond_resched() to avoid soft lockup and to provide a better
> system responding.
> 
> Signed-off-by: Shijie Luo <luoshijie1@huawei.com>

Thanks for the patch! It looks good. You can add:

Reviewed-by: Jan Kara <jack@suse.cz>

I was just wondering how come we didn't hit other scheduling points (e.g.
when reading directory blocks from the disk) but likely the directory
including new blocks was staying in cache...

								Honza

> ---
>  fs/ext4/namei.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c
> index 129d2ebae00d..9a1836632f51 100644
> --- a/fs/ext4/namei.c
> +++ b/fs/ext4/namei.c
> @@ -1511,6 +1511,7 @@ static struct buffer_head *__ext4_find_entry(struct inode *dir,
>  		/*
>  		 * We deal with the read-ahead logic here.
>  		 */
> +		cond_resched();
>  		if (ra_ptr >= ra_max) {
>  			/* Refill the readahead buffer */
>  			ra_ptr = 0;
> -- 
> 2.19.1
> 
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [PATCH] ext4: add cond_resched() to __ext4_find_entry()
  2020-02-15  8:02 [PATCH] ext4: add cond_resched() to __ext4_find_entry() Shijie Luo
  2020-02-17 10:31 ` Jan Kara
@ 2020-02-20  4:55 ` Theodore Y. Ts'o
  1 sibling, 0 replies; 3+ messages in thread
From: Theodore Y. Ts'o @ 2020-02-20  4:55 UTC (permalink / raw)
  To: Shijie Luo; +Cc: linux-ext4, jack, lutianxiong

On Sat, Feb 15, 2020 at 03:02:06AM -0500, Shijie Luo wrote:
> We tested a soft lockup problem in linux 4.19 which could also
> be found in linux 5.x.
> 
> When dir inode takes up a large number of blocks, and if the
> directory is growing when we are searching, it's possible the
> restart branch could be called many times, and the do while loop
> could hold cpu a long time.
> 
> Here is the call trace in linux 4.19.
> 
> [  473.756186] Call trace:
> [  473.756196]  dump_backtrace+0x0/0x198
> [  473.756199]  show_stack+0x24/0x30
> [  473.756205]  dump_stack+0xa4/0xcc
> [  473.756210]  watchdog_timer_fn+0x300/0x3e8
> [  473.756215]  __hrtimer_run_queues+0x114/0x358
> [  473.756217]  hrtimer_interrupt+0x104/0x2d8
> [  473.756222]  arch_timer_handler_virt+0x38/0x58
> [  473.756226]  handle_percpu_devid_irq+0x90/0x248
> [  473.756231]  generic_handle_irq+0x34/0x50
> [  473.756234]  __handle_domain_irq+0x68/0xc0
> [  473.756236]  gic_handle_irq+0x6c/0x150
> [  473.756238]  el1_irq+0xb8/0x140
> [  473.756286]  ext4_es_lookup_extent+0xdc/0x258 [ext4]
> [  473.756310]  ext4_map_blocks+0x64/0x5c0 [ext4]
> [  473.756333]  ext4_getblk+0x6c/0x1d0 [ext4]
> [  473.756356]  ext4_bread_batch+0x7c/0x1f8 [ext4]
> [  473.756379]  ext4_find_entry+0x124/0x3f8 [ext4]
> [  473.756402]  ext4_lookup+0x8c/0x258 [ext4]
> [  473.756407]  __lookup_hash+0x8c/0xe8
> [  473.756411]  filename_create+0xa0/0x170
> [  473.756413]  do_mkdirat+0x6c/0x140
> [  473.756415]  __arm64_sys_mkdirat+0x28/0x38
> [  473.756419]  el0_svc_common+0x78/0x130
> [  473.756421]  el0_svc_handler+0x38/0x78
> [  473.756423]  el0_svc+0x8/0xc
> [  485.755156] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [tmp:5149]
> 
> Add cond_resched() to avoid soft lockup and to provide a better
> system responding.
> 
> Signed-off-by: Shijie Luo <luoshijie1@huawei.com>

Thanks, applied.

					- Ted

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, back to index

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-15  8:02 [PATCH] ext4: add cond_resched() to __ext4_find_entry() Shijie Luo
2020-02-17 10:31 ` Jan Kara
2020-02-20  4:55 ` Theodore Y. Ts'o

Linux-ext4 Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-ext4/0 linux-ext4/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-ext4 linux-ext4/ https://lore.kernel.org/linux-ext4 \
		linux-ext4@vger.kernel.org
	public-inbox-index linux-ext4

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-ext4


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git