* [PATCH] fuse: fix the deadlock in race of reclaim path with kswapd
@ 2022-09-05 7:17 Kassey Li
2022-09-05 13:28 ` Miklos Szeredi
0 siblings, 1 reply; 3+ messages in thread
From: Kassey Li @ 2022-09-05 7:17 UTC (permalink / raw)
To: miklos, linux-fsdevel; +Cc: Kassey Li, quic_maow, linux-kernel
Task A wait for writeback, while writeback Task B send request to fuse.
Task C is expected to serve this request, here it is in direct reclaim
path cause deadlock when system is in low memory.
without __GFP_FS in Task_C break throttle_direct_reclaim with an
HZ timeout.
kswpad (Task_A): writeback(Task_B):
__switch_to+0x14c schedule+0x70
__schedule+0xb5c __fuse_request_send+0x154
schedule+0x70 fuse_simple_request+0x184
bit_wait+0x18 fuse_flush_times+0x114
__wait_on_bit+0x74 fuse_write_inode+0x60
inode_wait_for_writeback+0xa4 __writeback_single_inode+0x3d8
evict+0xa8 writeback_sb_inodes+0x4c0
iput+0x248 __writeback_inodes_wb+0xb0
dentry_unlink_inode+0xdc wb_writeback+0x270
__dentry_kill[jt]+0x110 wb_workfn+0x37c
shrink_dentry_list+0x17c process_one_work+0x284
prune_dcache_sb+0x5c
super_cache_scan+0x11c
do_shrink_slab+0x248
shrink_slab+0x260
shrink_node+0x678
kswapd+0x8ec
kthread+0x140
ret_from_fork+0x10
Task_C:
__switch_to+0x14c
__schedule+0xb5c
schedule+0x70
throttle_direct_reclaim
try_to_free_pages
__perform_reclaim
__alloc_pages_direct_reclaim
__alloc_pages_slowpath
__alloc_pages_nodemask
alloc_pages
fuse_copy_fill+0x168
fuse_dev_do_read+0x37c
fuse_dev_splice_read+0x94
Suggested-by: Wang Mao <quic_maow@quicinc.com>
Signed-off-by: Kassey Li <quic_yingangl@quicinc.com>
---
fs/fuse/dev.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/fs/fuse/dev.c b/fs/fuse/dev.c
index 51897427a534..0df7234840c3 100644
--- a/fs/fuse/dev.c
+++ b/fs/fuse/dev.c
@@ -713,7 +713,7 @@ static int fuse_copy_fill(struct fuse_copy_state *cs)
if (cs->nr_segs >= cs->pipe->max_usage)
return -EIO;
- page = alloc_page(GFP_HIGHUSER);
+ page = alloc_page(GFP_HIGHUSER & ~__GFP_FS);
if (!page)
return -ENOMEM;
--
2.17.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH] fuse: fix the deadlock in race of reclaim path with kswapd
2022-09-05 7:17 [PATCH] fuse: fix the deadlock in race of reclaim path with kswapd Kassey Li
@ 2022-09-05 13:28 ` Miklos Szeredi
2022-09-06 0:29 ` Kassey Li
0 siblings, 1 reply; 3+ messages in thread
From: Miklos Szeredi @ 2022-09-05 13:28 UTC (permalink / raw)
To: Kassey Li; +Cc: linux-fsdevel, quic_maow, linux-kernel
On Mon, 5 Sept 2022 at 09:17, Kassey Li <quic_yingangl@quicinc.com> wrote:
>
> Task A wait for writeback, while writeback Task B send request to fuse.
> Task C is expected to serve this request, here it is in direct reclaim
> path cause deadlock when system is in low memory.
>
> without __GFP_FS in Task_C break throttle_direct_reclaim with an
> HZ timeout.
>
> kswpad (Task_A): writeback(Task_B):
> __switch_to+0x14c schedule+0x70
> __schedule+0xb5c __fuse_request_send+0x154
> schedule+0x70 fuse_simple_request+0x184
> bit_wait+0x18 fuse_flush_times+0x114
> __wait_on_bit+0x74 fuse_write_inode+0x60
> inode_wait_for_writeback+0xa4 __writeback_single_inode+0x3d8
> evict+0xa8 writeback_sb_inodes+0x4c0
> iput+0x248 __writeback_inodes_wb+0xb0
> dentry_unlink_inode+0xdc wb_writeback+0x270
> __dentry_kill[jt]+0x110 wb_workfn+0x37c
> shrink_dentry_list+0x17c process_one_work+0x284
> prune_dcache_sb+0x5c
> super_cache_scan+0x11c
> do_shrink_slab+0x248
> shrink_slab+0x260
> shrink_node+0x678
> kswapd+0x8ec
> kthread+0x140
> ret_from_fork+0x10
>
> Task_C:
> __switch_to+0x14c
> __schedule+0xb5c
> schedule+0x70
> throttle_direct_reclaim
> try_to_free_pages
> __perform_reclaim
> __alloc_pages_direct_reclaim
> __alloc_pages_slowpath
> __alloc_pages_nodemask
> alloc_pages
> fuse_copy_fill+0x168
> fuse_dev_do_read+0x37c
> fuse_dev_splice_read+0x94
Should already be fixed in v5.16 by commit 5c791fe1e2a4 ("fuse: make
sure reclaim doesn't write the inode").
Thanks,
Miklos
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH] fuse: fix the deadlock in race of reclaim path with kswapd
2022-09-05 13:28 ` Miklos Szeredi
@ 2022-09-06 0:29 ` Kassey Li
0 siblings, 0 replies; 3+ messages in thread
From: Kassey Li @ 2022-09-06 0:29 UTC (permalink / raw)
To: Miklos Szeredi; +Cc: linux-fsdevel, quic_maow, linux-kernel
On 9/5/2022 9:28 PM, Miklos Szeredi wrote:
> On Mon, 5 Sept 2022 at 09:17, Kassey Li <quic_yingangl@quicinc.com> wrote:
>>
>> Task A wait for writeback, while writeback Task B send request to fuse.
>> Task C is expected to serve this request, here it is in direct reclaim
>> path cause deadlock when system is in low memory.
>>
>> without __GFP_FS in Task_C break throttle_direct_reclaim with an
>> HZ timeout.
>>
>> kswpad (Task_A): writeback(Task_B):
>> __switch_to+0x14c schedule+0x70
>> __schedule+0xb5c __fuse_request_send+0x154
>> schedule+0x70 fuse_simple_request+0x184
>> bit_wait+0x18 fuse_flush_times+0x114
>> __wait_on_bit+0x74 fuse_write_inode+0x60
>> inode_wait_for_writeback+0xa4 __writeback_single_inode+0x3d8
>> evict+0xa8 writeback_sb_inodes+0x4c0
>> iput+0x248 __writeback_inodes_wb+0xb0
>> dentry_unlink_inode+0xdc wb_writeback+0x270
>> __dentry_kill[jt]+0x110 wb_workfn+0x37c
>> shrink_dentry_list+0x17c process_one_work+0x284
>> prune_dcache_sb+0x5c
>> super_cache_scan+0x11c
>> do_shrink_slab+0x248
>> shrink_slab+0x260
>> shrink_node+0x678
>> kswapd+0x8ec
>> kthread+0x140
>> ret_from_fork+0x10
>>
>> Task_C:
>> __switch_to+0x14c
>> __schedule+0xb5c
>> schedule+0x70
>> throttle_direct_reclaim
>> try_to_free_pages
>> __perform_reclaim
>> __alloc_pages_direct_reclaim
>> __alloc_pages_slowpath
>> __alloc_pages_nodemask
>> alloc_pages
>> fuse_copy_fill+0x168
>> fuse_dev_do_read+0x37c
>> fuse_dev_splice_read+0x94
>
> Should already be fixed in v5.16 by commit 5c791fe1e2a4 ("fuse: make
> sure reclaim doesn't write the inode").
thanks for this info.
>
> Thanks,
> Miklos
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2022-09-06 0:29 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-09-05 7:17 [PATCH] fuse: fix the deadlock in race of reclaim path with kswapd Kassey Li
2022-09-05 13:28 ` Miklos Szeredi
2022-09-06 0:29 ` Kassey Li
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).