linux-erofs.lists.ozlabs.org archive mirror
 help / color / mirror / Atom feed
* [PATCH] erofs: fix deadlock when shrink erofs slab
@ 2021-11-18 13:58 Huang Jianan via Linux-erofs
  2021-11-19  1:32 ` Chao Yu
  2021-11-19  2:19 ` Gao Xiang
  0 siblings, 2 replies; 3+ messages in thread
From: Huang Jianan via Linux-erofs @ 2021-11-18 13:58 UTC (permalink / raw)
  To: linux-erofs, xiang; +Cc: yh, guoweichao, zhangshiming, guanyuwei

We observed the following deadlock in the stress test under low
memory scenario:

Thread A                               Thread B
- erofs_shrink_scan
 - erofs_try_to_release_workgroup
  - erofs_workgroup_try_to_freeze -- A
                                       - z_erofs_do_read_page
                                        - z_erofs_collection_begin
                                         - z_erofs_register_collection
                                          - erofs_insert_workgroup
                                           - xa_lock(&sbi->managed_pslots) -- B
                                           - erofs_workgroup_get
                                            - erofs_wait_on_workgroup_freezed -- A
  - xa_erase
   - xa_lock(&sbi->managed_pslots) -- B

To fix this, it need to hold the xa lock before freeze the workgroup
beacuse we will operate xarry. So let's hold the lock before access
each workgroup, just like when we using the radix tree before.

Fixes: 64094a04414f ("erofs: convert workstn to XArray")
Signed-off-by: Huang Jianan <huangjianan@oppo.com>
---
 fs/erofs/utils.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/fs/erofs/utils.c b/fs/erofs/utils.c
index 84da2c280012..84a59f075dd1 100644
--- a/fs/erofs/utils.c
+++ b/fs/erofs/utils.c
@@ -150,7 +150,7 @@ static bool erofs_try_to_release_workgroup(struct erofs_sb_info *sbi,
 	 * however in order to avoid some race conditions, add a
 	 * DBG_BUGON to observe this in advance.
 	 */
-	DBG_BUGON(xa_erase(&sbi->managed_pslots, grp->index) != grp);
+	DBG_BUGON(__xa_erase(&sbi->managed_pslots, grp->index) != grp);
 
 	/* last refcount should be connected with its managed pslot.  */
 	erofs_workgroup_unfreeze(grp, 0);
@@ -165,15 +165,20 @@ static unsigned long erofs_shrink_workstation(struct erofs_sb_info *sbi,
 	unsigned int freed = 0;
 	unsigned long index;
 
+	xa_lock(&sbi->managed_pslots);
 	xa_for_each(&sbi->managed_pslots, index, grp) {
 		/* try to shrink each valid workgroup */
 		if (!erofs_try_to_release_workgroup(sbi, grp))
 			continue;
+		xa_unlock(&sbi->managed_pslots);
 
 		++freed;
 		if (!--nr_shrink)
-			break;
+			return freed;
+		xa_lock(&sbi->managed_pslots);
 	}
+	xa_unlock(&sbi->managed_pslots);
+
 	return freed;
 }
 
-- 
2.25.1


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [PATCH] erofs: fix deadlock when shrink erofs slab
  2021-11-18 13:58 [PATCH] erofs: fix deadlock when shrink erofs slab Huang Jianan via Linux-erofs
@ 2021-11-19  1:32 ` Chao Yu
  2021-11-19  2:19 ` Gao Xiang
  1 sibling, 0 replies; 3+ messages in thread
From: Chao Yu @ 2021-11-19  1:32 UTC (permalink / raw)
  To: Huang Jianan, linux-erofs, xiang; +Cc: yh, guoweichao, zhangshiming, guanyuwei

On 2021/11/18 21:58, Huang Jianan via Linux-erofs wrote:
> We observed the following deadlock in the stress test under low
> memory scenario:
> 
> Thread A                               Thread B
> - erofs_shrink_scan
>   - erofs_try_to_release_workgroup
>    - erofs_workgroup_try_to_freeze -- A
>                                         - z_erofs_do_read_page
>                                          - z_erofs_collection_begin
>                                           - z_erofs_register_collection
>                                            - erofs_insert_workgroup
>                                             - xa_lock(&sbi->managed_pslots) -- B
>                                             - erofs_workgroup_get
>                                              - erofs_wait_on_workgroup_freezed -- A
>    - xa_erase
>     - xa_lock(&sbi->managed_pslots) -- B
> 
> To fix this, it need to hold the xa lock before freeze the workgroup
> beacuse we will operate xarry. So let's hold the lock before access
> each workgroup, just like when we using the radix tree before.
> 
> Fixes: 64094a04414f ("erofs: convert workstn to XArray")
> Signed-off-by: Huang Jianan <huangjianan@oppo.com>

Reviewed-by: Chao Yu <chao@kernel.org>

Thanks,

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [PATCH] erofs: fix deadlock when shrink erofs slab
  2021-11-18 13:58 [PATCH] erofs: fix deadlock when shrink erofs slab Huang Jianan via Linux-erofs
  2021-11-19  1:32 ` Chao Yu
@ 2021-11-19  2:19 ` Gao Xiang
  1 sibling, 0 replies; 3+ messages in thread
From: Gao Xiang @ 2021-11-19  2:19 UTC (permalink / raw)
  To: Huang Jianan; +Cc: zhangshiming, yh, guanyuwei, guoweichao, linux-erofs

On Thu, Nov 18, 2021 at 09:58:44PM +0800, Huang Jianan wrote:
> We observed the following deadlock in the stress test under low
> memory scenario:
> 
> Thread A                               Thread B
> - erofs_shrink_scan
>  - erofs_try_to_release_workgroup
>   - erofs_workgroup_try_to_freeze -- A
>                                        - z_erofs_do_read_page
>                                         - z_erofs_collection_begin
>                                          - z_erofs_register_collection
>                                           - erofs_insert_workgroup
>                                            - xa_lock(&sbi->managed_pslots) -- B
>                                            - erofs_workgroup_get
>                                             - erofs_wait_on_workgroup_freezed -- A
>   - xa_erase
>    - xa_lock(&sbi->managed_pslots) -- B
> 
> To fix this, it need to hold the xa lock before freeze the workgroup
> beacuse we will operate xarry. So let's hold the lock before access
  ^ because               ^ xarray

> each workgroup, just like when we using the radix tree before.
> 
> Fixes: 64094a04414f ("erofs: convert workstn to XArray")
> Signed-off-by: Huang Jianan <huangjianan@oppo.com>

Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>

Thanks,
Gao Xiang


^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2021-11-19  2:19 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-11-18 13:58 [PATCH] erofs: fix deadlock when shrink erofs slab Huang Jianan via Linux-erofs
2021-11-19  1:32 ` Chao Yu
2021-11-19  2:19 ` Gao Xiang

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).