All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v2] ocfs2: ocfs2_downconvert_lock failure results in deadlock
@ 2021-08-30  4:46 ` Gang He
  0 siblings, 0 replies; 4+ messages in thread
From: Gang He @ 2021-08-30  4:46 UTC (permalink / raw)
  To: mark, jlbec, joseph.qi; +Cc: Gang He, linux-kernel, ocfs2-devel, akpm

Usually, ocfs2_downconvert_lock() function always downconverts
dlm lock to the expected level for satisfy dlm bast requests
from the other nodes.
But there is a rare situation. When dlm lock conversion is being
canceled, ocfs2_downconvert_lock() function will return -EBUSY.
You need to be aware that ocfs2_cancel_convert() function is
asynchronous in fsdlm implementation.
If we does not requeue this lockres entry, ocfs2 downconvert
thread no longer handles this dlm lock bast request. Then, the
other nodes will not get the dlm lock again, the current node's
process will be blocked when acquire this dlm lock again.

Signed-off-by: Gang He <ghe@suse.com>
---
v2: keep error print in ocfs2_downconvert_lock function, add more detailed
comments in code, add msleep to avoid repeated attempts. 
---
 fs/ocfs2/dlmglue.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/fs/ocfs2/dlmglue.c b/fs/ocfs2/dlmglue.c
index 48fd369c29a4..f8f561850470 100644
--- a/fs/ocfs2/dlmglue.c
+++ b/fs/ocfs2/dlmglue.c
@@ -16,6 +16,7 @@
 #include <linux/debugfs.h>
 #include <linux/seq_file.h>
 #include <linux/time.h>
+#include <linux/delay.h>
 #include <linux/quotaops.h>
 #include <linux/sched/signal.h>
 
@@ -3912,6 +3913,17 @@ static int ocfs2_unblock_lock(struct ocfs2_super *osb,
 	spin_unlock_irqrestore(&lockres->l_lock, flags);
 	ret = ocfs2_downconvert_lock(osb, lockres, new_level, set_lvb,
 				     gen);
+	/* The dlm lock convert is being cancelled in background,
+	 * ocfs2_cancel_convert() is asynchronous in fs/dlm,
+	 * requeue it, try again later.
+	 */
+	if (ret == -EBUSY) {
+		ctl->requeue = 1;
+		mlog(ML_BASTS, "lockres %s, ReQ: Downconvert busy\n",
+		     lockres->l_name);
+		ret = 0;
+		msleep(20);
+	}
 
 leave:
 	if (ret)
-- 
2.12.3


^ permalink raw reply related	[flat|nested] 4+ messages in thread
* [Ocfs2-devel] [PATCH v2] ocfs2: ocfs2_downconvert_lock failure results in deadlock
@ 2021-08-30  4:46 ` Gang He
  0 siblings, 0 replies; 4+ messages in thread
From: Gang He @ 2021-08-30  4:46 UTC (permalink / raw)
  To: mark, jlbec, joseph.qi; +Cc: linux-kernel, ocfs2-devel

Usually, ocfs2_downconvert_lock() function always downconverts
dlm lock to the expected level for satisfy dlm bast requests
from the other nodes.
But there is a rare situation. When dlm lock conversion is being
canceled, ocfs2_downconvert_lock() function will return -EBUSY.
You need to be aware that ocfs2_cancel_convert() function is
asynchronous in fsdlm implementation.
If we does not requeue this lockres entry, ocfs2 downconvert
thread no longer handles this dlm lock bast request. Then, the
other nodes will not get the dlm lock again, the current node's
process will be blocked when acquire this dlm lock again.

Signed-off-by: Gang He <ghe@suse.com>
---
v2: keep error print in ocfs2_downconvert_lock function, add more detailed
comments in code, add msleep to avoid repeated attempts. 
---
 fs/ocfs2/dlmglue.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/fs/ocfs2/dlmglue.c b/fs/ocfs2/dlmglue.c
index 48fd369c29a4..f8f561850470 100644
--- a/fs/ocfs2/dlmglue.c
+++ b/fs/ocfs2/dlmglue.c
@@ -16,6 +16,7 @@
 #include <linux/debugfs.h>
 #include <linux/seq_file.h>
 #include <linux/time.h>
+#include <linux/delay.h>
 #include <linux/quotaops.h>
 #include <linux/sched/signal.h>
 
@@ -3912,6 +3913,17 @@ static int ocfs2_unblock_lock(struct ocfs2_super *osb,
 	spin_unlock_irqrestore(&lockres->l_lock, flags);
 	ret = ocfs2_downconvert_lock(osb, lockres, new_level, set_lvb,
 				     gen);
+	/* The dlm lock convert is being cancelled in background,
+	 * ocfs2_cancel_convert() is asynchronous in fs/dlm,
+	 * requeue it, try again later.
+	 */
+	if (ret == -EBUSY) {
+		ctl->requeue = 1;
+		mlog(ML_BASTS, "lockres %s, ReQ: Downconvert busy\n",
+		     lockres->l_name);
+		ret = 0;
+		msleep(20);
+	}
 
 leave:
 	if (ret)
-- 
2.12.3


_______________________________________________
Ocfs2-devel mailing list
Ocfs2-devel@oss.oracle.com
https://oss.oracle.com/mailman/listinfo/ocfs2-devel

^ permalink raw reply related	[flat|nested] 4+ messages in thread
* Re: [PATCH v2] ocfs2: ocfs2_downconvert_lock failure results in deadlock
  2021-08-30  4:46 ` [Ocfs2-devel] " Gang He
@ 2021-08-30 11:05   ` Joseph Qi
  -1 siblings, 0 replies; 4+ messages in thread
From: Joseph Qi @ 2021-08-30 11:05 UTC (permalink / raw)
  To: Gang He, mark, jlbec; +Cc: linux-kernel, ocfs2-devel, akpm



On 8/30/21 12:46 PM, Gang He wrote:
> Usually, ocfs2_downconvert_lock() function always downconverts
> dlm lock to the expected level for satisfy dlm bast requests
> from the other nodes.
> But there is a rare situation. When dlm lock conversion is being
> canceled, ocfs2_downconvert_lock() function will return -EBUSY.
> You need to be aware that ocfs2_cancel_convert() function is
> asynchronous in fsdlm implementation.
> If we does not requeue this lockres entry, ocfs2 downconvert
> thread no longer handles this dlm lock bast request. Then, the
> other nodes will not get the dlm lock again, the current node's
> process will be blocked when acquire this dlm lock again.
> 
> Signed-off-by: Gang He <ghe@suse.com>

Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com>
> ---
> v2: keep error print in ocfs2_downconvert_lock function, add more detailed
> comments in code, add msleep to avoid repeated attempts. 
> ---
>  fs/ocfs2/dlmglue.c | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/fs/ocfs2/dlmglue.c b/fs/ocfs2/dlmglue.c
> index 48fd369c29a4..f8f561850470 100644
> --- a/fs/ocfs2/dlmglue.c
> +++ b/fs/ocfs2/dlmglue.c
> @@ -16,6 +16,7 @@
>  #include <linux/debugfs.h>
>  #include <linux/seq_file.h>
>  #include <linux/time.h>
> +#include <linux/delay.h>
>  #include <linux/quotaops.h>
>  #include <linux/sched/signal.h>
>  
> @@ -3912,6 +3913,17 @@ static int ocfs2_unblock_lock(struct ocfs2_super *osb,
>  	spin_unlock_irqrestore(&lockres->l_lock, flags);
>  	ret = ocfs2_downconvert_lock(osb, lockres, new_level, set_lvb,
>  				     gen);
> +	/* The dlm lock convert is being cancelled in background,
> +	 * ocfs2_cancel_convert() is asynchronous in fs/dlm,
> +	 * requeue it, try again later.
> +	 */
> +	if (ret == -EBUSY) {
> +		ctl->requeue = 1;
> +		mlog(ML_BASTS, "lockres %s, ReQ: Downconvert busy\n",
> +		     lockres->l_name);
> +		ret = 0;
> +		msleep(20);
> +	}
>  
>  leave:
>  	if (ret)
> 

^ permalink raw reply	[flat|nested] 4+ messages in thread
* Re: [Ocfs2-devel] [PATCH v2] ocfs2: ocfs2_downconvert_lock failure results in deadlock
@ 2021-08-30 11:05   ` Joseph Qi
  0 siblings, 0 replies; 4+ messages in thread
From: Joseph Qi @ 2021-08-30 11:05 UTC (permalink / raw)
  To: Gang He, mark, jlbec; +Cc: linux-kernel, ocfs2-devel



On 8/30/21 12:46 PM, Gang He wrote:
> Usually, ocfs2_downconvert_lock() function always downconverts
> dlm lock to the expected level for satisfy dlm bast requests
> from the other nodes.
> But there is a rare situation. When dlm lock conversion is being
> canceled, ocfs2_downconvert_lock() function will return -EBUSY.
> You need to be aware that ocfs2_cancel_convert() function is
> asynchronous in fsdlm implementation.
> If we does not requeue this lockres entry, ocfs2 downconvert
> thread no longer handles this dlm lock bast request. Then, the
> other nodes will not get the dlm lock again, the current node's
> process will be blocked when acquire this dlm lock again.
> 
> Signed-off-by: Gang He <ghe@suse.com>

Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com>
> ---
> v2: keep error print in ocfs2_downconvert_lock function, add more detailed
> comments in code, add msleep to avoid repeated attempts. 
> ---
>  fs/ocfs2/dlmglue.c | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/fs/ocfs2/dlmglue.c b/fs/ocfs2/dlmglue.c
> index 48fd369c29a4..f8f561850470 100644
> --- a/fs/ocfs2/dlmglue.c
> +++ b/fs/ocfs2/dlmglue.c
> @@ -16,6 +16,7 @@
>  #include <linux/debugfs.h>
>  #include <linux/seq_file.h>
>  #include <linux/time.h>
> +#include <linux/delay.h>
>  #include <linux/quotaops.h>
>  #include <linux/sched/signal.h>
>  
> @@ -3912,6 +3913,17 @@ static int ocfs2_unblock_lock(struct ocfs2_super *osb,
>  	spin_unlock_irqrestore(&lockres->l_lock, flags);
>  	ret = ocfs2_downconvert_lock(osb, lockres, new_level, set_lvb,
>  				     gen);
> +	/* The dlm lock convert is being cancelled in background,
> +	 * ocfs2_cancel_convert() is asynchronous in fs/dlm,
> +	 * requeue it, try again later.
> +	 */
> +	if (ret == -EBUSY) {
> +		ctl->requeue = 1;
> +		mlog(ML_BASTS, "lockres %s, ReQ: Downconvert busy\n",
> +		     lockres->l_name);
> +		ret = 0;
> +		msleep(20);
> +	}
>  
>  leave:
>  	if (ret)
> 

_______________________________________________
Ocfs2-devel mailing list
Ocfs2-devel@oss.oracle.com
https://oss.oracle.com/mailman/listinfo/ocfs2-devel

^ permalink raw reply	[flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-08-30 11:05 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-30  4:46 [PATCH v2] ocfs2: ocfs2_downconvert_lock failure results in deadlock Gang He
2021-08-30  4:46 ` [Ocfs2-devel] " Gang He
2021-08-30 11:05 ` Joseph Qi
2021-08-30 11:05   ` [Ocfs2-devel] " Joseph Qi

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.