archive mirror
 help / color / mirror / Atom feed
From: Vivek Goyal <>
To: Linux fsdevel mailing list <>,
	Jan Kara <>, Dan Williams <>,
Cc: virtio-fs-list <>,
	Sergio Lopez <>, Miklos Szeredi <>,,
	linux kernel mailing list <>
Subject: [PATCH][v2] dax: Fix missed wakeup during dax entry invalidation
Date: Mon, 19 Apr 2021 14:45:16 -0400	[thread overview]
Message-ID: <> (raw)

This is V2 of the patch. Posted V1 here.

Based on feedback from Dan and Jan, modified the patch to wake up 
all waiters when dax entry is invalidated. This solves the issues
of missed wakeups.

I am seeing missed wakeups which ultimately lead to a deadlock when I am
using virtiofs with DAX enabled and running "make -j". I had to mount
virtiofs as rootfs and also reduce to dax window size to 256M to reproduce
the problem consistently.

So here is the problem. put_unlocked_entry() wakes up waiters only
if entry is not null as well as !dax_is_conflict(entry). But if I
call multiple instances of invalidate_inode_pages2() in parallel,
then I can run into a situation where there are waiters on
this index but nobody will wait these.

        __dax_invalidate_entry() {
                entry = get_unlocked_entry(&xas, 0);
                dax_disassociate_entry(entry, mapping, trunc);
                xas_store(&xas, NULL);
                put_unlocked_entry(&xas, entry);

Say a fault in in progress and it has locked entry at offset say "0x1c".
Now say three instances of invalidate_inode_pages2() are in progress
(A, B, C) and they all try to invalidate entry at offset "0x1c". Given
dax entry is locked, all tree instances A, B, C will wait in wait queue.

When dax fault finishes, say A is woken up. It will store NULL entry
at index "0x1c" and wake up B. When B comes along it will find "entry=0"
at page offset 0x1c and it will call put_unlocked_entry(&xas, 0). And
this means put_unlocked_entry() will not wake up next waiter, given
the current code. And that means C continues to wait and is not woken

This patch fixes the issue by waking up all waiters when a dax entry
has been invalidated. This seems to fix the deadlock I am facing
and I can make forward progress.

Reported-by: Sergio Lopez <>
Signed-off-by: Vivek Goyal <>
 fs/dax.c |   12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

Index: redhat-linux/fs/dax.c
--- redhat-linux.orig/fs/dax.c	2021-04-16 14:16:44.332140543 -0400
+++ redhat-linux/fs/dax.c	2021-04-19 11:24:11.465213474 -0400
@@ -264,11 +264,11 @@ static void wait_entry_unlocked(struct x
 	finish_wait(wq, &ewait.wait);
-static void put_unlocked_entry(struct xa_state *xas, void *entry)
+static void put_unlocked_entry(struct xa_state *xas, void *entry, bool wake_all)
 	/* If we were the only waiter woken, wake the next one */
 	if (entry && !dax_is_conflict(entry))
-		dax_wake_entry(xas, entry, false);
+		dax_wake_entry(xas, entry, wake_all);
@@ -622,7 +622,7 @@ struct page *dax_layout_busy_page_range(
 			entry = get_unlocked_entry(&xas, 0);
 		if (entry)
 			page = dax_busy_page(entry);
-		put_unlocked_entry(&xas, entry);
+		put_unlocked_entry(&xas, entry, false);
 		if (page)
 		if (++scanned % XA_CHECK_SCHED)
@@ -664,7 +664,7 @@ static int __dax_invalidate_entry(struct
 	ret = 1;
-	put_unlocked_entry(&xas, entry);
+	put_unlocked_entry(&xas, entry, true);
 	return ret;
@@ -943,7 +943,7 @@ static int dax_writeback_one(struct xa_s
 	return ret;
-	put_unlocked_entry(xas, entry);
+	put_unlocked_entry(xas, entry, false);
 	return ret;
@@ -1684,7 +1684,7 @@ dax_insert_pfn_mkwrite(struct vm_fault *
 	/* Did we race with someone splitting entry or so? */
 	if (!entry || dax_is_conflict(entry) ||
 	    (order == 0 && !dax_is_pte_entry(entry))) {
-		put_unlocked_entry(&xas, entry);
+		put_unlocked_entry(&xas, entry, false);
 		trace_dax_insert_pfn_mkwrite_no_entry(mapping->host, vmf,

             reply	other threads:[~2021-04-19 18:45 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-19 18:45 Vivek Goyal [this message]
2021-04-19 19:48 ` [PATCH][v2] dax: Fix missed wakeup during dax entry invalidation Dan Williams
2021-04-19 20:39   ` Vivek Goyal
2021-04-19 20:42     ` Vivek Goyal

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \ \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).