Hi, This is V3 of patches. Posted V2 here. https://lore.kernel.org/linux-fsdevel/20210419184516.GC1472665@redhat.com/ Changes since v2: - Broke down patch in to a patch series (Dan) - Added an enum to communicate wake mode (Dan) Thanks Vivek Vivek Goyal (3): dax: Add an enum for specifying dax wakup mode dax: Add a wakeup mode parameter to put_unlocked_entry() dax: Wake up all waiters after invalidating dax entry fs/dax.c | 34 +++++++++++++++++++++++----------- 1 file changed, 23 insertions(+), 11 deletions(-) -- 2.25.4
Dan mentioned that he is not very fond of passing around a boolean true/false to specify if only next waiter should be woken up or all waiters should be woken up. He instead prefers that we introduce an enum and make it very explicity at the callsite itself. Easier to read code. This patch should not introduce any change of behavior. Suggested-by: Dan Williams <dan.j.williams@intel.com> Signed-off-by: Vivek Goyal <vgoyal@redhat.com> --- fs/dax.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/fs/dax.c b/fs/dax.c index b3d27fdc6775..00978d0838b1 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -144,6 +144,16 @@ struct wait_exceptional_entry_queue { struct exceptional_entry_key key; }; +/** + * enum dax_entry_wake_mode: waitqueue wakeup toggle + * @WAKE_NEXT: entry was not mutated + * @WAKE_ALL: entry was invalidated, or resized + */ +enum dax_entry_wake_mode { + WAKE_NEXT, + WAKE_ALL, +}; + static wait_queue_head_t *dax_entry_waitqueue(struct xa_state *xas, void *entry, struct exceptional_entry_key *key) { @@ -182,7 +192,8 @@ static int wake_exceptional_entry_func(wait_queue_entry_t *wait, * The important information it's conveying is whether the entry at * this index used to be a PMD entry. */ -static void dax_wake_entry(struct xa_state *xas, void *entry, bool wake_all) +static void dax_wake_entry(struct xa_state *xas, void *entry, + enum dax_entry_wake_mode mode) { struct exceptional_entry_key key; wait_queue_head_t *wq; @@ -196,7 +207,7 @@ static void dax_wake_entry(struct xa_state *xas, void *entry, bool wake_all) * must be in the waitqueue and the following check will see them. */ if (waitqueue_active(wq)) - __wake_up(wq, TASK_NORMAL, wake_all ? 0 : 1, &key); + __wake_up(wq, TASK_NORMAL, mode == WAKE_ALL ? 0 : 1, &key); } /* @@ -268,7 +279,7 @@ static void put_unlocked_entry(struct xa_state *xas, void *entry) { /* If we were the only waiter woken, wake the next one */ if (entry && !dax_is_conflict(entry)) - dax_wake_entry(xas, entry, false); + dax_wake_entry(xas, entry, WAKE_NEXT); } /* @@ -286,7 +297,7 @@ static void dax_unlock_entry(struct xa_state *xas, void *entry) old = xas_store(xas, entry); xas_unlock_irq(xas); BUG_ON(!dax_is_locked(old)); - dax_wake_entry(xas, entry, false); + dax_wake_entry(xas, entry, WAKE_NEXT); } /* @@ -524,7 +535,7 @@ static void *grab_mapping_entry(struct xa_state *xas, dax_disassociate_entry(entry, mapping, false); xas_store(xas, NULL); /* undo the PMD join */ - dax_wake_entry(xas, entry, true); + dax_wake_entry(xas, entry, WAKE_ALL); mapping->nrexceptional--; entry = NULL; xas_set(xas, index); @@ -937,7 +948,7 @@ static int dax_writeback_one(struct xa_state *xas, struct dax_device *dax_dev, xas_lock_irq(xas); xas_store(xas, entry); xas_clear_mark(xas, PAGECACHE_TAG_DIRTY); - dax_wake_entry(xas, entry, false); + dax_wake_entry(xas, entry, WAKE_NEXT); trace_dax_writeback_one(mapping->host, index, count); return ret; -- 2.25.4
As of now put_unlocked_entry() always wakes up next waiter. In next patches we want to wake up all waiters at one callsite. Hence, add a parameter to the function. This patch does not introduce any change of behavior. Suggested-by: Dan Williams <dan.j.williams@intel.com> Signed-off-by: Vivek Goyal <vgoyal@redhat.com> --- fs/dax.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/fs/dax.c b/fs/dax.c index 00978d0838b1..f19d76a6a493 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -275,11 +275,12 @@ static void wait_entry_unlocked(struct xa_state *xas, void *entry) finish_wait(wq, &ewait.wait); } -static void put_unlocked_entry(struct xa_state *xas, void *entry) +static void put_unlocked_entry(struct xa_state *xas, void *entry, + enum dax_entry_wake_mode mode) { /* If we were the only waiter woken, wake the next one */ if (entry && !dax_is_conflict(entry)) - dax_wake_entry(xas, entry, WAKE_NEXT); + dax_wake_entry(xas, entry, mode); } /* @@ -633,7 +634,7 @@ struct page *dax_layout_busy_page_range(struct address_space *mapping, entry = get_unlocked_entry(&xas, 0); if (entry) page = dax_busy_page(entry); - put_unlocked_entry(&xas, entry); + put_unlocked_entry(&xas, entry, WAKE_NEXT); if (page) break; if (++scanned % XA_CHECK_SCHED) @@ -675,7 +676,7 @@ static int __dax_invalidate_entry(struct address_space *mapping, mapping->nrexceptional--; ret = 1; out: - put_unlocked_entry(&xas, entry); + put_unlocked_entry(&xas, entry, WAKE_NEXT); xas_unlock_irq(&xas); return ret; } @@ -954,7 +955,7 @@ static int dax_writeback_one(struct xa_state *xas, struct dax_device *dax_dev, return ret; put_unlocked: - put_unlocked_entry(xas, entry); + put_unlocked_entry(xas, entry, WAKE_NEXT); return ret; } @@ -1695,7 +1696,7 @@ dax_insert_pfn_mkwrite(struct vm_fault *vmf, pfn_t pfn, unsigned int order) /* Did we race with someone splitting entry or so? */ if (!entry || dax_is_conflict(entry) || (order == 0 && !dax_is_pte_entry(entry))) { - put_unlocked_entry(&xas, entry); + put_unlocked_entry(&xas, entry, WAKE_NEXT); xas_unlock_irq(&xas); trace_dax_insert_pfn_mkwrite_no_entry(mapping->host, vmf, VM_FAULT_NOPAGE); -- 2.25.4
I am seeing missed wakeups which ultimately lead to a deadlock when I am using virtiofs with DAX enabled and running "make -j". I had to mount virtiofs as rootfs and also reduce to dax window size to 256M to reproduce the problem consistently. So here is the problem. put_unlocked_entry() wakes up waiters only if entry is not null as well as !dax_is_conflict(entry). But if I call multiple instances of invalidate_inode_pages2() in parallel, then I can run into a situation where there are waiters on this index but nobody will wait these. invalidate_inode_pages2() invalidate_inode_pages2_range() invalidate_exceptional_entry2() dax_invalidate_mapping_entry_sync() __dax_invalidate_entry() { xas_lock_irq(&xas); entry = get_unlocked_entry(&xas, 0); ... ... dax_disassociate_entry(entry, mapping, trunc); xas_store(&xas, NULL); ... ... put_unlocked_entry(&xas, entry); xas_unlock_irq(&xas); } Say a fault in in progress and it has locked entry at offset say "0x1c". Now say three instances of invalidate_inode_pages2() are in progress (A, B, C) and they all try to invalidate entry at offset "0x1c". Given dax entry is locked, all tree instances A, B, C will wait in wait queue. When dax fault finishes, say A is woken up. It will store NULL entry at index "0x1c" and wake up B. When B comes along it will find "entry=0" at page offset 0x1c and it will call put_unlocked_entry(&xas, 0). And this means put_unlocked_entry() will not wake up next waiter, given the current code. And that means C continues to wait and is not woken up. This patch fixes the issue by waking up all waiters when a dax entry has been invalidated. This seems to fix the deadlock I am facing and I can make forward progress. Reported-by: Sergio Lopez <slp@redhat.com> Fixes: ac401cc78242 ("dax: New fault locking") Suggested-by: Dan Williams <dan.j.williams@intel.com> Signed-off-by: Vivek Goyal <vgoyal@redhat.com> --- fs/dax.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/dax.c b/fs/dax.c index f19d76a6a493..cc497519be83 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -676,7 +676,7 @@ static int __dax_invalidate_entry(struct address_space *mapping, mapping->nrexceptional--; ret = 1; out: - put_unlocked_entry(&xas, entry, WAKE_NEXT); + put_unlocked_entry(&xas, entry, WAKE_ALL); xas_unlock_irq(&xas); return ret; } -- 2.25.4
On Mon, 19 Apr 2021 17:36:34 -0400 Vivek Goyal <vgoyal@redhat.com> wrote: > Dan mentioned that he is not very fond of passing around a boolean true/false > to specify if only next waiter should be woken up or all waiters should be > woken up. He instead prefers that we introduce an enum and make it very > explicity at the callsite itself. Easier to read code. > > This patch should not introduce any change of behavior. > > Suggested-by: Dan Williams <dan.j.williams@intel.com> > Signed-off-by: Vivek Goyal <vgoyal@redhat.com> > --- Reviewed-by: Greg Kurz <groug@kaod.org> > fs/dax.c | 23 +++++++++++++++++------ > 1 file changed, 17 insertions(+), 6 deletions(-) > > diff --git a/fs/dax.c b/fs/dax.c > index b3d27fdc6775..00978d0838b1 100644 > --- a/fs/dax.c > +++ b/fs/dax.c > @@ -144,6 +144,16 @@ struct wait_exceptional_entry_queue { > struct exceptional_entry_key key; > }; > > +/** > + * enum dax_entry_wake_mode: waitqueue wakeup toggle > + * @WAKE_NEXT: entry was not mutated > + * @WAKE_ALL: entry was invalidated, or resized > + */ > +enum dax_entry_wake_mode { > + WAKE_NEXT, > + WAKE_ALL, > +}; > + > static wait_queue_head_t *dax_entry_waitqueue(struct xa_state *xas, > void *entry, struct exceptional_entry_key *key) > { > @@ -182,7 +192,8 @@ static int wake_exceptional_entry_func(wait_queue_entry_t *wait, > * The important information it's conveying is whether the entry at > * this index used to be a PMD entry. > */ > -static void dax_wake_entry(struct xa_state *xas, void *entry, bool wake_all) > +static void dax_wake_entry(struct xa_state *xas, void *entry, > + enum dax_entry_wake_mode mode) > { > struct exceptional_entry_key key; > wait_queue_head_t *wq; > @@ -196,7 +207,7 @@ static void dax_wake_entry(struct xa_state *xas, void *entry, bool wake_all) > * must be in the waitqueue and the following check will see them. > */ > if (waitqueue_active(wq)) > - __wake_up(wq, TASK_NORMAL, wake_all ? 0 : 1, &key); > + __wake_up(wq, TASK_NORMAL, mode == WAKE_ALL ? 0 : 1, &key); > } > > /* > @@ -268,7 +279,7 @@ static void put_unlocked_entry(struct xa_state *xas, void *entry) > { > /* If we were the only waiter woken, wake the next one */ > if (entry && !dax_is_conflict(entry)) > - dax_wake_entry(xas, entry, false); > + dax_wake_entry(xas, entry, WAKE_NEXT); > } > > /* > @@ -286,7 +297,7 @@ static void dax_unlock_entry(struct xa_state *xas, void *entry) > old = xas_store(xas, entry); > xas_unlock_irq(xas); > BUG_ON(!dax_is_locked(old)); > - dax_wake_entry(xas, entry, false); > + dax_wake_entry(xas, entry, WAKE_NEXT); > } > > /* > @@ -524,7 +535,7 @@ static void *grab_mapping_entry(struct xa_state *xas, > > dax_disassociate_entry(entry, mapping, false); > xas_store(xas, NULL); /* undo the PMD join */ > - dax_wake_entry(xas, entry, true); > + dax_wake_entry(xas, entry, WAKE_ALL); > mapping->nrexceptional--; > entry = NULL; > xas_set(xas, index); > @@ -937,7 +948,7 @@ static int dax_writeback_one(struct xa_state *xas, struct dax_device *dax_dev, > xas_lock_irq(xas); > xas_store(xas, entry); > xas_clear_mark(xas, PAGECACHE_TAG_DIRTY); > - dax_wake_entry(xas, entry, false); > + dax_wake_entry(xas, entry, WAKE_NEXT); > > trace_dax_writeback_one(mapping->host, index, count); > return ret;
On Mon, 19 Apr 2021 17:36:35 -0400 Vivek Goyal <vgoyal@redhat.com> wrote: > As of now put_unlocked_entry() always wakes up next waiter. In next > patches we want to wake up all waiters at one callsite. Hence, add a > parameter to the function. > > This patch does not introduce any change of behavior. > > Suggested-by: Dan Williams <dan.j.williams@intel.com> > Signed-off-by: Vivek Goyal <vgoyal@redhat.com> > --- > fs/dax.c | 13 +++++++------ > 1 file changed, 7 insertions(+), 6 deletions(-) > > diff --git a/fs/dax.c b/fs/dax.c > index 00978d0838b1..f19d76a6a493 100644 > --- a/fs/dax.c > +++ b/fs/dax.c > @@ -275,11 +275,12 @@ static void wait_entry_unlocked(struct xa_state *xas, void *entry) > finish_wait(wq, &ewait.wait); > } > > -static void put_unlocked_entry(struct xa_state *xas, void *entry) > +static void put_unlocked_entry(struct xa_state *xas, void *entry, > + enum dax_entry_wake_mode mode) > { > /* If we were the only waiter woken, wake the next one */ With this change, the comment is no longer accurate since the function can now wake all waiters if passed mode == WAKE_ALL. Also, it paraphrases the code which is simple enough, so I'd simply drop it. This is minor though and it shouldn't prevent this fix to go forward. Reviewed-by: Greg Kurz <groug@kaod.org> > if (entry && !dax_is_conflict(entry)) > - dax_wake_entry(xas, entry, WAKE_NEXT); > + dax_wake_entry(xas, entry, mode); > } > > /* > @@ -633,7 +634,7 @@ struct page *dax_layout_busy_page_range(struct address_space *mapping, > entry = get_unlocked_entry(&xas, 0); > if (entry) > page = dax_busy_page(entry); > - put_unlocked_entry(&xas, entry); > + put_unlocked_entry(&xas, entry, WAKE_NEXT); > if (page) > break; > if (++scanned % XA_CHECK_SCHED) > @@ -675,7 +676,7 @@ static int __dax_invalidate_entry(struct address_space *mapping, > mapping->nrexceptional--; > ret = 1; > out: > - put_unlocked_entry(&xas, entry); > + put_unlocked_entry(&xas, entry, WAKE_NEXT); > xas_unlock_irq(&xas); > return ret; > } > @@ -954,7 +955,7 @@ static int dax_writeback_one(struct xa_state *xas, struct dax_device *dax_dev, > return ret; > > put_unlocked: > - put_unlocked_entry(xas, entry); > + put_unlocked_entry(xas, entry, WAKE_NEXT); > return ret; > } > > @@ -1695,7 +1696,7 @@ dax_insert_pfn_mkwrite(struct vm_fault *vmf, pfn_t pfn, unsigned int order) > /* Did we race with someone splitting entry or so? */ > if (!entry || dax_is_conflict(entry) || > (order == 0 && !dax_is_pte_entry(entry))) { > - put_unlocked_entry(&xas, entry); > + put_unlocked_entry(&xas, entry, WAKE_NEXT); > xas_unlock_irq(&xas); > trace_dax_insert_pfn_mkwrite_no_entry(mapping->host, vmf, > VM_FAULT_NOPAGE);
On Tue, Apr 20, 2021 at 09:34:20AM +0200, Greg Kurz wrote: > On Mon, 19 Apr 2021 17:36:35 -0400 > Vivek Goyal <vgoyal@redhat.com> wrote: > > > As of now put_unlocked_entry() always wakes up next waiter. In next > > patches we want to wake up all waiters at one callsite. Hence, add a > > parameter to the function. > > > > This patch does not introduce any change of behavior. > > > > Suggested-by: Dan Williams <dan.j.williams@intel.com> > > Signed-off-by: Vivek Goyal <vgoyal@redhat.com> > > --- > > fs/dax.c | 13 +++++++------ > > 1 file changed, 7 insertions(+), 6 deletions(-) > > > > diff --git a/fs/dax.c b/fs/dax.c > > index 00978d0838b1..f19d76a6a493 100644 > > --- a/fs/dax.c > > +++ b/fs/dax.c > > @@ -275,11 +275,12 @@ static void wait_entry_unlocked(struct xa_state *xas, void *entry) > > finish_wait(wq, &ewait.wait); > > } > > > > -static void put_unlocked_entry(struct xa_state *xas, void *entry) > > +static void put_unlocked_entry(struct xa_state *xas, void *entry, > > + enum dax_entry_wake_mode mode) > > { > > /* If we were the only waiter woken, wake the next one */ > > With this change, the comment is no longer accurate since the > function can now wake all waiters if passed mode == WAKE_ALL. > Also, it paraphrases the code which is simple enough, so I'd > simply drop it. > > This is minor though and it shouldn't prevent this fix to go > forward. > > Reviewed-by: Greg Kurz <groug@kaod.org> Ok, here is the updated patch which drops that comment line. Vivek Subject: dax: Add a wakeup mode parameter to put_unlocked_entry() As of now put_unlocked_entry() always wakes up next waiter. In next patches we want to wake up all waiters at one callsite. Hence, add a parameter to the function. This patch does not introduce any change of behavior. Suggested-by: Dan Williams <dan.j.williams@intel.com> Signed-off-by: Vivek Goyal <vgoyal@redhat.com> --- fs/dax.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) Index: redhat-linux/fs/dax.c =================================================================== --- redhat-linux.orig/fs/dax.c 2021-04-20 09:55:45.105069893 -0400 +++ redhat-linux/fs/dax.c 2021-04-20 09:56:27.685822730 -0400 @@ -275,11 +275,11 @@ static void wait_entry_unlocked(struct x finish_wait(wq, &ewait.wait); } -static void put_unlocked_entry(struct xa_state *xas, void *entry) +static void put_unlocked_entry(struct xa_state *xas, void *entry, + enum dax_entry_wake_mode mode) { - /* If we were the only waiter woken, wake the next one */ if (entry && !dax_is_conflict(entry)) - dax_wake_entry(xas, entry, WAKE_NEXT); + dax_wake_entry(xas, entry, mode); } /* @@ -633,7 +633,7 @@ struct page *dax_layout_busy_page_range( entry = get_unlocked_entry(&xas, 0); if (entry) page = dax_busy_page(entry); - put_unlocked_entry(&xas, entry); + put_unlocked_entry(&xas, entry, WAKE_NEXT); if (page) break; if (++scanned % XA_CHECK_SCHED) @@ -675,7 +675,7 @@ static int __dax_invalidate_entry(struct mapping->nrexceptional--; ret = 1; out: - put_unlocked_entry(&xas, entry); + put_unlocked_entry(&xas, entry, WAKE_NEXT); xas_unlock_irq(&xas); return ret; } @@ -954,7 +954,7 @@ static int dax_writeback_one(struct xa_s return ret; put_unlocked: - put_unlocked_entry(xas, entry); + put_unlocked_entry(xas, entry, WAKE_NEXT); return ret; } @@ -1695,7 +1695,7 @@ dax_insert_pfn_mkwrite(struct vm_fault * /* Did we race with someone splitting entry or so? */ if (!entry || dax_is_conflict(entry) || (order == 0 && !dax_is_pte_entry(entry))) { - put_unlocked_entry(&xas, entry); + put_unlocked_entry(&xas, entry, WAKE_NEXT); xas_unlock_irq(&xas); trace_dax_insert_pfn_mkwrite_no_entry(mapping->host, vmf, VM_FAULT_NOPAGE);
On Mon 19-04-21 17:36:34, Vivek Goyal wrote: > Dan mentioned that he is not very fond of passing around a boolean true/false > to specify if only next waiter should be woken up or all waiters should be > woken up. He instead prefers that we introduce an enum and make it very > explicity at the callsite itself. Easier to read code. > > This patch should not introduce any change of behavior. > > Suggested-by: Dan Williams <dan.j.williams@intel.com> > Signed-off-by: Vivek Goyal <vgoyal@redhat.com> > --- > fs/dax.c | 23 +++++++++++++++++------ > 1 file changed, 17 insertions(+), 6 deletions(-) > > diff --git a/fs/dax.c b/fs/dax.c > index b3d27fdc6775..00978d0838b1 100644 > --- a/fs/dax.c > +++ b/fs/dax.c > @@ -144,6 +144,16 @@ struct wait_exceptional_entry_queue { > struct exceptional_entry_key key; > }; > > +/** > + * enum dax_entry_wake_mode: waitqueue wakeup toggle > + * @WAKE_NEXT: entry was not mutated > + * @WAKE_ALL: entry was invalidated, or resized Let's document the constants in terms of what they do, not when they are expected to be called. So something like: @WAKE_NEXT: wake only the first waiter in the waitqueue @WAKE_ALL: wake all waiters in the waitqueue Otherwise the patch looks good so feel free to add: Reviewed-by: Jan Kara <jack@suse.cz> Honza > + */ > +enum dax_entry_wake_mode { > + WAKE_NEXT, > + WAKE_ALL, > +}; > + > static wait_queue_head_t *dax_entry_waitqueue(struct xa_state *xas, > void *entry, struct exceptional_entry_key *key) > { > @@ -182,7 +192,8 @@ static int wake_exceptional_entry_func(wait_queue_entry_t *wait, > * The important information it's conveying is whether the entry at > * this index used to be a PMD entry. > */ > -static void dax_wake_entry(struct xa_state *xas, void *entry, bool wake_all) > +static void dax_wake_entry(struct xa_state *xas, void *entry, > + enum dax_entry_wake_mode mode) > { > struct exceptional_entry_key key; > wait_queue_head_t *wq; > @@ -196,7 +207,7 @@ static void dax_wake_entry(struct xa_state *xas, void *entry, bool wake_all) > * must be in the waitqueue and the following check will see them. > */ > if (waitqueue_active(wq)) > - __wake_up(wq, TASK_NORMAL, wake_all ? 0 : 1, &key); > + __wake_up(wq, TASK_NORMAL, mode == WAKE_ALL ? 0 : 1, &key); > } > > /* > @@ -268,7 +279,7 @@ static void put_unlocked_entry(struct xa_state *xas, void *entry) > { > /* If we were the only waiter woken, wake the next one */ > if (entry && !dax_is_conflict(entry)) > - dax_wake_entry(xas, entry, false); > + dax_wake_entry(xas, entry, WAKE_NEXT); > } > > /* > @@ -286,7 +297,7 @@ static void dax_unlock_entry(struct xa_state *xas, void *entry) > old = xas_store(xas, entry); > xas_unlock_irq(xas); > BUG_ON(!dax_is_locked(old)); > - dax_wake_entry(xas, entry, false); > + dax_wake_entry(xas, entry, WAKE_NEXT); > } > > /* > @@ -524,7 +535,7 @@ static void *grab_mapping_entry(struct xa_state *xas, > > dax_disassociate_entry(entry, mapping, false); > xas_store(xas, NULL); /* undo the PMD join */ > - dax_wake_entry(xas, entry, true); > + dax_wake_entry(xas, entry, WAKE_ALL); > mapping->nrexceptional--; > entry = NULL; > xas_set(xas, index); > @@ -937,7 +948,7 @@ static int dax_writeback_one(struct xa_state *xas, struct dax_device *dax_dev, > xas_lock_irq(xas); > xas_store(xas, entry); > xas_clear_mark(xas, PAGECACHE_TAG_DIRTY); > - dax_wake_entry(xas, entry, false); > + dax_wake_entry(xas, entry, WAKE_NEXT); > > trace_dax_writeback_one(mapping->host, index, count); > return ret; > -- > 2.25.4 > -- Jan Kara <jack@suse.com> SUSE Labs, CR
On Mon 19-04-21 17:36:35, Vivek Goyal wrote: > As of now put_unlocked_entry() always wakes up next waiter. In next > patches we want to wake up all waiters at one callsite. Hence, add a > parameter to the function. > > This patch does not introduce any change of behavior. > > Suggested-by: Dan Williams <dan.j.williams@intel.com> > Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Looks good. You can add: Reviewed-by: Jan Kara <jack@suse.cz> Honza > --- > fs/dax.c | 13 +++++++------ > 1 file changed, 7 insertions(+), 6 deletions(-) > > diff --git a/fs/dax.c b/fs/dax.c > index 00978d0838b1..f19d76a6a493 100644 > --- a/fs/dax.c > +++ b/fs/dax.c > @@ -275,11 +275,12 @@ static void wait_entry_unlocked(struct xa_state *xas, void *entry) > finish_wait(wq, &ewait.wait); > } > > -static void put_unlocked_entry(struct xa_state *xas, void *entry) > +static void put_unlocked_entry(struct xa_state *xas, void *entry, > + enum dax_entry_wake_mode mode) > { > /* If we were the only waiter woken, wake the next one */ > if (entry && !dax_is_conflict(entry)) > - dax_wake_entry(xas, entry, WAKE_NEXT); > + dax_wake_entry(xas, entry, mode); > } > > /* > @@ -633,7 +634,7 @@ struct page *dax_layout_busy_page_range(struct address_space *mapping, > entry = get_unlocked_entry(&xas, 0); > if (entry) > page = dax_busy_page(entry); > - put_unlocked_entry(&xas, entry); > + put_unlocked_entry(&xas, entry, WAKE_NEXT); > if (page) > break; > if (++scanned % XA_CHECK_SCHED) > @@ -675,7 +676,7 @@ static int __dax_invalidate_entry(struct address_space *mapping, > mapping->nrexceptional--; > ret = 1; > out: > - put_unlocked_entry(&xas, entry); > + put_unlocked_entry(&xas, entry, WAKE_NEXT); > xas_unlock_irq(&xas); > return ret; > } > @@ -954,7 +955,7 @@ static int dax_writeback_one(struct xa_state *xas, struct dax_device *dax_dev, > return ret; > > put_unlocked: > - put_unlocked_entry(xas, entry); > + put_unlocked_entry(xas, entry, WAKE_NEXT); > return ret; > } > > @@ -1695,7 +1696,7 @@ dax_insert_pfn_mkwrite(struct vm_fault *vmf, pfn_t pfn, unsigned int order) > /* Did we race with someone splitting entry or so? */ > if (!entry || dax_is_conflict(entry) || > (order == 0 && !dax_is_pte_entry(entry))) { > - put_unlocked_entry(&xas, entry); > + put_unlocked_entry(&xas, entry, WAKE_NEXT); > xas_unlock_irq(&xas); > trace_dax_insert_pfn_mkwrite_no_entry(mapping->host, vmf, > VM_FAULT_NOPAGE); > -- > 2.25.4 > -- Jan Kara <jack@suse.com> SUSE Labs, CR
On Mon 19-04-21 17:36:36, Vivek Goyal wrote: > I am seeing missed wakeups which ultimately lead to a deadlock when I am > using virtiofs with DAX enabled and running "make -j". I had to mount > virtiofs as rootfs and also reduce to dax window size to 256M to reproduce > the problem consistently. > > So here is the problem. put_unlocked_entry() wakes up waiters only > if entry is not null as well as !dax_is_conflict(entry). But if I > call multiple instances of invalidate_inode_pages2() in parallel, > then I can run into a situation where there are waiters on > this index but nobody will wait these. > > invalidate_inode_pages2() > invalidate_inode_pages2_range() > invalidate_exceptional_entry2() > dax_invalidate_mapping_entry_sync() > __dax_invalidate_entry() { > xas_lock_irq(&xas); > entry = get_unlocked_entry(&xas, 0); > ... > ... > dax_disassociate_entry(entry, mapping, trunc); > xas_store(&xas, NULL); > ... > ... > put_unlocked_entry(&xas, entry); > xas_unlock_irq(&xas); > } > > Say a fault in in progress and it has locked entry at offset say "0x1c". > Now say three instances of invalidate_inode_pages2() are in progress > (A, B, C) and they all try to invalidate entry at offset "0x1c". Given > dax entry is locked, all tree instances A, B, C will wait in wait queue. > > When dax fault finishes, say A is woken up. It will store NULL entry > at index "0x1c" and wake up B. When B comes along it will find "entry=0" > at page offset 0x1c and it will call put_unlocked_entry(&xas, 0). And > this means put_unlocked_entry() will not wake up next waiter, given > the current code. And that means C continues to wait and is not woken > up. > > This patch fixes the issue by waking up all waiters when a dax entry > has been invalidated. This seems to fix the deadlock I am facing > and I can make forward progress. > > Reported-by: Sergio Lopez <slp@redhat.com> > Fixes: ac401cc78242 ("dax: New fault locking") > Suggested-by: Dan Williams <dan.j.williams@intel.com> > Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Looks good to me. Thanks for fixing this! Feel free to add: Reviewed-by: Jan Kara <jack@suse.cz> Honza > --- > fs/dax.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/fs/dax.c b/fs/dax.c > index f19d76a6a493..cc497519be83 100644 > --- a/fs/dax.c > +++ b/fs/dax.c > @@ -676,7 +676,7 @@ static int __dax_invalidate_entry(struct address_space *mapping, > mapping->nrexceptional--; > ret = 1; > out: > - put_unlocked_entry(&xas, entry, WAKE_NEXT); > + put_unlocked_entry(&xas, entry, WAKE_ALL); > xas_unlock_irq(&xas); > return ret; > } > -- > 2.25.4 > -- Jan Kara <jack@suse.com> SUSE Labs, CR
On Wed, Apr 21, 2021 at 11:24:40AM +0200, Jan Kara wrote: > On Mon 19-04-21 17:36:34, Vivek Goyal wrote: > > Dan mentioned that he is not very fond of passing around a boolean true/false > > to specify if only next waiter should be woken up or all waiters should be > > woken up. He instead prefers that we introduce an enum and make it very > > explicity at the callsite itself. Easier to read code. > > > > This patch should not introduce any change of behavior. > > > > Suggested-by: Dan Williams <dan.j.williams@intel.com> > > Signed-off-by: Vivek Goyal <vgoyal@redhat.com> > > --- > > fs/dax.c | 23 +++++++++++++++++------ > > 1 file changed, 17 insertions(+), 6 deletions(-) > > > > diff --git a/fs/dax.c b/fs/dax.c > > index b3d27fdc6775..00978d0838b1 100644 > > --- a/fs/dax.c > > +++ b/fs/dax.c > > @@ -144,6 +144,16 @@ struct wait_exceptional_entry_queue { > > struct exceptional_entry_key key; > > }; > > > > +/** > > + * enum dax_entry_wake_mode: waitqueue wakeup toggle > > + * @WAKE_NEXT: entry was not mutated > > + * @WAKE_ALL: entry was invalidated, or resized > > Let's document the constants in terms of what they do, not when they are > expected to be called. So something like: > > @WAKE_NEXT: wake only the first waiter in the waitqueue > @WAKE_ALL: wake all waiters in the waitqueue > > Otherwise the patch looks good so feel free to add: > > Reviewed-by: Jan Kara <jack@suse.cz> > Hi Jan, Here is the updated patch based on your feedback. Thanks Vivek Subject: dax: Add an enum for specifying dax wakup mode Dan mentioned that he is not very fond of passing around a boolean true/false to specify if only next waiter should be woken up or all waiters should be woken up. He instead prefers that we introduce an enum and make it very explicity at the callsite itself. Easier to read code. This patch should not introduce any change of behavior. Suggested-by: Dan Williams <dan.j.williams@intel.com> Signed-off-by: Vivek Goyal <vgoyal@redhat.com> --- fs/dax.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) Index: redhat-linux/fs/dax.c =================================================================== --- redhat-linux.orig/fs/dax.c 2021-04-21 11:51:04.716289502 -0400 +++ redhat-linux/fs/dax.c 2021-04-21 11:52:10.298010850 -0400 @@ -144,6 +144,16 @@ struct wait_exceptional_entry_queue { struct exceptional_entry_key key; }; +/** + * enum dax_entry_wake_mode: waitqueue wakeup toggle + * @WAKE_NEXT: wake only the first waiter in the waitqueue + * @WAKE_ALL: wake all waiters in the waitqueue + */ +enum dax_entry_wake_mode { + WAKE_NEXT, + WAKE_ALL, +}; + static wait_queue_head_t *dax_entry_waitqueue(struct xa_state *xas, void *entry, struct exceptional_entry_key *key) { @@ -182,7 +192,8 @@ static int wake_exceptional_entry_func(w * The important information it's conveying is whether the entry at * this index used to be a PMD entry. */ -static void dax_wake_entry(struct xa_state *xas, void *entry, bool wake_all) +static void dax_wake_entry(struct xa_state *xas, void *entry, + enum dax_entry_wake_mode mode) { struct exceptional_entry_key key; wait_queue_head_t *wq; @@ -196,7 +207,7 @@ static void dax_wake_entry(struct xa_sta * must be in the waitqueue and the following check will see them. */ if (waitqueue_active(wq)) - __wake_up(wq, TASK_NORMAL, wake_all ? 0 : 1, &key); + __wake_up(wq, TASK_NORMAL, mode == WAKE_ALL ? 0 : 1, &key); } /* @@ -268,7 +279,7 @@ static void put_unlocked_entry(struct xa { /* If we were the only waiter woken, wake the next one */ if (entry && !dax_is_conflict(entry)) - dax_wake_entry(xas, entry, false); + dax_wake_entry(xas, entry, WAKE_NEXT); } /* @@ -286,7 +297,7 @@ static void dax_unlock_entry(struct xa_s old = xas_store(xas, entry); xas_unlock_irq(xas); BUG_ON(!dax_is_locked(old)); - dax_wake_entry(xas, entry, false); + dax_wake_entry(xas, entry, WAKE_NEXT); } /* @@ -524,7 +535,7 @@ retry: dax_disassociate_entry(entry, mapping, false); xas_store(xas, NULL); /* undo the PMD join */ - dax_wake_entry(xas, entry, true); + dax_wake_entry(xas, entry, WAKE_ALL); mapping->nrexceptional--; entry = NULL; xas_set(xas, index); @@ -937,7 +948,7 @@ static int dax_writeback_one(struct xa_s xas_lock_irq(xas); xas_store(xas, entry); xas_clear_mark(xas, PAGECACHE_TAG_DIRTY); - dax_wake_entry(xas, entry, false); + dax_wake_entry(xas, entry, WAKE_NEXT); trace_dax_writeback_one(mapping->host, index, count); return ret;
On Wed, Apr 21, 2021 at 11:56:31AM -0400, Vivek Goyal wrote: > +/** > + * enum dax_entry_wake_mode: waitqueue wakeup toggle s/toggle/behaviour/ ? > + * @WAKE_NEXT: wake only the first waiter in the waitqueue > + * @WAKE_ALL: wake all waiters in the waitqueue > + */ > +enum dax_entry_wake_mode { > + WAKE_NEXT, > + WAKE_ALL, > +}; > + > static wait_queue_head_t *dax_entry_waitqueue(struct xa_state *xas, > void *entry, struct exceptional_entry_key *key) > { > @@ -182,7 +192,8 @@ static int wake_exceptional_entry_func(w > * The important information it's conveying is whether the entry at > * this index used to be a PMD entry. > */ > -static void dax_wake_entry(struct xa_state *xas, void *entry, bool wake_all) > +static void dax_wake_entry(struct xa_state *xas, void *entry, > + enum dax_entry_wake_mode mode) It's an awfully verbose name. 'dax_wake_mode'?
On Wed, Apr 21, 2021 at 05:16:24PM +0100, Matthew Wilcox wrote: > On Wed, Apr 21, 2021 at 11:56:31AM -0400, Vivek Goyal wrote: > > +/** > > + * enum dax_entry_wake_mode: waitqueue wakeup toggle > > s/toggle/behaviour/ ? Will do. > > > + * @WAKE_NEXT: wake only the first waiter in the waitqueue > > + * @WAKE_ALL: wake all waiters in the waitqueue > > + */ > > +enum dax_entry_wake_mode { > > + WAKE_NEXT, > > + WAKE_ALL, > > +}; > > + > > static wait_queue_head_t *dax_entry_waitqueue(struct xa_state *xas, > > void *entry, struct exceptional_entry_key *key) > > { > > @@ -182,7 +192,8 @@ static int wake_exceptional_entry_func(w > > * The important information it's conveying is whether the entry at > > * this index used to be a PMD entry. > > */ > > -static void dax_wake_entry(struct xa_state *xas, void *entry, bool wake_all) > > +static void dax_wake_entry(struct xa_state *xas, void *entry, > > + enum dax_entry_wake_mode mode) > > It's an awfully verbose name. 'dax_wake_mode'? Sure. Will change. Vivek >
On Tue, Apr 20, 2021 at 09:34:20AM +0200, Greg Kurz wrote: > On Mon, 19 Apr 2021 17:36:35 -0400 > Vivek Goyal <vgoyal@redhat.com> wrote: > > > As of now put_unlocked_entry() always wakes up next waiter. In next > > patches we want to wake up all waiters at one callsite. Hence, add a > > parameter to the function. > > > > This patch does not introduce any change of behavior. > > > > Suggested-by: Dan Williams <dan.j.williams@intel.com> > > Signed-off-by: Vivek Goyal <vgoyal@redhat.com> > > --- > > fs/dax.c | 13 +++++++------ > > 1 file changed, 7 insertions(+), 6 deletions(-) > > > > diff --git a/fs/dax.c b/fs/dax.c > > index 00978d0838b1..f19d76a6a493 100644 > > --- a/fs/dax.c > > +++ b/fs/dax.c > > @@ -275,11 +275,12 @@ static void wait_entry_unlocked(struct xa_state *xas, void *entry) > > finish_wait(wq, &ewait.wait); > > } > > > > -static void put_unlocked_entry(struct xa_state *xas, void *entry) > > +static void put_unlocked_entry(struct xa_state *xas, void *entry, > > + enum dax_entry_wake_mode mode) > > { > > /* If we were the only waiter woken, wake the next one */ > > With this change, the comment is no longer accurate since the > function can now wake all waiters if passed mode == WAKE_ALL. > Also, it paraphrases the code which is simple enough, so I'd > simply drop it. Ok, I will get rid of this comment. Agreed that code is simple enough. And frankly speaking I don't even understand "If we were the only waiter woken" part. How do we know that only this caller was woken. Vivek > > This is minor though and it shouldn't prevent this fix to go > forward. > > Reviewed-by: Greg Kurz <groug@kaod.org> > > > if (entry && !dax_is_conflict(entry)) > > - dax_wake_entry(xas, entry, WAKE_NEXT); > > + dax_wake_entry(xas, entry, mode); > > } > > > > /* > > @@ -633,7 +634,7 @@ struct page *dax_layout_busy_page_range(struct address_space *mapping, > > entry = get_unlocked_entry(&xas, 0); > > if (entry) > > page = dax_busy_page(entry); > > - put_unlocked_entry(&xas, entry); > > + put_unlocked_entry(&xas, entry, WAKE_NEXT); > > if (page) > > break; > > if (++scanned % XA_CHECK_SCHED) > > @@ -675,7 +676,7 @@ static int __dax_invalidate_entry(struct address_space *mapping, > > mapping->nrexceptional--; > > ret = 1; > > out: > > - put_unlocked_entry(&xas, entry); > > + put_unlocked_entry(&xas, entry, WAKE_NEXT); > > xas_unlock_irq(&xas); > > return ret; > > } > > @@ -954,7 +955,7 @@ static int dax_writeback_one(struct xa_state *xas, struct dax_device *dax_dev, > > return ret; > > > > put_unlocked: > > - put_unlocked_entry(xas, entry); > > + put_unlocked_entry(xas, entry, WAKE_NEXT); > > return ret; > > } > > > > @@ -1695,7 +1696,7 @@ dax_insert_pfn_mkwrite(struct vm_fault *vmf, pfn_t pfn, unsigned int order) > > /* Did we race with someone splitting entry or so? */ > > if (!entry || dax_is_conflict(entry) || > > (order == 0 && !dax_is_pte_entry(entry))) { > > - put_unlocked_entry(&xas, entry); > > + put_unlocked_entry(&xas, entry, WAKE_NEXT); > > xas_unlock_irq(&xas); > > trace_dax_insert_pfn_mkwrite_no_entry(mapping->host, vmf, > > VM_FAULT_NOPAGE); >
On Tue, Apr 20, 2021 at 7:01 AM Vivek Goyal <vgoyal@redhat.com> wrote: > > On Tue, Apr 20, 2021 at 09:34:20AM +0200, Greg Kurz wrote: > > On Mon, 19 Apr 2021 17:36:35 -0400 > > Vivek Goyal <vgoyal@redhat.com> wrote: > > > > > As of now put_unlocked_entry() always wakes up next waiter. In next > > > patches we want to wake up all waiters at one callsite. Hence, add a > > > parameter to the function. > > > > > > This patch does not introduce any change of behavior. > > > > > > Suggested-by: Dan Williams <dan.j.williams@intel.com> > > > Signed-off-by: Vivek Goyal <vgoyal@redhat.com> > > > --- > > > fs/dax.c | 13 +++++++------ > > > 1 file changed, 7 insertions(+), 6 deletions(-) > > > > > > diff --git a/fs/dax.c b/fs/dax.c > > > index 00978d0838b1..f19d76a6a493 100644 > > > --- a/fs/dax.c > > > +++ b/fs/dax.c > > > @@ -275,11 +275,12 @@ static void wait_entry_unlocked(struct xa_state *xas, void *entry) > > > finish_wait(wq, &ewait.wait); > > > } > > > > > > -static void put_unlocked_entry(struct xa_state *xas, void *entry) > > > +static void put_unlocked_entry(struct xa_state *xas, void *entry, > > > + enum dax_entry_wake_mode mode) > > > { > > > /* If we were the only waiter woken, wake the next one */ > > > > With this change, the comment is no longer accurate since the > > function can now wake all waiters if passed mode == WAKE_ALL. > > Also, it paraphrases the code which is simple enough, so I'd > > simply drop it. > > > > This is minor though and it shouldn't prevent this fix to go > > forward. > > > > Reviewed-by: Greg Kurz <groug@kaod.org> > > Ok, here is the updated patch which drops that comment line. > > Vivek Hi Vivek, Can you get in the habit of not replying inline with new patches like this? Collect the review feedback, take a pause, and resend the full series so tooling like b4 and patchwork can track when a new posting supersedes a previous one. As is, this inline style inflicts manual effort on the maintainer. > > Subject: dax: Add a wakeup mode parameter to put_unlocked_entry() > > As of now put_unlocked_entry() always wakes up next waiter. In next > patches we want to wake up all waiters at one callsite. Hence, add a > parameter to the function. > > This patch does not introduce any change of behavior. > > Suggested-by: Dan Williams <dan.j.williams@intel.com> > Signed-off-by: Vivek Goyal <vgoyal@redhat.com> > --- > fs/dax.c | 14 +++++++------- > 1 file changed, 7 insertions(+), 7 deletions(-) > > Index: redhat-linux/fs/dax.c > =================================================================== > --- redhat-linux.orig/fs/dax.c 2021-04-20 09:55:45.105069893 -0400 > +++ redhat-linux/fs/dax.c 2021-04-20 09:56:27.685822730 -0400 > @@ -275,11 +275,11 @@ static void wait_entry_unlocked(struct x > finish_wait(wq, &ewait.wait); > } > > -static void put_unlocked_entry(struct xa_state *xas, void *entry) > +static void put_unlocked_entry(struct xa_state *xas, void *entry, > + enum dax_entry_wake_mode mode) > { > - /* If we were the only waiter woken, wake the next one */ > if (entry && !dax_is_conflict(entry)) > - dax_wake_entry(xas, entry, WAKE_NEXT); > + dax_wake_entry(xas, entry, mode); > } > > /* > @@ -633,7 +633,7 @@ struct page *dax_layout_busy_page_range( > entry = get_unlocked_entry(&xas, 0); > if (entry) > page = dax_busy_page(entry); > - put_unlocked_entry(&xas, entry); > + put_unlocked_entry(&xas, entry, WAKE_NEXT); > if (page) > break; > if (++scanned % XA_CHECK_SCHED) > @@ -675,7 +675,7 @@ static int __dax_invalidate_entry(struct > mapping->nrexceptional--; > ret = 1; > out: > - put_unlocked_entry(&xas, entry); > + put_unlocked_entry(&xas, entry, WAKE_NEXT); > xas_unlock_irq(&xas); > return ret; > } > @@ -954,7 +954,7 @@ static int dax_writeback_one(struct xa_s > return ret; > > put_unlocked: > - put_unlocked_entry(xas, entry); > + put_unlocked_entry(xas, entry, WAKE_NEXT); > return ret; > } > > @@ -1695,7 +1695,7 @@ dax_insert_pfn_mkwrite(struct vm_fault * > /* Did we race with someone splitting entry or so? */ > if (!entry || dax_is_conflict(entry) || > (order == 0 && !dax_is_pte_entry(entry))) { > - put_unlocked_entry(&xas, entry); > + put_unlocked_entry(&xas, entry, WAKE_NEXT); > xas_unlock_irq(&xas); > trace_dax_insert_pfn_mkwrite_no_entry(mapping->host, vmf, > VM_FAULT_NOPAGE); >
On Wed, Apr 21, 2021 at 12:09:54PM -0700, Dan Williams wrote: > On Tue, Apr 20, 2021 at 7:01 AM Vivek Goyal <vgoyal@redhat.com> wrote: > > > > On Tue, Apr 20, 2021 at 09:34:20AM +0200, Greg Kurz wrote: > > > On Mon, 19 Apr 2021 17:36:35 -0400 > > > Vivek Goyal <vgoyal@redhat.com> wrote: > > > > > > > As of now put_unlocked_entry() always wakes up next waiter. In next > > > > patches we want to wake up all waiters at one callsite. Hence, add a > > > > parameter to the function. > > > > > > > > This patch does not introduce any change of behavior. > > > > > > > > Suggested-by: Dan Williams <dan.j.williams@intel.com> > > > > Signed-off-by: Vivek Goyal <vgoyal@redhat.com> > > > > --- > > > > fs/dax.c | 13 +++++++------ > > > > 1 file changed, 7 insertions(+), 6 deletions(-) > > > > > > > > diff --git a/fs/dax.c b/fs/dax.c > > > > index 00978d0838b1..f19d76a6a493 100644 > > > > --- a/fs/dax.c > > > > +++ b/fs/dax.c > > > > @@ -275,11 +275,12 @@ static void wait_entry_unlocked(struct xa_state *xas, void *entry) > > > > finish_wait(wq, &ewait.wait); > > > > } > > > > > > > > -static void put_unlocked_entry(struct xa_state *xas, void *entry) > > > > +static void put_unlocked_entry(struct xa_state *xas, void *entry, > > > > + enum dax_entry_wake_mode mode) > > > > { > > > > /* If we were the only waiter woken, wake the next one */ > > > > > > With this change, the comment is no longer accurate since the > > > function can now wake all waiters if passed mode == WAKE_ALL. > > > Also, it paraphrases the code which is simple enough, so I'd > > > simply drop it. > > > > > > This is minor though and it shouldn't prevent this fix to go > > > forward. > > > > > > Reviewed-by: Greg Kurz <groug@kaod.org> > > > > Ok, here is the updated patch which drops that comment line. > > > > Vivek > > Hi Vivek, > > Can you get in the habit of not replying inline with new patches like > this? Collect the review feedback, take a pause, and resend the full > series so tooling like b4 and patchwork can track when a new posting > supersedes a previous one. As is, this inline style inflicts manual > effort on the maintainer. Hi Dan, Sure. I will avoid doing this updated inline patch style. I will post new version of patch series. Thanks Vivek > > > > > Subject: dax: Add a wakeup mode parameter to put_unlocked_entry() > > > > As of now put_unlocked_entry() always wakes up next waiter. In next > > patches we want to wake up all waiters at one callsite. Hence, add a > > parameter to the function. > > > > This patch does not introduce any change of behavior. > > > > Suggested-by: Dan Williams <dan.j.williams@intel.com> > > Signed-off-by: Vivek Goyal <vgoyal@redhat.com> > > --- > > fs/dax.c | 14 +++++++------- > > 1 file changed, 7 insertions(+), 7 deletions(-) > > > > Index: redhat-linux/fs/dax.c > > =================================================================== > > --- redhat-linux.orig/fs/dax.c 2021-04-20 09:55:45.105069893 -0400 > > +++ redhat-linux/fs/dax.c 2021-04-20 09:56:27.685822730 -0400 > > @@ -275,11 +275,11 @@ static void wait_entry_unlocked(struct x > > finish_wait(wq, &ewait.wait); > > } > > > > -static void put_unlocked_entry(struct xa_state *xas, void *entry) > > +static void put_unlocked_entry(struct xa_state *xas, void *entry, > > + enum dax_entry_wake_mode mode) > > { > > - /* If we were the only waiter woken, wake the next one */ > > if (entry && !dax_is_conflict(entry)) > > - dax_wake_entry(xas, entry, WAKE_NEXT); > > + dax_wake_entry(xas, entry, mode); > > } > > > > /* > > @@ -633,7 +633,7 @@ struct page *dax_layout_busy_page_range( > > entry = get_unlocked_entry(&xas, 0); > > if (entry) > > page = dax_busy_page(entry); > > - put_unlocked_entry(&xas, entry); > > + put_unlocked_entry(&xas, entry, WAKE_NEXT); > > if (page) > > break; > > if (++scanned % XA_CHECK_SCHED) > > @@ -675,7 +675,7 @@ static int __dax_invalidate_entry(struct > > mapping->nrexceptional--; > > ret = 1; > > out: > > - put_unlocked_entry(&xas, entry); > > + put_unlocked_entry(&xas, entry, WAKE_NEXT); > > xas_unlock_irq(&xas); > > return ret; > > } > > @@ -954,7 +954,7 @@ static int dax_writeback_one(struct xa_s > > return ret; > > > > put_unlocked: > > - put_unlocked_entry(xas, entry); > > + put_unlocked_entry(xas, entry, WAKE_NEXT); > > return ret; > > } > > > > @@ -1695,7 +1695,7 @@ dax_insert_pfn_mkwrite(struct vm_fault * > > /* Did we race with someone splitting entry or so? */ > > if (!entry || dax_is_conflict(entry) || > > (order == 0 && !dax_is_pte_entry(entry))) { > > - put_unlocked_entry(&xas, entry); > > + put_unlocked_entry(&xas, entry, WAKE_NEXT); > > xas_unlock_irq(&xas); > > trace_dax_insert_pfn_mkwrite_no_entry(mapping->host, vmf, > > VM_FAULT_NOPAGE); > > >
On Wed, Apr 21, 2021 at 12:09:54PM -0700, Dan Williams wrote:
> Can you get in the habit of not replying inline with new patches like
> this? Collect the review feedback, take a pause, and resend the full
> series so tooling like b4 and patchwork can track when a new posting
> supersedes a previous one. As is, this inline style inflicts manual
> effort on the maintainer.
Honestly I don't mind it at all. If you shiny new tooling can't handle
it maybe you should fix your shiny new tooling instead of changing
everyones workflow?
On Thu, Apr 22, 2021 at 07:24:58AM +0100, Christoph Hellwig wrote:
> On Wed, Apr 21, 2021 at 12:09:54PM -0700, Dan Williams wrote:
> > Can you get in the habit of not replying inline with new patches like
> > this? Collect the review feedback, take a pause, and resend the full
> > series so tooling like b4 and patchwork can track when a new posting
> > supersedes a previous one. As is, this inline style inflicts manual
> > effort on the maintainer.
>
> Honestly I don't mind it at all. If you shiny new tooling can't handle
> it maybe you should fix your shiny new tooling instead of changing
> everyones workflow?
Just speaking for XFS here, but I don't like inline resubmissions
because that makes it /really/ hard to find the original patch 6 months
later when everything has paged out of my brain but random enterprise
distro backporters start asking questions ("is this an actively
exploited security fix?" "what were you smoking?" etc).
At least change the subject line to something that screams "new patch!"
so that mutt and lore will make it stand out.
(Granted this isn't XFS, so I am not the enforcer here ;))
--D
On Wed, Apr 21, 2021 at 11:25 PM Christoph Hellwig <hch@infradead.org> wrote:
>
> On Wed, Apr 21, 2021 at 12:09:54PM -0700, Dan Williams wrote:
> > Can you get in the habit of not replying inline with new patches like
> > this? Collect the review feedback, take a pause, and resend the full
> > series so tooling like b4 and patchwork can track when a new posting
> > supersedes a previous one. As is, this inline style inflicts manual
> > effort on the maintainer.
>
> Honestly I don't mind it at all. If you shiny new tooling can't handle
> it maybe you should fix your shiny new tooling instead of changing
> everyones workflow?
I think asking a submitter to resend a series is par for the course,
especially for poor saps like me burdened by corporate email systems.
Vivek, if this is too onerous a request just give me a heads up and
I'll manually pull out the patch content from your replies.
On Thu, Apr 22, 2021 at 01:01:15PM -0700, Dan Williams wrote:
> On Wed, Apr 21, 2021 at 11:25 PM Christoph Hellwig <hch@infradead.org> wrote:
> >
> > On Wed, Apr 21, 2021 at 12:09:54PM -0700, Dan Williams wrote:
> > > Can you get in the habit of not replying inline with new patches like
> > > this? Collect the review feedback, take a pause, and resend the full
> > > series so tooling like b4 and patchwork can track when a new posting
> > > supersedes a previous one. As is, this inline style inflicts manual
> > > effort on the maintainer.
> >
> > Honestly I don't mind it at all. If you shiny new tooling can't handle
> > it maybe you should fix your shiny new tooling instead of changing
> > everyones workflow?
>
> I think asking a submitter to resend a series is par for the course,
> especially for poor saps like me burdened by corporate email systems.
> Vivek, if this is too onerous a request just give me a heads up and
> I'll manually pull out the patch content from your replies.
I am fine with posting new version. Initially I thought that there
were only 1-2 minor cleanup comments so I posted inline, thinking
it might preferred method instead of posting full patch series again.
But then more comments came along. So posting another version makes
more sense now.
Thanks
Vivek
On Wed, Apr 21, 2021 at 11:25 PM Christoph Hellwig <hch@infradead.org> wrote: > > On Wed, Apr 21, 2021 at 12:09:54PM -0700, Dan Williams wrote: > > Can you get in the habit of not replying inline with new patches like > > this? Collect the review feedback, take a pause, and resend the full > > series so tooling like b4 and patchwork can track when a new posting > > supersedes a previous one. As is, this inline style inflicts manual > > effort on the maintainer. > > Honestly I don't mind it at all. If you shiny new tooling can't handle > it maybe you should fix your shiny new tooling instead of changing > everyones workflow? Fyi, shiny new tooling has been fixed: http://lore.kernel.org/r/20210517161317.teawoh5qovxpmqdc@nitro.local ...it still requires properly formatted patches with commentary below the "---" break line, but this should cut down on re-rolls. Hat tip to Konstantin.