* Re: [RFC PATCH] mm: swap: remove lru drain waiters [not found] <20200601143734.9572-1-hdanton@sina.com> @ 2020-06-01 15:41 ` Konstantin Khlebnikov 2020-06-03 8:21 ` Sebastian Andrzej Siewior 1 sibling, 0 replies; 3+ messages in thread From: Konstantin Khlebnikov @ 2020-06-01 15:41 UTC (permalink / raw) To: Hillf Danton, linux-mm; +Cc: LKML, Sebastian Andrzej Siewior On 01/06/2020 17.37, Hillf Danton wrote: > > After updating the lru drain sequence, new comers avoid waiting for > the current drainer, because he is flushing works on each online CPU, > by trying to lock the mutex; the drainer OTOH tries to do works for > those who fail to acquire the lock by checking the lru drain sequence > after releasing lock. > > See eef1a429f234 ("mm/swap.c: piggyback lru_add_drain_all() calls") > for reasons why we can skip waiting for the lock. That patch tells nothing about such change in behaviour. Callers like invalidate_bdev() really need synchronous drain to be sure that pages have no extra reference from per-cpu vectors. > > The memory barriers around the sequence and the lock come together > to remove waiters without their drain works bandoned. > > Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de> > Cc: Konstantin Khlebnikov <khlebnikov@yandex-team.ru> > Signed-off-by: Hillf Danton <hdanton@sina.com> > --- > This is inspired by one of the works from Sebastian. > > --- a/mm/swap.c > +++ b/mm/swap.c > @@ -714,10 +714,11 @@ static void lru_add_drain_per_cpu(struct > */ > void lru_add_drain_all(void) > { > - static seqcount_t seqcount = SEQCNT_ZERO(seqcount); > + static unsigned int lru_drain_seq; > static DEFINE_MUTEX(lock); > static struct cpumask has_work; > - int cpu, seq; > + int cpu; > + unsigned int seq; > > /* > * Make sure nobody triggers this path before mm_percpu_wq is fully > @@ -726,18 +727,16 @@ void lru_add_drain_all(void) > if (WARN_ON(!mm_percpu_wq)) > return; > > - seq = raw_read_seqcount_latch(&seqcount); > + lru_drain_seq++; > + smp_mb(); > > - mutex_lock(&lock); > +more_work: > > - /* > - * Piggyback on drain started and finished while we waited for lock: > - * all pages pended at the time of our enter were drained from vectors. > - */ > - if (__read_seqcount_retry(&seqcount, seq)) > - goto done; > + if (!mutex_trylock(&lock)) > + return; > > - raw_write_seqcount_latch(&seqcount); > + smp_mb(); > + seq = lru_drain_seq; > > cpumask_clear(&has_work); > > @@ -759,8 +758,11 @@ void lru_add_drain_all(void) > for_each_cpu(cpu, &has_work) > flush_work(&per_cpu(lru_add_drain_work, cpu)); > > -done: > mutex_unlock(&lock); > + > + smp_mb(); > + if (seq != lru_drain_seq) > + goto more_work; > } > #else > void lru_add_drain_all(void) > -- > ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [RFC PATCH] mm: swap: remove lru drain waiters [not found] <20200601143734.9572-1-hdanton@sina.com> 2020-06-01 15:41 ` [RFC PATCH] mm: swap: remove lru drain waiters Konstantin Khlebnikov @ 2020-06-03 8:21 ` Sebastian Andrzej Siewior 2020-06-03 10:24 ` Ahmed S. Darwish 1 sibling, 1 reply; 3+ messages in thread From: Sebastian Andrzej Siewior @ 2020-06-03 8:21 UTC (permalink / raw) To: Hillf Danton Cc: linux-mm, LKML, Konstantin Khlebnikov, Ahmed S. Darwish, Peter Zijlstra, Thomas Gleixner On 2020-06-01 22:37:34 [+0800], Hillf Danton wrote: > > After updating the lru drain sequence, new comers avoid waiting for > the current drainer, because he is flushing works on each online CPU, > by trying to lock the mutex; the drainer OTOH tries to do works for > those who fail to acquire the lock by checking the lru drain sequence > after releasing lock. > > See eef1a429f234 ("mm/swap.c: piggyback lru_add_drain_all() calls") > for reasons why we can skip waiting for the lock. > > The memory barriers around the sequence and the lock come together > to remove waiters without their drain works bandoned. > > Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de> > Cc: Konstantin Khlebnikov <khlebnikov@yandex-team.ru> > Signed-off-by: Hillf Danton <hdanton@sina.com> > --- > This is inspired by one of the works from Sebastian. Not me, it was Ahmed. > --- a/mm/swap.c > +++ b/mm/swap.c > @@ -714,10 +714,11 @@ static void lru_add_drain_per_cpu(struct > */ > void lru_add_drain_all(void) > { > - static seqcount_t seqcount = SEQCNT_ZERO(seqcount); > + static unsigned int lru_drain_seq; > static DEFINE_MUTEX(lock); > static struct cpumask has_work; > - int cpu, seq; > + int cpu; > + unsigned int seq; > > /* > * Make sure nobody triggers this path before mm_percpu_wq is fully > @@ -726,18 +727,16 @@ void lru_add_drain_all(void) > if (WARN_ON(!mm_percpu_wq)) > return; > > - seq = raw_read_seqcount_latch(&seqcount); > + lru_drain_seq++; > + smp_mb(); > > - mutex_lock(&lock); > +more_work: > > - /* > - * Piggyback on drain started and finished while we waited for lock: > - * all pages pended at the time of our enter were drained from vectors. > - */ > - if (__read_seqcount_retry(&seqcount, seq)) > - goto done; > + if (!mutex_trylock(&lock)) > + return; > > - raw_write_seqcount_latch(&seqcount); > + smp_mb(); > + seq = lru_drain_seq; > > cpumask_clear(&has_work); > > @@ -759,8 +758,11 @@ void lru_add_drain_all(void) > for_each_cpu(cpu, &has_work) > flush_work(&per_cpu(lru_add_drain_work, cpu)); > > -done: > mutex_unlock(&lock); > + > + smp_mb(); > + if (seq != lru_drain_seq) > + goto more_work; > } > #else > void lru_add_drain_all(void) > -- Sebastian ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [RFC PATCH] mm: swap: remove lru drain waiters 2020-06-03 8:21 ` Sebastian Andrzej Siewior @ 2020-06-03 10:24 ` Ahmed S. Darwish 0 siblings, 0 replies; 3+ messages in thread From: Ahmed S. Darwish @ 2020-06-03 10:24 UTC (permalink / raw) To: Hillf Danton Cc: Sebastian Andrzej Siewior, LKML, linux-mm, Konstantin Khlebnikov, Peter Zijlstra, Thomas Gleixner Hi Hillf, For some reason, **all of your posts** from <hdanton@sina.com> do not appear on lore.kernel.org. Check, for example, https://lore.kernel.org/lkml/?q=hdanton%40sina.com, where thread replies are there but not the actual posts. Just wanted to let you know... Please continue below. On Wed, Jun 03, 2020 at 10:21:45AM +0200, Sebastian Andrzej Siewior wrote: > On 2020-06-01 22:37:34 [+0800], Hillf Danton wrote: > > > > After updating the lru drain sequence, new comers avoid waiting for > > the current drainer, because he is flushing works on each online CPU, > > by trying to lock the mutex; the drainer OTOH tries to do works for > > those who fail to acquire the lock by checking the lru drain sequence > > after releasing lock. > > > > See eef1a429f234 ("mm/swap.c: piggyback lru_add_drain_all() calls") > > for reasons why we can skip waiting for the lock. > > > > The memory barriers around the sequence and the lock come together > > to remove waiters without their drain works bandoned. > > > > Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de> > > Cc: Konstantin Khlebnikov <khlebnikov@yandex-team.ru> > > Signed-off-by: Hillf Danton <hdanton@sina.com> > > --- > > This is inspired by one of the works from Sebastian. > > Not me, it was Ahmed. > > > --- a/mm/swap.c > > +++ b/mm/swap.c > > @@ -714,10 +714,11 @@ static void lru_add_drain_per_cpu(struct > > */ > > void lru_add_drain_all(void) > > { > > - static seqcount_t seqcount = SEQCNT_ZERO(seqcount); > > + static unsigned int lru_drain_seq; > > static DEFINE_MUTEX(lock); > > static struct cpumask has_work; > > - int cpu, seq; > > + int cpu; > > + unsigned int seq; > > > > /* > > * Make sure nobody triggers this path before mm_percpu_wq is fully > > @@ -726,18 +727,16 @@ void lru_add_drain_all(void) > > if (WARN_ON(!mm_percpu_wq)) > > return; > > > > - seq = raw_read_seqcount_latch(&seqcount); > > + lru_drain_seq++; > > + smp_mb(); > > > > - mutex_lock(&lock); > > +more_work: > > > > - /* > > - * Piggyback on drain started and finished while we waited for lock: > > - * all pages pended at the time of our enter were drained from vectors. > > - */ > > - if (__read_seqcount_retry(&seqcount, seq)) > > - goto done; > > + if (!mutex_trylock(&lock)) > > + return; > > The patch I've posted makes sure to preserve the existing draining logic. It only fixes an erroneous usage of seqcount_t latching, plus a memory barriers bugfix, found by John, and is to be included in v2: https://lkml.kernel.org/r/87y2pg9erj.fsf@vostro.fn.ogness.net On the other hand, you're making the draining operation completely asynchronous for a number of callers. This is such a huge change, and I fail to see: 1) any rationale for it in the changelog, 2) whether it's been verified that call-sites won't be affected. Thanks, -- Ahmed S. Darwish Linutronix GmbH ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2020-06-03 10:24 UTC | newest] Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <20200601143734.9572-1-hdanton@sina.com> 2020-06-01 15:41 ` [RFC PATCH] mm: swap: remove lru drain waiters Konstantin Khlebnikov 2020-06-03 8:21 ` Sebastian Andrzej Siewior 2020-06-03 10:24 ` Ahmed S. Darwish
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).