* [patch 0/2] re-enable remote per-cpu-pages draining
@ 2020-06-16 16:11 Marcelo Tosatti
2020-06-16 16:11 ` [patch 1/2] rt: add local_lock_on and local_lock_irqsave_on to locallock.h Marcelo Tosatti
2020-06-16 16:11 ` [patch 2/2] mm: page_alloc: drain pages remotely Marcelo Tosatti
0 siblings, 2 replies; 11+ messages in thread
From: Marcelo Tosatti @ 2020-06-16 16:11 UTC (permalink / raw)
To: linux-rt-users; +Cc: Sebastian Andrzej Siewior, Juri Lelli, Thomas Gleixner
Remote draining of pages was removed from 5.6-rt.
Unfortunately its necessary for use-cases which have a busy spinning
SCHED_FIFO thread on isolated CPU:
[ 7475.821066] INFO: task ld:274531 blocked for more than 600 seconds.
[ 7475.822157] Not tainted 4.18.0-208.rt5.20.el8.x86_64 #1
[ 7475.823094] echo 0 /proc/sys/kernel/hung_task_timeout_secs disables this message.
[ 7475.824392] ld D 0 274531 274530 0x00084080
[ 7475.825307] Call Trace:
[ 7475.825761] __schedule+0x342/0x850
[ 7475.826377] schedule+0x39/0xd0
[ 7475.826923] schedule_timeout+0x20e/0x410
[ 7475.827610] ? __schedule+0x34a/0x850
[ 7475.828247] ? ___preempt_schedule+0x16/0x18
[ 7475.828953] wait_for_completion+0x85/0xe0
[ 7475.829653] flush_work+0x11a/0x1c0
[ 7475.830313] ? flush_workqueue_prep_pwqs+0x130/0x130
[ 7475.831148] drain_all_pages+0x140/0x190
[ 7475.831803] __alloc_pages_slowpath+0x3f8/0xe20
[ 7475.832571] ? mem_cgroup_commit_charge+0xcb/0x510
[ 7475.833371] __alloc_pages_nodemask+0x1ca/0x2b0
[ 7475.834134] pagecache_get_page+0xb5/0x2d0
[ 7475.834814] ? account_page_dirtied+0x11a/0x220
[ 7475.835579] grab_cache_page_write_begin+0x1f/0x40
[ 7475.836379] iomap_write_begin.constprop.44+0x1c1/0x370
[ 7475.837241] ? iomap_write_end+0x91/0x290
[ 7475.837911] iomap_write_actor+0x92/0x170
See individual patches for details.
^ permalink raw reply [flat|nested] 11+ messages in thread
* [patch 1/2] rt: add local_lock_on and local_lock_irqsave_on to locallock.h
2020-06-16 16:11 [patch 0/2] re-enable remote per-cpu-pages draining Marcelo Tosatti
@ 2020-06-16 16:11 ` Marcelo Tosatti
2020-06-16 16:11 ` [patch 2/2] mm: page_alloc: drain pages remotely Marcelo Tosatti
1 sibling, 0 replies; 11+ messages in thread
From: Marcelo Tosatti @ 2020-06-16 16:11 UTC (permalink / raw)
To: linux-rt-users
Cc: Sebastian Andrzej Siewior, Juri Lelli, Thomas Gleixner, Marcelo Tosatti
Re-add local_lock_on type functions to locallock.h (from the 5.4-RT
tree).
To be used by remote page draining.
The original commit message is:
Introduce locallock. For !RT this maps to preempt_disable()/
local_irq_disable() so there is not much that changes. For RT this will
map to a spinlock. This makes preemption possible and locked "ressource"
gets the lockdep anotation it wouldn't have otherwise. The locks are
recursive for owner == current. Also, all locks user migrate_disable()
which ensures that the task is not migrated to another CPU while the lock
is held and the owner is preempted.
From: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
---
include/linux/locallock.h | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
Index: linux-rt-devel/include/linux/locallock.h
===================================================================
--- linux-rt-devel.orig/include/linux/locallock.h
+++ linux-rt-devel/include/linux/locallock.h
@@ -51,6 +51,9 @@ static inline void __local_lock(struct l
#define local_lock(lvar) \
do { __local_lock(&get_local_var(lvar)); } while (0)
+#define local_lock_on(lvar, cpu) \
+ do { __local_lock(&per_cpu(lvar, cpu)); } while (0)
+
static inline int __local_trylock(struct local_irq_lock *lv)
{
if (lv->owner != current && spin_trylock(&lv->lock)) {
@@ -92,6 +95,9 @@ static inline void __local_unlock(struct
put_local_var(lvar); \
} while (0)
+#define local_unlock_on(lvar, cpu) \
+ do { __local_unlock(&per_cpu(lvar, cpu)); } while (0)
+
static inline void __local_lock_irq(struct local_irq_lock *lv)
{
spin_lock_irqsave(&lv->lock, lv->flags);
@@ -104,6 +110,9 @@ static inline void __local_lock_irq(stru
#define local_lock_irq(lvar) \
do { __local_lock_irq(&get_local_var(lvar)); } while (0)
+#define local_lock_irq_on(lvar, cpu) \
+ do { __local_lock_irq(&per_cpu(lvar, cpu)); } while (0)
+
static inline void __local_unlock_irq(struct local_irq_lock *lv)
{
LL_WARN(!lv->nestcnt);
@@ -119,6 +128,11 @@ static inline void __local_unlock_irq(st
put_local_var(lvar); \
} while (0)
+#define local_unlock_irq_on(lvar, cpu) \
+ do { \
+ __local_unlock_irq(&per_cpu(lvar, cpu)); \
+ } while (0)
+
static inline int __local_lock_irqsave(struct local_irq_lock *lv)
{
if (lv->owner != current) {
@@ -137,6 +151,12 @@ static inline int __local_lock_irqsave(s
_flags = __this_cpu_read(lvar.flags); \
} while (0)
+#define local_lock_irqsave_on(lvar, _flags, cpu) \
+ do { \
+ __local_lock_irqsave(&per_cpu(lvar, cpu)); \
+ _flags = per_cpu(lvar, cpu).flags; \
+ } while (0)
+
static inline int __local_unlock_irqrestore(struct local_irq_lock *lv,
unsigned long flags)
{
@@ -156,6 +176,11 @@ static inline int __local_unlock_irqrest
put_local_var(lvar); \
} while (0)
+#define local_unlock_irqrestore_on(lvar, flags, cpu) \
+ do { \
+ __local_unlock_irqrestore(&per_cpu(lvar, cpu), flags); \
+ } while (0)
+
#define local_spin_trylock_irq(lvar, lock) \
({ \
int __locked; \
^ permalink raw reply [flat|nested] 11+ messages in thread
* [patch 2/2] mm: page_alloc: drain pages remotely
2020-06-16 16:11 [patch 0/2] re-enable remote per-cpu-pages draining Marcelo Tosatti
2020-06-16 16:11 ` [patch 1/2] rt: add local_lock_on and local_lock_irqsave_on to locallock.h Marcelo Tosatti
@ 2020-06-16 16:11 ` Marcelo Tosatti
2020-06-16 16:32 ` Sebastian Andrzej Siewior
1 sibling, 1 reply; 11+ messages in thread
From: Marcelo Tosatti @ 2020-06-16 16:11 UTC (permalink / raw)
To: linux-rt-users
Cc: Sebastian Andrzej Siewior, Juri Lelli, Thomas Gleixner, Marcelo Tosatti
Remote draining of pages was removed from 5.6-rt.
Unfortunately its necessary for use-cases which have a busy spinning
SCHED_FIFO thread on isolated CPU:
[ 7475.821066] INFO: task ld:274531 blocked for more than 600 seconds.
[ 7475.822157] Not tainted 4.18.0-208.rt5.20.el8.x86_64 #1
[ 7475.823094] echo 0 /proc/sys/kernel/hung_task_timeout_secs disables this message.
[ 7475.824392] ld D 0 274531 274530 0x00084080
[ 7475.825307] Call Trace:
[ 7475.825761] __schedule+0x342/0x850
[ 7475.826377] schedule+0x39/0xd0
[ 7475.826923] schedule_timeout+0x20e/0x410
[ 7475.827610] ? __schedule+0x34a/0x850
[ 7475.828247] ? ___preempt_schedule+0x16/0x18
[ 7475.828953] wait_for_completion+0x85/0xe0
[ 7475.829653] flush_work+0x11a/0x1c0
[ 7475.830313] ? flush_workqueue_prep_pwqs+0x130/0x130
[ 7475.831148] drain_all_pages+0x140/0x190
[ 7475.831803] __alloc_pages_slowpath+0x3f8/0xe20
[ 7475.832571] ? mem_cgroup_commit_charge+0xcb/0x510
[ 7475.833371] __alloc_pages_nodemask+0x1ca/0x2b0
[ 7475.834134] pagecache_get_page+0xb5/0x2d0
[ 7475.834814] ? account_page_dirtied+0x11a/0x220
[ 7475.835579] grab_cache_page_write_begin+0x1f/0x40
[ 7475.836379] iomap_write_begin.constprop.44+0x1c1/0x370
[ 7475.837241] ? iomap_write_end+0x91/0x290
[ 7475.837911] iomap_write_actor+0x92/0x170
...
So enable remote draining again.
The original commit message is:
mm: page_alloc: rt-friendly per-cpu pages
rt-friendly per-cpu pages: convert the irqs-off per-cpu locking
method into a preemptible, explicit-per-cpu-locks method.
Contains fixes from:
Peter Zijlstra <a.p.zijlstra@chello.nl>
Thomas Gleixner <tglx@linutronix.de>
From: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
---
mm/page_alloc.c | 21 +++++++++++++++++++--
1 file changed, 19 insertions(+), 2 deletions(-)
Index: linux-rt-devel/mm/page_alloc.c
===================================================================
--- linux-rt-devel.orig/mm/page_alloc.c
+++ linux-rt-devel/mm/page_alloc.c
@@ -360,6 +360,16 @@ EXPORT_SYMBOL(nr_online_nodes);
static DEFINE_LOCAL_IRQ_LOCK(pa_lock);
+#ifdef CONFIG_PREEMPT_RT
+# define cpu_lock_irqsave(cpu, flags) \
+ local_lock_irqsave_on(pa_lock, flags, cpu)
+# define cpu_unlock_irqrestore(cpu, flags) \
+ local_unlock_irqrestore_on(pa_lock, flags, cpu)
+#else
+# define cpu_lock_irqsave(cpu, flags) local_irq_save(flags)
+# define cpu_unlock_irqrestore(cpu, flags) local_irq_restore(flags)
+#endif
+
int page_group_by_mobility_disabled __read_mostly;
#ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT
@@ -2852,7 +2862,7 @@ static void drain_pages_zone(unsigned in
LIST_HEAD(dst);
int count;
- local_lock_irqsave(pa_lock, flags);
+ cpu_lock_irqsave(cpu, flags);
pset = per_cpu_ptr(zone->pageset, cpu);
pcp = &pset->pcp;
@@ -2860,7 +2870,7 @@ static void drain_pages_zone(unsigned in
if (count)
isolate_pcp_pages(count, pcp, &dst);
- local_unlock_irqrestore(pa_lock, flags);
+ cpu_unlock_irqrestore(cpu, flags);
if (count)
free_pcppages_bulk(zone, &dst, false);
@@ -2898,6 +2908,7 @@ void drain_local_pages(struct zone *zone
drain_pages(cpu);
}
+#ifndef CONFIG_PREEMPT_RT
static void drain_local_pages_wq(struct work_struct *work)
{
struct pcpu_drain *drain;
@@ -2915,6 +2926,7 @@ static void drain_local_pages_wq(struct
drain_local_pages(drain->zone);
migrate_enable();
}
+#endif
/*
* Spill all the per-cpu pages from all CPUs back into the buddy allocator.
@@ -2982,6 +2994,7 @@ void drain_all_pages(struct zone *zone)
cpumask_clear_cpu(cpu, &cpus_with_pcps);
}
+#ifndef CONFIG_PREEMPT_RT
for_each_cpu(cpu, &cpus_with_pcps) {
struct pcpu_drain *drain = per_cpu_ptr(&pcpu_drain, cpu);
@@ -2991,6 +3004,10 @@ void drain_all_pages(struct zone *zone)
}
for_each_cpu(cpu, &cpus_with_pcps)
flush_work(&per_cpu_ptr(&pcpu_drain, cpu)->work);
+#else
+ for_each_cpu(cpu, &cpus_with_pcps)
+ drain_pages(cpu);
+#endif
mutex_unlock(&pcpu_drain_mutex);
}
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [patch 2/2] mm: page_alloc: drain pages remotely
2020-06-16 16:11 ` [patch 2/2] mm: page_alloc: drain pages remotely Marcelo Tosatti
@ 2020-06-16 16:32 ` Sebastian Andrzej Siewior
2020-06-16 16:55 ` Marcelo Tosatti
2020-06-16 17:04 ` Marcelo Tosatti
0 siblings, 2 replies; 11+ messages in thread
From: Sebastian Andrzej Siewior @ 2020-06-16 16:32 UTC (permalink / raw)
To: Marcelo Tosatti
Cc: linux-rt-users, Juri Lelli, Thomas Gleixner, Frederic Weisbecker
On 2020-06-16 13:11:51 [-0300], Marcelo Tosatti wrote:
> Remote draining of pages was removed from 5.6-rt.
>
> Unfortunately its necessary for use-cases which have a busy spinning
> SCHED_FIFO thread on isolated CPU:
>
> [ 7475.821066] INFO: task ld:274531 blocked for more than 600 seconds.
> [ 7475.822157] Not tainted 4.18.0-208.rt5.20.el8.x86_64 #1
> [ 7475.823094] echo 0 /proc/sys/kernel/hung_task_timeout_secs disables this message.
> [ 7475.824392] ld D 0 274531 274530 0x00084080
> [ 7475.825307] Call Trace:
> [ 7475.825761] __schedule+0x342/0x850
> [ 7475.826377] schedule+0x39/0xd0
> [ 7475.826923] schedule_timeout+0x20e/0x410
> [ 7475.827610] ? __schedule+0x34a/0x850
> [ 7475.828247] ? ___preempt_schedule+0x16/0x18
> [ 7475.828953] wait_for_completion+0x85/0xe0
> [ 7475.829653] flush_work+0x11a/0x1c0
> [ 7475.830313] ? flush_workqueue_prep_pwqs+0x130/0x130
> [ 7475.831148] drain_all_pages+0x140/0x190
> [ 7475.831803] __alloc_pages_slowpath+0x3f8/0xe20
> [ 7475.832571] ? mem_cgroup_commit_charge+0xcb/0x510
> [ 7475.833371] __alloc_pages_nodemask+0x1ca/0x2b0
> [ 7475.834134] pagecache_get_page+0xb5/0x2d0
> [ 7475.834814] ? account_page_dirtied+0x11a/0x220
> [ 7475.835579] grab_cache_page_write_begin+0x1f/0x40
> [ 7475.836379] iomap_write_begin.constprop.44+0x1c1/0x370
> [ 7475.837241] ? iomap_write_end+0x91/0x290
> [ 7475.837911] iomap_write_actor+0x92/0x170
> ...
>
> So enable remote draining again.
Is upstream affected by this? And if not, why not?
> Index: linux-rt-devel/mm/page_alloc.c
> ===================================================================
> --- linux-rt-devel.orig/mm/page_alloc.c
> +++ linux-rt-devel/mm/page_alloc.c
> @@ -360,6 +360,16 @@ EXPORT_SYMBOL(nr_online_nodes);
>
> static DEFINE_LOCAL_IRQ_LOCK(pa_lock);
>
> +#ifdef CONFIG_PREEMPT_RT
> +# define cpu_lock_irqsave(cpu, flags) \
> + local_lock_irqsave_on(pa_lock, flags, cpu)
> +# define cpu_unlock_irqrestore(cpu, flags) \
> + local_unlock_irqrestore_on(pa_lock, flags, cpu)
> +#else
> +# define cpu_lock_irqsave(cpu, flags) local_irq_save(flags)
> +# define cpu_unlock_irqrestore(cpu, flags) local_irq_restore(flags)
> +#endif
This is going to be tough. I removed the cross-CPU local-locks from RT
because it does something different for !RT. Furthermore we have
local_locks in upstream as of v5.8-rc1, see commit
91710728d1725 ("locking: Introduce local_lock()")
so whatever happens here should have upstream blessing or I will be
forced to drop the patch again while moving forward.
Before this, I looked for cases where remote drain is useful / needed
and didn't find one. I talked to Frederick and for the NO_HZ_FULL people
it is not a problem because they don't go to kernel and so they never
got anything on their per-CPU list.
We had this
https://lore.kernel.org/linux-mm/20190424111208.24459-1-bigeasy@linutronix.de/
Sebastian
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [patch 2/2] mm: page_alloc: drain pages remotely
2020-06-16 16:32 ` Sebastian Andrzej Siewior
@ 2020-06-16 16:55 ` Marcelo Tosatti
2020-06-17 7:42 ` Sebastian Andrzej Siewior
2020-06-16 17:04 ` Marcelo Tosatti
1 sibling, 1 reply; 11+ messages in thread
From: Marcelo Tosatti @ 2020-06-16 16:55 UTC (permalink / raw)
To: Sebastian Andrzej Siewior
Cc: linux-rt-users, Juri Lelli, Thomas Gleixner, Frederic Weisbecker
On Tue, Jun 16, 2020 at 06:32:48PM +0200, Sebastian Andrzej Siewior wrote:
> On 2020-06-16 13:11:51 [-0300], Marcelo Tosatti wrote:
> > Remote draining of pages was removed from 5.6-rt.
> >
> > Unfortunately its necessary for use-cases which have a busy spinning
> > SCHED_FIFO thread on isolated CPU:
> >
> > [ 7475.821066] INFO: task ld:274531 blocked for more than 600 seconds.
> > [ 7475.822157] Not tainted 4.18.0-208.rt5.20.el8.x86_64 #1
> > [ 7475.823094] echo 0 /proc/sys/kernel/hung_task_timeout_secs disables this message.
> > [ 7475.824392] ld D 0 274531 274530 0x00084080
> > [ 7475.825307] Call Trace:
> > [ 7475.825761] __schedule+0x342/0x850
> > [ 7475.826377] schedule+0x39/0xd0
> > [ 7475.826923] schedule_timeout+0x20e/0x410
> > [ 7475.827610] ? __schedule+0x34a/0x850
> > [ 7475.828247] ? ___preempt_schedule+0x16/0x18
> > [ 7475.828953] wait_for_completion+0x85/0xe0
> > [ 7475.829653] flush_work+0x11a/0x1c0
> > [ 7475.830313] ? flush_workqueue_prep_pwqs+0x130/0x130
> > [ 7475.831148] drain_all_pages+0x140/0x190
> > [ 7475.831803] __alloc_pages_slowpath+0x3f8/0xe20
> > [ 7475.832571] ? mem_cgroup_commit_charge+0xcb/0x510
> > [ 7475.833371] __alloc_pages_nodemask+0x1ca/0x2b0
> > [ 7475.834134] pagecache_get_page+0xb5/0x2d0
> > [ 7475.834814] ? account_page_dirtied+0x11a/0x220
> > [ 7475.835579] grab_cache_page_write_begin+0x1f/0x40
> > [ 7475.836379] iomap_write_begin.constprop.44+0x1c1/0x370
> > [ 7475.837241] ? iomap_write_end+0x91/0x290
> > [ 7475.837911] iomap_write_actor+0x92/0x170
> > ...
> >
> > So enable remote draining again.
>
> Is upstream affected by this? And if not, why not?
>
> > Index: linux-rt-devel/mm/page_alloc.c
> > ===================================================================
> > --- linux-rt-devel.orig/mm/page_alloc.c
> > +++ linux-rt-devel/mm/page_alloc.c
> > @@ -360,6 +360,16 @@ EXPORT_SYMBOL(nr_online_nodes);
> >
> > static DEFINE_LOCAL_IRQ_LOCK(pa_lock);
> >
> > +#ifdef CONFIG_PREEMPT_RT
> > +# define cpu_lock_irqsave(cpu, flags) \
> > + local_lock_irqsave_on(pa_lock, flags, cpu)
> > +# define cpu_unlock_irqrestore(cpu, flags) \
> > + local_unlock_irqrestore_on(pa_lock, flags, cpu)
> > +#else
> > +# define cpu_lock_irqsave(cpu, flags) local_irq_save(flags)
> > +# define cpu_unlock_irqrestore(cpu, flags) local_irq_restore(flags)
> > +#endif
>
> This is going to be tough. I removed the cross-CPU local-locks from RT
> because it does something different for !RT. Furthermore we have
> local_locks in upstream as of v5.8-rc1, see commit
> 91710728d1725 ("locking: Introduce local_lock()")
>
> so whatever happens here should have upstream blessing or I will be
> forced to drop the patch again while moving forward.
Understood.
> Before this, I looked for cases where remote drain is useful / needed
> and didn't find one.
Just pointed out one.
> I talked to Frederick and for the NO_HZ_FULL people
> it is not a problem because they don't go to kernel and so they never
> got anything on their per-CPU list.
People are using NOHZ_FULL CPUs to run both SCHED_FIFO realtime
workloads and normal workloads. Moreover, even with syscall-less
applications:
1) Setup application (malloc buffers, etc).
2) Set SCHED_FIFO priority.
3) sched_setaffinity() to NOHZ_FULL CPU.
Per-CPU buffers will be large and must be shrunk.
> We had this
> https://lore.kernel.org/linux-mm/20190424111208.24459-1-bigeasy@linutronix.de/
Will reply to that thread. Do you want to refresh/resend that patchset or
should I?
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [patch 2/2] mm: page_alloc: drain pages remotely
2020-06-16 16:32 ` Sebastian Andrzej Siewior
2020-06-16 16:55 ` Marcelo Tosatti
@ 2020-06-16 17:04 ` Marcelo Tosatti
2020-06-17 7:46 ` Sebastian Andrzej Siewior
1 sibling, 1 reply; 11+ messages in thread
From: Marcelo Tosatti @ 2020-06-16 17:04 UTC (permalink / raw)
To: Sebastian Andrzej Siewior
Cc: linux-rt-users, Juri Lelli, Thomas Gleixner, Frederic Weisbecker
On Tue, Jun 16, 2020 at 06:32:48PM +0200, Sebastian Andrzej Siewior wrote:
> On 2020-06-16 13:11:51 [-0300], Marcelo Tosatti wrote:
> > Remote draining of pages was removed from 5.6-rt.
> >
> > Unfortunately its necessary for use-cases which have a busy spinning
> > SCHED_FIFO thread on isolated CPU:
> >
> > [ 7475.821066] INFO: task ld:274531 blocked for more than 600 seconds.
> > [ 7475.822157] Not tainted 4.18.0-208.rt5.20.el8.x86_64 #1
> > [ 7475.823094] echo 0 /proc/sys/kernel/hung_task_timeout_secs disables this message.
> > [ 7475.824392] ld D 0 274531 274530 0x00084080
> > [ 7475.825307] Call Trace:
> > [ 7475.825761] __schedule+0x342/0x850
> > [ 7475.826377] schedule+0x39/0xd0
> > [ 7475.826923] schedule_timeout+0x20e/0x410
> > [ 7475.827610] ? __schedule+0x34a/0x850
> > [ 7475.828247] ? ___preempt_schedule+0x16/0x18
> > [ 7475.828953] wait_for_completion+0x85/0xe0
> > [ 7475.829653] flush_work+0x11a/0x1c0
> > [ 7475.830313] ? flush_workqueue_prep_pwqs+0x130/0x130
> > [ 7475.831148] drain_all_pages+0x140/0x190
> > [ 7475.831803] __alloc_pages_slowpath+0x3f8/0xe20
> > [ 7475.832571] ? mem_cgroup_commit_charge+0xcb/0x510
> > [ 7475.833371] __alloc_pages_nodemask+0x1ca/0x2b0
> > [ 7475.834134] pagecache_get_page+0xb5/0x2d0
> > [ 7475.834814] ? account_page_dirtied+0x11a/0x220
> > [ 7475.835579] grab_cache_page_write_begin+0x1f/0x40
> > [ 7475.836379] iomap_write_begin.constprop.44+0x1c1/0x370
> > [ 7475.837241] ? iomap_write_end+0x91/0x290
> > [ 7475.837911] iomap_write_actor+0x92/0x170
> > ...
> >
> > So enable remote draining again.
>
> Is upstream affected by this? And if not, why not?
It is affected. The workqueue thread for a given CPU is SCHED_OTHER, which can't
interrupt a SCHED_FIFO task which does not sleep.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [patch 2/2] mm: page_alloc: drain pages remotely
2020-06-16 16:55 ` Marcelo Tosatti
@ 2020-06-17 7:42 ` Sebastian Andrzej Siewior
2020-06-17 10:11 ` Marcelo Tosatti
0 siblings, 1 reply; 11+ messages in thread
From: Sebastian Andrzej Siewior @ 2020-06-17 7:42 UTC (permalink / raw)
To: Marcelo Tosatti
Cc: linux-rt-users, Juri Lelli, Thomas Gleixner, Frederic Weisbecker
On 2020-06-16 13:55:36 [-0300], Marcelo Tosatti wrote:
> > We had this
> > https://lore.kernel.org/linux-mm/20190424111208.24459-1-bigeasy@linutronix.de/
>
> Will reply to that thread. Do you want to refresh/resend that patchset or
> should I?
I don't plan to refresh/resend it because what we have now with local-locks
in v5.8-rc1 works for RT.
I dropped that from RT in v5.4.26-rt17 so the previous release should have
the last version of this. Please be aware that Joe Korty reported that
this patch somehow breaks a workload for him. The report never made it
to the list but he confirmed in
https://lkml.kernel.org/r/20200330154110.GA2975@zipoli.concurrent-rt.com
that is problem is gone with the removal of the patch.
Sebastian
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [patch 2/2] mm: page_alloc: drain pages remotely
2020-06-16 17:04 ` Marcelo Tosatti
@ 2020-06-17 7:46 ` Sebastian Andrzej Siewior
2020-06-17 10:12 ` Marcelo Tosatti
0 siblings, 1 reply; 11+ messages in thread
From: Sebastian Andrzej Siewior @ 2020-06-17 7:46 UTC (permalink / raw)
To: Marcelo Tosatti
Cc: linux-rt-users, Juri Lelli, Thomas Gleixner, Frederic Weisbecker
On 2020-06-16 14:04:25 [-0300], Marcelo Tosatti wrote:
> It is affected. The workqueue thread for a given CPU is SCHED_OTHER, which can't
> interrupt a SCHED_FIFO task which does not sleep.
Okay. So it is a problem which exists independently of -RT.
As I pointed out to Juri, vmstat_update() should have the very same
problem, correct?
Sebastian
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [patch 2/2] mm: page_alloc: drain pages remotely
2020-06-17 7:42 ` Sebastian Andrzej Siewior
@ 2020-06-17 10:11 ` Marcelo Tosatti
2020-06-17 12:01 ` Sebastian Andrzej Siewior
0 siblings, 1 reply; 11+ messages in thread
From: Marcelo Tosatti @ 2020-06-17 10:11 UTC (permalink / raw)
To: Sebastian Andrzej Siewior
Cc: linux-rt-users, Juri Lelli, Thomas Gleixner, Frederic Weisbecker
On Wed, Jun 17, 2020 at 09:42:23AM +0200, Sebastian Andrzej Siewior wrote:
> On 2020-06-16 13:55:36 [-0300], Marcelo Tosatti wrote:
> > > We had this
> > > https://lore.kernel.org/linux-mm/20190424111208.24459-1-bigeasy@linutronix.de/
> >
> > Will reply to that thread. Do you want to refresh/resend that patchset or
> > should I?
>
> I don't plan to refresh/resend it because what we have now with local-locks
> in v5.8-rc1 works for RT.
If your application busy-spins with DPDK (such as FlexRAN does):
https://www.slideshare.net/LuminaNetworks/osn-bay-area-feb-2019-meetup-intel-5g-flexran-solution-131647692
Then v5.8-rc1 does not work, right? (remote pagevec locking + remote
draining would be necessary).
> I dropped that from RT in v5.4.26-rt17 so the previous release should have
> the last version of this. Please be aware that Joe Korty reported that
> this patch somehow breaks a workload for him. The report never made it
> to the list but he confirmed in
> https://lkml.kernel.org/r/20200330154110.GA2975@zipoli.concurrent-rt.com
OK, will see if i can reproduce it and assist in the pagevec locking
upstreaming effort.
Thanks!
>
> that is problem is gone with the removal of the patch.
>
> Sebastian
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [patch 2/2] mm: page_alloc: drain pages remotely
2020-06-17 7:46 ` Sebastian Andrzej Siewior
@ 2020-06-17 10:12 ` Marcelo Tosatti
0 siblings, 0 replies; 11+ messages in thread
From: Marcelo Tosatti @ 2020-06-17 10:12 UTC (permalink / raw)
To: Sebastian Andrzej Siewior
Cc: linux-rt-users, Juri Lelli, Thomas Gleixner, Frederic Weisbecker
On Wed, Jun 17, 2020 at 09:46:17AM +0200, Sebastian Andrzej Siewior wrote:
> On 2020-06-16 14:04:25 [-0300], Marcelo Tosatti wrote:
> > It is affected. The workqueue thread for a given CPU is SCHED_OTHER, which can't
> > interrupt a SCHED_FIFO task which does not sleep.
>
> Okay. So it is a problem which exists independently of -RT.
> As I pointed out to Juri, vmstat_update() should have the very same
> problem, correct?
Yes.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [patch 2/2] mm: page_alloc: drain pages remotely
2020-06-17 10:11 ` Marcelo Tosatti
@ 2020-06-17 12:01 ` Sebastian Andrzej Siewior
0 siblings, 0 replies; 11+ messages in thread
From: Sebastian Andrzej Siewior @ 2020-06-17 12:01 UTC (permalink / raw)
To: Marcelo Tosatti
Cc: linux-rt-users, Juri Lelli, Thomas Gleixner, Frederic Weisbecker
On 2020-06-17 07:11:36 [-0300], Marcelo Tosatti wrote:
> If your application busy-spins with DPDK (such as FlexRAN does):
>
> https://www.slideshare.net/LuminaNetworks/osn-bay-area-feb-2019-meetup-intel-5g-flexran-solution-131647692
I see.
> Then v5.8-rc1 does not work, right? (remote pagevec locking + remote
> draining would be necessary).
No, it is what we have in latest rt-devel: A per-CPU spinlock
replacement for preempt_disable() on CONFIG_PREEMPT_RT. We use the
schedule_work_on() as mainline does.
Sebastian
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2020-06-17 12:01 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-06-16 16:11 [patch 0/2] re-enable remote per-cpu-pages draining Marcelo Tosatti
2020-06-16 16:11 ` [patch 1/2] rt: add local_lock_on and local_lock_irqsave_on to locallock.h Marcelo Tosatti
2020-06-16 16:11 ` [patch 2/2] mm: page_alloc: drain pages remotely Marcelo Tosatti
2020-06-16 16:32 ` Sebastian Andrzej Siewior
2020-06-16 16:55 ` Marcelo Tosatti
2020-06-17 7:42 ` Sebastian Andrzej Siewior
2020-06-17 10:11 ` Marcelo Tosatti
2020-06-17 12:01 ` Sebastian Andrzej Siewior
2020-06-16 17:04 ` Marcelo Tosatti
2020-06-17 7:46 ` Sebastian Andrzej Siewior
2020-06-17 10:12 ` Marcelo Tosatti
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).