* [RFC 0/3] rt: Some fixes for migrate_disable/enable
@ 2017-06-16 10:39 Daniel Bristot de Oliveira
2017-06-16 10:39 ` [RFC 1/3] rt: Increase/decrease the nr of migratable tasks when enabling/disabling migration Daniel Bristot de Oliveira
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Daniel Bristot de Oliveira @ 2017-06-16 10:39 UTC (permalink / raw)
To: linux-rt-users
Cc: Luis Claudio R . Goncalves, Clark Williams, Luiz Capitulino,
Sebastian Andrzej Siewior, Thomas Gleixner, Steven Rostedt,
Peter Zijlstra, LKML
This RFC suggest three changes from the migrate disable/enable
mechanism. Two of them are fixes, and the last one is a
suggestion/improvement.
As migrate_disable/enable is RT specific, this patch set is
RT specific.
Daniel Bristot de Oliveira (3):
rt: Increase/decrease the nr of migratable tasks when
enabling/disabling migration
rt: Update nr_cpus_allowed if the affinity of a task changes while its
migration is disabled
rt: Checks if task needs migration when re-enabling migration
kernel/sched/core.c | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 57 insertions(+), 1 deletion(-)
--
2.9.4
^ permalink raw reply [flat|nested] 6+ messages in thread* [RFC 1/3] rt: Increase/decrease the nr of migratable tasks when enabling/disabling migration 2017-06-16 10:39 [RFC 0/3] rt: Some fixes for migrate_disable/enable Daniel Bristot de Oliveira @ 2017-06-16 10:39 ` Daniel Bristot de Oliveira 2017-06-16 10:39 ` [RFC 2/3] rt: Update nr_cpus_allowed if the affinity of a task changes while its migration is disabled Daniel Bristot de Oliveira 2017-06-16 10:39 ` [RFC 3/3] rt: Check if the task needs to migrate when re-enabling migration Daniel Bristot de Oliveira 2 siblings, 0 replies; 6+ messages in thread From: Daniel Bristot de Oliveira @ 2017-06-16 10:39 UTC (permalink / raw) To: linux-rt-users Cc: Luis Claudio R . Goncalves, Clark Williams, Luiz Capitulino, Sebastian Andrzej Siewior, Thomas Gleixner, Steven Rostedt, Peter Zijlstra, LKML There is a problem in the migrate_disable()/enable() implementation regarding the number of migratable tasks in the rt/dl RQs. The problem is the following: When a task is attached to the rt runqueue, it is checked if it either can run in more than one CPU, or if it is with migration disable. If either check is true, the rt_rq->rt_nr_migratory counter is not increased. The counter increases otherwise. When the task is detached, the same check is done. If either check is true, the rt_rq->rt_nr_migratory counter is not decreased. The counter decreases otherwise. The same check is done in the dl scheduler. One important thing is that, migrate disable/enable does not touch this counter for tasks attached to the rt rq. So suppose the following chain of events. Assumptions: Task A is the only runnable task in A Task B runs on the CPU B Task A runs on CFS (non-rt) Task B has RT priority Thus, rt_nr_migratory is 0 B is running Task A can run on all CPUS. Timeline: CPU A/TASK A CPU B/TASK B A takes the rt mutex X . A disables migration . . B tries to take the rt mutex X . As it is held by A { . A inherits the rt priority of B . A is dequeued from CFS RQ of CPU A . A is enqueued in the RT RQ of CPU A . As migration is disabled . rt_nr_migratory in A is not increased . A enables migration A releases the rt mutex X { A returns to its original priority A ask to be dequeued from RT RQ { As migration is now enabled and it can run on all CPUS { rt_nr_migratory should be decreased As rt_nr_migratory is 0, rt_nr_migratory under flows } } This variable is important because it notifies if there are more than one runnable & migratable task in the runqueue. If there are more than one tasks, the rt_rq is set as overloaded, and then tries to migrate some tasks. This rule is important to keep the scheduler working conserving, that is, in a system with M CPUs, the M highest priority tasks should be running. As rt_nr_migratory is unsigned, it will become > 0, notifying that the RQ is overloaded, activating pushing mechanism without need. This patch fixes this problem by decreasing/increasing the rt/dl_nr_migratory in the migrate disable/enable operations. Reported-by: Pei Zhang <pezhang@redhat.com> Reported-by: Luiz Capitulino <lcapitulino@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@redhat.com> Cc: Luis Claudio R. Goncalves <lgoncalv@redhat.com> Cc: Clark Williams <williams@redhat.com> Cc: Luiz Capitulino <lcapitulino@redhat.com> Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: LKML <linux-kernel@vger.kernel.org> Cc: linux-rt-users <linux-rt-users@vger.kernel.org> --- kernel/sched/core.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 10e832d..aeb3e12 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -3376,6 +3376,8 @@ static inline void schedule_debug(struct task_struct *prev) void migrate_disable(void) { struct task_struct *p = current; + struct rq_flags rf; + struct rq *rq; if (in_atomic() || irqs_disabled()) { #ifdef CONFIG_SCHED_DEBUG @@ -3399,7 +3401,19 @@ void migrate_disable(void) preempt_disable(); preempt_lazy_disable(); pin_current_cpu(); + rq = task_rq_lock(p, &rf); p->migrate_disable = 1; + + if (unlikely((p->sched_class == &rt_sched_class || + p->sched_class == &dl_sched_class) && + p->nr_cpus_allowed > 1)) { + if (p->sched_class == &rt_sched_class) + task_rq(p)->rt.rt_nr_migratory--; + else + task_rq(p)->dl.dl_nr_migratory--; + } + + task_rq_unlock(rq, p, &rf); preempt_enable(); } EXPORT_SYMBOL(migrate_disable); @@ -3407,6 +3421,8 @@ EXPORT_SYMBOL(migrate_disable); void migrate_enable(void) { struct task_struct *p = current; + struct rq_flags rf; + struct rq *rq; if (in_atomic() || irqs_disabled()) { #ifdef CONFIG_SCHED_DEBUG @@ -3429,12 +3445,24 @@ void migrate_enable(void) } preempt_disable(); + rq = task_rq_lock(p, &rf); + /* * Clearing migrate_disable causes tsk_cpus_allowed to * show the tasks original cpu affinity. */ p->migrate_disable = 0; + if (unlikely((p->sched_class == &rt_sched_class || + p->sched_class == &dl_sched_class) && + p->nr_cpus_allowed > 1)) { + if (p->sched_class == &rt_sched_class) + task_rq(p)->rt.rt_nr_migratory++; + else + task_rq(p)->dl.dl_nr_migratory++; + } + + task_rq_unlock(rq, p, &rf); unpin_current_cpu(); preempt_enable(); preempt_lazy_enable(); -- 2.9.4 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* [RFC 2/3] rt: Update nr_cpus_allowed if the affinity of a task changes while its migration is disabled 2017-06-16 10:39 [RFC 0/3] rt: Some fixes for migrate_disable/enable Daniel Bristot de Oliveira 2017-06-16 10:39 ` [RFC 1/3] rt: Increase/decrease the nr of migratable tasks when enabling/disabling migration Daniel Bristot de Oliveira @ 2017-06-16 10:39 ` Daniel Bristot de Oliveira 2017-06-16 10:39 ` [RFC 3/3] rt: Check if the task needs to migrate when re-enabling migration Daniel Bristot de Oliveira 2 siblings, 0 replies; 6+ messages in thread From: Daniel Bristot de Oliveira @ 2017-06-16 10:39 UTC (permalink / raw) To: linux-rt-users Cc: Luis Claudio R . Goncalves, Clark Williams, Luiz Capitulino, Sebastian Andrzej Siewior, Thomas Gleixner, Steven Rostedt, Peter Zijlstra, LKML Currently, if the affinity of a task changes when it is with migration disabled, the nr_cpus_allowed is not being updated, creating an inconsistency between nr_cpus_allowed and cpumask_weight(cpus_allowed) This patch fixes this problem by calling set_cpus_allowed_common() if the cpumask of a task changes while it is with migration disable. Signed-off-by: Daniel Bristot de Oliveira <bristot@redhat.com> Cc: Luis Claudio R. Goncalves <lgoncalv@redhat.com> Cc: Clark Williams <williams@redhat.com> Cc: Luiz Capitulino <lcapitulino@redhat.com> Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: LKML <linux-kernel@vger.kernel.org> Cc: linux-rt-users <linux-rt-users@vger.kernel.org> --- kernel/sched/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index aeb3e12..0396bf2 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -1146,7 +1146,7 @@ void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new_mask) lockdep_assert_held(&p->pi_lock); if (__migrate_disabled(p)) { - cpumask_copy(&p->cpus_allowed, new_mask); + set_cpus_allowed_common(p, new_mask); return; } -- 2.9.4 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* [RFC 3/3] rt: Check if the task needs to migrate when re-enabling migration 2017-06-16 10:39 [RFC 0/3] rt: Some fixes for migrate_disable/enable Daniel Bristot de Oliveira 2017-06-16 10:39 ` [RFC 1/3] rt: Increase/decrease the nr of migratable tasks when enabling/disabling migration Daniel Bristot de Oliveira 2017-06-16 10:39 ` [RFC 2/3] rt: Update nr_cpus_allowed if the affinity of a task changes while its migration is disabled Daniel Bristot de Oliveira @ 2017-06-16 10:39 ` Daniel Bristot de Oliveira 2017-06-16 16:58 ` Sebastian Andrzej Siewior 2 siblings, 1 reply; 6+ messages in thread From: Daniel Bristot de Oliveira @ 2017-06-16 10:39 UTC (permalink / raw) To: linux-rt-users Cc: Luis Claudio R . Goncalves, Clark Williams, Luiz Capitulino, Sebastian Andrzej Siewior, Thomas Gleixner, Steven Rostedt, Peter Zijlstra, LKML In the case of an affinity change during a migrate_disable section, __set_cpus_allowed_ptr will not try to move the task from a CPU in which it cannot execute anymore. So, after enabling migration, if the current task cannot execute in the current CPU anymore, migrate it away. Signed-off-by: Daniel Bristot de Oliveira <bristot@redhat.com> Cc: Luis Claudio R. Goncalves <lgoncalv@redhat.com> Cc: Clark Williams <williams@redhat.com> Cc: Luiz Capitulino <lcapitulino@redhat.com> Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: LKML <linux-kernel@vger.kernel.org> Cc: linux-rt-users <linux-rt-users@vger.kernel.org> --- kernel/sched/core.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 0396bf2..207bc85 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -3462,6 +3462,34 @@ void migrate_enable(void) task_rq(p)->dl.dl_nr_migratory++; } + /* + * Check if the task can still run on this CPU. In the case of an + * affinity change during a migrate_disable section, + * __set_cpus_allowed_ptr will not try to move the task from a CPU + * that the task cannot execute anymore. + * + * So, if the current task cannot execute in the current CPU anymore, + * migrate it away. + */ + if (unlikely(!cpumask_test_cpu(task_cpu(p), &p->cpus_allowed))) { + const struct cpumask *cpu_mask = (p->flags & PF_KTHREAD) ? + cpu_online_mask : cpu_active_mask; + + int dest_cpu = cpumask_any_and(cpu_mask, &p->cpus_allowed); + struct migration_arg arg = {p, dest_cpu}; + + /* Need help from migration thread: drop lock and wait. */ + task_rq_unlock(rq, p, &rf); + unpin_current_cpu(); + preempt_enable(); + preempt_lazy_enable(); + + stop_one_cpu(cpu_of(rq), migration_cpu_stop, &arg); + tlb_migrate_finish(p->mm); + + return; + } + task_rq_unlock(rq, p, &rf); unpin_current_cpu(); preempt_enable(); -- 2.9.4 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [RFC 3/3] rt: Check if the task needs to migrate when re-enabling migration 2017-06-16 10:39 ` [RFC 3/3] rt: Check if the task needs to migrate when re-enabling migration Daniel Bristot de Oliveira @ 2017-06-16 16:58 ` Sebastian Andrzej Siewior 2017-06-23 13:14 ` Sebastian Andrzej Siewior 0 siblings, 1 reply; 6+ messages in thread From: Sebastian Andrzej Siewior @ 2017-06-16 16:58 UTC (permalink / raw) To: Daniel Bristot de Oliveira Cc: linux-rt-users, Luis Claudio R . Goncalves, Clark Williams, Luiz Capitulino, Thomas Gleixner, Steven Rostedt, Peter Zijlstra, LKML On 2017-06-16 12:39:48 [+0200], Daniel Bristot de Oliveira wrote: > In the case of an affinity change during a migrate_disable section, > __set_cpus_allowed_ptr will not try to move the task from a CPU > in which it cannot execute anymore. > > So, after enabling migration, if the current task cannot execute in > the current CPU anymore, migrate it away. > > Signed-off-by: Daniel Bristot de Oliveira <bristot@redhat.com> > Cc: Luis Claudio R. Goncalves <lgoncalv@redhat.com> > Cc: Clark Williams <williams@redhat.com> > Cc: Luiz Capitulino <lcapitulino@redhat.com> > Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de> > Cc: Thomas Gleixner <tglx@linutronix.de> > Cc: Steven Rostedt <rostedt@goodmis.org> > Cc: Peter Zijlstra <peterz@infradead.org> > Cc: LKML <linux-kernel@vger.kernel.org> > Cc: linux-rt-users <linux-rt-users@vger.kernel.org> > --- > kernel/sched/core.c | 28 ++++++++++++++++++++++++++++ > 1 file changed, 28 insertions(+) > > diff --git a/kernel/sched/core.c b/kernel/sched/core.c > index 0396bf2..207bc85 100644 > --- a/kernel/sched/core.c > +++ b/kernel/sched/core.c > @@ -3462,6 +3462,34 @@ void migrate_enable(void) > task_rq(p)->dl.dl_nr_migratory++; > } > > + /* > + * Check if the task can still run on this CPU. In the case of an > + * affinity change during a migrate_disable section, > + * __set_cpus_allowed_ptr will not try to move the task from a CPU > + * that the task cannot execute anymore. > + * > + * So, if the current task cannot execute in the current CPU anymore, > + * migrate it away. > + */ > + if (unlikely(!cpumask_test_cpu(task_cpu(p), &p->cpus_allowed))) { > + const struct cpumask *cpu_mask = (p->flags & PF_KTHREAD) ? > + cpu_online_mask : cpu_active_mask; > + > + int dest_cpu = cpumask_any_and(cpu_mask, &p->cpus_allowed); > + struct migration_arg arg = {p, dest_cpu}; > + > + /* Need help from migration thread: drop lock and wait. */ > + task_rq_unlock(rq, p, &rf); > + unpin_current_cpu(); > + preempt_enable(); > + preempt_lazy_enable(); > + > + stop_one_cpu(cpu_of(rq), migration_cpu_stop, &arg); > + tlb_migrate_finish(p->mm); > + > + return; > + } I noticed this problem (and the one you pointed out in 2/3) while redoing this whole thing for v4.11. In v4.11 I added a field current->migrate_disable_update [0]. Once it is set, I do what you do here + __do_set_cpus_allowed_tail() which is the whole part that got skipped in do_set_cpus_allowed(). Invoking set_cpus_allowed_common() as you do in 2/3 works in general but sched_DL does a few extra things so I would suggest to backport that part from v4.11. Also I have no idea if it is okay to run that hook for DL if the task is still limited to only one CPU (instead in migrate_enable()). So this is the backport for v4.9: diff --git a/include/linux/sched.h b/include/linux/sched.h --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1535,6 +1535,7 @@ struct task_struct { unsigned int policy; #ifdef CONFIG_PREEMPT_RT_FULL int migrate_disable; + int migrate_disable_update; # ifdef CONFIG_SCHED_DEBUG int migrate_disable_atomic; # endif diff --git a/kernel/sched/core.c b/kernel/sched/core.c --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -1138,18 +1138,14 @@ void set_cpus_allowed_common(struct task_struct *p, const struct cpumask *new_ma p->nr_cpus_allowed = cpumask_weight(new_mask); } -void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new_mask) +static void __do_set_cpus_allowed_tail(struct task_struct *p, + const struct cpumask *new_mask) { struct rq *rq = task_rq(p); bool queued, running; lockdep_assert_held(&p->pi_lock); - if (__migrate_disabled(p)) { - cpumask_copy(&p->cpus_allowed, new_mask); - return; - } - queued = task_on_rq_queued(p); running = task_current(rq, p); @@ -1172,6 +1168,20 @@ void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new_mask) set_curr_task(rq, p); } +void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new_mask) +{ + if (__migrate_disabled(p)) { + lockdep_assert_held(&p->pi_lock); + + cpumask_copy(&p->cpus_allowed, new_mask); +#if defined(CONFIG_PREEMPT_RT_FULL) && defined(CONFIG_SMP) + p->migrate_disable_update = 1; +#endif + return; + } + __do_set_cpus_allowed_tail(p, new_mask); +} + static DEFINE_PER_CPU(struct cpumask, sched_cpumasks); static DEFINE_MUTEX(sched_down_mutex); static cpumask_t sched_down_cpumask; @@ -1307,9 +1317,16 @@ static int __set_cpus_allowed_ptr(struct task_struct *p, } /* Can the task run on the task's current CPU? If so, we're done */ - if (cpumask_test_cpu(task_cpu(p), new_mask) || __migrate_disabled(p)) + if (cpumask_test_cpu(task_cpu(p), new_mask)) goto out; +#if defined(CONFIG_PREEMPT_RT_FULL) && defined(CONFIG_SMP) + if (__migrate_disabled(p)) { + p->migrate_disable_update = 1; + goto out; + } +#endif + dest_cpu = cpumask_any_and(cpu_valid_mask, new_mask); if (task_running(rq, p) || p->state == TASK_WAKING) { struct migration_arg arg = { p, dest_cpu }; @@ -3435,6 +3452,43 @@ void migrate_enable(void) */ p->migrate_disable = 0; + if (p->migrate_disable_update) { + struct rq *rq; + struct rq_flags rf; + + rq = task_rq_lock(p, &rf); + update_rq_clock(rq); + + __do_set_cpus_allowed_tail(p, &p->cpus_allowed); + task_rq_unlock(rq, p, &rf); + + p->migrate_disable_update = 0; + + WARN_ON(smp_processor_id() != task_cpu(p)); + if (!cpumask_test_cpu(task_cpu(p), &p->cpus_allowed)) { + const struct cpumask *cpu_valid_mask = cpu_active_mask; + struct migration_arg arg; + unsigned int dest_cpu; + + if (p->flags & PF_KTHREAD) { + /* + * Kernel threads are allowed on online && !active CPUs + */ + cpu_valid_mask = cpu_online_mask; + } + dest_cpu = cpumask_any_and(cpu_valid_mask, &p->cpus_allowed); + arg.task = p; + arg.dest_cpu = dest_cpu; + + unpin_current_cpu(); + preempt_lazy_enable(); + preempt_enable(); + stop_one_cpu(task_cpu(p), migration_cpu_stop, &arg); + tlb_migrate_finish(p->mm); + return; + } + } + unpin_current_cpu(); preempt_enable(); preempt_lazy_enable(); Any objections? > task_rq_unlock(rq, p, &rf); > unpin_current_cpu(); > preempt_enable(); [0] https://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-rt-devel.git/tree/kernel/sched/core.c?h=v4.11.5-rt1#n7635 Sebastian ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [RFC 3/3] rt: Check if the task needs to migrate when re-enabling migration 2017-06-16 16:58 ` Sebastian Andrzej Siewior @ 2017-06-23 13:14 ` Sebastian Andrzej Siewior 0 siblings, 0 replies; 6+ messages in thread From: Sebastian Andrzej Siewior @ 2017-06-23 13:14 UTC (permalink / raw) To: Daniel Bristot de Oliveira Cc: linux-rt-users, Luis Claudio R . Goncalves, Clark Williams, Luiz Capitulino, Thomas Gleixner, Steven Rostedt, Peter Zijlstra, LKML On 2017-06-16 18:58:15 [+0200], To Daniel Bristot de Oliveira wrote: > Any objections? Okay, taking this for v4.9 then (mostly the same, except for one superfluous check): Subject: [PATCH] sched/migrate disable: handle updated task-mask mg-dis section If task's cpumask changes while in the task is in a migrate_disable() section then we don't react on it after a migrate_enable(). It matters however if current CPU is no longer part of the cpumask. We also miss the ->set_cpus_allowed() callback. This patch fixes it by setting task->migrate_disable_update once we this "delayed" hook. This bug was introduced while fixing unrelated issue in migrate_disable() in v4.4-rt3 (update_migrate_disable() got removed during that). Cc: stable-rt@vger.kernel.org Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> --- include/linux/sched.h | 1 + kernel/sched/core.c | 59 +++++++++++++++++++++++++++++++++++++++++++++------ 2 files changed, 54 insertions(+), 6 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 4d779486ad6b..89b453dc7c14 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1535,6 +1535,7 @@ struct task_struct { unsigned int policy; #ifdef CONFIG_PREEMPT_RT_FULL int migrate_disable; + int migrate_disable_update; # ifdef CONFIG_SCHED_DEBUG int migrate_disable_atomic; # endif diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 10e832da70b6..cadc1e14073d 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -1138,18 +1138,14 @@ void set_cpus_allowed_common(struct task_struct *p, const struct cpumask *new_ma p->nr_cpus_allowed = cpumask_weight(new_mask); } -void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new_mask) +static void __do_set_cpus_allowed_tail(struct task_struct *p, + const struct cpumask *new_mask) { struct rq *rq = task_rq(p); bool queued, running; lockdep_assert_held(&p->pi_lock); - if (__migrate_disabled(p)) { - cpumask_copy(&p->cpus_allowed, new_mask); - return; - } - queued = task_on_rq_queued(p); running = task_current(rq, p); @@ -1172,6 +1168,20 @@ void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new_mask) set_curr_task(rq, p); } +void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new_mask) +{ + if (__migrate_disabled(p)) { + lockdep_assert_held(&p->pi_lock); + + cpumask_copy(&p->cpus_allowed, new_mask); +#if defined(CONFIG_PREEMPT_RT_FULL) && defined(CONFIG_SMP) + p->migrate_disable_update = 1; +#endif + return; + } + __do_set_cpus_allowed_tail(p, new_mask); +} + static DEFINE_PER_CPU(struct cpumask, sched_cpumasks); static DEFINE_MUTEX(sched_down_mutex); static cpumask_t sched_down_cpumask; @@ -3435,6 +3445,43 @@ void migrate_enable(void) */ p->migrate_disable = 0; + if (p->migrate_disable_update) { + struct rq *rq; + struct rq_flags rf; + + rq = task_rq_lock(p, &rf); + update_rq_clock(rq); + + __do_set_cpus_allowed_tail(p, &p->cpus_allowed); + task_rq_unlock(rq, p, &rf); + + p->migrate_disable_update = 0; + + WARN_ON(smp_processor_id() != task_cpu(p)); + if (!cpumask_test_cpu(task_cpu(p), &p->cpus_allowed)) { + const struct cpumask *cpu_valid_mask = cpu_active_mask; + struct migration_arg arg; + unsigned int dest_cpu; + + if (p->flags & PF_KTHREAD) { + /* + * Kernel threads are allowed on online && !active CPUs + */ + cpu_valid_mask = cpu_online_mask; + } + dest_cpu = cpumask_any_and(cpu_valid_mask, &p->cpus_allowed); + arg.task = p; + arg.dest_cpu = dest_cpu; + + unpin_current_cpu(); + preempt_lazy_enable(); + preempt_enable(); + stop_one_cpu(task_cpu(p), migration_cpu_stop, &arg); + tlb_migrate_finish(p->mm); + return; + } + } + unpin_current_cpu(); preempt_enable(); preempt_lazy_enable(); -- 2.13.1 ^ permalink raw reply related [flat|nested] 6+ messages in thread
end of thread, other threads:[~2017-06-23 13:15 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-06-16 10:39 [RFC 0/3] rt: Some fixes for migrate_disable/enable Daniel Bristot de Oliveira 2017-06-16 10:39 ` [RFC 1/3] rt: Increase/decrease the nr of migratable tasks when enabling/disabling migration Daniel Bristot de Oliveira 2017-06-16 10:39 ` [RFC 2/3] rt: Update nr_cpus_allowed if the affinity of a task changes while its migration is disabled Daniel Bristot de Oliveira 2017-06-16 10:39 ` [RFC 3/3] rt: Check if the task needs to migrate when re-enabling migration Daniel Bristot de Oliveira 2017-06-16 16:58 ` Sebastian Andrzej Siewior 2017-06-23 13:14 ` Sebastian Andrzej Siewior
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.