From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>, Tim Deegan <tim@xen.org>,
Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
George Dunlap <George.Dunlap@eu.citrix.com>,
Andrew Cooper <andrew.cooper3@citrix.com>,
Ian Jackson <ian.jackson@eu.citrix.com>,
Dario Faggioli <dfaggioli@suse.com>,
Julien Grall <julien.grall@arm.com>,
Jan Beulich <jbeulich@suse.com>
Subject: [Xen-devel] [PATCH v2 42/48] xen/sched: split schedule_cpu_switch()
Date: Fri, 9 Aug 2019 16:58:27 +0200 [thread overview]
Message-ID: <20190809145833.1020-43-jgross@suse.com> (raw)
In-Reply-To: <20190809145833.1020-1-jgross@suse.com>
Instead of letting schedule_cpu_switch() handle moving cpus from and
to cpupools, split it into schedule_cpu_add() and schedule_cpu_rm().
This will allow us to drop allocating/freeing scheduler data for free
cpus as the idle scheduler doesn't need such data.
Signed-off-by: Juergen Gross <jgross@suse.com>
---
V1: new patch
---
xen/common/cpupool.c | 4 +-
xen/common/schedule.c | 125 +++++++++++++++++++++++++++---------------------
xen/include/xen/sched.h | 3 +-
3 files changed, 74 insertions(+), 58 deletions(-)
diff --git a/xen/common/cpupool.c b/xen/common/cpupool.c
index 8789fde3c4..4749ead846 100644
--- a/xen/common/cpupool.c
+++ b/xen/common/cpupool.c
@@ -268,7 +268,7 @@ static int cpupool_assign_cpu_locked(struct cpupool *c, unsigned int cpu)
if ( (cpupool_moving_cpu == cpu) && (c != cpupool_cpu_moving) )
return -EADDRNOTAVAIL;
- ret = schedule_cpu_switch(cpu, c);
+ ret = schedule_cpu_add(cpu, c);
if ( ret )
return ret;
@@ -318,7 +318,7 @@ static int cpupool_unassign_cpu_epilogue(struct cpupool *c)
*/
if ( !ret )
{
- ret = schedule_cpu_switch(cpu, NULL);
+ ret = schedule_cpu_rm(cpu);
if ( ret )
cpumask_clear_cpu(cpu, &cpupool_free_cpus);
else
diff --git a/xen/common/schedule.c b/xen/common/schedule.c
index 7823b48e32..999f6e347b 100644
--- a/xen/common/schedule.c
+++ b/xen/common/schedule.c
@@ -83,15 +83,6 @@ extern const struct scheduler *__start_schedulers_array[], *__end_schedulers_arr
static struct scheduler __read_mostly ops;
-static spinlock_t *
-sched_idle_switch_sched(struct scheduler *new_ops, unsigned int cpu,
- void *pdata, void *vdata)
-{
- sched_idle_unit(cpu)->priv = NULL;
-
- return &sched_free_cpu_lock;
-}
-
static struct sched_resource *
sched_idle_res_pick(const struct scheduler *ops, struct sched_unit *unit)
{
@@ -131,7 +122,6 @@ static struct scheduler sched_idle_ops = {
.alloc_vdata = sched_idle_alloc_vdata,
.free_vdata = sched_idle_free_vdata,
- .switch_sched = sched_idle_switch_sched,
};
static inline struct vcpu *unit2vcpu_cpu(struct sched_unit *unit,
@@ -2492,36 +2482,22 @@ void __init scheduler_init(void)
}
/*
- * Move a pCPU outside of the influence of the scheduler of its current
- * cpupool, or subject it to the scheduler of a new cpupool.
- *
- * For the pCPUs that are removed from their cpupool, their scheduler becomes
- * &sched_idle_ops (the idle scheduler).
+ * Move a pCPU from free cpus (running the idle scheduler) to a cpupool
+ * using any "real" scheduler.
+ * The cpu is still marked as "free" and not yet valid for its cpupool.
*/
-int schedule_cpu_switch(unsigned int cpu, struct cpupool *c)
+int schedule_cpu_add(unsigned int cpu, struct cpupool *c)
{
struct vcpu *idle;
- void *ppriv, *ppriv_old, *vpriv, *vpriv_old;
- struct scheduler *old_ops = get_sched_res(cpu)->scheduler;
- struct scheduler *new_ops = (c == NULL) ? &sched_idle_ops : c->sched;
+ void *ppriv, *vpriv;
+ struct scheduler *new_ops = c->sched;
struct sched_resource *sd = get_sched_res(cpu);
- struct cpupool *old_pool = sd->cpupool;
spinlock_t *old_lock, *new_lock;
unsigned long flags;
- /*
- * pCPUs only move from a valid cpupool to free (i.e., out of any pool),
- * or from free to a valid cpupool. In the former case (which happens when
- * c is NULL), we want the CPU to have been marked as free already, as
- * well as to not be valid for the source pool any longer, when we get to
- * here. In the latter case (which happens when c is a valid cpupool), we
- * want the CPU to still be marked as free, as well as to not yet be valid
- * for the destination pool.
- */
- ASSERT(c != old_pool && (c != NULL || old_pool != NULL));
ASSERT(cpumask_test_cpu(cpu, &cpupool_free_cpus));
- ASSERT((c == NULL && !cpumask_test_cpu(cpu, old_pool->cpu_valid)) ||
- (c != NULL && !cpumask_test_cpu(cpu, c->cpu_valid)));
+ ASSERT(!cpumask_test_cpu(cpu, c->cpu_valid));
+ ASSERT(get_sched_res(cpu)->cpupool == NULL);
/*
* To setup the cpu for the new scheduler we need:
@@ -2546,52 +2522,91 @@ int schedule_cpu_switch(unsigned int cpu, struct cpupool *c)
return -ENOMEM;
}
- sched_do_tick_suspend(old_ops, cpu);
-
/*
- * The actual switch, including (if necessary) the rerouting of the
- * scheduler lock to whatever new_ops prefers, needs to happen in one
- * critical section, protected by old_ops' lock, or races are possible.
- * It is, in fact, the lock of another scheduler that we are taking (the
- * scheduler of the cpupool that cpu still belongs to). But that is ok
- * as, anyone trying to schedule on this cpu will spin until when we
- * release that lock (bottom of this function). When he'll get the lock
- * --thanks to the loop inside *_schedule_lock() functions-- he'll notice
- * that the lock itself changed, and retry acquiring the new one (which
- * will be the correct, remapped one, at that point).
+ * The actual switch, including the rerouting of the scheduler lock to
+ * whatever new_ops prefers, needs to happen in one critical section,
+ * protected by old_ops' lock, or races are possible.
+ * It is, in fact, the lock of the idle scheduler that we are taking.
+ * But that is ok as anyone trying to schedule on this cpu will spin until
+ * when we release that lock (bottom of this function). When he'll get the
+ * lock --thanks to the loop inside *_schedule_lock() functions-- he'll
+ * notice that the lock itself changed, and retry acquiring the new one
+ * (which will be the correct, remapped one, at that point).
*/
old_lock = pcpu_schedule_lock_irqsave(cpu, &flags);
- vpriv_old = idle->sched_unit->priv;
- ppriv_old = sd->sched_priv;
new_lock = sched_switch_sched(new_ops, cpu, ppriv, vpriv);
sd->scheduler = new_ops;
sd->sched_priv = ppriv;
/*
- * The data above is protected under new_lock, which may be unlocked.
- * Another CPU can take new_lock as soon as sd->schedule_lock is visible,
- * and must observe all prior initialisation.
+ * Reroute the lock to the per pCPU lock as /last/ thing. In fact,
+ * if it is free (and it can be) we want that anyone that manages
+ * taking it, finds all the initializations we've done above in place.
*/
smp_wmb();
sd->schedule_lock = new_lock;
- /* _Not_ pcpu_schedule_unlock(): schedule_lock may have changed! */
+ /* _Not_ pcpu_schedule_unlock(): schedule_lock has changed! */
spin_unlock_irqrestore(old_lock, flags);
sched_do_tick_resume(new_ops, cpu);
+ sd->granularity = c->granularity;
+ sd->cpupool = c;
+ /* The cpu is added to a pool, trigger it to go pick up some work */
+ cpu_raise_softirq(cpu, SCHEDULE_SOFTIRQ);
+
+ return 0;
+}
+
+/*
+ * Remove a pCPU from its cpupool. Its scheduler becomes &sched_idle_ops
+ * (the idle scheduler).
+ * The cpu is already marked as "free" and not valid any longer for its
+ * cpupool.
+ */
+int schedule_cpu_rm(unsigned int cpu)
+{
+ struct vcpu *idle;
+ void *ppriv_old, *vpriv_old;
+ struct sched_resource *sd = get_sched_res(cpu);
+ struct scheduler *old_ops = sd->scheduler;
+ spinlock_t *old_lock;
+ unsigned long flags;
+
+ ASSERT(sd->cpupool != NULL);
+ ASSERT(cpumask_test_cpu(cpu, &cpupool_free_cpus));
+ ASSERT(!cpumask_test_cpu(cpu, sd->cpupool->cpu_valid));
+
+ idle = idle_vcpu[cpu];
+
+ sched_do_tick_suspend(old_ops, cpu);
+
+ /* See comment in schedule_cpu_add() regarding lock switching. */
+ old_lock = pcpu_schedule_lock_irqsave(cpu, &flags);
+
+ vpriv_old = idle->sched_unit->priv;
+ ppriv_old = sd->sched_priv;
+
+ idle->sched_unit->priv = NULL;
+ sd->scheduler = &sched_idle_ops;
+ sd->sched_priv = NULL;
+
+ smp_mb();
+ sd->schedule_lock = &sched_free_cpu_lock;
+
+ /* _Not_ pcpu_schedule_unlock(): schedule_lock may have changed! */
+ spin_unlock_irqrestore(old_lock, flags);
+
sched_deinit_pdata(old_ops, ppriv_old, cpu);
sched_free_vdata(old_ops, vpriv_old);
sched_free_pdata(old_ops, ppriv_old, cpu);
- get_sched_res(cpu)->granularity = c ? c->granularity : 1;
- get_sched_res(cpu)->cpupool = c;
- /* When a cpu is added to a pool, trigger it to go pick up some work */
- if ( c != NULL )
- cpu_raise_softirq(cpu, SCHEDULE_SOFTIRQ);
+ sd->granularity = 1;
+ sd->cpupool = NULL;
return 0;
}
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 39355f5d67..346e564e05 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -910,7 +910,8 @@ struct scheduler;
struct scheduler *scheduler_get_default(void);
struct scheduler *scheduler_alloc(unsigned int sched_id, int *perr);
void scheduler_free(struct scheduler *sched);
-int schedule_cpu_switch(unsigned int cpu, struct cpupool *c);
+int schedule_cpu_add(unsigned int cpu, struct cpupool *c);
+int schedule_cpu_rm(unsigned int cpu);
void vcpu_set_periodic_timer(struct vcpu *v, s_time_t value);
int cpu_disable_scheduler(unsigned int cpu);
/* We need it in dom0_setup_vcpu */
--
2.16.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
next prev parent reply other threads:[~2019-08-09 14:59 UTC|newest]
Thread overview: 126+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-08-09 14:57 [Xen-devel] [PATCH v2 00/48] xen: add core scheduling support Juergen Gross
2019-08-09 14:57 ` [Xen-devel] [PATCH v2 01/48] xen/sched: use new sched_unit instead of vcpu in scheduler interfaces Juergen Gross
2019-09-02 9:07 ` Jan Beulich
2019-09-09 5:26 ` Juergen Gross
2019-08-09 14:57 ` [Xen-devel] [PATCH v2 02/48] xen/sched: move per-vcpu scheduler private data pointer to sched_unit Juergen Gross
2019-08-23 10:47 ` Dario Faggioli
2019-08-09 14:57 ` [Xen-devel] [PATCH v2 03/48] xen/sched: build a linked list of struct sched_unit Juergen Gross
2019-08-23 10:52 ` Dario Faggioli
2019-08-09 14:57 ` [Xen-devel] [PATCH v2 04/48] xen/sched: introduce struct sched_resource Juergen Gross
2019-08-23 10:54 ` Dario Faggioli
2019-09-04 13:10 ` Jan Beulich
2019-09-09 5:31 ` Juergen Gross
2019-08-09 14:57 ` [Xen-devel] [PATCH v2 05/48] xen/sched: let pick_cpu return a scheduler resource Juergen Gross
2019-09-04 13:34 ` Jan Beulich
2019-09-09 5:43 ` Juergen Gross
2019-08-09 14:57 ` [Xen-devel] [PATCH v2 06/48] xen/sched: switch schedule_data.curr to point at sched_unit Juergen Gross
2019-09-04 13:36 ` Jan Beulich
2019-09-09 5:46 ` Juergen Gross
2019-08-09 14:57 ` [Xen-devel] [PATCH v2 07/48] xen/sched: move per cpu scheduler private data into struct sched_resource Juergen Gross
2019-09-04 13:48 ` Jan Beulich
2019-09-05 7:13 ` Juergen Gross
2019-09-05 7:38 ` Jan Beulich
2019-09-09 13:03 ` Dario Faggioli
2019-08-09 14:57 ` [Xen-devel] [PATCH v2 08/48] xen/sched: switch vcpu_schedule_lock to unit_schedule_lock Juergen Gross
2019-09-04 14:02 ` Jan Beulich
2019-09-04 14:41 ` Juergen Gross
2019-09-04 14:54 ` Jan Beulich
2019-09-04 15:02 ` Juergen Gross
2019-09-11 16:02 ` Dario Faggioli
2019-08-09 14:57 ` [Xen-devel] [PATCH v2 09/48] xen/sched: move some per-vcpu items to struct sched_unit Juergen Gross
2019-09-04 14:16 ` Jan Beulich
2019-09-09 6:39 ` Juergen Gross
2019-09-09 6:55 ` Jan Beulich
2019-09-09 7:05 ` Juergen Gross
2019-08-09 14:57 ` [Xen-devel] [PATCH v2 10/48] xen/sched: add scheduler helpers hiding vcpu Juergen Gross
2019-09-04 14:49 ` Jan Beulich
2019-09-11 13:22 ` Juergen Gross
2019-08-09 14:57 ` [Xen-devel] [PATCH v2 11/48] xen/sched: rename scheduler related perf counters Juergen Gross
2019-08-09 14:57 ` [Xen-devel] [PATCH v2 12/48] xen/sched: switch struct task_slice from vcpu to sched_unit Juergen Gross
2019-08-09 14:57 ` [Xen-devel] [PATCH v2 13/48] xen/sched: add is_running indicator to struct sched_unit Juergen Gross
2019-09-04 15:06 ` Jan Beulich
2019-09-11 13:44 ` Juergen Gross
2019-09-11 15:06 ` Jan Beulich
2019-09-11 15:32 ` Juergen Gross
2019-08-09 14:57 ` [Xen-devel] [PATCH v2 14/48] xen/sched: make null scheduler vcpu agnostic Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 15/48] xen/sched: make rt " Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 16/48] xen/sched: make credit " Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 17/48] xen/sched: make credit2 " Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 18/48] xen/sched: make arinc653 " Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 19/48] xen: add sched_unit_pause_nosync() and sched_unit_unpause() Juergen Gross
2019-09-09 13:34 ` Jan Beulich
2019-09-11 14:15 ` Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 20/48] xen: let vcpu_create() select processor Juergen Gross
2019-08-23 16:42 ` Julien Grall
2019-09-09 13:38 ` Jan Beulich
2019-09-11 14:22 ` Juergen Gross
2019-09-11 17:20 ` Dario Faggioli
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 21/48] xen/sched: use sched_resource cpu instead smp_processor_id in schedulers Juergen Gross
2019-09-09 14:17 ` Jan Beulich
2019-09-12 9:34 ` Juergen Gross
2019-09-12 10:04 ` Jan Beulich
2019-09-12 11:03 ` Juergen Gross
2019-09-12 11:17 ` Juergen Gross
2019-09-12 11:46 ` Jan Beulich
2019-09-12 11:53 ` Juergen Gross
2019-09-12 12:08 ` Jan Beulich
2019-09-12 12:13 ` Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 22/48] xen/sched: switch schedule() from vcpus to sched_units Juergen Gross
2019-09-09 14:35 ` Jan Beulich
2019-09-12 13:44 ` Juergen Gross
2019-09-12 14:34 ` Jan Beulich
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 23/48] xen/sched: switch sched_move_irqs() to take sched_unit as parameter Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 24/48] xen: switch from for_each_vcpu() to for_each_sched_unit() Juergen Gross
2019-09-09 15:14 ` Jan Beulich
2019-09-12 14:02 ` Juergen Gross
2019-09-12 14:40 ` Jan Beulich
2019-09-12 14:47 ` Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 25/48] xen/sched: add runstate counters to struct sched_unit Juergen Gross
2019-09-09 14:30 ` Jan Beulich
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 26/48] xen/sched: rework and rename vcpu_force_reschedule() Juergen Gross
2019-09-10 14:06 ` Jan Beulich
2019-09-13 9:33 ` Juergen Gross
2019-09-13 9:40 ` Jan Beulich
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 27/48] xen/sched: Change vcpu_migrate_*() to operate on schedule unit Juergen Gross
2019-09-10 15:11 ` Jan Beulich
2019-09-13 12:33 ` Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 28/48] xen/sched: move struct task_slice into struct sched_unit Juergen Gross
2019-09-10 15:18 ` Jan Beulich
2019-09-13 12:56 ` Juergen Gross
2019-09-12 8:13 ` Dario Faggioli
2019-09-12 8:21 ` Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 29/48] xen/sched: add code to sync scheduling of all vcpus of a sched unit Juergen Gross
2019-09-10 15:36 ` Jan Beulich
2019-09-13 13:12 ` Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 30/48] xen/sched: introduce unit_runnable_state() Juergen Gross
2019-09-11 10:30 ` Jan Beulich
2019-09-12 10:22 ` Dario Faggioli
2019-09-13 14:07 ` Juergen Gross
2019-09-13 14:44 ` Jan Beulich
2019-09-13 15:23 ` Juergen Gross
2019-09-12 10:24 ` Dario Faggioli
2019-09-13 14:14 ` Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 31/48] xen/sched: add support for multiple vcpus per sched unit where missing Juergen Gross
2019-09-11 10:43 ` Jan Beulich
2019-09-13 15:01 ` Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 32/48] xen/sched: modify cpupool_domain_cpumask() to be an unit mask Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 33/48] xen/sched: support allocating multiple vcpus into one sched unit Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 34/48] xen/sched: add a percpu resource index Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 35/48] xen/sched: add fall back to idle vcpu when scheduling unit Juergen Gross
2019-09-11 11:33 ` Julien Grall
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 36/48] xen/sched: make vcpu_wake() and vcpu_sleep() core scheduling aware Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 37/48] xen/sched: carve out freeing sched_unit memory into dedicated function Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 38/48] xen/sched: move per-cpu variable scheduler to struct sched_resource Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 39/48] xen/sched: move per-cpu variable cpupool " Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 40/48] xen/sched: reject switching smt on/off with core scheduling active Juergen Gross
2019-09-10 15:47 ` Jan Beulich
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 41/48] xen/sched: prepare per-cpupool scheduling granularity Juergen Gross
2019-08-09 14:58 ` Juergen Gross [this message]
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 43/48] xen/sched: protect scheduling resource via rcu Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 44/48] xen/sched: support multiple cpus per scheduling resource Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 45/48] xen/sched: support differing granularity in schedule_cpu_[add/rm]() Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 46/48] xen/sched: support core scheduling for moving cpus to/from cpupools Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 47/48] xen/sched: disable scheduling when entering ACPI deep sleep states Juergen Gross
2019-08-09 14:58 ` [Xen-devel] [PATCH v2 48/48] xen/sched: add scheduling granularity enum Juergen Gross
2019-08-15 10:17 ` [Xen-devel] [PATCH v2 00/48] xen: add core scheduling support Sergey Dyasli
2019-09-05 6:22 ` Juergen Gross
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190809145833.1020-43-jgross@suse.com \
--to=jgross@suse.com \
--cc=George.Dunlap@eu.citrix.com \
--cc=andrew.cooper3@citrix.com \
--cc=dfaggioli@suse.com \
--cc=ian.jackson@eu.citrix.com \
--cc=jbeulich@suse.com \
--cc=julien.grall@arm.com \
--cc=konrad.wilk@oracle.com \
--cc=sstabellini@kernel.org \
--cc=tim@xen.org \
--cc=wl@xen.org \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).