From: George Dunlap <george.dunlap@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Dario Faggioli <dario.faggioli@citrix.com>,
Jan Beulich <jbeulich@suse.com>,
George Dunlap <george.dunlap@citrix.com>,
Anshul Makkar <anshul.makkar@citrix.com>,
Meng Xu <mengxu@cis.upenn.edu>
Subject: [PATCH 2/3] xen: Have schedulers revise initial placement
Date: Fri, 15 Jul 2016 19:02:01 +0100 [thread overview]
Message-ID: <1468605722-24239-2-git-send-email-george.dunlap@citrix.com> (raw)
In-Reply-To: <1468605722-24239-1-git-send-email-george.dunlap@citrix.com>
The generic domain creation logic in
xen/common/domctl.c:default_vcpu0_location() attempts to try to do
initial placement load-balancing by placing vcpu 0 on the least-busy
non-primary hyperthread available. Unfortunately, the logic can end
up picking a pcpu that's not in the online mask. When this is passed
to a scheduler such which assumes that the initial assignment is
valid, it causes a null pointer dereference looking up the runqueue.
Furthermore, this initial placement doesn't take into account hard or
soft affinity, or any scheduler-specific knowledge (such as historic
runqueue load, as in credit2).
To solve this, when inserting a vcpu, always call the per-scheduler
"pick" function to revise the initial placement. This will
automatically take all knowledge the scheduler has into account.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
---
CC: Dario Faggioli <dario.faggioli@citrix.com>
CC: Anshul Makkar <anshul.makkar@citrix.com>
CC: Meng Xu <mengxu@cis.upenn.edu>
CC: Jan Beulich <jbeulich@suse.com>
---
xen/common/sched_credit.c | 3 +++
xen/common/sched_credit2.c | 12 ++++++++++--
xen/common/sched_rt.c | 3 +++
3 files changed, 16 insertions(+), 2 deletions(-)
diff --git a/xen/common/sched_credit.c b/xen/common/sched_credit.c
index ac22746..a3ef00a 100644
--- a/xen/common/sched_credit.c
+++ b/xen/common/sched_credit.c
@@ -998,6 +998,9 @@ csched_vcpu_insert(const struct scheduler *ops, struct vcpu *vc)
BUG_ON( is_idle_vcpu(vc) );
+ /* This is safe because vc isn't yet being scheduled */
+ vc->processor = csched_cpu_pick(ops, vc);
+
lock = vcpu_schedule_lock_irq(vc);
if ( !__vcpu_on_runq(svc) && vcpu_runnable(vc) && !vc->is_running )
diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
index 3b9aa27..5a04985 100644
--- a/xen/common/sched_credit2.c
+++ b/xen/common/sched_credit2.c
@@ -1620,15 +1620,23 @@ csched2_vcpu_insert(const struct scheduler *ops, struct vcpu *vc)
BUG_ON(is_idle_vcpu(vc));
+ /* Locks in cpu_pick expect irqs to be disabled */
+ local_irq_disable();
+
+ /* This is safe because vc isn't yet being scheduled */
+ vc->processor = csched2_cpu_pick(ops, vc);
+
/* Add vcpu to runqueue of initial processor */
- lock = vcpu_schedule_lock_irq(vc);
+ lock = vcpu_schedule_lock(vc);
runq_assign(ops, vc);
- vcpu_schedule_unlock_irq(lock, vc);
+ vcpu_schedule_unlock(lock, vc);
sdom->nr_vcpus++;
+ local_irq_enable();
+
SCHED_STAT_CRANK(vcpu_insert);
CSCHED2_VCPU_CHECK(vc);
diff --git a/xen/common/sched_rt.c b/xen/common/sched_rt.c
index bd3a2a0..41c61a7 100644
--- a/xen/common/sched_rt.c
+++ b/xen/common/sched_rt.c
@@ -874,6 +874,9 @@ rt_vcpu_insert(const struct scheduler *ops, struct vcpu *vc)
BUG_ON( is_idle_vcpu(vc) );
+ /* This is safe because vc isn't yet being scheduled */
+ vc->processor = rt_cpu_pick(ops, vc);
+
lock = vcpu_schedule_lock_irq(vc);
now = NOW();
--
2.1.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
next prev parent reply other threads:[~2016-07-15 18:02 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-07-15 18:02 [PATCH 1/3] xen: Some code motion to avoid having to do forward-declaration George Dunlap
2016-07-15 18:02 ` George Dunlap [this message]
2016-07-15 18:07 ` [PATCH 2/3] xen: Have schedulers revise initial placement Andrew Cooper
2016-07-16 14:12 ` Dario Faggioli
2016-07-18 18:10 ` Andrew Cooper
2016-07-18 18:55 ` Dario Faggioli
2016-07-18 21:36 ` Andrew Cooper
2016-07-19 7:14 ` Dario Faggioli
2016-07-18 10:28 ` Dario Faggioli
2016-07-25 11:17 ` George Dunlap
2016-07-25 14:36 ` Meng Xu
2016-07-26 9:17 ` Dario Faggioli
2016-07-25 14:35 ` Meng Xu
2016-08-01 10:40 ` Jan Beulich
2016-08-01 12:32 ` Dario Faggioli
2016-08-05 13:24 ` Jan Beulich
2016-08-05 14:09 ` Dario Faggioli
2016-08-05 14:44 ` Jan Beulich
2016-08-11 14:59 ` Dario Faggioli
2016-08-11 15:51 ` Andrew Cooper
2016-08-11 23:35 ` Dario Faggioli
2016-08-12 1:59 ` dependences for backporting to 4.6 [was: Re: [PATCH 2/3] xen: Have schedulers revise initial placement] Dario Faggioli
2016-08-12 13:53 ` Jan Beulich
2016-08-16 10:21 ` Dario Faggioli
2016-08-16 11:21 ` Jan Beulich
2016-08-12 8:58 ` dependences for backporting to 4.5 " Dario Faggioli
2016-07-15 18:02 ` [PATCH 3/3] xen: Remove buggy initial placement algorithm George Dunlap
2016-07-15 18:10 ` Andrew Cooper
2016-07-16 13:55 ` Dario Faggioli
2016-07-18 10:03 ` George Dunlap
2016-07-16 15:48 ` [PATCH 1/3] xen: Some code motion to avoid having to do forward-declaration Meng Xu
2016-07-18 9:58 ` Dario Faggioli
2016-07-18 10:06 ` George Dunlap
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1468605722-24239-2-git-send-email-george.dunlap@citrix.com \
--to=george.dunlap@citrix.com \
--cc=anshul.makkar@citrix.com \
--cc=dario.faggioli@citrix.com \
--cc=jbeulich@suse.com \
--cc=mengxu@cis.upenn.edu \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).