From: Dario Faggioli <dario.faggioli@citrix.com>
To: xen-devel <xen-devel@lists.xenproject.org>
Cc: george.dunlap@eu.citrix.com,
Andrew Cooper <andrew.cooper3@citrix.com>,
keir@xen.org, Jan Beulich <JBeulich@suse.com>
Subject: [PATCH 0/2] Credit2: fix per-socket runqueue setup
Date: Fri, 22 Aug 2014 19:15:26 +0200 [thread overview]
Message-ID: <20140822165628.32764.15082.stgit@Solace.lan> (raw)
Hi everyone,
Code is already there in the credit2 scheduler, for setting up one runqueue per
each socket found in the system, despite the fact that comments in
sched_credit2.c still report this (well, something like this )to be a TODO
item.
However, such code is not functional. In fact, on the following host:
cpu_topology :
cpu: core socket node
0: 0 1 0
1: 0 1 0
2: 1 1 0
3: 1 1 0
4: 2 1 0
5: 2 1 0
6: 8 1 0
7: 8 1 0
8: 9 1 0
9: 9 1 0
10: 10 1 0
11: 10 1 0
12: 0 0 1
13: 0 0 1
14: 1 0 1
15: 1 0 1
16: 2 0 1
17: 2 0 1
18: 8 0 1
19: 8 0 1
20: 9 0 1
21: 9 0 1
22: 10 0 1
23: 10 0 1
numa_info :
node: memsize memfree distances
0: 18432 17488 10,20
1: 19200 17932 20,10
What I see during boot and scheduler initialization is as follows (for all the
24 pCPUs):
(XEN) init_pcpu: Strange, cpu 1 already initialized!
(XEN) init_pcpu: Strange, cpu 2 already initialized!
...
(XEN) init_pcpu: Strange, cpu 23 already initialized!
Also, once the system booted, here's what happened:
root@tg03:~# xl dmesg |grep -i runqueue
(XEN) Adding cpu 0 to runqueue 0
(XEN) First cpu on runqueue, activating
(XEN) Adding cpu 1 to runqueue 0
(XEN) Adding cpu 2 to runqueue 0
...
(XEN) Adding cpu 23 to runqueue 0
I.e., only one runqueue was created, despite the host having 2 sockets, and all
the pCPUs have been assigned to it.
This series fixes that by doing the following three things:
- anticipate the identification of the topology of the boot CPU to happen
_before_ the scheduler's initialization. This is necessary to make sure that,
when credit2 will try to figure out on which socket CPU 0 it, it will find the
correct info (patch 1);
- properly initialize the cpuinfo_x86 data structure, and more specifically the
phys_proc_id field, that hosts the socket, to something invalid (-1). In fact,
having that field in all elements set to 0 would induce credit2 to think that
the pCPU have already been initialized, and that all are on socket 0 (patch 1);
- use the info made available by early boot CPU identification in credit2
(patch 2).
After this series is applied, no more "Strange, cpu X already initialized!" can
be found in the logs. Also, here's how the scheduler is being setup:
root@tg03:~# xl dmesg |grep -i runqueue
(XEN) Adding cpu 0 to runqueue 1
(XEN) First cpu on runqueue, activating
(XEN) Adding cpu 1 to runqueue 1
(XEN) Adding cpu 2 to runqueue 1
(XEN) Adding cpu 3 to runqueue 1
(XEN) Adding cpu 4 to runqueue 1
(XEN) Adding cpu 5 to runqueue 1
(XEN) Adding cpu 6 to runqueue 1
(XEN) Adding cpu 7 to runqueue 1
(XEN) Adding cpu 8 to runqueue 1
(XEN) Adding cpu 9 to runqueue 1
(XEN) Adding cpu 10 to runqueue 1
(XEN) Adding cpu 11 to runqueue 1
(XEN) Adding cpu 12 to runqueue 0
(XEN) First cpu on runqueue, activating
(XEN) Adding cpu 13 to runqueue 0
(XEN) Adding cpu 14 to runqueue 0
(XEN) Adding cpu 15 to runqueue 0
(XEN) Adding cpu 16 to runqueue 0
(XEN) Adding cpu 17 to runqueue 0
(XEN) Adding cpu 18 to runqueue 0
(XEN) Adding cpu 19 to runqueue 0
(XEN) Adding cpu 20 to runqueue 0
(XEN) Adding cpu 21 to runqueue 0
(XEN) Adding cpu 22 to runqueue 0
(XEN) Adding cpu 23 to runqueue 0
Which makes a lot more sense. :-)
Regards,
Dario
---
Dario Faggioli (2):
x86: during boot, anticipate identifying the boot cpu
sched: credit2: use boot CPU info for CPU #0
xen/arch/x86/setup.c | 8 ++++++--
xen/arch/x86/smpboot.c | 3 ++-
xen/common/sched_credit2.c | 12 +++++-------
xen/include/asm-x86/processor.h | 6 ++++--
4 files changed, 17 insertions(+), 12 deletions(-)
--
s happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
next reply other threads:[~2014-08-22 17:15 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-08-22 17:15 Dario Faggioli [this message]
2014-08-22 17:15 ` [PATCH 1/2] x86: during boot, anticipate identifying the boot cpu Dario Faggioli
2014-08-22 17:28 ` Andrew Cooper
2014-08-22 18:40 ` Dario Faggioli
2014-08-25 8:35 ` Jan Beulich
2014-08-25 8:39 ` Jan Beulich
2014-09-01 15:12 ` George Dunlap
2014-09-01 15:24 ` Jan Beulich
2014-08-22 17:15 ` [PATCH 2/2] sched: credit2: use boot CPU info for CPU #0 Dario Faggioli
2014-08-25 8:41 ` Jan Beulich
2014-08-25 8:31 ` [PATCH 0/2] Credit2: fix per-socket runqueue setup Jan Beulich
2014-09-01 13:59 ` George Dunlap
2014-09-02 16:46 ` Dario Faggioli
2014-09-03 10:00 ` George Dunlap
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20140822165628.32764.15082.stgit@Solace.lan \
--to=dario.faggioli@citrix.com \
--cc=JBeulich@suse.com \
--cc=andrew.cooper3@citrix.com \
--cc=george.dunlap@eu.citrix.com \
--cc=keir@xen.org \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.