From: Dario Faggioli <dfaggioli@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
Andrew Cooper <andrew.cooper3@citrix.com>,
George Dunlap <george.dunlap@citrix.com>,
Jan Beulich <jbeulich@suse.com>, Paul Durrant <paul@xen.org>
Subject: Re: [PATCH 0/2] xen: credit2: limit the number of CPUs per runqueue
Date: Fri, 29 May 2020 17:06:46 +0200 [thread overview]
Message-ID: <60c7224bf3f502e62a21deeb4d45f0aec9cdd943.camel@suse.com> (raw)
In-Reply-To: <ab810b293ca8324ca3fba22476401a58435243fa.camel@suse.com>
[-- Attachment #1: Type: text/plain, Size: 5283 bytes --]
On Fri, 2020-05-29 at 13:46 +0200, Dario Faggioli wrote:
> Basically, if we just consider patches 1 and 4 we will end up, right
> after boot, with a system that has smaller runqueues.
>
Actually, to be fully precise, given how I reorganized the series, it's
not patches 1 and 4, it's patches 1, 3 and 4.
Hopefully, that is not a big deal, but it patch 3 is really a problem,
I can re-arrange patch 4 for working without it.
Apart from this, and for adding more information, on a system with 96
CPUs in 2 sockets, this is how the runqueues looks like (with these
patches:
(XEN) Online Cpus: 0-95
(XEN) Cpupool 0:
(XEN) Cpus: 0-95
(XEN) Scheduling granularity: cpu, 1 CPU per sched-resource
(XEN) Scheduler: SMP Credit Scheduler rev2 (credit2)
(XEN) Active queues: 6
(XEN) default-weight = 256
(XEN) Runqueue 0:
(XEN) ncpus = 16
(XEN) cpus = 0-15
(XEN) max_weight = 256
(XEN) pick_bias = 0
(XEN) instload = 0
(XEN) aveload = 1223 (~0%)
(XEN) idlers: 00000000,00000000,00000000,00000000,00000000,00000000,0000ffff
(XEN) tickled: 00000000,00000000,00000000,00000000,00000000,00000000,00000000
(XEN) fully idle cores: 00000000,00000000,00000000,00000000,00000000,00000000,0000ffff
(XEN) Runqueue 1:
(XEN) ncpus = 16
(XEN) cpus = 16-31
(XEN) max_weight = 256
(XEN) pick_bias = 16
(XEN) instload = 0
(XEN) aveload = 3324 (~1%)
(XEN) idlers: 00000000,00000000,00000000,00000000,00000000,00000000,ffff0000
(XEN) tickled: 00000000,00000000,00000000,00000000,00000000,00000000,00000000
(XEN) fully idle cores: 00000000,00000000,00000000,00000000,00000000,00000000,ffff0000
(XEN) Runqueue 2:
(XEN) ncpus = 16
(XEN) cpus = 32-47
(XEN) max_weight = 256
(XEN) pick_bias = 32
(XEN) instload = 1
(XEN) aveload = 8996 (~3%)
(XEN) idlers: 00000000,00000000,00000000,00000000,00000000,0000feff,00000000
(XEN) tickled: 00000000,00000000,00000000,00000000,00000000,00000000,00000000
(XEN) fully idle cores: 00000000,00000000,00000000,00000000,00000000,0000fcff,00000000
(XEN) Runqueue 3:
(XEN) ncpus = 16
(XEN) cpus = 48-63
(XEN) max_weight = 256
(XEN) pick_bias = 48
(XEN) instload = 0
(XEN) aveload = 2424 (~0%)
(XEN) idlers: 00000000,00000000,00000000,00000000,00000000,ffff0000,00000000
(XEN) tickled: 00000000,00000000,00000000,00000000,00000000,00000000,00000000
(XEN) fully idle cores: 00000000,00000000,00000000,00000000,00000000,ffff0000,00000000
(XEN) Runqueue 4:
(XEN) ncpus = 16
(XEN) cpus = 64-79
(XEN) max_weight = 256
(XEN) pick_bias = 66
(XEN) instload = 0
(XEN) aveload = 1070 (~0%)
(XEN) idlers: 00000000,00000000,00000000,00000000,0000ffff,00000000,00000000
(XEN) tickled: 00000000,00000000,00000000,00000000,00000000,00000000,00000000
(XEN) fully idle cores: 00000000,00000000,00000000,00000000,0000ffff,00000000,00000000
(XEN) Runqueue 5:
(XEN) ncpus = 16
(XEN) cpus = 80-95
(XEN) max_weight = 256
(XEN) pick_bias = 82
(XEN) instload = 0
(XEN) aveload = 425 (~0%)
(XEN) idlers: 00000000,00000000,00000000,00000000,ffff0000,00000000,00000000
(XEN) tickled: 00000000,00000000,00000000,00000000,00000000,00000000,00000000
(XEN) fully idle cores: 00000000,00000000,00000000,00000000,ffff0000,00000000,00000000
Without the patches, there would be just 2 of them (on with CPUs 0-47
and another with CPUs 48-95).
On a system with "just" 16 CPUs, in 2 sockets, they look like this:
(XEN) Online Cpus: 0-15
(XEN) Cpupool 0:
(XEN) Cpus: 0-15
(XEN) Scheduling granularity: cpu, 1 CPU per sched-resource
(XEN) Scheduler: SMP Credit Scheduler rev2 (credit2)
(XEN) Active queues: 2
(XEN) default-weight = 256
(XEN) Runqueue 0:
(XEN) ncpus = 8
(XEN) cpus = 0-7
(XEN) max_weight = 256
(XEN) pick_bias = 0
(XEN) instload = 0
(XEN) aveload = 7077 (~2%)
(XEN) idlers: 00000000,000000ff
(XEN) tickled: 00000000,00000000
(XEN) fully idle cores: 00000000,000000ff
(XEN) Runqueue 1:
(XEN) ncpus = 8
(XEN) cpus = 8-15
(XEN) max_weight = 256
(XEN) pick_bias = 8
(XEN) instload = 1
(XEN) aveload = 11848 (~4%)
(XEN) idlers: 00000000,0000fe00
(XEN) tickled: 00000000,00000000
(XEN) fully idle cores: 00000000,0000fc00
There are still 2, because there are 2 sockets, and we still honor the
topology (and 8 CPUs in a runqueue is fine, because is lower than 16).
I'll share the same output on a 256 CPU system, as soon as I finish
installing it.
Regards
--
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
next prev parent reply other threads:[~2020-05-29 15:07 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-04-29 17:36 [PATCH 0/2] xen: credit2: limit the number of CPUs per runqueue Dario Faggioli
2020-04-29 17:36 ` [PATCH 1/2] xen: credit2: factor cpu to runqueue matching in a function Dario Faggioli
2020-04-29 17:36 ` [PATCH 2/2] xen: credit2: limit the max number of CPUs in a runqueue Dario Faggioli
2020-04-30 6:45 ` Jan Beulich
2020-05-26 22:00 ` Dario Faggioli
2020-05-27 4:26 ` Jürgen Groß
2020-05-28 9:32 ` Dario Faggioli
2020-05-27 6:17 ` Jan Beulich
2020-05-28 14:55 ` Dario Faggioli
2020-05-29 9:58 ` Jan Beulich
2020-05-29 10:19 ` Dario Faggioli
2020-04-30 7:35 ` Jürgen Groß
2020-04-30 12:28 ` Dario Faggioli
2020-04-30 12:52 ` Jürgen Groß
2020-04-30 14:01 ` Dario Faggioli
2020-05-26 21:18 ` Dario Faggioli
2020-05-27 6:22 ` Jan Beulich
2020-05-28 9:36 ` Dario Faggioli
2020-05-29 11:46 ` [PATCH 0/2] xen: credit2: limit the number of CPUs per runqueue Dario Faggioli
2020-05-29 15:06 ` Dario Faggioli [this message]
2020-05-29 16:15 ` George Dunlap
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=60c7224bf3f502e62a21deeb4d45f0aec9cdd943.camel@suse.com \
--to=dfaggioli@suse.com \
--cc=andrew.cooper3@citrix.com \
--cc=george.dunlap@citrix.com \
--cc=jbeulich@suse.com \
--cc=jgross@suse.com \
--cc=paul@xen.org \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).