* [PATCH v2 0/3 for 4.8] docs: feature documents for the schedulers
@ 2016-10-14 9:59 Dario Faggioli
2016-10-14 10:00 ` [PATCH v2 1/3] docs: Credit1 feature document Dario Faggioli
` (3 more replies)
0 siblings, 4 replies; 10+ messages in thread
From: Dario Faggioli @ 2016-10-14 9:59 UTC (permalink / raw)
To: xen-devel
Cc: Lars Kurth, Stefano Stabellini, Wei Liu, George Dunlap,
Andrew Cooper, Anshul Makkar, Ian Jackson, Meng Xu, Jan Beulich
Hi,
Here's v2.
The differences are:
- review comments taken into account.
- I'm *NOT* claiming 'SuppOrted' status for Credit2. I'll do that in a
separate patch.
v1 is here:
https://lists.xen.org/archives/html/xen-devel/2016-10/msg00857.html
And there is a git branch for this new version:
git://xenbits.xen.org/people/dariof/xen.git rel/sched/feature-docs
http://xenbits.xen.org/gitweb/?p=people/dariof/xen.git;a=shortlog;h=refs/heads/rel/sched/feature-docs
Thanks and Regards,
Dario
---
Dario Faggioli (3):
docs: Credit1 feature document.
docs: Credit2 feature document.
docs: RTDS feature document.
docs/features/sched_credit.pandoc | 97 +++++++++++++++++++++++++++++
docs/features/sched_credit2.pandoc | 118 +++++++++++++++++++++++++++++++++++
docs/features/sched_rtds.pandoc | 121 ++++++++++++++++++++++++++++++++++++
3 files changed, 336 insertions(+)
create mode 100644 docs/features/sched_credit.pandoc
create mode 100644 docs/features/sched_credit2.pandoc
create mode 100644 docs/features/sched_rtds.pandoc
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v2 1/3] docs: Credit1 feature document.
2016-10-14 9:59 [PATCH v2 0/3 for 4.8] docs: feature documents for the schedulers Dario Faggioli
@ 2016-10-14 10:00 ` Dario Faggioli
2016-10-14 10:01 ` [PATCH v2 2/3] docs: Credit2 " Dario Faggioli
` (2 subsequent siblings)
3 siblings, 0 replies; 10+ messages in thread
From: Dario Faggioli @ 2016-10-14 10:00 UTC (permalink / raw)
To: xen-devel
Cc: Lars Kurth, Stefano Stabellini, Wei Liu, George Dunlap,
Andrew Cooper, Ian Jackson, Jan Beulich
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
---
Cc: George Dunlap <George.Dunlap@eu.citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
Cc: Lars Kurth <lars.kurth@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>
---
Changes from v1:
* file renamed from credit.pandoc to sched_credit.pandoc;
* feature name is now 'Credit Scheduler';
* kill the 'e.g.'-s in the header (sorry!);
* removed 'Architecture' line from header, as this is arch independent;
* removed some text that was duplicated in other sched feature documents;
it's redundant wrt what's in the wiki about schedulers, and the page
is referenced.
---
docs/features/sched_credit.pandoc | 97 +++++++++++++++++++++++++++++++++++++
1 file changed, 97 insertions(+)
create mode 100644 docs/features/sched_credit.pandoc
diff --git a/docs/features/sched_credit.pandoc b/docs/features/sched_credit.pandoc
new file mode 100644
index 0000000..79587d8
--- /dev/null
+++ b/docs/features/sched_credit.pandoc
@@ -0,0 +1,97 @@
+% Credit Scheduler
+% Revision 1
+
+\clearpage
+
+# Basics
+---------------- ----------------------------------------------------
+ Status: **Supported**
+
+ Component: Hypervisor
+---------------- ----------------------------------------------------
+
+# Overview
+
+Credit (also known as Credit1) is the default virtual CPU (vCPU) scheduler
+of the Xen hypervisor.
+
+It is a general purpose, weighted fair-share scheduler.
+
+# User details
+
+Xen supports multiple schedulers. As said, Credit is the default, so it
+is used automatically, unless the `sched=$SCHED` (with `$SCHED` different
+than `credit`) parameter is passed to Xen via the bootloader.
+
+Once the system is live, for creating a cpupool with Credit as its
+scheduler, either compile a cpupool configuration file, as described
+in `docs/man/xlcpupool.cfg.pod.5` (and as exemplified in
+`tools/examples/cpupool`), or use just `xl` directly:
+
+ xl cpupool-create name=\"pool1\" sched=\"credit\" cpus=[4,8]
+
+Two kind of interactions with the scheduler are possible:
+
+* checking or changing the global parameters, via, e.g.:
+ * `xl sched-credit -s`
+ * `xl sched-credit -s -p pool1`
+ * `xl sched-credit -s -t 20`
+* checking or changing a VM's scheduling parameters, via, e.g.:
+ * `xl sched-credit -d vm1`
+ * `xl sched-credit -d vm1 -w 512`
+
+# Technical details
+
+Implementation entirely lives in the hypervisor. Xen has a pluggable,
+hook based, architecture for schedulers. Thanks to this, Credit code
+is all contained in `xen/common/sched_credit.c`.
+
+# Limitations
+
+In Credit, a vCPU has a priority, a status (i.e., active or inactive),
+a weight and some credits... and all these things interact in a rather
+involved way. Also, with years of use, things have gotten even more
+complex (due to, e.g., the introduction of boosting, caps and vCPU
+soft-affinity).
+
+Dealing with such complexity is starting to be an issue. Odd behavior
+or subtle scheduling anomalies, that is not always possible to act upon,
+have been identified already. [1][2][3]
+
+A certain lack of scalability and difficulties and weakness in dealing
+with mixed workloads and VMs with low latency requirements are other
+known problems. [4] For all these reasons, effort is ongoing to have
+Credit2 become the new default scheduler.
+
+# Testing
+
+Any change to Credit code must to be tested by doing at least the following:
+
+* create a few virtual machine and verify that they boot and can
+ run some basic workload (e.g., login into them and run simple commands),
+* shutdown/reboot the virtual machines,
+* shutdown the system.
+
+Ideally, all the above steps should **also** be performed in a configuration
+that includes cpupools, better if with pools using different schedulers, and
+by also doing the following:
+
+* move the virtual machines between cpupools.
+
+# References
+
+* [potential non-ideal behavior on hyperthreaded systems](https://lists.xenproject.org/archives/html/xen-devel/2014-07/msg01848.html) [1]
+* [long standing BOOST vs. migration bug](https://lists.xen.org/archives/html/xen-devel/2015-10/msg02851.html) [2]
+* [priority handling issues](https://lists.xenproject.org/archives/html/xen-devel/2016-05/msg01362.html) [3]
+* "Scheduler development update", XenSummit Asia 2009 [whitepaper](http://www-archive.xenproject.org/files/xensummit_intel09/George_Dunlap.pdf) [4]
+* "Scheduling in Xen" [XPDS15 Presentation](http://events.linuxfoundation.org/sites/events/files/slides/Faggioli_XenSummit.pdf)
+* "The Credit Scheduler" [on the Xen-Project wiki](https://wiki.xenproject.org/wiki/Credit_Scheduler)
+* "Xen Project Schedulers" [on the Xen-Project wiki](https://wiki.xenproject.org/wiki/Xen_Project_Schedulers)
+
+# History
+
+------------------------------------------------------------------------
+Date Revision Version Notes
+---------- -------- -------- -------------------------------------------
+2016-10-14 1 Xen 4.8 Document written
+---------- -------- -------- -------------------------------------------
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v2 2/3] docs: Credit2 feature document.
2016-10-14 9:59 [PATCH v2 0/3 for 4.8] docs: feature documents for the schedulers Dario Faggioli
2016-10-14 10:00 ` [PATCH v2 1/3] docs: Credit1 feature document Dario Faggioli
@ 2016-10-14 10:01 ` Dario Faggioli
2016-10-14 10:02 ` [PATCH v2 3/3] docs: RTDS " Dario Faggioli
2016-10-14 13:24 ` [PATCH v2 0/3 for 4.8] docs: feature documents for the schedulers Wei Liu
3 siblings, 0 replies; 10+ messages in thread
From: Dario Faggioli @ 2016-10-14 10:01 UTC (permalink / raw)
To: xen-devel
Cc: Lars Kurth, Stefano Stabellini, Wei Liu, George Dunlap,
Andrew Cooper, Anshul Makkar, Ian Jackson, Jan Beulich
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
---
Cc: George Dunlap <George.Dunlap@eu.citrix.com>
Cc: Anshul Makkar <anshul.makkar@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
Cc: Lars Kurth <lars.kurth@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>
---
Changes from v1:
* don't claim 'Supported' status;
* file renamed from credit2.pandoc to sched_credit2.pandoc;
* feature name is now 'Credit2 Scheduler';
* kill the 'e.g.'-s in the header (sorry!);
* removed 'Architecture' line from header, as this is arch independent;
* removed some text that was duplicated in other sched feature documents;
it's redundant wrt what's in the wiki about schedulers, and the page
is referenced.
---
docs/features/sched_credit2.pandoc | 118 ++++++++++++++++++++++++++++++++++++
1 file changed, 118 insertions(+)
create mode 100644 docs/features/sched_credit2.pandoc
diff --git a/docs/features/sched_credit2.pandoc b/docs/features/sched_credit2.pandoc
new file mode 100644
index 0000000..8609d9c
--- /dev/null
+++ b/docs/features/sched_credit2.pandoc
@@ -0,0 +1,118 @@
+% Credit2 Scheduler
+% Revision 1
+
+\clearpage
+
+# Basics
+---------------- ----------------------------------------------------
+ Status: **Experimental**
+
+ Component: Hypervisor
+---------------- ----------------------------------------------------
+
+# Overview
+
+Credit2 is one of the virtual CPU (vCPU) scheduler available in the
+Xen hypervisor.
+
+Credit2 was designed as a general purpose scheduler, with particular
+focus on improving handling of mixed workloads, scalability and
+support for low latency applications inside VMs, with respect to
+Credit1.
+
+# User details
+
+Credit2 is not in use by default. In order to use it as the Xen
+scheduler the following parameter should be passed to the hypervisor
+at boot:
+
+ `sched=credit2`
+
+Other parameters are available for tuning the behavior of Credit2
+(see `docs/misc/xen-command-line.markdown` for a complete list and
+for their meaning).
+
+Once the system is live, for creating a cpupool with Credit2 as
+its scheduler, either compile a cpupool configuration file, as
+described in `docs/man/xlcpupool.cfg.pod.5` (and as exemplified
+in `tools/examples/cpupool`), or use just `xl` directly:
+
+ xl cpupool-create name=\"pool1\" sched=\"credit2\" cpus=[1,2]
+
+Two kind of interactions with the scheduler are possible:
+
+* checking or changing the global parameters, via, e.g.:
+ * `xl sched-credit2 -s`
+ * `xl sched-credit2 -s -p pool1`
+ * `xl sched-credit2 -s -r 100`
+* checking or changing a VM scheduling parameters, via, e.g.:
+ * `xl sched-credit2 -d vm1`
+ * `xl sched-credit2 -d vm1 -w 1024`
+
+# Technical details
+
+Implementation entirely lives in the hypervisor. Xen has a pluggable,
+hook based, architecture for schedulers. Thanks to this, Credit2 code
+is all contained in `xen/common/sched_credit2.c`.
+
+Global scheduling parameters, such as context switching rate
+limiting, is only available from Xen 4.8 onward. In libxl, the
+`LIBXL\_HAVE\_SCHED\_CREDIT2\_PARAMS` symbol is introduced to
+indicate their availability.
+
+# Limitations
+
+The Credit1 scheduler comes with vCPU hard-affinity, vCPU soft-affinity
+and caps (see `docs/man/xl.pod.1.in` for more details). In Credit2,
+vCPU hard affinity is supported starting from Xen 4.8, while soft-affinity
+and caps, while being worked on, are not yet available in any released
+hypervisor.
+
+# Testing
+
+Any change done in Credit2 wants to be tested by doing at least the
+following:
+
+* boot the system with `sched=credit2`,
+* create a few virtual machine and verify that they boot and can
+ run some basic workload (e.g., login into them and run simple commands),
+* shutdown/reboot the virtual machines,
+* shutdown/reboot the system.
+
+Ideally, all the above steps should **also** be performed in a configuration
+where Credit2 is used as the scheduler of a cpupool, and by also doing the
+following:
+
+* move a virtual machine inside and outside a Credit2 cpupool.
+
+# Areas for improvement
+
+* Close the feature gap with Credit1 (i.e., finishing implementing vCPU
+ soft-affinity and caps);
+* vCPUs' reservations (similar to caps, but providing a vCPU with guarantees
+ about some pCPU time it will always be able to execute for);
+* benchmarking for assessing the best combination of values for the various
+ parameters (`sched\_credit2\_migrate\_resist`, `credit2\_balance\_over`,
+ `credit2\_balance\_under`)
+
+# Known issues
+
+* I/O oriented benchmarks (like network and disk throughput) have given
+ contradictory and non-conclusive results so far. Need to run more of
+ those.
+
+# References
+
+* "Scheduler development update", XenSummit Asia 2009 [whitepaper](http://www-archive.xenproject.org/files/xensummit_intel09/George_Dunlap.pdf)
+* "Scheduling in Xen" [XPDS15 Presentation](http://events.linuxfoundation.org/sites/events/files/slides/Faggioli_XenSummit.pdf)
+* "Scope and Performance of Credit-2 Scheduler" [XPDS16 Presentation](http://www.slideshare.net/xen_com_mgr/xpds16-scope-and-performance-of-credit2-scheduler-anshul-makkar-ctirix-systems-uk-ltd)
+* "The Credit2 Scheduler" [on the Xen-Project wiki](https://wiki.xenproject.org/wiki/Credit2_Scheduler_Development)
+* "Xen Project Schedulers" [on the Xen-Project wiki](https://wiki.xenproject.org/wiki/Xen_Project_Schedulers)
+
+# History
+
+------------------------------------------------------------------------
+Date Revision Version Notes
+---------- -------- -------- -------------------------------------------
+2016-10-14 1 Xen 4.8 Document written
+---------- -------- -------- -------------------------------------------
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v2 3/3] docs: RTDS feature document.
2016-10-14 9:59 [PATCH v2 0/3 for 4.8] docs: feature documents for the schedulers Dario Faggioli
2016-10-14 10:00 ` [PATCH v2 1/3] docs: Credit1 feature document Dario Faggioli
2016-10-14 10:01 ` [PATCH v2 2/3] docs: Credit2 " Dario Faggioli
@ 2016-10-14 10:02 ` Dario Faggioli
2016-10-18 17:49 ` Meng Xu
2016-10-14 13:24 ` [PATCH v2 0/3 for 4.8] docs: feature documents for the schedulers Wei Liu
3 siblings, 1 reply; 10+ messages in thread
From: Dario Faggioli @ 2016-10-14 10:02 UTC (permalink / raw)
To: xen-devel
Cc: Lars Kurth, Stefano Stabellini, Wei Liu, George Dunlap,
Andrew Cooper, Ian Jackson, Meng Xu, Jan Beulich
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
---
Cc: Meng Xu <mengxu@cis.upenn.edu>
Cc: George Dunlap <George.Dunlap@eu.citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
Cc: Lars Kurth <lars.kurth@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>
---
Changes from v1:
* file renamed from rtds.pandoc to sched_rtds.pandoc;
* feature name is now 'RTDS Scheduler';
* kill the 'e.g.'-s in the header (sorry!);
* removed 'Architecture' line from header, as this is arch independent;
* removed some text that was duplicated in other sched feature documents;
it's redundant wrt what's in the wiki about schedulers, and the page
is referenced.
---
docs/features/sched_rtds.pandoc | 121 +++++++++++++++++++++++++++++++++++++++
1 file changed, 121 insertions(+)
create mode 100644 docs/features/sched_rtds.pandoc
diff --git a/docs/features/sched_rtds.pandoc b/docs/features/sched_rtds.pandoc
new file mode 100644
index 0000000..2843d97
--- /dev/null
+++ b/docs/features/sched_rtds.pandoc
@@ -0,0 +1,121 @@
+% RTDS Scheduler
+% Revision 1
+
+\clearpage
+
+# Basics
+---------------- ----------------------------------------------------
+ Status: **Experimental**
+
+ Component: Hypervisor
+---------------- ----------------------------------------------------
+
+# Overview
+
+RTDS is one of the virtual CPU (vCPU) scheduler available in the Xen
+hypervisor.
+
+RTDS is a real--time scheduler, so its purpose is enabling
+**deterministic** scheduling of the virtual machine's vCPUs. It has
+been originally developed in the context of the RT-Xen project.
+
+# User details
+
+RTDS is not in use by default. In order to use it as the Xen scheduler
+the following parameter should be passed to the hypervisor at boot:
+
+ `sched=rtds`
+
+Once the system is live, for creating a cpupool with RTDS as its
+scheduler, either compile a cpupool configuration file, as described
+in `docs/man/xlcpupool.cfg.pod.5` (and as exemplified in
+`tools/examples/cpupool`), or use just `xl` directly:
+
+ xl cpupool-create name=\"pool-rt\" sched=\"rtds\" cpus=[4,5,6,8]
+
+For checking or changing a VM's scheduling parameters from xl, do
+as follows:
+ * `xl sched-rtds -d vm-rt`
+ * `xl sched-rtds -d vm-rt -t 10000 -b 25000`
+
+It is possible, for a multiple vCPUs VM, to change the parameters of
+each vCPU individually:
+ * `xl sched-rtds -d vm-rt -v 0 -p 20000 -b 10000 -v 1 -p 45000 -b 12000`
+
+# Technical details
+
+Implementation entirely lives in the hypervisor. Xen has a pluggable,
+hook based, architecture for schedulers. Thanks to this, RTDS code
+is all contained in `xen/common/sched_rtds.c`.
+
+In libxl, the availability of the RTDS scheduler is advertised by
+the presence of the LIBXL\_HAVE\_SCHED\_RTDS symbol. The ability of
+specifying different scheduling parameters for each vcpu has been
+introduced later, and is available if the following symbols are defined:
+ * `LIBXL\_HAVE\_VCPU\_SCHED\_PARAMS`,
+ * `LIBXL\_HAVE\_SCHED\_RTDS\_VCPU\_PARAMS`.
+
+# Limitations
+
+RTDS is a special purpose scheduling. This is by design, and not at
+all a limitation, but it is certainly something to keep in mind when
+thinking about using it. The purpose of the scheduler is enabling
+deterministic and statically analyzable behavior (as per the
+real-time academic literature), according to the scheduling parameters
+assigned to each vCPU.
+
+Using RTDS a the Xen scheduler, and/or for general purpose workloads
+is definitely possible, but the vCPU scheduling parameters (of both
+Domain0 and of the various VMs) would probably require tweaking, with
+respect to their default values.
+
+# Testing
+
+Any change done in RTDS must be tested by doing the following:
+
+* create a cpupool with RTDS as its scheduler,
+* create a few virtual machines a move them in and out of the pool,
+* create a few virtual machines, directly inside the pool, and verify
+ that they boot and can run some basic workload (e.g., login into them
+ and run simple commands),
+* shutdown/reboot the virtual machines,
+
+The fact that the system boots fine when passing `sched=rtds` to Xen
+should also be verified.
+
+Finally, to check that the scheduler is working properly (although only
+at a macroscopic level), the following should be done:
+
+* create a VM with 1 vCPU and put it in the RTDS cpupool,
+* set the scheduling parameters such as it has a 50% reservation, with
+ `xl sched-rtds -d vm -t 100000 -b 50000`,
+* run a CPU-burning process inside the VM (e.g., `yes`),
+* check with `xentop` (in Domain0) that the VM is getting no more than
+ 50% pCPU time.
+
+# Areas for improvement
+
+* Work-conserving mode to be added;
+* performance assessment, especially focusing on what level of real-time
+ behavior the scheduler enables.
+
+# Known issues
+
+* OSSTest reports occasional failures on ARM.
+
+# References
+
+* "RT-Xen: Real-Time Virtualization" [XPDS14 Presentation](http://events.linuxfoundation.org/sites/events/files/slides/2014_Xen_Developer_Summit_0.pdf)
+* "Scheduling in Xen" [XPDS15 Presentation](http://events.linuxfoundation.org/sites/events/files/slides/Faggioli_XenSummit.pdf)
+* [RT-Xen Project](https://sites.google.com/site/realtimexen/)
+* [RTDS-Based-Scheduler](https://wiki.xenproject.org/wiki/RTDS-Based-Scheduler)
+* "The RTDS Scheduler" [on the Xen-Project wiki](https://wiki.xenproject.org/wiki/RTDS-Based-Scheduler)
+* "Xen Project Schedulers" [on the Xen-Project wiki](https://wiki.xenproject.org/wiki/Xen_Project_Schedulers)
+
+# History
+
+------------------------------------------------------------------------
+Date Revision Version Notes
+---------- -------- -------- -------------------------------------------
+2016-10-14 1 Xen 4.8 Document written
+---------- -------- -------- -------------------------------------------
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH v2 0/3 for 4.8] docs: feature documents for the schedulers
2016-10-14 9:59 [PATCH v2 0/3 for 4.8] docs: feature documents for the schedulers Dario Faggioli
` (2 preceding siblings ...)
2016-10-14 10:02 ` [PATCH v2 3/3] docs: RTDS " Dario Faggioli
@ 2016-10-14 13:24 ` Wei Liu
2016-10-14 15:03 ` Andrew Cooper
3 siblings, 1 reply; 10+ messages in thread
From: Wei Liu @ 2016-10-14 13:24 UTC (permalink / raw)
To: Dario Faggioli
Cc: Lars Kurth, Stefano Stabellini, Wei Liu, George Dunlap,
Andrew Cooper, Anshul Makkar, Ian Jackson, Meng Xu, Jan Beulich,
xen-devel
On Fri, Oct 14, 2016 at 10:59:55AM +0100, Dario Faggioli wrote:
> Hi,
>
> Here's v2.
>
> The differences are:
> - review comments taken into account.
> - I'm *NOT* claiming 'SuppOrted' status for Credit2. I'll do that in a
> separate patch.
>
> v1 is here:
>
> https://lists.xen.org/archives/html/xen-devel/2016-10/msg00857.html
>
> And there is a git branch for this new version:
>
> git://xenbits.xen.org/people/dariof/xen.git rel/sched/feature-docs
> http://xenbits.xen.org/gitweb/?p=people/dariof/xen.git;a=shortlog;h=refs/heads/rel/sched/feature-docs
>
> Thanks and Regards,
> Dario
> ---
> Dario Faggioli (3):
> docs: Credit1 feature document.
> docs: Credit2 feature document.
> docs: RTDS feature document.
>
All three:
Acked-by: Wei Liu <wei.liu2@citrix.com>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2 0/3 for 4.8] docs: feature documents for the schedulers
2016-10-14 13:24 ` [PATCH v2 0/3 for 4.8] docs: feature documents for the schedulers Wei Liu
@ 2016-10-14 15:03 ` Andrew Cooper
2016-10-14 17:10 ` Wei Liu
0 siblings, 1 reply; 10+ messages in thread
From: Andrew Cooper @ 2016-10-14 15:03 UTC (permalink / raw)
To: Wei Liu, Dario Faggioli
Cc: Lars Kurth, Stefano Stabellini, George Dunlap, Anshul Makkar,
Ian Jackson, Meng Xu, Jan Beulich, xen-devel
On 14/10/16 14:24, Wei Liu wrote:
> On Fri, Oct 14, 2016 at 10:59:55AM +0100, Dario Faggioli wrote:
>> Hi,
>>
>> Here's v2.
>>
>> The differences are:
>> - review comments taken into account.
>> - I'm *NOT* claiming 'SuppOrted' status for Credit2. I'll do that in a
>> separate patch.
>>
>> v1 is here:
>>
>> https://lists.xen.org/archives/html/xen-devel/2016-10/msg00857.html
>>
>> And there is a git branch for this new version:
>>
>> git://xenbits.xen.org/people/dariof/xen.git rel/sched/feature-docs
>> http://xenbits.xen.org/gitweb/?p=people/dariof/xen.git;a=shortlog;h=refs/heads/rel/sched/feature-docs
>>
>> Thanks and Regards,
>> Dario
>> ---
>> Dario Faggioli (3):
>> docs: Credit1 feature document.
>> docs: Credit2 feature document.
>> docs: RTDS feature document.
>>
> All three:
>
> Acked-by: Wei Liu <wei.liu2@citrix.com>
All LGTM as well.
~Andrew
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2 0/3 for 4.8] docs: feature documents for the schedulers
2016-10-14 15:03 ` Andrew Cooper
@ 2016-10-14 17:10 ` Wei Liu
0 siblings, 0 replies; 10+ messages in thread
From: Wei Liu @ 2016-10-14 17:10 UTC (permalink / raw)
To: Andrew Cooper
Cc: Lars Kurth, Stefano Stabellini, Wei Liu, George Dunlap,
Dario Faggioli, Ian Jackson, Meng Xu, Jan Beulich, xen-devel,
Anshul Makkar
On Fri, Oct 14, 2016 at 04:03:18PM +0100, Andrew Cooper wrote:
> On 14/10/16 14:24, Wei Liu wrote:
> > On Fri, Oct 14, 2016 at 10:59:55AM +0100, Dario Faggioli wrote:
> >> Hi,
> >>
> >> Here's v2.
> >>
> >> The differences are:
> >> - review comments taken into account.
> >> - I'm *NOT* claiming 'SuppOrted' status for Credit2. I'll do that in a
> >> separate patch.
> >>
> >> v1 is here:
> >>
> >> https://lists.xen.org/archives/html/xen-devel/2016-10/msg00857.html
> >>
> >> And there is a git branch for this new version:
> >>
> >> git://xenbits.xen.org/people/dariof/xen.git rel/sched/feature-docs
> >> http://xenbits.xen.org/gitweb/?p=people/dariof/xen.git;a=shortlog;h=refs/heads/rel/sched/feature-docs
> >>
> >> Thanks and Regards,
> >> Dario
> >> ---
> >> Dario Faggioli (3):
> >> docs: Credit1 feature document.
> >> docs: Credit2 feature document.
> >> docs: RTDS feature document.
> >>
> > All three:
> >
> > Acked-by: Wei Liu <wei.liu2@citrix.com>
>
> All LGTM as well.
>
Series pushed.
> ~Andrew
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2 3/3] docs: RTDS feature document.
2016-10-14 10:02 ` [PATCH v2 3/3] docs: RTDS " Dario Faggioli
@ 2016-10-18 17:49 ` Meng Xu
2016-10-18 17:55 ` Wei Liu
2016-10-18 21:53 ` Dario Faggioli
0 siblings, 2 replies; 10+ messages in thread
From: Meng Xu @ 2016-10-18 17:49 UTC (permalink / raw)
To: Dario Faggioli
Cc: Lars Kurth, Stefano Stabellini, Wei Liu, George Dunlap,
Andrew Cooper, Ian Jackson, Jan Beulich, xen-devel
Hi Dario,
Thank you very much for writing this nice document!
Sorry for the late reply...
I found two typos in the command example.
I saw the patch has been committed. Maybe I (or you) should send a
patch to correct the typo?
>
>
> +
> +For checking or changing a VM's scheduling parameters from xl, do
> +as follows:
> + * `xl sched-rtds -d vm-rt`
It should be
xl sched-rtds -d vm-rt -v all
>
> + * `xl sched-rtds -d vm-rt -t 10000 -b 25000`
There are three typos:
1) We don't support "-t" option.
2) We must specify -v in the latest Xen. (Once the per-vcpu parameter
is supported, we won't support setting a VM's parameter like this. The
toolstack will reports error.)
3) The budget 25000 is larger than the period.
It should be:
sudo xl sched-rtds -d vm-rt -v all -p 10000 -b 2500
>
> +
> +It is possible, for a multiple vCPUs VM, to change the parameters of
> +each vCPU individually:
> + * `xl sched-rtds -d vm-rt -v 0 -p 20000 -b 10000 -v 1 -p 45000 -b 12000`
This is correct.
>
> +
> +# Technical details
> +
> +Implementation entirely lives in the hypervisor. Xen has a pluggable,
> +hook based, architecture for schedulers. Thanks to this, RTDS code
> +is all contained in `xen/common/sched_rtds.c`.
> +
> +In libxl, the availability of the RTDS scheduler is advertised by
> +the presence of the LIBXL\_HAVE\_SCHED\_RTDS symbol. The ability of
> +specifying different scheduling parameters for each vcpu has been
> +introduced later, and is available if the following symbols are defined:
> + * `LIBXL\_HAVE\_VCPU\_SCHED\_PARAMS`,
> + * `LIBXL\_HAVE\_SCHED\_RTDS\_VCPU\_PARAMS`.
> +
> +# Limitations
> +
> +RTDS is a special purpose scheduling. This is by design, and not at
> +all a limitation, but it is certainly something to keep in mind when
> +thinking about using it. The purpose of the scheduler is enabling
> +deterministic and statically analyzable behavior (as per the
> +real-time academic literature), according to the scheduling parameters
> +assigned to each vCPU.
> +
> +Using RTDS a the Xen scheduler, and/or for general purpose workloads
> +is definitely possible, but the vCPU scheduling parameters (of both
> +Domain0 and of the various VMs) would probably require tweaking, with
> +respect to their default values.
> +
> +# Testing
> +
> +Any change done in RTDS must be tested by doing the following:
> +
> +* create a cpupool with RTDS as its scheduler,
> +* create a few virtual machines a move them in and out of the pool,
> +* create a few virtual machines, directly inside the pool, and verify
> + that they boot and can run some basic workload (e.g., login into them
> + and run simple commands),
> +* shutdown/reboot the virtual machines,
> +
> +The fact that the system boots fine when passing `sched=rtds` to Xen
> +should also be verified.
> +
> +Finally, to check that the scheduler is working properly (although only
> +at a macroscopic level), the following should be done:
> +
> +* create a VM with 1 vCPU and put it in the RTDS cpupool,
> +* set the scheduling parameters such as it has a 50% reservation, with
> + `xl sched-rtds -d vm -t 100000 -b 50000`,
-t here should be -p.
The -v all should be specified here.
Thanks,
Meng
-----------
Meng Xu
PhD Student in Computer and Information Science
University of Pennsylvania
http://www.cis.upenn.edu/~mengxu/
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2 3/3] docs: RTDS feature document.
2016-10-18 17:49 ` Meng Xu
@ 2016-10-18 17:55 ` Wei Liu
2016-10-18 21:53 ` Dario Faggioli
1 sibling, 0 replies; 10+ messages in thread
From: Wei Liu @ 2016-10-18 17:55 UTC (permalink / raw)
To: Meng Xu
Cc: Lars Kurth, Stefano Stabellini, Wei Liu, George Dunlap,
Andrew Cooper, Dario Faggioli, Ian Jackson, Jan Beulich,
xen-devel
Hi Meng,
I've committed this patch. Feel free to send update when you and
Dario have all the details hashed out.
Wei.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2 3/3] docs: RTDS feature document.
2016-10-18 17:49 ` Meng Xu
2016-10-18 17:55 ` Wei Liu
@ 2016-10-18 21:53 ` Dario Faggioli
1 sibling, 0 replies; 10+ messages in thread
From: Dario Faggioli @ 2016-10-18 21:53 UTC (permalink / raw)
To: Meng Xu; +Cc: xen-devel, Wei Liu
[-- Attachment #1.1: Type: text/plain, Size: 800 bytes --]
[Trimming the Cc-list]
On Tue, 2016-10-18 at 13:49 -0400, Meng Xu wrote:
> > +For checking or changing a VM's scheduling parameters from xl, do
> > +as follows:
> > + * `xl sched-rtds -d vm-rt`
>
>
> It should be
> xl sched-rtds -d vm-rt -v all
>
Yeah, you're right.
I wrote this basing on my memory, then went there and check, but I
apparently have forgotten to update all the examples.
All you say in this mail makes sense. So, yes, do put together a patch
and send it here. :-)
Thanks and Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
[-- Attachment #2: Type: text/plain, Size: 127 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2016-10-18 21:53 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-10-14 9:59 [PATCH v2 0/3 for 4.8] docs: feature documents for the schedulers Dario Faggioli
2016-10-14 10:00 ` [PATCH v2 1/3] docs: Credit1 feature document Dario Faggioli
2016-10-14 10:01 ` [PATCH v2 2/3] docs: Credit2 " Dario Faggioli
2016-10-14 10:02 ` [PATCH v2 3/3] docs: RTDS " Dario Faggioli
2016-10-18 17:49 ` Meng Xu
2016-10-18 17:55 ` Wei Liu
2016-10-18 21:53 ` Dario Faggioli
2016-10-14 13:24 ` [PATCH v2 0/3 for 4.8] docs: feature documents for the schedulers Wei Liu
2016-10-14 15:03 ` Andrew Cooper
2016-10-14 17:10 ` Wei Liu
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.