[PATCH RFC 1/2] xen: credit2: flexible configuration of runqueues

[PATCH RFC 1/2] xen: credit2: flexible configuration of runqueues

[Praveen Kumar]

The idea is to give user more flexibility to configure runqueue further.
For most workloads and in most systems, using per-core means have too many
small runqueues. Using per-socket is almost always better, but it may result
in too few big runqueues.

OPTION 1 :
--------
The user can create runqueue per-cpu using Xen boot parameter like below:

 credit2_runqueue=cpu

which would mean the following:
 - pCPU 0 belong to runqueue 0
 - pCPU 1 belong to runqueue 1
 - pCPU 2 belong to runqueue 2
 and so on.

OPTION 2 :
--------
Further user can be allowed to say something shown below :

 credit2_runqueue=0,1,4,5;2,3,6,7;8,9,12,13;10,11,14,15

or (with exactly the same meaning, but a perhaps more clear syntax):

 credit2_runqueue=[[0,1,4,5][2,3,6,7][8,9,12,13][10,11,14,15]]

which would mean the following:
 - pCPUs 0, 1, 4 and 5 belong to runqueue 0
 - pCPUs 2, 3, 6 and 7 belong to runqueue 1
 - pCPUs 8, 9, 12 and 13 belong to runqueue 2
 - pCPUs 10, 11, 14 and 15 belong to runqueue 3

---
PATCH 1/2 enables to create runqueue per-cpu [OPTION 1].
---

Signed-off-by: Praveen Kumar <kpraveen.lkml@gmail.com>

---
diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
index af457c1..2bc0013 100644
--- a/xen/common/sched_credit2.c
+++ b/xen/common/sched_credit2.c
@@ -301,6 +301,9 @@ integer_param("credit2_balance_over", opt_overload_balance_tolerance);
  * want that to happen basing on topology. At the moment, it is possible
  * to choose to arrange runqueues to be:
  *
+ * - per-cpu: meaning that there will be one runqueue per logical cpu. This
+ *            will happen when if the opt_runqueue parameter is set to 'cpu'.
+ *
  * - per-core: meaning that there will be one runqueue per each physical
  *             core of the host. This will happen if the opt_runqueue
  *             parameter is set to 'core';
@@ -322,11 +325,13 @@ integer_param("credit2_balance_over", opt_overload_balance_tolerance);
  * either the same physical core, the same physical socket, the same NUMA
  * node, or just all of them, will be put together to form runqueues.
  */
-#define OPT_RUNQUEUE_CORE   0
-#define OPT_RUNQUEUE_SOCKET 1
-#define OPT_RUNQUEUE_NODE   2
-#define OPT_RUNQUEUE_ALL    3
+#define OPT_RUNQUEUE_CPU    0
+#define OPT_RUNQUEUE_CORE   1
+#define OPT_RUNQUEUE_SOCKET 2
+#define OPT_RUNQUEUE_NODE   3
+#define OPT_RUNQUEUE_ALL    4
 static const char *const opt_runqueue_str[] = {
+    [OPT_RUNQUEUE_CPU] = "cpu",
     [OPT_RUNQUEUE_CORE] = "core",
     [OPT_RUNQUEUE_SOCKET] = "socket",
     [OPT_RUNQUEUE_NODE] = "node",
@@ -682,6 +687,8 @@ cpu_to_runqueue(struct csched2_private *prv, unsigned int cpu)
         BUG_ON(cpu_to_socket(cpu) == XEN_INVALID_SOCKET_ID ||
                cpu_to_socket(peer_cpu) == XEN_INVALID_SOCKET_ID);
 
+        if (opt_runqueue == OPT_RUNQUEUE_CPU)
+            continue;
         if ( opt_runqueue == OPT_RUNQUEUE_ALL ||
              (opt_runqueue == OPT_RUNQUEUE_CORE && same_core(peer_cpu, cpu)) ||
              (opt_runqueue == OPT_RUNQUEUE_SOCKET && same_socket(peer_cpu, cpu)) ||

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH RFC 1/2] xen: credit2: flexible configuration of runqueues

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 3565 bytes --]

Hey Praveen,

Thanks for working on this and sending the patches!

A couple of things on the submission itself.

When we send more than 1 patch, we say that we send a "patch series".
That basically is a set of related patches. They may be related in
various ways. For instance, each patch may be a piece, or a step, for
implementing something (a feature, a bugfi, etc); or maybe they all do
something similar to either the same or different subsystems.

In any case, for making it better clear this fact that they are
related, these patch series always assume the following format:
- there is an email which acts as the introduction of the series, 
  which is often called cover letter, and is numbered as patch 0 of 
  the series;
- the emails containing actual patches, numbered starting from 1, are
  all sent as they were replies to the cover letter.

In the cover letter, one usually introduces an explains the work being
done, focusing on making it as easy as possible for the reviewers to
understand anything that it is important they know when looking at the
series.

There are several examples of patch series in the xen-devel mailing
list archives:
 https://lists.xen.org/archives/html/xen-devel/2017-02/threads.html

 https://lists.xen.org/archives/html/xen-devel/2017-02/msg03455.html
 https://lists.xen.org/archives/html/xen-devel/2017-02/msg01027.html
 https://lists.xen.org/archives/html/xen-devel/2017-01/msg02837.html

And the topic of sending patch series is covered here:
https://wiki.xen.org/wiki/Submitting_Xen_Project_Patches#Sending_the_patches_to_the_list

As per how to actually send the various emails in such a way  that they
get to the list in the proper format, there are tools.

I use stgit for development, and it supports doing that by means of
`stg mail'. A lot of people which use "just" git, do use
git-send-email.

On Fri, 2017-03-10 at 23:56 +0530, Praveen Kumar wrote:
> The idea is to give user more flexibility to configure runqueue
> further.
> For most workloads and in most systems, using per-core means have too
> many
> small runqueues. Using per-socket is almost always better, but it may
> result
> in too few big runqueues.
> 
> OPTION 1 :
> --------
> The user can create runqueue per-cpu using Xen boot parameter like
> below:
> 
>  credit2_runqueue=cpu
> 
> which would mean the following:
>  - pCPU 0 belong to runqueue 0
>  - pCPU 1 belong to runqueue 1
>  - pCPU 2 belong to runqueue 2
>  and so on.
> 
> OPTION 2 :
> --------
> Further user can be allowed to say something shown below :
> 
>  credit2_runqueue=0,1,4,5;2,3,6,7;8,9,12,13;10,11,14,15
> 
> or (with exactly the same meaning, but a perhaps more clear syntax):
> 
>  credit2_runqueue=[[0,1,4,5][2,3,6,7][8,9,12,13][10,11,14,15]]
> 
> which would mean the following:
>  - pCPUs 0, 1, 4 and 5 belong to runqueue 0
>  - pCPUs 2, 3, 6 and 7 belong to runqueue 1
>  - pCPUs 8, 9, 12 and 13 belong to runqueue 2
>  - pCPUs 10, 11, 14 and 15 belong to runqueue 3
> 
> ---
> PATCH 1/2 enables to create runqueue per-cpu [OPTION 1].
> ---
> 
So, this kind of high level summary about both patches 1 and 2 is
probably similar to what a cover letter for this series should contain.

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH RFC 1/2] xen: credit2: flexible configuration of runqueues

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1862 bytes --]

Ok, now about the code.

On Fri, 2017-03-10 at 23:56 +0530, Praveen Kumar wrote:
> The user can create runqueue per-cpu using Xen boot parameter like
> below:
> 
>  credit2_runqueue=cpu
> 
> which would mean the following:
>  - pCPU 0 belong to runqueue 0
>  - pCPU 1 belong to runqueue 1
>  - pCPU 2 belong to runqueue 2
>  and so on.
> 
> Signed-off-by: Praveen Kumar <kpraveen.lkml@gmail.com>
> 
> ---
> diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
> index af457c1..2bc0013 100644
> --- a/xen/common/sched_credit2.c
> +++ b/xen/common/sched_credit2.c
> @@ -301,6 +301,9 @@ integer_param("credit2_balance_over",
> opt_overload_balance_tolerance);
>   * want that to happen basing on topology. At the moment, it is
> possible
>   * to choose to arrange runqueues to be:
>   *
> + * - per-cpu: meaning that there will be one runqueue per logical
> cpu. This
> + *            will happen when if the opt_runqueue parameter is set
> to 'cpu'.
> + *
>   * - per-core: meaning that there will be one runqueue per each
> physical
>   *             core of the host. This will happen if the
> opt_runqueue
>   *             parameter is set to 'core';
> 
This is ok, but you also need to modify the "credit2\_runqueue" section
in docs/misc/xen-command-line.markdown .

In fact, in order to not end up with outdated and incorrect
documentation, we require that the docs are updated in the same patch
that introduces something new (or changes something existing).

Thanks and Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel