xen-devel.lists.xenproject.org archive mirror
 help / color / mirror / Atom feed
From: Dario Faggioli <dario.faggioli@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Anshul Makkar <anshul.makkar@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	David Vrabel <david.vrabel@citrix.com>
Subject: [PATCH v2 02/11] xen: credit2: prevent load balancing to go mad if time goes backwards
Date: Fri, 15 Jul 2016 16:49:18 +0200	[thread overview]
Message-ID: <146859415833.10217.1295698882153856098.stgit@Solace.fritz.box> (raw)
In-Reply-To: <146859397891.10217.10155969474613302167.stgit@Solace.fritz.box>

This really should not happen, but:
 1. it does happen! Some more info here:
    http://lists.xen.org/archives/html/xen-devel/2016-06/msg00922.html
 2. independently from 1, it makes sense and is easy enough
    to have a 'safety catch'.

The reason why this is particularly bad for Credit2 is that
negative values of delta mean out of scale high load (because
of the conversion to unsigned). This, for instance in the
case of runqueue load, results in a runqueue having its load
updated to values of the order of 10000% or so, which in turns
means that the load balancer will migrate everything off from
the pCPUs in the runqueue, and leave them idle until the load
gets back to something sane... which may indeed take a while!

This is not a fix for the problem of time going backwards. In
fact, if that happens a lot, load tracking accuracy is still
compromized, but at least the effect is a lot less bad than
before.

Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
---
Cc: Jan Beulich <JBeulich@suse.com>
Cc: Anshul Makkar <anshul.makkar@citrix.com>
Cc: David Vrabel <david.vrabel@citrix.com>
---
Leaving the patch as it was in v1, for reasons explained in these messages, by
both George and me:
 * <CAFLBxZZdS21UGW4TSDGvaDtn3LzD3X6L3f-rpDP4ySrHN0r7RQ@mail.gmail.com>
 * <a5b9e1c0-1d25-4cda-b1b6-a546189b43b0@citrix.com>
 * <1467888805.23985.62.camel@citrix.com>
---
 xen/common/sched_credit2.c |   12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
index 7e572bf..6cb06e8 100644
--- a/xen/common/sched_credit2.c
+++ b/xen/common/sched_credit2.c
@@ -404,6 +404,12 @@ __update_runq_load(const struct scheduler *ops,
     else
     {
         delta = now - rqd->load_last_update;
+        if ( unlikely(delta < 0) )
+        {
+            d2printk("%s: Time went backwards? now %"PRI_stime" llu %"PRI_stime"\n",
+                     __func__, now, rqd->load_last_update);
+            delta = 0;
+        }
 
         rqd->avgload =
             ( ( delta * ( (unsigned long long)rqd->load << prv->load_window_shift ) )
@@ -455,6 +461,12 @@ __update_svc_load(const struct scheduler *ops,
     else
     {
         delta = now - svc->load_last_update;
+        if ( unlikely(delta < 0) )
+        {
+            d2printk("%s: Time went backwards? now %"PRI_stime" llu %"PRI_stime"\n",
+                     __func__, now, svc->load_last_update);
+            delta = 0;
+        }
 
         svc->avgload =
             ( ( delta * ( (unsigned long long)vcpu_load << prv->load_window_shift ) )


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

  parent reply	other threads:[~2016-07-15 14:49 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-07-15 14:49 [PATCH v2 00/11] xen: sched: assorted fixes and improvements to Credit2 Dario Faggioli
2016-07-15 14:49 ` [PATCH v2 01/11] xen: sched: leave CPUs doing tasklet work alone Dario Faggioli
2016-07-18 14:35   ` George Dunlap
2016-07-15 14:49 ` Dario Faggioli [this message]
2016-07-15 14:49 ` [PATCH v2 03/11] xen: credit2: rework load tracking logic Dario Faggioli
2016-07-18 14:46   ` George Dunlap
2016-07-18 14:51     ` Dario Faggioli
2016-07-15 14:49 ` [PATCH v2 04/11] xen/tools: improve tracing of Credit2 load tracking events Dario Faggioli
2016-07-18 15:38   ` Wei Liu
2016-07-15 14:49 ` [PATCH v2 05/11] xen: credit2: use non-atomic cpumask and bit operations Dario Faggioli
2016-07-15 14:49 ` [PATCH v2 06/11] xen: credit2: make the code less experimental Dario Faggioli
2016-07-18 15:04   ` George Dunlap
2016-07-15 14:49 ` [PATCH v2 07/11] xen: credit2: add yet some more tracing Dario Faggioli
2016-07-15 14:50 ` [PATCH v2 08/11] xen: credit2: only marshall trace point arguments if tracing enabled Dario Faggioli
2016-07-18 15:10   ` George Dunlap
2016-07-15 14:50 ` [PATCH v2 09/11] tools: tracing: deal with new Credit2 events Dario Faggioli
2016-07-18 15:39   ` Wei Liu
2016-07-15 14:50 ` [PATCH v2 10/11] xen: credit2: the private scheduler lock can be an rwlock Dario Faggioli
2016-07-15 14:50 ` [PATCH v2 11/11] xen: credit2: implement true SMT support Dario Faggioli
2016-07-18 16:48   ` George Dunlap
2016-07-18 17:24     ` Dario Faggioli
2016-07-19  9:39       ` George Dunlap
2016-07-19  9:57         ` Dario Faggioli
2016-07-19 10:05           ` George Dunlap
2016-07-19 10:19             ` Dario Faggioli
2016-07-18 17:21 ` [PATCH v2 00/11] xen: sched: assorted fixes and improvements to Credit2 George Dunlap

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=146859415833.10217.1295698882153856098.stgit@Solace.fritz.box \
    --to=dario.faggioli@citrix.com \
    --cc=JBeulich@suse.com \
    --cc=anshul.makkar@citrix.com \
    --cc=david.vrabel@citrix.com \
    --cc=george.dunlap@citrix.com \
    --cc=xen-devel@lists.xenproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).