[PATCH 0/4] xen: sched: rework and add performance counters

[PATCH 0/4] xen: sched: rework and add performance counters

[Dario Faggioli]

A small series that refactors a few of the existing scheduling related
performance counters, making them generic and updating them from all
schedulers, rather than just in Credit1.

It also (in the last patch) add a few new counters, specific to Credit2.

Thanks and Regards,
Dario
---
Dario Faggioli (4):
      xen: sched: honour generic perf conuters in the RTDS scheduler
      xen: sched: make counters for vCPU sleep and wakeup generic
      xen: sched: make counters for vCPU tickling generic
      xen: credit2: add a few performance counters

 xen/common/sched_credit2.c   |   37 +++++++++++++++++++++++++++++++++----
 xen/common/sched_rt.c        |   18 ++++++++++++++++++
 xen/include/xen/perfc_defn.h |   29 +++++++++++++++++++++--------
 3 files changed, 72 insertions(+), 12 deletions(-)

--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

[PATCH 1/4] xen: sched: honour generic perf conuters in the RTDS scheduler

[Dario Faggioli]

more specifically, about vCPU initialization and destruction events,
in line with adb26c09f26e ("xen: sched: introduce a couple of counters
in credit2 and SEDF").

Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Cc: Meng Xu <xumengpanda@gmail.com>
Cc: George Dunlap <george.dunlap@eu.citrix.com>
Cc: Jan Beulich <JBeulich@suse.com>
Cc: Keir Fraser <keir@xen.org>
---
 xen/common/sched_rt.c |    4 ++++
 1 file changed, 4 insertions(+)

diff --git a/xen/common/sched_rt.c b/xen/common/sched_rt.c
index df4adac..58dd646 100644
--- a/xen/common/sched_rt.c
+++ b/xen/common/sched_rt.c
@@ -525,6 +525,8 @@ rt_alloc_vdata(const struct scheduler *ops, struct vcpu *vc, void *dd)
     if ( !is_idle_vcpu(vc) )
         svc->budget = RTDS_DEFAULT_BUDGET;
 
+    SCHED_STAT_CRANK(vcpu_init);
+
     return svc;
 }
 
@@ -574,6 +576,8 @@ rt_vcpu_remove(const struct scheduler *ops, struct vcpu *vc)
     struct rt_dom * const sdom = svc->sdom;
     spinlock_t *lock;
 
+    SCHED_STAT_CRANK(vcpu_destroy);
+
     BUG_ON( sdom == NULL );
 
     lock = vcpu_schedule_lock_irq(vc);

[PATCH 2/4] xen: sched: make counters for vCPU sleep and wakeup generic

[Dario Faggioli]

and update them from Credit2 and RTDS. In Credit2, while there,
remove some stale comments too.

Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Cc: George Dunlap <george.dunlap@eu.citrix.com>
Cc: Jan Beulich <JBeulich@suse.com>
Cc: Keir Fraser <keir@xen.org>
---
 xen/common/sched_credit2.c   |   12 ++++++++----
 xen/common/sched_rt.c        |   12 ++++++++++++
 xen/include/xen/perfc_defn.h |   10 +++++-----
 3 files changed, 25 insertions(+), 9 deletions(-)

diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
index ad0a5d4..2b852cc 100644
--- a/xen/common/sched_credit2.c
+++ b/xen/common/sched_credit2.c
@@ -931,6 +931,7 @@ csched2_vcpu_sleep(const struct scheduler *ops, struct vcpu *vc)
     struct csched2_vcpu * const svc = CSCHED2_VCPU(vc);
 
     BUG_ON( is_idle_vcpu(vc) );
+    SCHED_STAT_CRANK(vcpu_sleep);
 
     if ( per_cpu(schedule_data, vc->processor).curr == vc )
         cpu_raise_softirq(vc->processor, SCHEDULE_SOFTIRQ);
@@ -956,19 +957,22 @@ csched2_vcpu_wake(const struct scheduler *ops, struct vcpu *vc)
 
     BUG_ON( is_idle_vcpu(vc) );
 
-    /* Make sure svc priority mod happens before runq check */
     if ( unlikely(per_cpu(schedule_data, vc->processor).curr == vc) )
     {
+        SCHED_STAT_CRANK(vcpu_wake_running);
         goto out;
     }
-
     if ( unlikely(__vcpu_on_runq(svc)) )
     {
-        /* If we've boosted someone that's already on a runqueue, prioritize
-         * it and inform the cpu in question. */
+        SCHED_STAT_CRANK(vcpu_wake_onrunq);
         goto out;
     }
 
+    if ( likely(vcpu_runnable(vc)) )
+        SCHED_STAT_CRANK(vcpu_wake_runnable);
+    else
+        SCHED_STAT_CRANK(vcpu_wake_not_runnable);
+
     /* If the context hasn't been saved for this vcpu yet, we can't put it on
      * another runqueue.  Instead, we set a flag so that it will be put on the runqueue
      * after the context has been saved. */
diff --git a/xen/common/sched_rt.c b/xen/common/sched_rt.c
index 58dd646..49d1b83 100644
--- a/xen/common/sched_rt.c
+++ b/xen/common/sched_rt.c
@@ -851,6 +851,7 @@ rt_vcpu_sleep(const struct scheduler *ops, struct vcpu *vc)
     struct rt_vcpu * const svc = rt_vcpu(vc);
 
     BUG_ON( is_idle_vcpu(vc) );
+    SCHED_STAT_CRANK(vcpu_sleep);
 
     if ( curr_on_cpu(vc->processor) == vc )
         cpu_raise_softirq(vc->processor, SCHEDULE_SOFTIRQ);
@@ -966,11 +967,22 @@ rt_vcpu_wake(const struct scheduler *ops, struct vcpu *vc)
     BUG_ON( is_idle_vcpu(vc) );
 
     if ( unlikely(curr_on_cpu(vc->processor) == vc) )
+    {
+        SCHED_STAT_CRANK(vcpu_wake_running);
         return;
+    }
 
     /* on RunQ/DepletedQ, just update info is ok */
     if ( unlikely(__vcpu_on_q(svc)) )
+    {
+        SCHED_STAT_CRANK(vcpu_wake_onrunq);
         return;
+    }
+
+    if ( likely(vcpu_runnable(vc)) )
+        SCHED_STAT_CRANK(vcpu_wake_runnable);
+    else
+        SCHED_STAT_CRANK(vcpu_wake_not_runnable);
 
     /* If context hasn't been saved for this vcpu yet, we can't put it on
      * the Runqueue/DepletedQ. Instead, we set a flag so that it will be
diff --git a/xen/include/xen/perfc_defn.h b/xen/include/xen/perfc_defn.h
index 3ac7b45..2dc78fe 100644
--- a/xen/include/xen/perfc_defn.h
+++ b/xen/include/xen/perfc_defn.h
@@ -21,6 +21,11 @@ PERFCOUNTER(dom_init,               "sched: dom_init")
 PERFCOUNTER(dom_destroy,            "sched: dom_destroy")
 PERFCOUNTER(vcpu_init,              "sched: vcpu_init")
 PERFCOUNTER(vcpu_destroy,           "sched: vcpu_destroy")
+PERFCOUNTER(vcpu_sleep,             "sched: vcpu_sleep")
+PERFCOUNTER(vcpu_wake_running,      "sched: vcpu_wake_running")
+PERFCOUNTER(vcpu_wake_onrunq,       "sched: vcpu_wake_onrunq")
+PERFCOUNTER(vcpu_wake_runnable,     "sched: vcpu_wake_runnable")
+PERFCOUNTER(vcpu_wake_not_runnable, "sched: vcpu_wake_not_runnable")
 
 /* credit specific counters */
 PERFCOUNTER(delay_ms,               "csched: delay")
@@ -32,11 +37,6 @@ PERFCOUNTER(acct_reorder,           "csched: acct_reorder")
 PERFCOUNTER(acct_min_credit,        "csched: acct_min_credit")
 PERFCOUNTER(acct_vcpu_active,       "csched: acct_vcpu_active")
 PERFCOUNTER(acct_vcpu_idle,         "csched: acct_vcpu_idle")
-PERFCOUNTER(vcpu_sleep,             "csched: vcpu_sleep")
-PERFCOUNTER(vcpu_wake_running,      "csched: vcpu_wake_running")
-PERFCOUNTER(vcpu_wake_onrunq,       "csched: vcpu_wake_onrunq")
-PERFCOUNTER(vcpu_wake_runnable,     "csched: vcpu_wake_runnable")
-PERFCOUNTER(vcpu_wake_not_runnable, "csched: vcpu_wake_not_runnable")
 PERFCOUNTER(vcpu_park,              "csched: vcpu_park")
 PERFCOUNTER(vcpu_unpark,            "csched: vcpu_unpark")
 PERFCOUNTER(tickle_idlers_none,     "csched: tickle_idlers_none")

[PATCH 3/4] xen: sched: make counters for vCPU tickling generic

[Dario Faggioli]

and update them from Credit2 and RTDS schedulers.

Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Cc: Meng Xu <xumengpanda@gmail.com>
Cc: George Dunlap <george.dunlap@eu.citrix.com>
Cc: Jan Beulich <JBeulich@suse.com>
Cc: Keir Fraser <keir@xen.org>
---
 xen/common/sched_credit2.c   |    2 ++
 xen/common/sched_rt.c        |    2 ++
 xen/include/xen/perfc_defn.h |    4 ++--
 3 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
index 2b852cc..bf13a84 100644
--- a/xen/common/sched_credit2.c
+++ b/xen/common/sched_credit2.c
@@ -571,9 +571,11 @@ tickle:
                   (unsigned char *)&d);
     }
     cpumask_set_cpu(ipid, &rqd->tickled);
+    SCHED_STAT_CRANK(tickle_idlers_some);
     cpu_raise_softirq(ipid, SCHEDULE_SOFTIRQ);
 
 no_tickle:
+    SCHED_STAT_CRANK(tickle_idlers_none);
     return;
 }
 
diff --git a/xen/common/sched_rt.c b/xen/common/sched_rt.c
index 49d1b83..2ad0c68 100644
--- a/xen/common/sched_rt.c
+++ b/xen/common/sched_rt.c
@@ -929,6 +929,7 @@ runq_tickle(const struct scheduler *ops, struct rt_vcpu *new)
     }
 
     /* didn't tickle any cpu */
+    SCHED_STAT_CRANK(tickle_idlers_none);
     return;
 out:
     /* TRACE */
@@ -944,6 +945,7 @@ out:
     }
 
     cpumask_set_cpu(cpu_to_tickle, &prv->tickled);
+    SCHED_STAT_CRANK(tickle_idlers_some);
     cpu_raise_softirq(cpu_to_tickle, SCHEDULE_SOFTIRQ);
     return;
 }
diff --git a/xen/include/xen/perfc_defn.h b/xen/include/xen/perfc_defn.h
index 2dc78fe..f754331 100644
--- a/xen/include/xen/perfc_defn.h
+++ b/xen/include/xen/perfc_defn.h
@@ -26,6 +26,8 @@ PERFCOUNTER(vcpu_wake_running,      "sched: vcpu_wake_running")
 PERFCOUNTER(vcpu_wake_onrunq,       "sched: vcpu_wake_onrunq")
 PERFCOUNTER(vcpu_wake_runnable,     "sched: vcpu_wake_runnable")
 PERFCOUNTER(vcpu_wake_not_runnable, "sched: vcpu_wake_not_runnable")
+PERFCOUNTER(tickle_idlers_none,     "sched: tickle_idlers_none")
+PERFCOUNTER(tickle_idlers_some,     "sched: tickle_idlers_some")
 
 /* credit specific counters */
 PERFCOUNTER(delay_ms,               "csched: delay")
@@ -39,8 +41,6 @@ PERFCOUNTER(acct_vcpu_active,       "csched: acct_vcpu_active")
 PERFCOUNTER(acct_vcpu_idle,         "csched: acct_vcpu_idle")
 PERFCOUNTER(vcpu_park,              "csched: vcpu_park")
 PERFCOUNTER(vcpu_unpark,            "csched: vcpu_unpark")
-PERFCOUNTER(tickle_idlers_none,     "csched: tickle_idlers_none")
-PERFCOUNTER(tickle_idlers_some,     "csched: tickle_idlers_some")
 PERFCOUNTER(load_balance_idle,      "csched: load_balance_idle")
 PERFCOUNTER(load_balance_over,      "csched: load_balance_over")
 PERFCOUNTER(load_balance_other,     "csched: load_balance_other")

[PATCH 4/4] xen: credit2: add a few performance counters

[Dario Faggioli]

for events that are specific to Credit2 (as it happens
for Credit1 already).

Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Cc: George Dunlap <george.dunlap@eu.citrix.com>
Cc: Jan Beulich <JBeulich@suse.com>
Cc: Keir Fraser <keir@xen.org>
---
 xen/common/sched_credit2.c   |   23 +++++++++++++++++++++++
 xen/include/xen/perfc_defn.h |   15 ++++++++++++++-
 2 files changed, 37 insertions(+), 1 deletion(-)

diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
index bf13a84..fa764f9 100644
--- a/xen/common/sched_credit2.c
+++ b/xen/common/sched_credit2.c
@@ -652,6 +652,8 @@ static void reset_credit(const struct scheduler *ops, int cpu, s_time_t now,
         }
     }
 
+    SCHED_STAT_CRANK(credit_reset);
+
     /* No need to resort runqueue, as everyone's order should be the same. */
 }
 
@@ -671,6 +673,7 @@ void burn_credits(struct csched2_runqueue_data *rqd, struct csched2_vcpu *svc, s
     delta = now - svc->start_time;
 
     if ( delta > 0 ) {
+        SCHED_STAT_CRANK(burn_credits_t2c);
         t2c_update(rqd, delta, svc);
         svc->start_time = now;
 
@@ -711,6 +714,7 @@ static void update_max_weight(struct csched2_runqueue_data *rqd, int new_weight,
     {
         rqd->max_weight = new_weight;
         d2printk("%s: Runqueue id %d max weight %d\n", __func__, rqd->id, rqd->max_weight);
+        SCHED_STAT_CRANK(upd_max_weight_quick);
     }
     else if ( old_weight == rqd->max_weight )
     {
@@ -727,6 +731,7 @@ static void update_max_weight(struct csched2_runqueue_data *rqd, int new_weight,
 
         rqd->max_weight = max_weight;
         d2printk("%s: Runqueue %d max weight %d\n", __func__, rqd->id, rqd->max_weight);
+        SCHED_STAT_CRANK(upd_max_weight_full);
     }
 }
 
@@ -748,6 +753,7 @@ __csched2_vcpu_check(struct vcpu *vc)
     {
         BUG_ON( !is_idle_vcpu(vc) );
     }
+    SCHED_STAT_CRANK(vcpu_check);
 }
 #define CSCHED2_VCPU_CHECK(_vc)  (__csched2_vcpu_check(_vc))
 #else
@@ -1201,6 +1207,7 @@ static void migrate(const struct scheduler *ops,
         svc->migrate_rqd = trqd;
         set_bit(_VPF_migrating, &svc->vcpu->pause_flags);
         set_bit(__CSFLAG_runq_migrate_request, &svc->flags);
+        SCHED_STAT_CRANK(migrate_requested);
     }
     else
     {
@@ -1221,7 +1228,10 @@ static void migrate(const struct scheduler *ops,
             update_load(ops, svc->rqd, svc, 1, now);
             runq_insert(ops, svc->vcpu->processor, svc);
             runq_tickle(ops, svc->vcpu->processor, svc, now);
+            SCHED_STAT_CRANK(migrate_on_runq);
         }
+        else
+            SCHED_STAT_CRANK(migrate_no_runq);
     }
 }
 
@@ -1575,7 +1585,10 @@ csched2_runtime(const struct scheduler *ops, int cpu, struct csched2_vcpu *snext
     /* The next guy may actually have a higher credit, if we've tried to
      * avoid migrating him from a different cpu.  DTRT.  */
     if ( rt_credit <= 0 )
+    {
         time = CSCHED2_MIN_TIMER;
+        SCHED_STAT_CRANK(runtime_min_timer);
+    }
     else
     {
         /* FIXME: See if we can eliminate this conversion if we know time
@@ -1586,9 +1599,15 @@ csched2_runtime(const struct scheduler *ops, int cpu, struct csched2_vcpu *snext
 
         /* Check limits */
         if ( time < CSCHED2_MIN_TIMER )
+        {
             time = CSCHED2_MIN_TIMER;
+            SCHED_STAT_CRANK(runtime_min_timer);
+        }
         else if ( time > CSCHED2_MAX_TIMER )
+        {
             time = CSCHED2_MAX_TIMER;
+            SCHED_STAT_CRANK(runtime_max_timer);
+        }
     }
 
     return time;
@@ -1621,7 +1640,10 @@ runq_candidate(struct csched2_runqueue_data *rqd,
          * its credit is at least CSCHED2_MIGRATE_RESIST higher. */
         if ( svc->vcpu->processor != cpu
              && snext->credit + CSCHED2_MIGRATE_RESIST > svc->credit )
+        {
+            SCHED_STAT_CRANK(migrate_resisted);
             continue;
+        }
 
         /* If the next one on the list has more credit than current
          * (or idle, if current is not runnable), choose it. */
@@ -1766,6 +1788,7 @@ csched2_schedule(
         {
             snext->credit += CSCHED2_MIGRATE_COMPENSATION;
             snext->vcpu->processor = cpu;
+            SCHED_STAT_CRANK(migrated);
             ret.migrated = 1;
         }
     }
diff --git a/xen/include/xen/perfc_defn.h b/xen/include/xen/perfc_defn.h
index f754331..d3af41f 100644
--- a/xen/include/xen/perfc_defn.h
+++ b/xen/include/xen/perfc_defn.h
@@ -28,10 +28,10 @@ PERFCOUNTER(vcpu_wake_runnable,     "sched: vcpu_wake_runnable")
 PERFCOUNTER(vcpu_wake_not_runnable, "sched: vcpu_wake_not_runnable")
 PERFCOUNTER(tickle_idlers_none,     "sched: tickle_idlers_none")
 PERFCOUNTER(tickle_idlers_some,     "sched: tickle_idlers_some")
+PERFCOUNTER(vcpu_check,             "sched: vcpu_check")
 
 /* credit specific counters */
 PERFCOUNTER(delay_ms,               "csched: delay")
-PERFCOUNTER(vcpu_check,             "csched: vcpu_check")
 PERFCOUNTER(acct_run,               "csched: acct_run")
 PERFCOUNTER(acct_no_work,           "csched: acct_no_work")
 PERFCOUNTER(acct_balance,           "csched: acct_balance")
@@ -51,6 +51,19 @@ PERFCOUNTER(migrate_running,        "csched: migrate_running")
 PERFCOUNTER(migrate_kicked_away,    "csched: migrate_kicked_away")
 PERFCOUNTER(vcpu_hot,               "csched: vcpu_hot")
 
+/* credit2 specific counters */
+PERFCOUNTER(burn_credits_t2c,       "csched2: burn_credits_t2c")
+PERFCOUNTER(upd_max_weight_quick,   "cshced2: update_max_weight_quick")
+PERFCOUNTER(upd_max_weight_full,    "cshced2: update_max_weight_full")
+PERFCOUNTER(migrate_requested,      "cshced2: migrate_requested")
+PERFCOUNTER(migrate_on_runq,        "cshced2: migrate_on_runq")
+PERFCOUNTER(migrate_no_runq,        "cshced2: migrate_no_runq")
+PERFCOUNTER(runtime_min_timer,      "csched2: runtime_min_timer")
+PERFCOUNTER(runtime_max_timer,      "csched2: runtime_max_timer")
+PERFCOUNTER(migrated,               "csched2: migrated")
+PERFCOUNTER(migrate_resisted,       "csched2: migrate_resisted")
+PERFCOUNTER(credit_reset,           "csched2: credit_reset")
+
 PERFCOUNTER(need_flush_tlb_flush,   "PG_need_flush tlb flushes")
 
 /*#endif*/ /* __XEN_PERFC_DEFN_H__ */

Re: [PATCH 3/4] xen: sched: make counters for vCPU tickling generic

[Jan Beulich]

>>> On 26.02.15 at 14:37, <dario.faggioli@citrix.com> wrote:
> --- a/xen/common/sched_credit2.c
> +++ b/xen/common/sched_credit2.c
> @@ -571,9 +571,11 @@ tickle:
>                    (unsigned char *)&d);
>      }
>      cpumask_set_cpu(ipid, &rqd->tickled);
> +    SCHED_STAT_CRANK(tickle_idlers_some);
>      cpu_raise_softirq(ipid, SCHEDULE_SOFTIRQ);
>  
>  no_tickle:
> +    SCHED_STAT_CRANK(tickle_idlers_none);
>      return;
>  }

Isn't there a return statement missing ahead of "no_tickle:" now?

Jan

Re: [PATCH 4/4] xen: credit2: add a few performance counters

[Jan Beulich]

>>> On 26.02.15 at 14:37, <dario.faggioli@citrix.com> wrote:
> @@ -51,6 +51,19 @@ PERFCOUNTER(migrate_running,        "csched: migrate_running")
>  PERFCOUNTER(migrate_kicked_away,    "csched: migrate_kicked_away")
>  PERFCOUNTER(vcpu_hot,               "csched: vcpu_hot")
>  
> +/* credit2 specific counters */
> +PERFCOUNTER(burn_credits_t2c,       "csched2: burn_credits_t2c")
> +PERFCOUNTER(upd_max_weight_quick,   "cshced2: update_max_weight_quick")
> +PERFCOUNTER(upd_max_weight_full,    "cshced2: update_max_weight_full")
> +PERFCOUNTER(migrate_requested,      "cshced2: migrate_requested")
> +PERFCOUNTER(migrate_on_runq,        "cshced2: migrate_on_runq")
> +PERFCOUNTER(migrate_no_runq,        "cshced2: migrate_no_runq")

Repeated typos.

Jan

Re: [PATCH 0/4] xen: sched: rework and add performance counters

[Jan Beulich]

>>> On 26.02.15 at 14:36, <dario.faggioli@citrix.com> wrote:
> A small series that refactors a few of the existing scheduling related
> performance counters, making them generic and updating them from all
> schedulers, rather than just in Credit1.
> 
> It also (in the last patch) add a few new counters, specific to Credit2.
> 
> Thanks and Regards,
> Dario
> ---
> Dario Faggioli (4):
>       xen: sched: honour generic perf conuters in the RTDS scheduler
>       xen: sched: make counters for vCPU sleep and wakeup generic
>       xen: sched: make counters for vCPU tickling generic
>       xen: credit2: add a few performance counters
> 
>  xen/common/sched_credit2.c   |   37 +++++++++++++++++++++++++++++++++----
>  xen/common/sched_rt.c        |   18 ++++++++++++++++++
>  xen/include/xen/perfc_defn.h |   29 +++++++++++++++++++++--------

With the typos in patch 4 fixed, all changes to this file in this series
Acked-by: Jan Beulich <jbeulich@suse.com>

Jan

Re: [PATCH 3/4] xen: sched: make counters for vCPU tickling generic

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 729 bytes --]

On Thu, 2015-02-26 at 15:22 +0000, Jan Beulich wrote:
> >>> On 26.02.15 at 14:37, <dario.faggioli@citrix.com> wrote:
> > --- a/xen/common/sched_credit2.c
> > +++ b/xen/common/sched_credit2.c
> > @@ -571,9 +571,11 @@ tickle:
> >                    (unsigned char *)&d);
> >      }
> >      cpumask_set_cpu(ipid, &rqd->tickled);
> > +    SCHED_STAT_CRANK(tickle_idlers_some);
> >      cpu_raise_softirq(ipid, SCHEDULE_SOFTIRQ);
> >  
> >  no_tickle:
> > +    SCHED_STAT_CRANK(tickle_idlers_none);
> >      return;
> >  }
> 
> Isn't there a return statement missing ahead of "no_tickle:" now?
> 
There is. I reworked this last minute, and overlooked this... sorry.
Will fix for v2.

Thanks and Regards,
Dario

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH 1/4] xen: sched: honour generic perf conuters in the RTDS scheduler

[Meng Xu]

[-- Attachment #1.1: Type: text/plain, Size: 1560 bytes --]

Not sure if I should comment with Reviewed-by, I will just do it. Please
just ignore if I should not add Reviewed-by.

2015-02-26 8:36 GMT-05:00 Dario Faggioli <dario.faggioli@citrix.com>:

> more specifically, about vCPU initialization and destruction events,
> in line with adb26c09f26e ("xen: sched: introduce a couple of counters
> in credit2 and SEDF").
>
> Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
> Cc: Meng Xu <xumengpanda@gmail.com>
> Cc: George Dunlap <george.dunlap@eu.citrix.com>
> Cc: Jan Beulich <JBeulich@suse.com>
> Cc: Keir Fraser <keir@xen.org>
> ---
>  xen/common/sched_rt.c |    4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/xen/common/sched_rt.c b/xen/common/sched_rt.c
> index df4adac..58dd646 100644
> --- a/xen/common/sched_rt.c
> +++ b/xen/common/sched_rt.c
> @@ -525,6 +525,8 @@ rt_alloc_vdata(const struct scheduler *ops, struct
> vcpu *vc, void *dd)
>      if ( !is_idle_vcpu(vc) )
>          svc->budget = RTDS_DEFAULT_BUDGET;
>
> +    SCHED_STAT_CRANK(vcpu_init);
> +
>      return svc;
>  }
>
> @@ -574,6 +576,8 @@ rt_vcpu_remove(const struct scheduler *ops, struct
> vcpu *vc)
>      struct rt_dom * const sdom = svc->sdom;
>      spinlock_t *lock;
>
> +    SCHED_STAT_CRANK(vcpu_destroy);
> +
>      BUG_ON( sdom == NULL );
>
>      lock = vcpu_schedule_lock_irq(vc);
>
>
​Reviewed-by: Meng Xu <mengxu@cis.upenn.edu>​

​Thanks,

Meng​



-----------
Meng Xu
PhD Student in Computer and Information Science
University of Pennsylvania

[-- Attachment #1.2: Type: text/html, Size: 2643 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH 3/4] xen: sched: make counters for vCPU tickling generic

[Meng Xu]

[-- Attachment #1.1: Type: text/plain, Size: 1844 bytes --]

2015-02-26 8:37 GMT-05:00 Dario Faggioli <dario.faggioli@citrix.com>:

> and update them from Credit2 and RTDS schedulers.
>
> Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
> Cc: Meng Xu <xumengpanda@gmail.com>
> Cc: George Dunlap <george.dunlap@eu.citrix.com>
> Cc: Jan Beulich <JBeulich@suse.com>
> Cc: Keir Fraser <keir@xen.org>
> ---
>  xen/common/sched_credit2.c   |    2 ++
>  xen/common/sched_rt.c        |    2 ++
>  xen/include/xen/perfc_defn.h |    4 ++--
>  3 files changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
> index 2b852cc..bf13a84 100644
> --- a/xen/common/sched_credit2.c
> +++ b/xen/common/sched_credit2.c
> @@ -571,9 +571,11 @@ tickle:
>                    (unsigned char *)&d);
>      }
>      cpumask_set_cpu(ipid, &rqd->tickled);
> +    SCHED_STAT_CRANK(tickle_idlers_some);
>      cpu_raise_softirq(ipid, SCHEDULE_SOFTIRQ);
>
>  no_tickle:
> +    SCHED_STAT_CRANK(tickle_idlers_none);
>      return;
>  }
>
> diff --git a/xen/common/sched_rt.c b/xen/common/sched_rt.c
> index 49d1b83..2ad0c68 100644
> --- a/xen/common/sched_rt.c
> +++ b/xen/common/sched_rt.c
> @@ -929,6 +929,7 @@ runq_tickle(const struct scheduler *ops, struct
> rt_vcpu *new)
>      }
>
>      /* didn't tickle any cpu */
> +    SCHED_STAT_CRANK(tickle_idlers_none);
>      return;
>  out:
>      /* TRACE */
> @@ -944,6 +945,7 @@ out:
>      }
>
>      cpumask_set_cpu(cpu_to_tickle, &prv->tickled);
> +    SCHED_STAT_CRANK(tickle_idlers_some);
>      cpu_raise_softirq(cpu_to_tickle, SCHEDULE_SOFTIRQ);
>      return;
>  }
>


​The change for RTDS scheduler looks good to me.

Thanks,

Meng​


-- 


-----------
Meng Xu
PhD Student in Computer and Information Science
University of Pennsylvania

[-- Attachment #1.2: Type: text/html, Size: 2981 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH 1/4] xen: sched: honour generic perf conuters in the RTDS scheduler

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 441 bytes --]

On Fri, 2015-02-27 at 00:43 -0500, Meng Xu wrote:
> Not sure if I should comment with Reviewed-by, I will just do it.
>
Provided you have actually reviewed the change, you definitely
should! :-D :-D

>  Please just ignore if I should not add Reviewed-by. 
>
No reason to! Maybe, next time, do it without using HTML emails. :-P

> ​Reviewed-by: Meng Xu <mengxu@cis.upenn.edu>​

> ​Thanks,
> 
Thanks and Regards,
Dario


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH 3/4] xen: sched: make counters for vCPU tickling generic

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1237 bytes --]

On Fri, 2015-02-27 at 00:47 -0500, Meng Xu wrote:

> 2015-02-26 8:37 GMT-05:00 Dario Faggioli <dario.faggioli@citrix.com>:
>         and update them from Credit2 and RTDS schedulers.
>         
>         Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
>         Cc: Meng Xu <xumengpanda@gmail.com>
>         Cc: George Dunlap <george.dunlap@eu.citrix.com>
>         Cc: Jan Beulich <JBeulich@suse.com>
>         Cc: Keir Fraser <keir@xen.org>
>         ---
>          xen/common/sched_credit2.c   |    2 ++
>          xen/common/sched_rt.c        |    2 ++
>          xen/include/xen/perfc_defn.h |    4 ++--
>          3 files changed, 6 insertions(+), 2 deletions(-)
> 
> ​The change for RTDS scheduler looks good to me.
>
Does this count as a "Reviewed-by: Meng Xu <mengxu@cis.upenn.edu>" ?

Also, if yes, does it also apply to patch #2 ? That is unclear as
sched_rt.c is modified in patches #1, #2 ad #3, while what you did is:
 - you explicitly provided the tag for patch #1
 - you said "looks good" for this for patch #3
 - you said nothing for patch #2

The bottom line of all this being: with Ack-s/Reviewed-by-s, it's always
better be pretty explicit! :-D

Thanks and Regards,
Dario


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH 0/4] xen: sched: rework and add performance counters

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1297 bytes --]

On Thu, 2015-02-26 at 15:25 +0000, Jan Beulich wrote:
> >>> On 26.02.15 at 14:36, <dario.faggioli@citrix.com> wrote:
> > A small series that refactors a few of the existing scheduling related
> > performance counters, making them generic and updating them from all
> > schedulers, rather than just in Credit1.
> > 
> > It also (in the last patch) add a few new counters, specific to Credit2.
> > 
> > Thanks and Regards,
> > Dario
> > ---
> > Dario Faggioli (4):
> >       xen: sched: honour generic perf conuters in the RTDS scheduler
> >       xen: sched: make counters for vCPU sleep and wakeup generic
> >       xen: sched: make counters for vCPU tickling generic
> >       xen: credit2: add a few performance counters
> > 
> >  xen/common/sched_credit2.c   |   37 +++++++++++++++++++++++++++++++++----
> >  xen/common/sched_rt.c        |   18 ++++++++++++++++++
> >  xen/include/xen/perfc_defn.h |   29 +++++++++++++++++++++--------
> 
> With the typos in patch 4 fixed, all changes to this file in this series
> Acked-by: Jan Beulich <jbeulich@suse.com>
> 
Thanks.

So, when resending (as soon as hearing back from Meng), I'll stick your
Ack to the patches touching perfc_defn.h (patch 2, 3 and 4). Hope this
is what you meant with this line.

Regards,
Dario

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH 3/4] xen: sched: make counters for vCPU tickling generic

[Meng Xu]

2015-02-27 5:53 GMT-05:00 Dario Faggioli <dario.faggioli@citrix.com>:
> On Fri, 2015-02-27 at 00:47 -0500, Meng Xu wrote:
>
>> 2015-02-26 8:37 GMT-05:00 Dario Faggioli <dario.faggioli@citrix.com>:
>>         and update them from Credit2 and RTDS schedulers.
>>
>>         Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
>>         Cc: Meng Xu <xumengpanda@gmail.com>
>>         Cc: George Dunlap <george.dunlap@eu.citrix.com>
>>         Cc: Jan Beulich <JBeulich@suse.com>
>>         Cc: Keir Fraser <keir@xen.org>
>>         ---
>>          xen/common/sched_credit2.c   |    2 ++
>>          xen/common/sched_rt.c        |    2 ++
>>          xen/include/xen/perfc_defn.h |    4 ++--
>>          3 files changed, 6 insertions(+), 2 deletions(-)
>>
>> The change for RTDS scheduler looks good to me.
>>
> Does this count as a "Reviewed-by: Meng Xu <mengxu@cis.upenn.edu>" ?
>
> Also, if yes, does it also apply to patch #2 ? That is unclear as
> sched_rt.c is modified in patches #1, #2 ad #3, while what you did is:
>  - you explicitly provided the tag for patch #1
>  - you said "looks good" for this for patch #3
>  - you said nothing for patch #2
>
> The bottom line of all this being: with Ack-s/Reviewed-by-s, it's always
> better be pretty explicit! :-D

I see. Thank you very much, Dario, for explaining this to me! :-)

After you add return before no_tickle:, this patch is good to go, IMHO.

So after the this change,
Reviewed-by: Meng Xu <mengxu@cis.upenn.edu>

Thank you very much!

Best,

Meng



-- 


-----------
Meng Xu
PhD Student in Computer and Information Science
University of Pennsylvania

Re: [PATCH 2/4] xen: sched: make counters for vCPU sleep and wakeup generic

[Meng Xu]

[I see the reason why I neglected this patch: my gmail just filter it
into the Forum category and I didn't see it. :-)
Dario, Do you have any suggestion of the email client (maybe the one
you guys are using)?  ]

2015-02-26 8:37 GMT-05:00 Dario Faggioli <dario.faggioli@citrix.com>:
> and update them from Credit2 and RTDS. In Credit2, while there,
> remove some stale comments too.
>
> Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
> Cc: George Dunlap <george.dunlap@eu.citrix.com>
> Cc: Jan Beulich <JBeulich@suse.com>
> Cc: Keir Fraser <keir@xen.org>
> ---
>  xen/common/sched_credit2.c   |   12 ++++++++----
>  xen/common/sched_rt.c        |   12 ++++++++++++
>  xen/include/xen/perfc_defn.h |   10 +++++-----
>  3 files changed, 25 insertions(+), 9 deletions(-)
>
> diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
> index ad0a5d4..2b852cc 100644
> --- a/xen/common/sched_credit2.c
> +++ b/xen/common/sched_credit2.c
> @@ -931,6 +931,7 @@ csched2_vcpu_sleep(const struct scheduler *ops, struct vcpu *vc)
>      struct csched2_vcpu * const svc = CSCHED2_VCPU(vc);
>
>      BUG_ON( is_idle_vcpu(vc) );
> +    SCHED_STAT_CRANK(vcpu_sleep);
>
>      if ( per_cpu(schedule_data, vc->processor).curr == vc )
>          cpu_raise_softirq(vc->processor, SCHEDULE_SOFTIRQ);
> @@ -956,19 +957,22 @@ csched2_vcpu_wake(const struct scheduler *ops, struct vcpu *vc)
>
>      BUG_ON( is_idle_vcpu(vc) );
>
> -    /* Make sure svc priority mod happens before runq check */
>      if ( unlikely(per_cpu(schedule_data, vc->processor).curr == vc) )
>      {
> +        SCHED_STAT_CRANK(vcpu_wake_running);
>          goto out;
>      }
> -
>      if ( unlikely(__vcpu_on_runq(svc)) )
>      {
> -        /* If we've boosted someone that's already on a runqueue, prioritize
> -         * it and inform the cpu in question. */
> +        SCHED_STAT_CRANK(vcpu_wake_onrunq);
>          goto out;
>      }
>
> +    if ( likely(vcpu_runnable(vc)) )
> +        SCHED_STAT_CRANK(vcpu_wake_runnable);
> +    else
> +        SCHED_STAT_CRANK(vcpu_wake_not_runnable);
> +
>      /* If the context hasn't been saved for this vcpu yet, we can't put it on
>       * another runqueue.  Instead, we set a flag so that it will be put on the runqueue
>       * after the context has been saved. */
> diff --git a/xen/common/sched_rt.c b/xen/common/sched_rt.c
> index 58dd646..49d1b83 100644
> --- a/xen/common/sched_rt.c
> +++ b/xen/common/sched_rt.c
> @@ -851,6 +851,7 @@ rt_vcpu_sleep(const struct scheduler *ops, struct vcpu *vc)
>      struct rt_vcpu * const svc = rt_vcpu(vc);
>
>      BUG_ON( is_idle_vcpu(vc) );
> +    SCHED_STAT_CRANK(vcpu_sleep);
>
>      if ( curr_on_cpu(vc->processor) == vc )
>          cpu_raise_softirq(vc->processor, SCHEDULE_SOFTIRQ);
> @@ -966,11 +967,22 @@ rt_vcpu_wake(const struct scheduler *ops, struct vcpu *vc)
>      BUG_ON( is_idle_vcpu(vc) );
>
>      if ( unlikely(curr_on_cpu(vc->processor) == vc) )
> +    {
> +        SCHED_STAT_CRANK(vcpu_wake_running);
>          return;
> +    }
>
>      /* on RunQ/DepletedQ, just update info is ok */
>      if ( unlikely(__vcpu_on_q(svc)) )
> +    {
> +        SCHED_STAT_CRANK(vcpu_wake_onrunq);
>          return;
> +    }
> +
> +    if ( likely(vcpu_runnable(vc)) )
> +        SCHED_STAT_CRANK(vcpu_wake_runnable);
> +    else
> +        SCHED_STAT_CRANK(vcpu_wake_not_runnable);
>
>      /* If context hasn't been saved for this vcpu yet, we can't put it on
>       * the Runqueue/DepletedQ. Instead, we set a flag so that it will be
> diff --git a/xen/include/xen/perfc_defn.h b/xen/include/xen/perfc_defn.h
> index 3ac7b45..2dc78fe 100644
> --- a/xen/include/xen/perfc_defn.h
> +++ b/xen/include/xen/perfc_defn.h
> @@ -21,6 +21,11 @@ PERFCOUNTER(dom_init,               "sched: dom_init")
>  PERFCOUNTER(dom_destroy,            "sched: dom_destroy")
>  PERFCOUNTER(vcpu_init,              "sched: vcpu_init")
>  PERFCOUNTER(vcpu_destroy,           "sched: vcpu_destroy")
> +PERFCOUNTER(vcpu_sleep,             "sched: vcpu_sleep")
> +PERFCOUNTER(vcpu_wake_running,      "sched: vcpu_wake_running")
> +PERFCOUNTER(vcpu_wake_onrunq,       "sched: vcpu_wake_onrunq")
> +PERFCOUNTER(vcpu_wake_runnable,     "sched: vcpu_wake_runnable")
> +PERFCOUNTER(vcpu_wake_not_runnable, "sched: vcpu_wake_not_runnable")
>
>  /* credit specific counters */
>  PERFCOUNTER(delay_ms,               "csched: delay")
> @@ -32,11 +37,6 @@ PERFCOUNTER(acct_reorder,           "csched: acct_reorder")
>  PERFCOUNTER(acct_min_credit,        "csched: acct_min_credit")
>  PERFCOUNTER(acct_vcpu_active,       "csched: acct_vcpu_active")
>  PERFCOUNTER(acct_vcpu_idle,         "csched: acct_vcpu_idle")
> -PERFCOUNTER(vcpu_sleep,             "csched: vcpu_sleep")
> -PERFCOUNTER(vcpu_wake_running,      "csched: vcpu_wake_running")
> -PERFCOUNTER(vcpu_wake_onrunq,       "csched: vcpu_wake_onrunq")
> -PERFCOUNTER(vcpu_wake_runnable,     "csched: vcpu_wake_runnable")
> -PERFCOUNTER(vcpu_wake_not_runnable, "csched: vcpu_wake_not_runnable")
>  PERFCOUNTER(vcpu_park,              "csched: vcpu_park")
>  PERFCOUNTER(vcpu_unpark,            "csched: vcpu_unpark")
>  PERFCOUNTER(tickle_idlers_none,     "csched: tickle_idlers_none")
>

Reviewed-by: Meng Xu <mengxu@cis.upenn.edu>

Thanks,

Meng


-----------
Meng Xu
PhD Student in Computer and Information Science
University of Pennsylvania

Re: [PATCH 2/4] xen: sched: make counters for vCPU sleep and wakeup generic

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1319 bytes --]

[Removing others, to avoi bothering them]

On Fri, 2015-02-27 at 10:01 -0500, Meng Xu wrote:
> [I see the reason why I neglected this patch: my gmail just filter it
> into the Forum category and I didn't see it. :-)
> Dario, Do you have any suggestion of the email client (maybe the one
> you guys are using)?  ]
> 
These appear to be at least partially my fault, as ...

> 2015-02-26 8:37 GMT-05:00 Dario Faggioli <dario.faggioli@citrix.com>:
> > and update them from Credit2 and RTDS. In Credit2, while there,
> > remove some stale comments too.
> >
> > Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
> > Cc: George Dunlap <george.dunlap@eu.citrix.com>
> > Cc: Jan Beulich <JBeulich@suse.com>
> > Cc: Keir Fraser <keir@xen.org>
>
... as you can see from here, it looks like I haven't Cc-ed you to this
email, despite sched_rt.c being changed in the patch. Sorry for this.

About email client, that's not something you can easily recommend, it's
too personal. I use evolution because it's integrated with the GNOME
desktop, but I wouldn't call it great. A lot of people are happy with
Mutt, and I'm sure there are many other solutions (including people
using gmail directly) so, really, you should try and use whatever you
find out to be more comfortable for you. :-)

Dario


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel