* [RFC PATCH v2 0/7] Repurpose SEDF Scheduler for Real-time Use
@ 2014-07-09 20:55 Josh Whitehead
2014-07-09 20:55 ` [RFC PATCH v2 1/7] Removed all code from sedf not needed for basic EDF functionality Josh Whitehead
` (7 more replies)
0 siblings, 8 replies; 30+ messages in thread
From: Josh Whitehead @ 2014-07-09 20:55 UTC (permalink / raw)
To: Xen-devel
Cc: Ian Campbell, Stefano Stabellini, George Dunlap, Dario Faggioli,
Ian Jackson, Robert VanVossen, Nathan Studer, Josh Whitehead
This patch series has been updated in response to the discussion on the previous
patch series. The first patch was split in to three parts to make it easier to
review; the first patch now only removes unneeded code, the 2nd updates some
formatting and variable names, and the 3rd only adds the new functionality.
Additionally the other patches modifying the toolstack have been divided to more
logically coincide (as best as we were able to determine) with the various
pieces of the toolstack. As a side note: Would it be possible to specify,
perhaps in the MAINTAINERS file, where (which files) the split should occur for
each piece of the toolstack? We used what was mentioned in the discussion to
make the division, howeve having that documented somewhere would be very helpful
in the future.
Also in regard to previous discussion, attempts have been made to protect the
interface versioning through the use of if-defs etc. More likely needs to be
done with the versioning, please let us know. We have tried to indicate which
files we most unsure of in the change-logs for each patch.
Finally, the amount of name changes has been significantly reduced, the most
prominent being to keep the "sedf" name and not switch to a new name as was
also decided through previous discussion.
Hopefully these changes have made the series easier to review, we look forward
to everyone's comments and suggestions.
Below is the original patch cover letter (with minor updates to reflect the lack
of name changes) for reference:
----------------------------------------
NEED:
With the increased interest in embedded Xen, there is a need for a suitable
real-time scheduler. The arinc653 scheduler currently only supports a
single core and has limited niche appeal, while the sedf scheduler is
widely consider deprecated and is currently a mess. This patchset
repurposes the current sedf scheduler and adds a more capable and robust
real-time scheduler suitable for embedded use to the Xen repertoire.
PROPOSED SOLUTION:
Repurposing of the sedf scheduler was accomplished by implementing the
Constant Bandwidth Server (CBS) algorithm (as originally proposed for us in
Xen by Dario Faggioli) which is capable of properly handling mixed soft
real-time and hard real-time tasks (domains/vcpus) on the same system.
If all domains in the system are set as hard real-time domains (the default
behavior) the scheduler acts as a very simple EDF scheduler, akin to the
original version of the SEDF scheduler. If the "soft" flag is set for a
domain it enables the CBS algorithm to set the deadlines of a domain. The
primary advantage to this algorithm being that it allows a domain/vcpu to
consume additional CPU resources when they're available (work conserving)
while not affecting the deadlines of other domains (temporal isolation) but
still enforcing the domain receives a guaranteed minimum level of service.
This functionality allows hard and soft tasks to interact in a safe and
predictable fashion (one type is not adversely impacting the performance of
the other), while still making efficient use of all available system
resources, i.e. unlike a pure EDF system there is little or no wasted time
where the CPU is sitting idle.
A full discussion of the Constant Bandwidth Server algorithm and its
behavior is located here:
http://retis.sssup.it/~giorgio/paps/2004/rtsj04-cbs.pdf
USAGE INSTRUCTIONS:
After applying the patches the scheduler can be used just as before by
specifying the "sched=sedf" boot parameter. Though capable of booting Dom0
with "sched=sedf", the scheduler is best for testing and development purposes
in a separate CPU pool. This is because the scheduler is being targeted at
embedded real-time systems. Performance as a general purpose operating system
would be good, though noticeably less than the current credit schedulers.
To achieve its full potential the CBS scheduler would need to be applied to a
specific real-time system using a real-time OS where the system designer had
some foreknowledge of that systems workload.
CBS uses a resource reservation scheme where the resources provided to each
domain/vcpu are specified in milliseconds using the "period=" and "budget="
parameters in the domain's config file, or at the command line using the
"xl sched-sedf" command. The "bandwidth" or percentage of available
processing power that is guaranteed to a domain is simply the ratio of
budget / period. The total of these ratios for all domains being operated
by the CBS scheduler should never be >1. Currently there are no checks
implemented preventing a user from specifying a total of ratios >1, but
failure to follow this rule will likely result in the domains in question
becoming unstable and crashing as this would be akin to specifying more
than 100% of the total available processing power.
As an example, let's look at a setup of 4 CBS operated domains: 1 hard RT
and 3 soft RT, each with a period of 40 and a budget of 10. Each domain
has a bandwidth of 0.25 (10/40), which with 4 domains means our total
bandwidth is 1.0 meeting that requirement. In this scenario, the hard RT
domain would be guaranteed 25% of the CPU time, but would also never be
able to use more than that. The 3 soft RT domains would be able to use
up to 100% of the CPU depending on system load across all the domains,
but would each be guaranteed to receive 25% of the processor even under
fully loaded conditions. While the 3 soft RT domains could make use of
all of the CPU, the CBS algorithm prevents them from causing the hard RT
domain to miss any deadlines.
Instructions for using the "xl sched-sedf" command and modifying a domains
period and budget parameters are located in the xl man page.
FUTURE DEVELOPMENT:
Though useful in its current state, there are a few additional features
and modifications we'd like to make in the future:
1) More efficient multiprocessor support that makes better use of
available cores. This would likely be accomplished through the use of a
global EDF queue for all available PCPUs. Though the current version
does recognize and use multiple cores it takes some creative assigning
and pinning by the user to use them efficiently, the goal of this change
would be to have this load balancing occur seamlessly and automatically.
2) VCPU level parameter assignments instead of the current domain level
assignments. This would allow for more accurate and efficient division of
resources amongst domains and cpu pools. Currently all vcpus within a
domain must use the same period and budget parameters.
3) Alternate scheduling algorithms (e.g. Deferrable Server) could now
be implemented in this scheduler with relative ease given the
simplification of the underlying EDF scheduler that was done as part of
this patchset. This would further expand the capabilities of Xen in
embedded real-time systems and give developers more options to fit their
individual system requirements.
Josh Whitehead (3):
Removed all code from sedf not needed for basic EDF functionality
Fixed formatting and misleading comments/variables. Added comments
and renamed variables to accurately reflect modern terminology
Added constant bandwidth server functionality to sedf scheduler
Robbie VanVossen (4):
Add cbs parameter support and removed sedf parameters from libxc
Add cbs parameter support and removed sedf parameters with a
LIBXL_API_VERSION gate from libxl.
Changed slice to budget in libxc for the sedf scheduler
Changed slice to budget in libxl for the sedf scheduler
docs/man/xl.cfg.pod.5 | 13 +-
tools/libxc/xc_sedf.c | 24 +-
tools/libxc/xenctrl.h | 12 +-
tools/libxl/libxl.c | 36 +-
tools/libxl/libxl.h | 2 +
tools/libxl/libxl_create.c | 61 --
tools/libxl/libxl_types.idl | 2 +
tools/libxl/xl_cmdimpl.c | 72 +++
tools/libxl/xl_cmdtable.c | 7 +
tools/python/xen/lowlevel/xc/xc.c | 42 +-
xen/common/sched_sedf.c | 1280 ++++++++++---------------------------
xen/include/public/domctl.h | 6 +-
12 files changed, 487 insertions(+), 1070 deletions(-)
--
1.7.9.5
^ permalink raw reply [flat|nested] 30+ messages in thread
* [RFC PATCH v2 1/7] Removed all code from sedf not needed for basic EDF functionality
2014-07-09 20:55 [RFC PATCH v2 0/7] Repurpose SEDF Scheduler for Real-time Use Josh Whitehead
@ 2014-07-09 20:55 ` Josh Whitehead
2014-07-11 7:05 ` Dario Faggioli
2014-07-09 20:55 ` [RFC PATCH v2 2/7] Fixed formatting and misleading comments/variables. Added comments and renamed variables to accurately reflect modern terminology Josh Whitehead
` (6 subsequent siblings)
7 siblings, 1 reply; 30+ messages in thread
From: Josh Whitehead @ 2014-07-09 20:55 UTC (permalink / raw)
To: Xen-devel
Cc: Ian Campbell, Stefano Stabellini, George Dunlap, Dario Faggioli,
Ian Jackson, Robert VanVossen, Nathan Studer, Joshua Whitehead
A large amount of code had been added to the sedf scheduler over the years that
had made it everything but a "simple" EDF scheduler. This patch removes all the
extra code build up, and adds a few lines needed to make the scheduler boot and
function normally.
Signed-off-by: Nathan Studer <nate.studer@gmail.com>
Signed-off-by: Joshua Whitehead <Josh.Whitehead@dornerworks.com>
---
xen/common/sched_sedf.c | 809 ++++-------------------------------------------
1 file changed, 56 insertions(+), 753 deletions(-)
diff --git a/xen/common/sched_sedf.c b/xen/common/sched_sedf.c
index 0c9011a..0cc3e06 100644
--- a/xen/common/sched_sedf.c
+++ b/xen/common/sched_sedf.c
@@ -25,18 +25,10 @@
#define CHECK(_p) ((void)0)
#endif
-#define EXTRA_NONE (0)
-#define EXTRA_AWARE (1)
-#define EXTRA_RUN_PEN (2)
-#define EXTRA_RUN_UTIL (4)
-#define EXTRA_WANT_PEN_Q (8)
-#define EXTRA_PEN_Q (0)
-#define EXTRA_UTIL_Q (1)
#define SEDF_ASLEEP (16)
-#define EXTRA_QUANTUM (MICROSECS(500))
-#define WEIGHT_PERIOD (MILLISECS(100))
-#define WEIGHT_SAFETY (MILLISECS(5))
+#define DEFAULT_PERIOD (MILLISECS(20))
+#define DEFAULT_SLICE (MILLISECS(10))
#define PERIOD_MAX MILLISECS(10000) /* 10s */
#define PERIOD_MIN (MICROSECS(10)) /* 10us */
@@ -58,24 +50,13 @@ struct sedf_priv_info {
struct sedf_vcpu_info {
struct vcpu *vcpu;
struct list_head list;
- struct list_head extralist[2];
/* Parameters for EDF */
s_time_t period; /* = relative deadline */
s_time_t slice; /* = worst case execution time */
- /* Advaced Parameters */
-
- /* Latency Scaling */
- s_time_t period_orig;
- s_time_t slice_orig;
- s_time_t latency;
-
/* Status of domain */
int status;
- /* Weights for "Scheduling for beginners/ lazy/ etc." ;) */
- short weight;
- short extraweight;
/* Bookkeeping */
s_time_t deadl_abs;
s_time_t sched_start_abs;
@@ -84,28 +65,17 @@ struct sedf_vcpu_info {
s_time_t block_abs;
s_time_t unblock_abs;
- /* Scores for {util, block penalty}-weighted extratime distribution */
- int score[2];
- s_time_t short_block_lost_tot;
-
- /* Statistics */
- s_time_t extra_time_tot;
-
#ifdef SEDF_STATS
s_time_t block_time_tot;
- s_time_t penalty_time_tot;
int block_tot;
int short_block_tot;
int long_block_tot;
- int pen_extra_blocks;
- int pen_extra_slices;
#endif
};
struct sedf_cpu_info {
struct list_head runnableq;
struct list_head waitq;
- struct list_head extraq[2];
s_time_t current_slice_expires;
};
@@ -115,103 +85,19 @@ struct sedf_cpu_info {
#define CPU_INFO(cpu) \
((struct sedf_cpu_info *)per_cpu(schedule_data, cpu).sched_priv)
#define LIST(d) (&EDOM_INFO(d)->list)
-#define EXTRALIST(d,i) (&(EDOM_INFO(d)->extralist[i]))
#define RUNQ(cpu) (&CPU_INFO(cpu)->runnableq)
#define WAITQ(cpu) (&CPU_INFO(cpu)->waitq)
-#define EXTRAQ(cpu,i) (&(CPU_INFO(cpu)->extraq[i]))
#define IDLETASK(cpu) (idle_vcpu[cpu])
#define PERIOD_BEGIN(inf) ((inf)->deadl_abs - (inf)->period)
#define DIV_UP(x,y) (((x) + (y) - 1) / y)
-#define extra_runs(inf) ((inf->status) & 6)
-#define extra_get_cur_q(inf) (((inf->status & 6) >> 1)-1)
#define sedf_runnable(edom) (!(EDOM_INFO(edom)->status & SEDF_ASLEEP))
static void sedf_dump_cpu_state(const struct scheduler *ops, int i);
-static inline int extraq_on(struct vcpu *d, int i)
-{
- return ((EXTRALIST(d,i)->next != NULL) &&
- (EXTRALIST(d,i)->next != EXTRALIST(d,i)));
-}
-
-static inline void extraq_del(struct vcpu *d, int i)
-{
- struct list_head *list = EXTRALIST(d,i);
- ASSERT(extraq_on(d,i));
- list_del(list);
- list->next = NULL;
- ASSERT(!extraq_on(d, i));
-}
-
-/*
- * Adds a domain to the queue of processes which are aware of extra time. List
- * is sorted by score, where a lower score means higher priority for an extra
- * slice. It also updates the score, by simply subtracting a fixed value from
- * each entry, in order to avoid overflow. The algorithm works by simply
- * charging each domain that recieved extratime with an inverse of its weight.
- */
-static inline void extraq_add_sort_update(struct vcpu *d, int i, int sub)
-{
- struct list_head *cur;
- struct sedf_vcpu_info *curinf;
-
- ASSERT(!extraq_on(d,i));
-
- /*
- * Iterate through all elements to find our "hole" and on our way
- * update all the other scores.
- */
- list_for_each ( cur, EXTRAQ(d->processor, i) )
- {
- curinf = list_entry(cur,struct sedf_vcpu_info,extralist[i]);
- curinf->score[i] -= sub;
- if ( EDOM_INFO(d)->score[i] < curinf->score[i] )
- break;
- }
-
- /* cur now contains the element, before which we'll enqueue */
- list_add(EXTRALIST(d,i),cur->prev);
-
- /* Continue updating the extraq */
- if ( (cur != EXTRAQ(d->processor,i)) && sub )
- {
- for ( cur = cur->next; cur != EXTRAQ(d->processor,i); cur = cur->next )
- {
- curinf = list_entry(cur,struct sedf_vcpu_info, extralist[i]);
- curinf->score[i] -= sub;
- }
- }
-
- ASSERT(extraq_on(d,i));
-}
-static inline void extraq_check(struct vcpu *d)
-{
- if ( extraq_on(d, EXTRA_UTIL_Q) )
- {
- if ( !(EDOM_INFO(d)->status & EXTRA_AWARE) &&
- !extra_runs(EDOM_INFO(d)) )
- extraq_del(d, EXTRA_UTIL_Q);
- }
- else
- {
- if ( (EDOM_INFO(d)->status & EXTRA_AWARE) && sedf_runnable(d) )
- extraq_add_sort_update(d, EXTRA_UTIL_Q, 0);
- }
-}
-
-static inline void extraq_check_add_unblocked(struct vcpu *d, int priority)
-{
- struct sedf_vcpu_info *inf = EDOM_INFO(d);
-
- if ( inf->status & EXTRA_AWARE )
- /* Put on the weighted extraq without updating any scores */
- extraq_add_sort_update(d, EXTRA_UTIL_Q, 0);
-}
-
static inline int __task_on_queue(struct vcpu *d)
{
return (((LIST(d))->next != NULL) && (LIST(d)->next != LIST(d)));
@@ -284,11 +170,7 @@ static inline void __add_to_runqueue_sort(struct vcpu *v)
static void sedf_insert_vcpu(const struct scheduler *ops, struct vcpu *v)
{
- if ( !is_idle_vcpu(v) )
- {
- extraq_check(v);
- }
- else
+ if ( is_idle_vcpu(v) )
{
EDOM_INFO(v)->deadl_abs = 0;
EDOM_INFO(v)->status &= ~SEDF_ASLEEP;
@@ -305,19 +187,22 @@ static void *sedf_alloc_vdata(const struct scheduler *ops, struct vcpu *v, void
inf->vcpu = v;
- /* Every VCPU gets an equal share of extratime by default */
inf->deadl_abs = 0;
- inf->latency = 0;
- inf->status = EXTRA_AWARE | SEDF_ASLEEP;
- inf->extraweight = 1;
- /* Upon creation all domain are best-effort */
- inf->period = WEIGHT_PERIOD;
- inf->slice = 0;
-
- inf->period_orig = inf->period; inf->slice_orig = inf->slice;
+ inf->status = SEDF_ASLEEP;
+
+ if (v->domain->domain_id == 0)
+ {
+ /* Domain 0, needs a slice to boot the machine */
+ inf->period = DEFAULT_PERIOD;
+ inf->slice = DEFAULT_SLICE;
+ }
+ else
+ {
+ inf->period = DEFAULT_PERIOD;
+ inf->slice = 0;
+ }
+
INIT_LIST_HEAD(&(inf->list));
- INIT_LIST_HEAD(&(inf->extralist[EXTRA_PEN_Q]));
- INIT_LIST_HEAD(&(inf->extralist[EXTRA_UTIL_Q]));
SCHED_STAT_CRANK(vcpu_init);
@@ -333,8 +218,6 @@ sedf_alloc_pdata(const struct scheduler *ops, int cpu)
BUG_ON(spc == NULL);
INIT_LIST_HEAD(&spc->waitq);
INIT_LIST_HEAD(&spc->runnableq);
- INIT_LIST_HEAD(&spc->extraq[EXTRA_PEN_Q]);
- INIT_LIST_HEAD(&spc->extraq[EXTRA_UTIL_Q]);
return (void *)spc;
}
@@ -417,20 +300,6 @@ static void desched_edf_dom(s_time_t now, struct vcpu* d)
if ( inf->cputime >= inf->slice )
{
inf->cputime -= inf->slice;
-
- if ( inf->period < inf->period_orig )
- {
- /* This domain runs in latency scaling or burst mode */
- inf->period *= 2;
- inf->slice *= 2;
- if ( (inf->period > inf->period_orig) ||
- (inf->slice > inf->slice_orig) )
- {
- /* Reset slice and period */
- inf->period = inf->period_orig;
- inf->slice = inf->slice_orig;
- }
- }
/* Set next deadline */
inf->deadl_abs += inf->period;
@@ -441,18 +310,8 @@ static void desched_edf_dom(s_time_t now, struct vcpu* d)
{
__add_to_waitqueue_sort(d);
}
- else
- {
- /* We have a blocked realtime task -> remove it from exqs too */
- if ( extraq_on(d, EXTRA_PEN_Q) )
- extraq_del(d, EXTRA_PEN_Q);
- if ( extraq_on(d, EXTRA_UTIL_Q) )
- extraq_del(d, EXTRA_UTIL_Q);
- }
ASSERT(EQ(sedf_runnable(d), __task_on_queue(d)));
- ASSERT(IMPLY(extraq_on(d, EXTRA_UTIL_Q) || extraq_on(d, EXTRA_PEN_Q),
- sedf_runnable(d)));
}
@@ -498,217 +357,12 @@ static void update_queues(
/* Put them back into the queue */
__add_to_waitqueue_sort(curinf->vcpu);
}
- else if ( unlikely((curinf->deadl_abs < now) ||
- (curinf->cputime > curinf->slice)) )
- {
- /*
- * We missed the deadline or the slice was already finished.
- * Might hapen because of dom_adj.
- */
- printk("\tDomain %i.%i exceeded it's deadline/"
- "slice (%"PRIu64" / %"PRIu64") now: %"PRIu64
- " cputime: %"PRIu64"\n",
- curinf->vcpu->domain->domain_id,
- curinf->vcpu->vcpu_id,
- curinf->deadl_abs, curinf->slice, now,
- curinf->cputime);
- __del_from_queue(curinf->vcpu);
-
- /* Common case: we miss one period */
- curinf->deadl_abs += curinf->period;
-
- /*
- * If we are still behind: modulo arithmetic, force deadline
- * to be in future and aligned to period borders.
- */
- if ( unlikely(curinf->deadl_abs < now) )
- curinf->deadl_abs +=
- DIV_UP(now - curinf->deadl_abs,
- curinf->period) * curinf->period;
- ASSERT(curinf->deadl_abs >= now);
-
- /* Give a fresh slice */
- curinf->cputime = 0;
- if ( PERIOD_BEGIN(curinf) > now )
- __add_to_waitqueue_sort(curinf->vcpu);
- else
- __add_to_runqueue_sort(curinf->vcpu);
- }
else
break;
}
}
-/*
- * removes a domain from the head of the according extraQ and
- * requeues it at a specified position:
- * round-robin extratime: end of extraQ
- * weighted ext.: insert in sorted list by score
- * if the domain is blocked / has regained its short-block-loss
- * time it is not put on any queue.
- */
-static void desched_extra_dom(s_time_t now, struct vcpu *d)
-{
- struct sedf_vcpu_info *inf = EDOM_INFO(d);
- int i = extra_get_cur_q(inf);
- unsigned long oldscore;
-
- ASSERT(extraq_on(d, i));
-
- /* Unset all running flags */
- inf->status &= ~(EXTRA_RUN_PEN | EXTRA_RUN_UTIL);
- /* Fresh slice for the next run */
- inf->cputime = 0;
- /* Accumulate total extratime */
- inf->extra_time_tot += now - inf->sched_start_abs;
- /* Remove extradomain from head of the queue. */
- extraq_del(d, i);
-
- /* Update the score */
- oldscore = inf->score[i];
- if ( i == EXTRA_PEN_Q )
- {
- /* Domain was running in L0 extraq */
- /* reduce block lost, probably more sophistication here!*/
- /*inf->short_block_lost_tot -= EXTRA_QUANTUM;*/
- inf->short_block_lost_tot -= now - inf->sched_start_abs;
-#if 0
- /* KAF: If we don't exit short-blocking state at this point
- * domain0 can steal all CPU for up to 10 seconds before
- * scheduling settles down (when competing against another
- * CPU-bound domain). Doing this seems to make things behave
- * nicely. Noone gets starved by default.
- */
- if ( inf->short_block_lost_tot <= 0 )
-#endif
- {
- /* We have (over-)compensated our block penalty */
- inf->short_block_lost_tot = 0;
- /* We don't want a place on the penalty queue anymore! */
- inf->status &= ~EXTRA_WANT_PEN_Q;
- goto check_extra_queues;
- }
-
- /*
- * We have to go again for another try in the block-extraq,
- * the score is not used incremantally here, as this is
- * already done by recalculating the block_lost
- */
- inf->score[EXTRA_PEN_Q] = (inf->period << 10) /
- inf->short_block_lost_tot;
- oldscore = 0;
- }
- else
- {
- /*
- * Domain was running in L1 extraq => score is inverse of
- * utilization and is used somewhat incremental!
- */
- if ( !inf->extraweight )
- {
- /* NB: use fixed point arithmetic with 10 bits */
- inf->score[EXTRA_UTIL_Q] = (inf->period << 10) /
- inf->slice;
- }
- else
- {
- /*
- * Conversion between realtime utilisation and extrawieght:
- * full (ie 100%) utilization is equivalent to 128 extraweight
- */
- inf->score[EXTRA_UTIL_Q] = (1<<17) / inf->extraweight;
- }
- }
-
- check_extra_queues:
- /* Adding a runnable domain to the right queue and removing blocked ones */
- if ( sedf_runnable(d) )
- {
- /* Add according to score: weighted round robin */
- if (((inf->status & EXTRA_AWARE) && (i == EXTRA_UTIL_Q)) ||
- ((inf->status & EXTRA_WANT_PEN_Q) && (i == EXTRA_PEN_Q)))
- extraq_add_sort_update(d, i, oldscore);
- }
- else
- {
- /* Remove this blocked domain from the waitq! */
- __del_from_queue(d);
- /* Make sure that we remove a blocked domain from the other
- * extraq too. */
- if ( i == EXTRA_PEN_Q )
- {
- if ( extraq_on(d, EXTRA_UTIL_Q) )
- extraq_del(d, EXTRA_UTIL_Q);
- }
- else
- {
- if ( extraq_on(d, EXTRA_PEN_Q) )
- extraq_del(d, EXTRA_PEN_Q);
- }
- }
-
- ASSERT(EQ(sedf_runnable(d), __task_on_queue(d)));
- ASSERT(IMPLY(extraq_on(d, EXTRA_UTIL_Q) || extraq_on(d, EXTRA_PEN_Q),
- sedf_runnable(d)));
-}
-
-
-static struct task_slice sedf_do_extra_schedule(
- s_time_t now, s_time_t end_xt, struct list_head *extraq[], int cpu)
-{
- struct task_slice ret = { 0 };
- struct sedf_vcpu_info *runinf;
- ASSERT(end_xt > now);
-
- /* Enough time left to use for extratime? */
- if ( end_xt - now < EXTRA_QUANTUM )
- goto return_idle;
-
- if ( !list_empty(extraq[EXTRA_PEN_Q]) )
- {
- /*
- * We still have elements on the level 0 extraq
- * => let those run first!
- */
- runinf = list_entry(extraq[EXTRA_PEN_Q]->next,
- struct sedf_vcpu_info, extralist[EXTRA_PEN_Q]);
- runinf->status |= EXTRA_RUN_PEN;
- ret.task = runinf->vcpu;
- ret.time = EXTRA_QUANTUM;
-#ifdef SEDF_STATS
- runinf->pen_extra_slices++;
-#endif
- }
- else
- {
- if ( !list_empty(extraq[EXTRA_UTIL_Q]) )
- {
- /* Use elements from the normal extraqueue */
- runinf = list_entry(extraq[EXTRA_UTIL_Q]->next,
- struct sedf_vcpu_info,
- extralist[EXTRA_UTIL_Q]);
- runinf->status |= EXTRA_RUN_UTIL;
- ret.task = runinf->vcpu;
- ret.time = EXTRA_QUANTUM;
- }
- else
- goto return_idle;
- }
-
- ASSERT(ret.time > 0);
- ASSERT(sedf_runnable(ret.task));
- return ret;
-
- return_idle:
- ret.task = IDLETASK(cpu);
- ret.time = end_xt - now;
- ASSERT(ret.time > 0);
- ASSERT(sedf_runnable(ret.task));
- return ret;
-}
-
-
static int sedf_init(struct scheduler *ops)
{
struct sedf_priv_info *prv;
@@ -748,8 +402,6 @@ static struct task_slice sedf_do_schedule(
struct list_head *runq = RUNQ(cpu);
struct list_head *waitq = WAITQ(cpu);
struct sedf_vcpu_info *inf = EDOM_INFO(current);
- struct list_head *extraq[] = {
- EXTRAQ(cpu, EXTRA_PEN_Q), EXTRAQ(cpu, EXTRA_UTIL_Q)};
struct sedf_vcpu_info *runinf, *waitinf;
struct task_slice ret;
@@ -770,15 +422,7 @@ static struct task_slice sedf_do_schedule(
if ( inf->status & SEDF_ASLEEP )
inf->block_abs = now;
- if ( unlikely(extra_runs(inf)) )
- {
- /* Special treatment of domains running in extra time */
- desched_extra_dom(now, current);
- }
- else
- {
- desched_edf_dom(now, current);
- }
+ desched_edf_dom(now, current);
check_waitq:
update_queues(now, runq, waitq);
@@ -820,12 +464,9 @@ static struct task_slice sedf_do_schedule(
else
{
waitinf = list_entry(waitq->next,struct sedf_vcpu_info, list);
- /*
- * We could not find any suitable domain
- * => look for domains that are aware of extratime
- */
- ret = sedf_do_extra_schedule(now, PERIOD_BEGIN(waitinf),
- extraq, cpu);
+
+ ret.task = IDLETASK(cpu);
+ ret.time = PERIOD_BEGIN(waitinf) - now;
}
/*
@@ -833,11 +474,8 @@ static struct task_slice sedf_do_schedule(
* still can happen!!!
*/
if ( ret.time < 0)
- {
printk("Ouch! We are seriously BEHIND schedule! %"PRIi64"\n",
ret.time);
- ret.time = EXTRA_QUANTUM;
- }
ret.migrated = 0;
@@ -848,7 +486,6 @@ static struct task_slice sedf_do_schedule(
return ret;
}
-
static void sedf_sleep(const struct scheduler *ops, struct vcpu *d)
{
if ( is_idle_vcpu(d) )
@@ -864,14 +501,9 @@ static void sedf_sleep(const struct scheduler *ops, struct vcpu *d)
{
if ( __task_on_queue(d) )
__del_from_queue(d);
- if ( extraq_on(d, EXTRA_UTIL_Q) )
- extraq_del(d, EXTRA_UTIL_Q);
- if ( extraq_on(d, EXTRA_PEN_Q) )
- extraq_del(d, EXTRA_PEN_Q);
}
}
-
/*
* This function wakes up a domain, i.e. moves them into the waitqueue
* things to mention are: admission control is taking place nowhere at
@@ -904,8 +536,6 @@ static void sedf_sleep(const struct scheduler *ops, struct vcpu *d)
*
* -this also doesn't disturb scheduling, but might lead to the fact, that
* the domain can't finish it's workload in the period
- * -in addition to that the domain can be treated prioritised when
- * extratime is available
* -addition: experiments have shown that this may have a HUGE impact on
* performance of other domains, becaus it can lead to excessive context
* switches
@@ -931,10 +561,6 @@ static void sedf_sleep(const struct scheduler *ops, struct vcpu *d)
* DRB______D___URRRR___D...<prev [Thread] next>
* (D) <- old deadline was here
* -problem: deadlines don't occur isochronous anymore
- * Part 2c (Improved Atropos design)
- * -when a domain unblocks it is given a very short period (=latency hint)
- * and slice length scaled accordingly
- * -both rise again to the original value (e.g. get doubled every period)
*
* 3. Unconservative (i.e. incorrect)
* -to boost the performance of I/O dependent domains it would be possible
@@ -944,100 +570,11 @@ static void sedf_sleep(const struct scheduler *ops, struct vcpu *d)
* -either behaviour can lead to missed deadlines in other domains as
* opposed to approaches 1,2a,2b
*/
-static void unblock_short_extra_support(
- struct sedf_vcpu_info* inf, s_time_t now)
-{
- /*
- * This unblocking scheme tries to support the domain, by assigning it
- * a priority in extratime distribution according to the loss of time
- * in this slice due to blocking
- */
- s_time_t pen;
-
- /* No more realtime execution in this period! */
- inf->deadl_abs += inf->period;
- if ( likely(inf->block_abs) )
- {
- /* Treat blocked time as consumed by the domain */
- /*inf->cputime += now - inf->block_abs;*/
- /*
- * Penalty is time the domain would have
- * had if it continued to run.
- */
- pen = (inf->slice - inf->cputime);
- if ( pen < 0 )
- pen = 0;
- /* Accumulate all penalties over the periods */
- /*inf->short_block_lost_tot += pen;*/
- /* Set penalty to the current value */
- inf->short_block_lost_tot = pen;
- /* Not sure which one is better.. but seems to work well... */
-
- if ( inf->short_block_lost_tot )
- {
- inf->score[0] = (inf->period << 10) /
- inf->short_block_lost_tot;
-#ifdef SEDF_STATS
- inf->pen_extra_blocks++;
-#endif
- if ( extraq_on(inf->vcpu, EXTRA_PEN_Q) )
- /* Remove domain for possible resorting! */
- extraq_del(inf->vcpu, EXTRA_PEN_Q);
- else
- /*
- * Remember that we want to be on the penalty q
- * so that we can continue when we (un-)block
- * in penalty-extratime
- */
- inf->status |= EXTRA_WANT_PEN_Q;
-
- /* (re-)add domain to the penalty extraq */
- extraq_add_sort_update(inf->vcpu, EXTRA_PEN_Q, 0);
- }
- }
-
- /* Give it a fresh slice in the next period! */
- inf->cputime = 0;
-}
-
-
-static void unblock_long_cons_b(struct sedf_vcpu_info* inf,s_time_t now)
-{
- /* Conservative 2b */
-
- /* Treat the unblocking time as a start of a new period */
- inf->deadl_abs = now + inf->period;
- inf->cputime = 0;
-}
-
-
-#define DOMAIN_EDF 1
-#define DOMAIN_EXTRA_PEN 2
-#define DOMAIN_EXTRA_UTIL 3
-#define DOMAIN_IDLE 4
-static inline int get_run_type(struct vcpu* d)
-{
- struct sedf_vcpu_info* inf = EDOM_INFO(d);
- if (is_idle_vcpu(d))
- return DOMAIN_IDLE;
- if (inf->status & EXTRA_RUN_PEN)
- return DOMAIN_EXTRA_PEN;
- if (inf->status & EXTRA_RUN_UTIL)
- return DOMAIN_EXTRA_UTIL;
- return DOMAIN_EDF;
-}
-
/*
* Compares two domains in the relation of whether the one is allowed to
* interrupt the others execution.
* It returns true (!=0) if a switch to the other domain is good.
- * Current Priority scheme is as follows:
- * EDF > L0 (penalty based) extra-time >
- * L1 (utilization) extra-time > idle-domain
- * In the same class priorities are assigned as following:
- * EDF: early deadline > late deadline
- * L0 extra-time: lower score > higher score
*/
static inline int should_switch(struct vcpu *cur,
struct vcpu *other,
@@ -1046,32 +583,17 @@ static inline int should_switch(struct vcpu *cur,
struct sedf_vcpu_info *cur_inf, *other_inf;
cur_inf = EDOM_INFO(cur);
other_inf = EDOM_INFO(other);
-
+
+ /* Always interrupt idle vcpu. */
+ if ( is_idle_vcpu(cur) )
+ return 1;
+
/* Check whether we need to make an earlier scheduling decision */
if ( PERIOD_BEGIN(other_inf) <
CPU_INFO(other->processor)->current_slice_expires )
return 1;
- /* No timing-based switches need to be taken into account here */
- switch ( get_run_type(cur) )
- {
- case DOMAIN_EDF:
- /* Do not interrupt a running EDF domain */
- return 0;
- case DOMAIN_EXTRA_PEN:
- /* Check whether we also want the L0 ex-q with lower score */
- return ((other_inf->status & EXTRA_WANT_PEN_Q) &&
- (other_inf->score[EXTRA_PEN_Q] <
- cur_inf->score[EXTRA_PEN_Q]));
- case DOMAIN_EXTRA_UTIL:
- /* Check whether we want the L0 extraq. Don't
- * switch if both domains want L1 extraq. */
- return !!(other_inf->status & EXTRA_WANT_PEN_Q);
- case DOMAIN_IDLE:
- return 1;
- }
-
- return 1;
+ return 0;
}
static void sedf_wake(const struct scheduler *ops, struct vcpu *d)
@@ -1087,8 +609,6 @@ static void sedf_wake(const struct scheduler *ops, struct vcpu *d)
ASSERT(!sedf_runnable(d));
inf->status &= ~SEDF_ASLEEP;
- ASSERT(!extraq_on(d, EXTRA_UTIL_Q));
- ASSERT(!extraq_on(d, EXTRA_PEN_Q));
if ( unlikely(inf->deadl_abs == 0) )
{
@@ -1100,43 +620,15 @@ static void sedf_wake(const struct scheduler *ops, struct vcpu *d)
inf->block_tot++;
#endif
- if ( unlikely(now < PERIOD_BEGIN(inf)) )
+ if ( now < inf->deadl_abs )
{
- /* Unblocking in extra-time! */
- if ( inf->status & EXTRA_WANT_PEN_Q )
- {
- /*
- * We have a domain that wants compensation
- * for block penalty and did just block in
- * its compensation time. Give it another
- * chance!
- */
- extraq_add_sort_update(d, EXTRA_PEN_Q, 0);
- }
- extraq_check_add_unblocked(d, 0);
- }
+ /* Short blocking */
+ inf->short_block_tot++;
+ }
else
- {
- if ( now < inf->deadl_abs )
- {
- /* Short blocking */
-#ifdef SEDF_STATS
- inf->short_block_tot++;
-#endif
- unblock_short_extra_support(inf, now);
-
- extraq_check_add_unblocked(d, 1);
- }
- else
- {
+ {
/* Long unblocking */
-#ifdef SEDF_STATS
- inf->long_block_tot++;
-#endif
- unblock_long_cons_b(inf, now);
-
- extraq_check_add_unblocked(d, 1);
- }
+ inf->long_block_tot++;
}
if ( PERIOD_BEGIN(inf) > now )
@@ -1149,13 +641,9 @@ static void sedf_wake(const struct scheduler *ops, struct vcpu *d)
if ( inf->block_abs != 0 )
{
inf->block_time_tot += now - inf->block_abs;
- inf->penalty_time_tot +=
- PERIOD_BEGIN(inf) + inf->cputime - inf->block_abs;
}
#endif
- /* Sanity check: make sure each extra-aware domain IS on the util-q! */
- ASSERT(IMPLY(inf->status & EXTRA_AWARE, extraq_on(d, EXTRA_UTIL_Q)));
ASSERT(__task_on_queue(d));
/*
* Check whether the awakened task needs to invoke the do_schedule
@@ -1176,29 +664,21 @@ static void sedf_dump_domain(struct vcpu *d)
{
printk("%i.%i has=%c ", d->domain->domain_id, d->vcpu_id,
d->is_running ? 'T':'F');
- printk("p=%"PRIu64" sl=%"PRIu64" ddl=%"PRIu64" w=%hu"
- " sc=%i xtr(%s)=%"PRIu64" ew=%hu",
- EDOM_INFO(d)->period, EDOM_INFO(d)->slice, EDOM_INFO(d)->deadl_abs,
- EDOM_INFO(d)->weight,
- EDOM_INFO(d)->score[EXTRA_UTIL_Q],
- (EDOM_INFO(d)->status & EXTRA_AWARE) ? "yes" : "no",
- EDOM_INFO(d)->extra_time_tot, EDOM_INFO(d)->extraweight);
+ printk("p=%"PRIu64" sl=%"PRIu64" ddl=%"PRIu64,
+ EDOM_INFO(d)->period, EDOM_INFO(d)->slice, EDOM_INFO(d)->deadl_abs);
#ifdef SEDF_STATS
if ( EDOM_INFO(d)->block_time_tot != 0 )
- printk(" pen=%"PRIu64"%%", (EDOM_INFO(d)->penalty_time_tot * 100) /
- EDOM_INFO(d)->block_time_tot);
+ printk(" pen=%"PRIu64"%%", EDOM_INFO(d)->block_time_tot);
if ( EDOM_INFO(d)->block_tot != 0 )
- printk("\n blks=%u sh=%u (%u%%) (shex=%i "\
- "shexsl=%i) l=%u (%u%%) avg: b=%"PRIu64" p=%"PRIu64"",
+ printk("\n blks=%u sh=%u (%u%%) "\
+ "l=%u (%u%%) avg: b=%"PRIu64" p=%d",
EDOM_INFO(d)->block_tot, EDOM_INFO(d)->short_block_tot,
(EDOM_INFO(d)->short_block_tot * 100) / EDOM_INFO(d)->block_tot,
- EDOM_INFO(d)->pen_extra_blocks,
- EDOM_INFO(d)->pen_extra_slices,
EDOM_INFO(d)->long_block_tot,
(EDOM_INFO(d)->long_block_tot * 100) / EDOM_INFO(d)->block_tot,
(EDOM_INFO(d)->block_time_tot) / EDOM_INFO(d)->block_tot,
- (EDOM_INFO(d)->penalty_time_tot) / EDOM_INFO(d)->block_tot);
+ EDOM_INFO(d)->block_tot);
#endif
printk("\n");
}
@@ -1234,30 +714,6 @@ static void sedf_dump_cpu_state(const struct scheduler *ops, int i)
sedf_dump_domain(d_inf->vcpu);
}
- queue = EXTRAQ(i,EXTRA_PEN_Q); loop = 0;
- printk("\nEXTRAQ (penalty) rq %lx n: %lx, p: %lx\n",
- (unsigned long)queue, (unsigned long) queue->next,
- (unsigned long) queue->prev);
- list_for_each_safe ( list, tmp, queue )
- {
- d_inf = list_entry(list, struct sedf_vcpu_info,
- extralist[EXTRA_PEN_Q]);
- printk("%3d: ",loop++);
- sedf_dump_domain(d_inf->vcpu);
- }
-
- queue = EXTRAQ(i,EXTRA_UTIL_Q); loop = 0;
- printk("\nEXTRAQ (utilization) rq %lx n: %lx, p: %lx\n",
- (unsigned long)queue, (unsigned long) queue->next,
- (unsigned long) queue->prev);
- list_for_each_safe ( list, tmp, queue )
- {
- d_inf = list_entry(list, struct sedf_vcpu_info,
- extralist[EXTRA_UTIL_Q]);
- printk("%3d: ",loop++);
- sedf_dump_domain(d_inf->vcpu);
- }
-
loop = 0;
printk("\nnot on Q\n");
@@ -1279,199 +735,52 @@ static void sedf_dump_cpu_state(const struct scheduler *ops, int i)
}
-/* Adjusts periods and slices of the domains accordingly to their weights */
-static int sedf_adjust_weights(struct cpupool *c, int nr_cpus, int *sumw, s_time_t *sumt)
-{
- struct vcpu *p;
- struct domain *d;
- unsigned int cpu;
-
- /*
- * Sum across all weights. Notice that no runq locking is needed
- * here: the caller holds sedf_priv_info.lock and we're not changing
- * anything that is accessed during scheduling.
- */
- rcu_read_lock(&domlist_read_lock);
- for_each_domain_in_cpupool( d, c )
- {
- for_each_vcpu( d, p )
- {
- if ( (cpu = p->processor) >= nr_cpus )
- continue;
-
- if ( EDOM_INFO(p)->weight )
- {
- sumw[cpu] += EDOM_INFO(p)->weight;
- }
- else
- {
- /*
- * Don't modify domains who don't have a weight, but sum
- * up the time they need, projected to a WEIGHT_PERIOD,
- * so that this time is not given to the weight-driven
- * domains
- */
-
- /* Check for overflows */
- ASSERT((WEIGHT_PERIOD < ULONG_MAX)
- && (EDOM_INFO(p)->slice_orig < ULONG_MAX));
- sumt[cpu] +=
- (WEIGHT_PERIOD * EDOM_INFO(p)->slice_orig) /
- EDOM_INFO(p)->period_orig;
- }
- }
- }
- rcu_read_unlock(&domlist_read_lock);
-
- /*
- * Adjust all slices (and periods) to the new weight. Unlike above, we
- * need to take thr runq lock for the various VCPUs: we're modyfing
- * slice and period which are referenced during scheduling.
- */
- rcu_read_lock(&domlist_read_lock);
- for_each_domain_in_cpupool( d, c )
- {
- for_each_vcpu ( d, p )
- {
- if ( (cpu = p->processor) >= nr_cpus )
- continue;
- if ( EDOM_INFO(p)->weight )
- {
- /* Interrupts already off */
- spinlock_t *lock = vcpu_schedule_lock(p);
-
- EDOM_INFO(p)->period_orig =
- EDOM_INFO(p)->period = WEIGHT_PERIOD;
- EDOM_INFO(p)->slice_orig =
- EDOM_INFO(p)->slice =
- (EDOM_INFO(p)->weight *
- (WEIGHT_PERIOD - WEIGHT_SAFETY - sumt[cpu])) / sumw[cpu];
-
- vcpu_schedule_unlock(lock, p);
- }
- }
- }
- rcu_read_unlock(&domlist_read_lock);
-
- return 0;
-}
-
-
/* Set or fetch domain scheduling parameters */
static int sedf_adjust(const struct scheduler *ops, struct domain *p, struct xen_domctl_scheduler_op *op)
{
struct sedf_priv_info *prv = SEDF_PRIV(ops);
unsigned long flags;
- unsigned int nr_cpus = cpumask_last(&cpu_online_map) + 1;
- int *sumw = xzalloc_array(int, nr_cpus);
- s_time_t *sumt = xzalloc_array(s_time_t, nr_cpus);
struct vcpu *v;
int rc = 0;
/*
* Serialize against the pluggable scheduler lock to protect from
* concurrent updates. We need to take the runq lock for the VCPUs
- * as well, since we are touching extraweight, weight, slice and
- * period. As in sched_credit2.c, runq locks nest inside the
- * pluggable scheduler lock.
+ * as well, since we are touching slice and period.
+ *
+ * As in sched_credit2.c, runq locks nest inside the pluggable scheduler
+ * lock.
*/
spin_lock_irqsave(&prv->lock, flags);
if ( op->cmd == XEN_DOMCTL_SCHEDOP_putinfo )
{
- /*
- * These are used in sedf_adjust_weights() but have to be allocated in
- * this function, as we need to avoid nesting xmem_pool_alloc's lock
- * within our prv->lock.
- */
- if ( !sumw || !sumt )
- {
- /* Check for errors here, the _getinfo branch doesn't care */
- rc = -ENOMEM;
- goto out;
- }
-
/* Check for sane parameters */
- if ( !op->u.sedf.period && !op->u.sedf.weight )
+ if ( !op->u.sedf.period )
{
rc = -EINVAL;
goto out;
}
- if ( op->u.sedf.weight )
- {
- if ( (op->u.sedf.extratime & EXTRA_AWARE) &&
- (!op->u.sedf.period) )
- {
- /* Weight-driven domains with extratime only */
- for_each_vcpu ( p, v )
- {
- /* (Here and everywhere in the following) IRQs are already off,
- * hence vcpu_spin_lock() is the one. */
- spinlock_t *lock = vcpu_schedule_lock(v);
-
- EDOM_INFO(v)->extraweight = op->u.sedf.weight;
- EDOM_INFO(v)->weight = 0;
- EDOM_INFO(v)->slice = 0;
- EDOM_INFO(v)->period = WEIGHT_PERIOD;
- vcpu_schedule_unlock(lock, v);
- }
- }
- else
- {
- /* Weight-driven domains with real-time execution */
- for_each_vcpu ( p, v )
- {
- spinlock_t *lock = vcpu_schedule_lock(v);
-
- EDOM_INFO(v)->weight = op->u.sedf.weight;
- vcpu_schedule_unlock(lock, v);
- }
- }
- }
- else
+ /*
+ * Sanity checking
+ */
+ if ( (op->u.sedf.period > PERIOD_MAX) ||
+ (op->u.sedf.period < PERIOD_MIN) ||
+ (op->u.sedf.slice > op->u.sedf.period) ||
+ (op->u.sedf.slice < SLICE_MIN) )
{
- /*
- * Sanity checking: note that disabling extra weight requires
- * that we set a non-zero slice.
- */
- if ( (op->u.sedf.period > PERIOD_MAX) ||
- (op->u.sedf.period < PERIOD_MIN) ||
- (op->u.sedf.slice > op->u.sedf.period) ||
- (op->u.sedf.slice < SLICE_MIN) )
- {
- rc = -EINVAL;
- goto out;
- }
-
- /* Time-driven domains */
- for_each_vcpu ( p, v )
- {
- spinlock_t *lock = vcpu_schedule_lock(v);
-
- EDOM_INFO(v)->weight = 0;
- EDOM_INFO(v)->extraweight = 0;
- EDOM_INFO(v)->period_orig =
- EDOM_INFO(v)->period = op->u.sedf.period;
- EDOM_INFO(v)->slice_orig =
- EDOM_INFO(v)->slice = op->u.sedf.slice;
- vcpu_schedule_unlock(lock, v);
- }
- }
-
- rc = sedf_adjust_weights(p->cpupool, nr_cpus, sumw, sumt);
- if ( rc )
+ rc = -EINVAL;
goto out;
+ }
+ /* Time-driven domains */
for_each_vcpu ( p, v )
{
spinlock_t *lock = vcpu_schedule_lock(v);
- EDOM_INFO(v)->status =
- (EDOM_INFO(v)->status &
- ~EXTRA_AWARE) | (op->u.sedf.extratime & EXTRA_AWARE);
- EDOM_INFO(v)->latency = op->u.sedf.latency;
- extraq_check(v);
+ EDOM_INFO(v)->period = op->u.sedf.period;
+ EDOM_INFO(v)->slice = op->u.sedf.slice;
vcpu_schedule_unlock(lock, v);
}
}
@@ -1485,17 +794,11 @@ static int sedf_adjust(const struct scheduler *ops, struct domain *p, struct xen
op->u.sedf.period = EDOM_INFO(p->vcpu[0])->period;
op->u.sedf.slice = EDOM_INFO(p->vcpu[0])->slice;
- op->u.sedf.extratime = EDOM_INFO(p->vcpu[0])->status & EXTRA_AWARE;
- op->u.sedf.latency = EDOM_INFO(p->vcpu[0])->latency;
- op->u.sedf.weight = EDOM_INFO(p->vcpu[0])->weight;
}
out:
spin_unlock_irqrestore(&prv->lock, flags);
- xfree(sumt);
- xfree(sumw);
-
return rc;
}
--
1.7.9.5
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [RFC PATCH v2 2/7] Fixed formatting and misleading comments/variables. Added comments and renamed variables to accurately reflect modern terminology
2014-07-09 20:55 [RFC PATCH v2 0/7] Repurpose SEDF Scheduler for Real-time Use Josh Whitehead
2014-07-09 20:55 ` [RFC PATCH v2 1/7] Removed all code from sedf not needed for basic EDF functionality Josh Whitehead
@ 2014-07-09 20:55 ` Josh Whitehead
2014-07-11 3:59 ` Dario Faggioli
2014-07-09 20:55 ` [RFC PATCH v2 3/7] Added constant bandwidth server functionality to sedf scheduler Josh Whitehead
` (5 subsequent siblings)
7 siblings, 1 reply; 30+ messages in thread
From: Josh Whitehead @ 2014-07-09 20:55 UTC (permalink / raw)
To: Xen-devel
Cc: Ian Campbell, Stefano Stabellini, George Dunlap, Dario Faggioli,
Ian Jackson, Robert VanVossen, Nathan Studer, Joshua Whitehead
Due to the age of the scheduler there were many incorrect/misleading comments
and variable names, the bulk of which centered around the fact that "VCPU" and
"Domain" used to be synonymous. Therefore a large portion of these modifcations
involve simply changing a variable "d" to a "v" or a "dom" to "vcpu" so that the
comments and variable names are accurate to what's being used.
A few other name changes were also made, the most significant being the change
from "slice" to "budget" to better reflect modern terminology used in real-time
algorithms such as CBS and deferrable server.
Some varible, function name, and macro name changes were also made to bring the
code inline with current Xen coding standards.
Signed-off-by: Nathan Studer <nate.studer@gmail.com>
Signed-off-by: Joshua Whitehead <Josh.Whitehead@dornerworks.com>
---
xen/common/sched_sedf.c | 412 ++++++++++++++++++++++-------------------------
1 file changed, 190 insertions(+), 222 deletions(-)
diff --git a/xen/common/sched_sedf.c b/xen/common/sched_sedf.c
index 0cc3e06..a8dd3e0 100644
--- a/xen/common/sched_sedf.c
+++ b/xen/common/sched_sedf.c
@@ -1,8 +1,29 @@
/******************************************************************************
- * Simple EDF scheduler for xen
+ * Simple EDF Scheduler for xen
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ *
+ * by DornerWorks Ltd. (C) 2014 Grand Rapids, MI
+ *
+ * Adapted from code by Stephan Diestelhorst (C) 2004 Cambridge University
+ * and Mark Williamson (C) 2004 Intel Research Cambridge
*
- * by Stephan Diestelhorst (C) 2004 Cambridge University
- * based on code by Mark Williamson (C) 2004 Intel Research Cambridge
*/
#include <xen/lib.h>
@@ -28,14 +49,13 @@
#define SEDF_ASLEEP (16)
#define DEFAULT_PERIOD (MILLISECS(20))
-#define DEFAULT_SLICE (MILLISECS(10))
+#define DEFAULT_BUDGET (MILLISECS(10))
#define PERIOD_MAX MILLISECS(10000) /* 10s */
#define PERIOD_MIN (MICROSECS(10)) /* 10us */
-#define SLICE_MIN (MICROSECS(5)) /* 5us */
+#define BUDGET_MIN (MICROSECS(5)) /* 5us */
-#define IMPLY(a, b) (!(a) || (b))
-#define EQ(a, b) ((!!(a)) == (!!(b)))
+#define EQ(_A, _B) ((!!(_A)) == (!!(_B)))
struct sedf_dom_info {
@@ -52,16 +72,16 @@ struct sedf_vcpu_info {
struct list_head list;
/* Parameters for EDF */
- s_time_t period; /* = relative deadline */
- s_time_t slice; /* = worst case execution time */
+ s_time_t period; /* = Server scheduling period */
+ s_time_t budget; /* = Guarenteed minimum CPU time per period */
- /* Status of domain */
+ /* Status of vcpu */
int status;
/* Bookkeeping */
s_time_t deadl_abs;
s_time_t sched_start_abs;
s_time_t cputime;
- /* Times the domain un-/blocked */
+ /* Times the vcpu un-/blocked */
s_time_t block_abs;
s_time_t unblock_abs;
@@ -81,35 +101,35 @@ struct sedf_cpu_info {
#define SEDF_PRIV(_ops) \
((struct sedf_priv_info *)((_ops)->sched_data))
-#define EDOM_INFO(d) ((struct sedf_vcpu_info *)((d)->sched_priv))
-#define CPU_INFO(cpu) \
- ((struct sedf_cpu_info *)per_cpu(schedule_data, cpu).sched_priv)
-#define LIST(d) (&EDOM_INFO(d)->list)
-#define RUNQ(cpu) (&CPU_INFO(cpu)->runnableq)
-#define WAITQ(cpu) (&CPU_INFO(cpu)->waitq)
-#define IDLETASK(cpu) (idle_vcpu[cpu])
+#define SEDF_VCPU(_vcpu) ((struct sedf_vcpu_info *)((_vcpu)->sched_priv))
+#define SEDF_PCPU(_cpu) \
+ ((struct sedf_cpu_info *)per_cpu(schedule_data, _cpu).sched_priv)
+#define LIST(_vcpu) (&SEDF_VCPU(_vcpu)->list)
+#define RUNQ(_cpu) (&SEDF_PCPU(_cpu)->runnableq)
+#define WAITQ(_cpu) (&SEDF_PCPU(_cpu)->waitq)
+#define IDLETASK(_cpu) (idle_vcpu[_cpu])
#define PERIOD_BEGIN(inf) ((inf)->deadl_abs - (inf)->period)
-#define DIV_UP(x,y) (((x) + (y) - 1) / y)
+#define DIV_UP(_X, _Y) (((_X) + (_Y) - 1) / _Y)
-#define sedf_runnable(edom) (!(EDOM_INFO(edom)->status & SEDF_ASLEEP))
+#define sedf_runnable(edom) (!(SEDF_VCPU(edom)->status & SEDF_ASLEEP))
-static void sedf_dump_cpu_state(const struct scheduler *ops, int i);
+static void sedf_dump_cpu_state(const struct scheduler *ops, int cpu);
-static inline int __task_on_queue(struct vcpu *d)
+static inline int __task_on_queue(struct vcpu *v)
{
- return (((LIST(d))->next != NULL) && (LIST(d)->next != LIST(d)));
+ return (((LIST(v))->next != NULL) && (LIST(v)->next != LIST(v)));
}
-static inline void __del_from_queue(struct vcpu *d)
+static inline void __del_from_queue(struct vcpu *v)
{
- struct list_head *list = LIST(d);
- ASSERT(__task_on_queue(d));
+ struct list_head *list = LIST(v);
+ ASSERT(__task_on_queue(v));
list_del(list);
list->next = NULL;
- ASSERT(!__task_on_queue(d));
+ ASSERT(!__task_on_queue(v));
}
typedef int(*list_comparer)(struct list_head* el1, struct list_head* el2);
@@ -128,12 +148,12 @@ static inline void list_insert_sort(
list_add(element, cur->prev);
}
-#define DOMAIN_COMPARER(name, field, comp1, comp2) \
+#define VCPU_COMPARER(name, field, comp1, comp2) \
static int name##_comp(struct list_head* el1, struct list_head* el2) \
{ \
- struct sedf_vcpu_info *d1, *d2; \
- d1 = list_entry(el1,struct sedf_vcpu_info, field); \
- d2 = list_entry(el2,struct sedf_vcpu_info, field); \
+ struct sedf_vcpu_info *v1, *v2; \
+ v1 = list_entry(el1, struct sedf_vcpu_info, field); \
+ v2 = list_entry(el2, struct sedf_vcpu_info, field); \
if ( (comp1) == (comp2) ) \
return 0; \
if ( (comp1) < (comp2) ) \
@@ -143,11 +163,11 @@ static int name##_comp(struct list_head* el1, struct list_head* el2) \
}
/*
- * Adds a domain to the queue of processes which wait for the beginning of the
- * next period; this list is therefore sortet by this time, which is simply
+ * Adds a vcpu to the queue of processes which wait for the beginning of the
+ * next period; this list is therefore sorted by this time, which is simply
* absol. deadline - period.
*/
-DOMAIN_COMPARER(waitq, list, PERIOD_BEGIN(d1), PERIOD_BEGIN(d2));
+VCPU_COMPARER(waitq, list, PERIOD_BEGIN(v1), PERIOD_BEGIN(v2));
static inline void __add_to_waitqueue_sort(struct vcpu *v)
{
ASSERT(!__task_on_queue(v));
@@ -156,12 +176,12 @@ static inline void __add_to_waitqueue_sort(struct vcpu *v)
}
/*
- * Adds a domain to the queue of processes which have started their current
+ * Adds a vcpu to the queue of processes which have started their current
* period and are runnable (i.e. not blocked, dieing,...). The first element
* on this list is running on the processor, if the list is empty the idle
* task will run. As we are implementing EDF, this list is sorted by deadlines.
*/
-DOMAIN_COMPARER(runq, list, d1->deadl_abs, d2->deadl_abs);
+VCPU_COMPARER(runq, list, v1->deadl_abs, v2->deadl_abs);
static inline void __add_to_runqueue_sort(struct vcpu *v)
{
list_insert_sort(RUNQ(v->processor), LIST(v), runq_comp);
@@ -172,8 +192,8 @@ static void sedf_insert_vcpu(const struct scheduler *ops, struct vcpu *v)
{
if ( is_idle_vcpu(v) )
{
- EDOM_INFO(v)->deadl_abs = 0;
- EDOM_INFO(v)->status &= ~SEDF_ASLEEP;
+ SEDF_VCPU(v)->deadl_abs = 0;
+ SEDF_VCPU(v)->status &= ~SEDF_ASLEEP;
}
}
@@ -187,19 +207,19 @@ static void *sedf_alloc_vdata(const struct scheduler *ops, struct vcpu *v, void
inf->vcpu = v;
- inf->deadl_abs = 0;
- inf->status = SEDF_ASLEEP;
+ inf->deadl_abs = 0;
+ inf->status = SEDF_ASLEEP;
if (v->domain->domain_id == 0)
{
- /* Domain 0, needs a slice to boot the machine */
- inf->period = DEFAULT_PERIOD;
- inf->slice = DEFAULT_SLICE;
+ /* Domain 0, needs a budget to boot the machine */
+ inf->period = DEFAULT_PERIOD;
+ inf->budget = DEFAULT_BUDGET;
}
else
{
- inf->period = DEFAULT_PERIOD;
- inf->slice = 0;
+ inf->period = DEFAULT_PERIOD;
+ inf->budget = 0;
}
INIT_LIST_HEAD(&(inf->list));
@@ -273,45 +293,45 @@ static int sedf_pick_cpu(const struct scheduler *ops, struct vcpu *v)
}
/*
- * Handles the rescheduling & bookkeeping of domains running in their
+ * Handles the rescheduling & bookkeeping of vcpus running in their
* guaranteed timeslice.
*/
-static void desched_edf_dom(s_time_t now, struct vcpu* d)
+static void desched_edf_vcpu(s_time_t now, struct vcpu *v)
{
- struct sedf_vcpu_info* inf = EDOM_INFO(d);
+ struct sedf_vcpu_info* inf = SEDF_VCPU(v);
- /* Current domain is running in real time mode */
- ASSERT(__task_on_queue(d));
+ /* Current vcpu is running in real time mode */
+ ASSERT(__task_on_queue(v));
- /* Update the domain's cputime */
+ /* Update the vcpu's cputime */
inf->cputime += now - inf->sched_start_abs;
- /* Scheduling decisions which don't remove the running domain from
+ /* Scheduling decisions which don't remove the running vcpu from
* the runq */
- if ( (inf->cputime < inf->slice) && sedf_runnable(d) )
+ if ( (inf->cputime < inf->budget) && sedf_runnable(v) )
return;
- __del_from_queue(d);
+ __del_from_queue(v);
/*
* Manage bookkeeping (i.e. calculate next deadline, memorise
- * overrun-time of slice) of finished domains.
+ * overrun-time of budget) of finished vcpus.
*/
- if ( inf->cputime >= inf->slice )
+ if ( inf->cputime >= inf->budget )
{
- inf->cputime -= inf->slice;
+ inf->cputime -= inf->budget;
/* Set next deadline */
inf->deadl_abs += inf->period;
}
- /* Add a runnable domain to the waitqueue */
- if ( sedf_runnable(d) )
+ /* Add a runnable vcpu to the appropriate queue */
+ if ( sedf_runnable(v) )
{
- __add_to_waitqueue_sort(d);
+ __add_to_waitqueue_sort(v);
}
- ASSERT(EQ(sedf_runnable(d), __task_on_queue(d)));
+ ASSERT(EQ(sedf_runnable(v), __task_on_queue(v)));
}
@@ -335,14 +355,14 @@ static void update_queues(
__add_to_runqueue_sort(curinf->vcpu);
}
- /* Process the runq, find domains that are on the runq that shouldn't */
+ /* Process the runq, find vcpus that are on the runq that shouldn't */
list_for_each_safe ( cur, tmp, runq )
{
- curinf = list_entry(cur,struct sedf_vcpu_info,list);
+ curinf = list_entry(cur, struct sedf_vcpu_info, list);
- if ( unlikely(curinf->slice == 0) )
+ if ( unlikely(curinf->budget == 0) )
{
- /* Ignore domains with empty slice */
+ /* Ignore vcpus with empty budget */
__del_from_queue(curinf->vcpu);
/* Move them to their next period */
@@ -391,9 +411,9 @@ static void sedf_deinit(const struct scheduler *ops)
/*
* Main scheduling function
* Reasons for calling this function are:
- * -timeslice for the current period used up
- * -domain on waitqueue has started it's period
- * -and various others ;) in general: determine which domain to run next
+ * -budget for the current period is used up
+ * -vcpu on waitqueue has started it's period
+ * -and various others ;) in general: determine which vcpu to run next
*/
static struct task_slice sedf_do_schedule(
const struct scheduler *ops, s_time_t now, bool_t tasklet_work_scheduled)
@@ -401,18 +421,18 @@ static struct task_slice sedf_do_schedule(
int cpu = smp_processor_id();
struct list_head *runq = RUNQ(cpu);
struct list_head *waitq = WAITQ(cpu);
- struct sedf_vcpu_info *inf = EDOM_INFO(current);
+ struct sedf_vcpu_info *inf = SEDF_VCPU(current);
struct sedf_vcpu_info *runinf, *waitinf;
struct task_slice ret;
SCHED_STAT_CRANK(schedule);
- /* Idle tasks don't need any of the following stuf */
+ /* Idle tasks don't need any of the following stuff */
if ( is_idle_vcpu(current) )
goto check_waitq;
/*
- * Create local state of the status of the domain, in order to avoid
+ * Create local state of the status of the vcpu, in order to avoid
* inconsistent state during scheduling decisions, because data for
* vcpu_runnable is not protected by the scheduling lock!
*/
@@ -422,12 +442,12 @@ static struct task_slice sedf_do_schedule(
if ( inf->status & SEDF_ASLEEP )
inf->block_abs = now;
- desched_edf_dom(now, current);
+ desched_edf_vcpu(now, current);
check_waitq:
update_queues(now, runq, waitq);
/*
- * Now simply pick the first domain from the runqueue, which has the
+ * Now simply pick the first vcpu from the runqueue, which has the
* earliest deadline, because the list is sorted
*
* Tasklet work (which runs in idle VCPU context) overrides all else.
@@ -442,28 +462,28 @@ static struct task_slice sedf_do_schedule(
}
else if ( !list_empty(runq) )
{
- runinf = list_entry(runq->next,struct sedf_vcpu_info,list);
+ runinf = list_entry(runq->next, struct sedf_vcpu_info, list);
ret.task = runinf->vcpu;
if ( !list_empty(waitq) )
{
waitinf = list_entry(waitq->next,
- struct sedf_vcpu_info,list);
+ struct sedf_vcpu_info, list);
/*
- * Rerun scheduler, when scheduled domain reaches it's
- * end of slice or the first domain from the waitqueue
+ * Rerun scheduler, when scheduled vcpu consumes
+ * its budget or the first vcpu from the waitqueue
* gets ready.
*/
- ret.time = MIN(now + runinf->slice - runinf->cputime,
+ ret.time = MIN(now + runinf->budget - runinf->cputime,
PERIOD_BEGIN(waitinf)) - now;
}
else
{
- ret.time = runinf->slice - runinf->cputime;
+ ret.time = runinf->budget - runinf->cputime;
}
}
else
{
- waitinf = list_entry(waitq->next,struct sedf_vcpu_info, list);
+ waitinf = list_entry(waitq->next, struct sedf_vcpu_info, list);
ret.task = IDLETASK(cpu);
ret.time = PERIOD_BEGIN(waitinf) - now;
@@ -479,141 +499,89 @@ static struct task_slice sedf_do_schedule(
ret.migrated = 0;
- EDOM_INFO(ret.task)->sched_start_abs = now;
+ SEDF_VCPU(ret.task)->sched_start_abs = now;
CHECK(ret.time > 0);
ASSERT(sedf_runnable(ret.task));
- CPU_INFO(cpu)->current_slice_expires = now + ret.time;
+ SEDF_PCPU(cpu)->current_slice_expires = now + ret.time;
return ret;
}
-static void sedf_sleep(const struct scheduler *ops, struct vcpu *d)
+static void sedf_sleep(const struct scheduler *ops, struct vcpu *v)
{
- if ( is_idle_vcpu(d) )
+ if ( is_idle_vcpu(v) )
return;
- EDOM_INFO(d)->status |= SEDF_ASLEEP;
+ SEDF_VCPU(v)->status |= SEDF_ASLEEP;
- if ( per_cpu(schedule_data, d->processor).curr == d )
+ if ( per_cpu(schedule_data, v->processor).curr == v )
{
- cpu_raise_softirq(d->processor, SCHEDULE_SOFTIRQ);
+ cpu_raise_softirq(v->processor, SCHEDULE_SOFTIRQ);
}
else
{
- if ( __task_on_queue(d) )
- __del_from_queue(d);
+ if ( __task_on_queue(v) )
+ __del_from_queue(v);
}
}
/*
- * This function wakes up a domain, i.e. moves them into the waitqueue
- * things to mention are: admission control is taking place nowhere at
- * the moment, so we can't be sure, whether it is safe to wake the domain
- * up at all. Anyway, even if it is safe (total cpu usage <=100%) there are
- * some considerations on when to allow the domain to wake up and have it's
- * first deadline...
- * I detected 3 cases, which could describe the possible behaviour of the
- * scheduler,
- * and I'll try to make them more clear:
- *
- * 1. Very conservative
- * -when a blocked domain unblocks, it is allowed to start execution at
- * the beginning of the next complete period
- * (D..deadline, R..running, B..blocking/sleeping, U..unblocking/waking up
- *
- * DRRB_____D__U_____DRRRRR___D________ ...
- *
- * -this causes the domain to miss a period (and a deadlline)
- * -doesn't disturb the schedule at all
- * -deadlines keep occuring isochronous
- *
- * 2. Conservative Part 1: Short Unblocking
- * -when a domain unblocks in the same period as it was blocked it
- * unblocks and may consume the rest of it's original time-slice minus
- * the time it was blocked
- * (assume period=9, slice=5)
- *
- * DRB_UR___DRRRRR___D...
- *
- * -this also doesn't disturb scheduling, but might lead to the fact, that
- * the domain can't finish it's workload in the period
- * -addition: experiments have shown that this may have a HUGE impact on
- * performance of other domains, becaus it can lead to excessive context
- * switches
- *
- * Part2: Long Unblocking
- * Part 2a
- * -it is obvious that such accounting of block time, applied when
- * unblocking is happening in later periods, works fine aswell
- * -the domain is treated as if it would have been running since the start
- * of its new period
- *
- * DRB______D___UR___D...
- *
- * Part 2b
- * -if one needs the full slice in the next period, it is necessary to
- * treat the unblocking time as the start of the new period, i.e. move
- * the deadline further back (later)
- * -this doesn't disturb scheduling as well, because for EDF periods can
- * be treated as minimal inter-release times and scheduling stays
- * correct, when deadlines are kept relative to the time the process
- * unblocks
- *
- * DRB______D___URRRR___D...<prev [Thread] next>
- * (D) <- old deadline was here
- * -problem: deadlines don't occur isochronous anymore
- *
- * 3. Unconservative (i.e. incorrect)
- * -to boost the performance of I/O dependent domains it would be possible
- * to put the domain into the runnable queue immediately, and let it run
- * for the remainder of the slice of the current period
- * (or even worse: allocate a new full slice for the domain)
- * -either behaviour can lead to missed deadlines in other domains as
- * opposed to approaches 1,2a,2b
- */
-
-/*
- * Compares two domains in the relation of whether the one is allowed to
+ * Compares two vcpus in the relation of whether the one is allowed to
* interrupt the others execution.
- * It returns true (!=0) if a switch to the other domain is good.
+ * It returns true (!=0) if a switch to the other vcpu is good.
+ * Priority scheme is as follows:
+ * EDF: early deadline > late deadline
*/
static inline int should_switch(struct vcpu *cur,
struct vcpu *other,
s_time_t now)
{
struct sedf_vcpu_info *cur_inf, *other_inf;
- cur_inf = EDOM_INFO(cur);
- other_inf = EDOM_INFO(other);
-
+ cur_inf = SEDF_VCPU(cur);
+ other_inf = SEDF_VCPU(other);
+
/* Always interrupt idle vcpu. */
if ( is_idle_vcpu(cur) )
return 1;
-
+
/* Check whether we need to make an earlier scheduling decision */
if ( PERIOD_BEGIN(other_inf) <
- CPU_INFO(other->processor)->current_slice_expires )
+ SEDF_PCPU(other->processor)->current_slice_expires )
return 1;
return 0;
}
-static void sedf_wake(const struct scheduler *ops, struct vcpu *d)
+/*
+ * This function wakes up a vcpu, i.e. moves them into the appropriate queue
+ *
+ * When a blocked vcpu unblocks, it is allowed to start execution at
+ * the beginning of the next complete period
+ * (D..deadline, R..running, B..blocking/sleeping, U..unblocking/waking up
+ *
+ * DRRB_____D__U_____DRRRRR___D________ ...
+ *
+ * - This causes the vcpu to miss a period (and a deadlline)
+ * - Doesn't disturb the schedule at all
+ * - Deadlines keep occuring isochronous
+ */
+static void sedf_wake(const struct scheduler *ops, struct vcpu *v)
{
s_time_t now = NOW();
- struct sedf_vcpu_info* inf = EDOM_INFO(d);
+ struct sedf_vcpu_info* inf = SEDF_VCPU(v);
- if ( unlikely(is_idle_vcpu(d)) )
+ if ( unlikely(is_idle_vcpu(v)) )
return;
- if ( unlikely(__task_on_queue(d)) )
+ if ( unlikely(__task_on_queue(v)) )
return;
- ASSERT(!sedf_runnable(d));
+ ASSERT(!sedf_runnable(v));
inf->status &= ~SEDF_ASLEEP;
if ( unlikely(inf->deadl_abs == 0) )
{
/* Initial setup of the deadline */
- inf->deadl_abs = now + inf->slice;
+ inf->deadl_abs = now + inf->budget;
}
#ifdef SEDF_STATS
@@ -627,14 +595,14 @@ static void sedf_wake(const struct scheduler *ops, struct vcpu *d)
}
else
{
- /* Long unblocking */
+ /* Long unblocking, someone is going to miss their deadline. */
inf->long_block_tot++;
}
if ( PERIOD_BEGIN(inf) > now )
- __add_to_waitqueue_sort(d);
+ __add_to_waitqueue_sort(v);
else
- __add_to_runqueue_sort(d);
+ __add_to_runqueue_sort(v);
#ifdef SEDF_STATS
/* Do some statistics here... */
@@ -644,74 +612,74 @@ static void sedf_wake(const struct scheduler *ops, struct vcpu *d)
}
#endif
- ASSERT(__task_on_queue(d));
+ ASSERT(__task_on_queue(v));
/*
* Check whether the awakened task needs to invoke the do_schedule
* routine. Try to avoid unnecessary runs but:
- * Save approximation: Always switch to scheduler!
+ * Safe approximation: Always switch to scheduler!
*/
- ASSERT(d->processor >= 0);
- ASSERT(d->processor < nr_cpu_ids);
- ASSERT(per_cpu(schedule_data, d->processor).curr);
+ ASSERT(v->processor >= 0);
+ ASSERT(v->processor < nr_cpu_ids);
+ ASSERT(per_cpu(schedule_data, v->processor).curr);
- if ( should_switch(per_cpu(schedule_data, d->processor).curr, d, now) )
- cpu_raise_softirq(d->processor, SCHEDULE_SOFTIRQ);
+ if ( should_switch(per_cpu(schedule_data, v->processor).curr, v, now) )
+ cpu_raise_softirq(v->processor, SCHEDULE_SOFTIRQ);
}
-
-/* Print a lot of useful information about a domains in the system */
-static void sedf_dump_domain(struct vcpu *d)
+/* Print a lot of useful information about a vcpus in the system */
+static void sedf_dump_vcpu(struct vcpu *v)
{
- printk("%i.%i has=%c ", d->domain->domain_id, d->vcpu_id,
- d->is_running ? 'T':'F');
+ printk("%i.%i has=%c ", v->domain->domain_id, v->vcpu_id,
+ v->is_running ? 'T':'F');
printk("p=%"PRIu64" sl=%"PRIu64" ddl=%"PRIu64,
- EDOM_INFO(d)->period, EDOM_INFO(d)->slice, EDOM_INFO(d)->deadl_abs);
+ SEDF_VCPU(v)->period, SEDF_VCPU(v)->budget, SEDF_VCPU(v)->deadl_abs);
#ifdef SEDF_STATS
- if ( EDOM_INFO(d)->block_time_tot != 0 )
- printk(" pen=%"PRIu64"%%", EDOM_INFO(d)->block_time_tot);
- if ( EDOM_INFO(d)->block_tot != 0 )
+ if ( SEDF_VCPU(v)->block_time_tot != 0 )
+ printk(" pen=%"PRIu64"%%", SEDF_VCPU(v)->block_time_tot);
+ if ( SEDF_VCPU(v)->block_tot != 0 )
printk("\n blks=%u sh=%u (%u%%) "\
"l=%u (%u%%) avg: b=%"PRIu64" p=%d",
- EDOM_INFO(d)->block_tot, EDOM_INFO(d)->short_block_tot,
- (EDOM_INFO(d)->short_block_tot * 100) / EDOM_INFO(d)->block_tot,
- EDOM_INFO(d)->long_block_tot,
- (EDOM_INFO(d)->long_block_tot * 100) / EDOM_INFO(d)->block_tot,
- (EDOM_INFO(d)->block_time_tot) / EDOM_INFO(d)->block_tot,
- EDOM_INFO(d)->block_tot);
+ SEDF_VCPU(v)->block_tot, SEDF_VCPU(v)->short_block_tot,
+ (SEDF_VCPU(v)->short_block_tot * 100) / SEDF_VCPU(v)->block_tot,
+ SEDF_VCPU(v)->long_block_tot,
+ (SEDF_VCPU(v)->long_block_tot * 100) / SEDF_VCPU(v)->block_tot,
+ (SEDF_VCPU(v)->block_time_tot) / SEDF_VCPU(v)->block_tot,
+ SEDF_VCPU(v)->block_tot);
#endif
printk("\n");
}
-/* Dumps all domains on the specified cpu */
-static void sedf_dump_cpu_state(const struct scheduler *ops, int i)
+/* Dumps all vcpus on the specified cpu */
+static void sedf_dump_cpu_state(const struct scheduler *ops, int cpu)
{
struct list_head *list, *queue, *tmp;
- struct sedf_vcpu_info *d_inf;
+ struct sedf_vcpu_info *v_inf;
struct domain *d;
- struct vcpu *ed;
+ struct vcpu *v;
int loop = 0;
- printk("now=%"PRIu64"\n",NOW());
- queue = RUNQ(i);
+ printk("now=%"PRIu64"\n", NOW());
+ queue = RUNQ(cpu);
printk("RUNQ rq %lx n: %lx, p: %lx\n", (unsigned long)queue,
(unsigned long) queue->next, (unsigned long) queue->prev);
list_for_each_safe ( list, tmp, queue )
{
- printk("%3d: ",loop++);
- d_inf = list_entry(list, struct sedf_vcpu_info, list);
- sedf_dump_domain(d_inf->vcpu);
+ printk("%3d: ", loop++);
+ v_inf = list_entry(list, struct sedf_vcpu_info, list);
+ sedf_dump_vcpu(v_inf->vcpu);
}
- queue = WAITQ(i); loop = 0;
+ queue = WAITQ(cpu);
+ loop = 0;
printk("\nWAITQ rq %lx n: %lx, p: %lx\n", (unsigned long)queue,
(unsigned long) queue->next, (unsigned long) queue->prev);
list_for_each_safe ( list, tmp, queue )
{
- printk("%3d: ",loop++);
- d_inf = list_entry(list, struct sedf_vcpu_info, list);
- sedf_dump_domain(d_inf->vcpu);
+ printk("%3d: ", loop++);
+ v_inf = list_entry(list, struct sedf_vcpu_info, list);
+ sedf_dump_vcpu(v_inf->vcpu);
}
loop = 0;
@@ -722,12 +690,12 @@ static void sedf_dump_cpu_state(const struct scheduler *ops, int i)
{
if ( (d->cpupool ? d->cpupool->sched : &sched_sedf_def) != ops )
continue;
- for_each_vcpu(d, ed)
+ for_each_vcpu(d, v)
{
- if ( !__task_on_queue(ed) && (ed->processor == i) )
+ if ( !__task_on_queue(v) && (v->processor == cpu) )
{
- printk("%3d: ",loop++);
- sedf_dump_domain(ed);
+ printk("%3d: ", loop++);
+ sedf_dump_vcpu(v);
}
}
}
@@ -736,7 +704,7 @@ static void sedf_dump_cpu_state(const struct scheduler *ops, int i)
/* Set or fetch domain scheduling parameters */
-static int sedf_adjust(const struct scheduler *ops, struct domain *p, struct xen_domctl_scheduler_op *op)
+static int sedf_adjust(const struct scheduler *ops, struct domain *d, struct xen_domctl_scheduler_op *op)
{
struct sedf_priv_info *prv = SEDF_PRIV(ops);
unsigned long flags;
@@ -746,8 +714,8 @@ static int sedf_adjust(const struct scheduler *ops, struct domain *p, struct xen
/*
* Serialize against the pluggable scheduler lock to protect from
* concurrent updates. We need to take the runq lock for the VCPUs
- * as well, since we are touching slice and period.
- *
+ * as well, since we are touching budget and period.
+ *
* As in sched_credit2.c, runq locks nest inside the pluggable scheduler
* lock.
*/
@@ -767,33 +735,33 @@ static int sedf_adjust(const struct scheduler *ops, struct domain *p, struct xen
*/
if ( (op->u.sedf.period > PERIOD_MAX) ||
(op->u.sedf.period < PERIOD_MIN) ||
- (op->u.sedf.slice > op->u.sedf.period) ||
- (op->u.sedf.slice < SLICE_MIN) )
+ (op->u.sedf.slice > op->u.sedf.slice) ||
+ (op->u.sedf.slice < BUDGET_MIN) )
{
rc = -EINVAL;
goto out;
}
/* Time-driven domains */
- for_each_vcpu ( p, v )
+ for_each_vcpu ( d, v )
{
spinlock_t *lock = vcpu_schedule_lock(v);
- EDOM_INFO(v)->period = op->u.sedf.period;
- EDOM_INFO(v)->slice = op->u.sedf.slice;
+ SEDF_VCPU(v)->period = op->u.sedf.period;
+ SEDF_VCPU(v)->budget = op->u.sedf.slice;
vcpu_schedule_unlock(lock, v);
}
}
else if ( op->cmd == XEN_DOMCTL_SCHEDOP_getinfo )
{
- if ( p->vcpu[0] == NULL )
+ if ( d->vcpu[0] == NULL )
{
rc = -EINVAL;
goto out;
}
- op->u.sedf.period = EDOM_INFO(p->vcpu[0])->period;
- op->u.sedf.slice = EDOM_INFO(p->vcpu[0])->slice;
+ op->u.sedf.period = SEDF_VCPU(d->vcpu[0])->period;
+ op->u.sedf.slice = SEDF_VCPU(d->vcpu[0])->budget;
}
out:
--
1.7.9.5
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [RFC PATCH v2 3/7] Added constant bandwidth server functionality to sedf scheduler
2014-07-09 20:55 [RFC PATCH v2 0/7] Repurpose SEDF Scheduler for Real-time Use Josh Whitehead
2014-07-09 20:55 ` [RFC PATCH v2 1/7] Removed all code from sedf not needed for basic EDF functionality Josh Whitehead
2014-07-09 20:55 ` [RFC PATCH v2 2/7] Fixed formatting and misleading comments/variables. Added comments and renamed variables to accurately reflect modern terminology Josh Whitehead
@ 2014-07-09 20:55 ` Josh Whitehead
2014-07-11 9:37 ` Dario Faggioli
2014-07-09 20:55 ` [RFC PATCH v2 4/7] Add cbs parameter support and removed sedf parameters from libxc Josh Whitehead
` (4 subsequent siblings)
7 siblings, 1 reply; 30+ messages in thread
From: Josh Whitehead @ 2014-07-09 20:55 UTC (permalink / raw)
To: Xen-devel
Cc: Ian Campbell, Stefano Stabellini, George Dunlap, Dario Faggioli,
Ian Jackson, Robert VanVossen, Nathan Studer, Joshua Whitehead
This patch adds the pieces needed for the CBS functionality. This includes a
"soft" flag to indicate whether a domain should be handled in a pure EDF manner
or handled by the CBS algorithm, the CBS algorithm itself to the wake function,
and few other bits and comments to track scheduler statistics, set parameters,
and document the functionality of the code itself.
Signed-off-by: Nathan Studer <nate.studer@gmail.com>
Signed-off-by: Joshua Whitehead <Josh.Whitehead@dornerworks.com>
---
xen/common/sched_sedf.c | 197 ++++++++++++++++++++++++++++++++++++-----------
1 file changed, 153 insertions(+), 44 deletions(-)
diff --git a/xen/common/sched_sedf.c b/xen/common/sched_sedf.c
index a8dd3e0..7333095 100644
--- a/xen/common/sched_sedf.c
+++ b/xen/common/sched_sedf.c
@@ -1,5 +1,5 @@
/******************************************************************************
- * Simple EDF Scheduler for xen
+ * Simple EDF Scheduler for Xen
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to
@@ -46,6 +46,7 @@
#define CHECK(_p) ((void)0)
#endif
+#define SEDF_SOFT_TASK (1)
#define SEDF_ASLEEP (16)
#define DEFAULT_PERIOD (MILLISECS(20))
@@ -73,7 +74,8 @@ struct sedf_vcpu_info {
/* Parameters for EDF */
s_time_t period; /* = Server scheduling period */
- s_time_t budget; /* = Guarenteed minimum CPU time per period */
+ s_time_t budget; /* = Guarenteed minimum CPU time per period */
+ /* Note: Server bandwidth = (budget / period) */
/* Status of vcpu */
int status;
@@ -90,6 +92,10 @@ struct sedf_vcpu_info {
int block_tot;
int short_block_tot;
int long_block_tot;
+ s_time_t miss_time;
+ s_time_t over_time;
+ int miss_tot;
+ int over_tot;
#endif
};
@@ -115,6 +121,7 @@ struct sedf_cpu_info {
#define sedf_runnable(edom) (!(SEDF_VCPU(edom)->status & SEDF_ASLEEP))
+#define sedf_soft(edom) (SEDF_VCPU(edom)->status & SEDF_SOFT_TASK)
static void sedf_dump_cpu_state(const struct scheduler *ops, int cpu);
@@ -208,6 +215,7 @@ static void *sedf_alloc_vdata(const struct scheduler *ops, struct vcpu *v, void
inf->vcpu = v;
inf->deadl_abs = 0;
+ inf->cputime = 0;
inf->status = SEDF_ASLEEP;
if (v->domain->domain_id == 0)
@@ -306,29 +314,64 @@ static void desched_edf_vcpu(s_time_t now, struct vcpu *v)
/* Update the vcpu's cputime */
inf->cputime += now - inf->sched_start_abs;
- /* Scheduling decisions which don't remove the running vcpu from
- * the runq */
+ /* If running vcpu has budget remaining, return */
if ( (inf->cputime < inf->budget) && sedf_runnable(v) )
return;
+ /* Current vcpu has consumed its budget for this period */
__del_from_queue(v);
- /*
- * Manage bookkeeping (i.e. calculate next deadline, memorise
- * overrun-time of budget) of finished vcpus.
- */
+#ifdef SEDF_STATS
+ /* Manage deadline misses */
+ if ( unlikely(inf->deadl_abs < now) )
+ {
+ inf->miss_tot++;
+ inf->miss_time += inf->cputime;
+ }
+#endif
+
+ /* Manage overruns */
if ( inf->cputime >= inf->budget )
{
inf->cputime -= inf->budget;
/* Set next deadline */
inf->deadl_abs += inf->period;
+
+ /* Ensure that the cputime is always less than budget */
+ if ( unlikely(inf->cputime > inf->budget) )
+ {
+#ifdef SEDF_STATS
+ inf->over_tot++;
+ inf->over_time += inf->cputime;
+#endif
+
+ /* Make up for the overage by pushing the deadline
+ into the future; ensure one domain doesn't consistently
+ consume extra time by overrunning its budget */
+ inf->deadl_abs += ((inf->cputime / inf->budget)
+ * inf->period);
+ inf->cputime -= (inf->cputime / inf->budget) * inf->budget;
+ }
+
+ /* Ensure that the start of the next period is in the future */
+ if ( unlikely(PERIOD_BEGIN(inf) < now) )
+ inf->deadl_abs +=
+ (DIV_UP(now - PERIOD_BEGIN(inf),
+ inf->period)) * inf->period;
}
/* Add a runnable vcpu to the appropriate queue */
if ( sedf_runnable(v) )
{
- __add_to_waitqueue_sort(v);
+ if( sedf_soft(v) )
+ {
+ __add_to_runqueue_sort(v);
+ }
+ else
+ {
+ __add_to_waitqueue_sort(v);
+ }
}
ASSERT(EQ(sedf_runnable(v), __task_on_queue(v)));
@@ -418,9 +461,9 @@ static void sedf_deinit(const struct scheduler *ops)
static struct task_slice sedf_do_schedule(
const struct scheduler *ops, s_time_t now, bool_t tasklet_work_scheduled)
{
- int cpu = smp_processor_id();
- struct list_head *runq = RUNQ(cpu);
- struct list_head *waitq = WAITQ(cpu);
+ int cpu = smp_processor_id();
+ struct list_head *runq = RUNQ(cpu);
+ struct list_head *waitq = WAITQ(cpu);
struct sedf_vcpu_info *inf = SEDF_VCPU(current);
struct sedf_vcpu_info *runinf, *waitinf;
struct task_slice ret;
@@ -469,7 +512,7 @@ static struct task_slice sedf_do_schedule(
waitinf = list_entry(waitq->next,
struct sedf_vcpu_info, list);
/*
- * Rerun scheduler, when scheduled vcpu consumes
+ * Rerun scheduler when scheduled vcpu consumes
* its budget or the first vcpu from the waitqueue
* gets ready.
*/
@@ -490,7 +533,7 @@ static struct task_slice sedf_do_schedule(
}
/*
- * TODO: Do something USEFUL when this happens and find out, why it
+ * TODO: Do something USEFUL when this happens and find out why it
* still can happen!!!
*/
if ( ret.time < 0)
@@ -554,15 +597,21 @@ static inline int should_switch(struct vcpu *cur,
/*
* This function wakes up a vcpu, i.e. moves them into the appropriate queue
*
- * When a blocked vcpu unblocks, it is allowed to start execution at
- * the beginning of the next complete period
- * (D..deadline, R..running, B..blocking/sleeping, U..unblocking/waking up
+ * For Hard Real-Time vcpus (soft = 0):
+ * -When a blocked vcpu unblocks, it is allowed to start execution at
+ * the beginning of the next complete period
+ * (D..deadline, R..running, B..blocking/sleeping, U..unblocking/waking up
+ *
+ * DRRB_____D__U_____DRRRRR___D________ ...
+ *
+ * -This causes the vcpu to miss a period (and a deadlline)
+ * -Doesn't disturb the schedule at all
+ * -Deadlines keep occuring isochronous
*
- * DRRB_____D__U_____DRRRRR___D________ ...
+ * For Soft Real-Time vcpus (soft = 1):
+ * -Deadlines are set and updated according to the Constant Bandwidth Server
+ * rule and vcpus are moved immediately to the run queue.
*
- * - This causes the vcpu to miss a period (and a deadlline)
- * - Doesn't disturb the schedule at all
- * - Deadlines keep occuring isochronous
*/
static void sedf_wake(const struct scheduler *ops, struct vcpu *v)
{
@@ -588,21 +637,62 @@ static void sedf_wake(const struct scheduler *ops, struct vcpu *v)
inf->block_tot++;
#endif
- if ( now < inf->deadl_abs )
- {
- /* Short blocking */
- inf->short_block_tot++;
- }
- else
+ if ( sedf_soft(v) )
{
- /* Long unblocking, someone is going to miss their deadline. */
- inf->long_block_tot++;
+ /*
+ * Apply CBS rule
+ * Where:
+ * c == Remaining server budget == (inf->budget - cpu_time)
+ * d == Server (vcpu) deadline == inf->deadl_abs
+ * r == Wake-up time of vcpu == now
+ * U == Server (vcpu) bandwidth == (inf->budget / inf->period)
+ *
+ * if c>=(d-r)*U --->
+ * (inf->budget - cputime) >=
+ * (inf->deadl_abs - now) * (inf->period / inf->period)
+ *
+ * If true, push deadline back by one period and refresh budget, else
+ * use current budget and deadline.
+ *
+ * Note: The 'if' statement below is equivalent to the above comments;
+ * it has been simplified to avoid the division operator
+ */
+ if ((inf->budget - inf->cputime) * inf->period >=
+ (inf->deadl_abs - now) * inf->budget)
+ {
+ /* Push back deadline by one period */
+ inf->deadl_abs += inf->period;
+ inf->cputime = 0;
+ }
+
+ /*
+ * In CBS we don't care if the period has begun,
+ * the task doesn't have to wait for its period
+ * because it'll never request more than its budget
+ * for any given period.
+ */
+ __add_to_runqueue_sort(v);
}
+ else {
+ /* Task is a hard task, treat accordingly */
+#ifdef SEDF_STATS
+ if ( now < inf->deadl_abs )
+ {
+ /* Short blocking */
+ inf->short_block_tot++;
+ }
+ else
+ {
+ /* Long unblocking, someone is going to miss their deadline. */
+ inf->long_block_tot++;
+ }
+#endif
- if ( PERIOD_BEGIN(inf) > now )
- __add_to_waitqueue_sort(v);
- else
- __add_to_runqueue_sort(v);
+ if ( PERIOD_BEGIN(inf) > now )
+ __add_to_waitqueue_sort(v);
+ else
+ __add_to_runqueue_sort(v);
+ }
#ifdef SEDF_STATS
/* Do some statistics here... */
@@ -626,7 +716,7 @@ static void sedf_wake(const struct scheduler *ops, struct vcpu *v)
cpu_raise_softirq(v->processor, SCHEDULE_SOFTIRQ);
}
-/* Print a lot of useful information about a vcpus in the system */
+/* Print a lot of useful information about a vcpu in the system */
static void sedf_dump_vcpu(struct vcpu *v)
{
printk("%i.%i has=%c ", v->domain->domain_id, v->vcpu_id,
@@ -635,17 +725,18 @@ static void sedf_dump_vcpu(struct vcpu *v)
SEDF_VCPU(v)->period, SEDF_VCPU(v)->budget, SEDF_VCPU(v)->deadl_abs);
#ifdef SEDF_STATS
- if ( SEDF_VCPU(v)->block_time_tot != 0 )
- printk(" pen=%"PRIu64"%%", SEDF_VCPU(v)->block_time_tot);
+ printk(" m=%u mt=%"PRIu64"o=%u ot=%"PRIu64,
+ SEDF_VCPU(v)->miss_tot, SEDF_VCPU(v)->miss_time,
+ SEDF_VCPU(v)->over_tot, SEDF_VCPU(v)->over_time);
+
if ( SEDF_VCPU(v)->block_tot != 0 )
printk("\n blks=%u sh=%u (%u%%) "\
- "l=%u (%u%%) avg: b=%"PRIu64" p=%d",
+ "l=%u (%u%%) avg: b=%"PRIu64,
SEDF_VCPU(v)->block_tot, SEDF_VCPU(v)->short_block_tot,
(SEDF_VCPU(v)->short_block_tot * 100) / SEDF_VCPU(v)->block_tot,
SEDF_VCPU(v)->long_block_tot,
(SEDF_VCPU(v)->long_block_tot * 100) / SEDF_VCPU(v)->block_tot,
- (SEDF_VCPU(v)->block_time_tot) / SEDF_VCPU(v)->block_tot,
- SEDF_VCPU(v)->block_tot);
+ (SEDF_VCPU(v)->block_time_tot) / SEDF_VCPU(v)->block_tot);
#endif
printk("\n");
}
@@ -708,6 +799,7 @@ static int sedf_adjust(const struct scheduler *ops, struct domain *d, struct xen
{
struct sedf_priv_info *prv = SEDF_PRIV(ops);
unsigned long flags;
+ s_time_t now = NOW();
struct vcpu *v;
int rc = 0;
@@ -726,6 +818,7 @@ static int sedf_adjust(const struct scheduler *ops, struct domain *d, struct xen
/* Check for sane parameters */
if ( !op->u.sedf.period )
{
+ printk("Period Not set");
rc = -EINVAL;
goto out;
}
@@ -735,9 +828,10 @@ static int sedf_adjust(const struct scheduler *ops, struct domain *d, struct xen
*/
if ( (op->u.sedf.period > PERIOD_MAX) ||
(op->u.sedf.period < PERIOD_MIN) ||
- (op->u.sedf.slice > op->u.sedf.slice) ||
- (op->u.sedf.slice < BUDGET_MIN) )
+ (op->u.sedf.budget > op->u.sedf.budget) ||
+ (op->u.sedf.budget < BUDGET_MIN) )
{
+ printk("Insane Parameters: period: %lu\tbudget: %lu\n", op->u.sedf.period, op->u.sedf.budget);
rc = -EINVAL;
goto out;
}
@@ -748,7 +842,21 @@ static int sedf_adjust(const struct scheduler *ops, struct domain *d, struct xen
spinlock_t *lock = vcpu_schedule_lock(v);
SEDF_VCPU(v)->period = op->u.sedf.period;
- SEDF_VCPU(v)->budget = op->u.sedf.slice;
+ SEDF_VCPU(v)->budget = op->u.sedf.budget;
+ if(op->u.sedf.soft)
+ {
+ SEDF_VCPU(v)->status |= SEDF_SOFT_TASK;
+ }
+ else
+ {
+ /* Correct deadline when switching from a soft to hard vcpu */
+ if( unlikely((SEDF_VCPU(v)->deadl_abs - now) >= (SEDF_VCPU(v)->period * 3)) )
+ {
+ SEDF_VCPU(v)->deadl_abs = (now - SEDF_VCPU(v)->cputime) + (2 * SEDF_VCPU(v)->period);
+ }
+
+ SEDF_VCPU(v)->status &= (~SEDF_SOFT_TASK);
+ }
vcpu_schedule_unlock(lock, v);
}
}
@@ -760,8 +868,9 @@ static int sedf_adjust(const struct scheduler *ops, struct domain *d, struct xen
goto out;
}
- op->u.sedf.period = SEDF_VCPU(d->vcpu[0])->period;
- op->u.sedf.slice = SEDF_VCPU(d->vcpu[0])->budget;
+ op->u.sedf.period = SEDF_VCPU(d->vcpu[0])->period;
+ op->u.sedf.budget = SEDF_VCPU(d->vcpu[0])->budget;
+ op->u.sedf.soft = sedf_soft(d->vcpu[0]);
}
out:
--
1.7.9.5
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [RFC PATCH v2 4/7] Add cbs parameter support and removed sedf parameters from libxc
2014-07-09 20:55 [RFC PATCH v2 0/7] Repurpose SEDF Scheduler for Real-time Use Josh Whitehead
` (2 preceding siblings ...)
2014-07-09 20:55 ` [RFC PATCH v2 3/7] Added constant bandwidth server functionality to sedf scheduler Josh Whitehead
@ 2014-07-09 20:55 ` Josh Whitehead
2014-07-10 23:17 ` Dario Faggioli
2014-07-09 20:55 ` [RFC PATCH v2 5/7] Add cbs parameter support and removed sedf parameters with a LIBXL_API_VERSION gate from libxl Josh Whitehead
` (3 subsequent siblings)
7 siblings, 1 reply; 30+ messages in thread
From: Josh Whitehead @ 2014-07-09 20:55 UTC (permalink / raw)
To: Xen-devel
Cc: Ian Campbell, Stefano Stabellini, George Dunlap, Dario Faggioli,
Ian Jackson, Robert VanVossen, Nathan Studer, Josh Whitehead
From: Robbie VanVossen <robert.vanvossen@dornerworks.com>
This file may also need something similar to the LIBXL_API_VERSION gate.
Signed-off-by: Robert VanVossen <Robert.VanVossen@dornerworks.com>
Signed-off-by: Nathan Studer <nate.studer@gmail.com>
---
tools/libxc/xc_sedf.c | 16 ++++------------
tools/libxc/xenctrl.h | 12 ++++++------
tools/python/xen/lowlevel/xc/xc.c | 34 +++++++++++++++-------------------
xen/include/public/domctl.h | 4 +---
4 files changed, 26 insertions(+), 40 deletions(-)
diff --git a/tools/libxc/xc_sedf.c b/tools/libxc/xc_sedf.c
index db372ca..81ff133 100644
--- a/tools/libxc/xc_sedf.c
+++ b/tools/libxc/xc_sedf.c
@@ -29,9 +29,7 @@ int xc_sedf_domain_set(
uint32_t domid,
uint64_t period,
uint64_t slice,
- uint64_t latency,
- uint16_t extratime,
- uint16_t weight)
+ uint16_t soft)
{
DECLARE_DOMCTL;
struct xen_domctl_sched_sedf *p = &domctl.u.scheduler_op.u.sedf;
@@ -43,9 +41,7 @@ int xc_sedf_domain_set(
p->period = period;
p->slice = slice;
- p->latency = latency;
- p->extratime = extratime;
- p->weight = weight;
+ p->soft = soft;
return do_domctl(xch, &domctl);
}
@@ -54,9 +50,7 @@ int xc_sedf_domain_get(
uint32_t domid,
uint64_t *period,
uint64_t *slice,
- uint64_t *latency,
- uint16_t *extratime,
- uint16_t *weight)
+ uint16_t *soft)
{
DECLARE_DOMCTL;
int ret;
@@ -71,8 +65,6 @@ int xc_sedf_domain_get(
*period = p->period;
*slice = p->slice;
- *latency = p->latency;
- *extratime = p->extratime;
- *weight = p->weight;
+ *soft = p->soft;
return ret;
}
diff --git a/tools/libxc/xenctrl.h b/tools/libxc/xenctrl.h
index 02129f7..d5cfdb8 100644
--- a/tools/libxc/xenctrl.h
+++ b/tools/libxc/xenctrl.h
@@ -765,15 +765,15 @@ int xc_shadow_control(xc_interface *xch,
int xc_sedf_domain_set(xc_interface *xch,
uint32_t domid,
- uint64_t period, uint64_t slice,
- uint64_t latency, uint16_t extratime,
- uint16_t weight);
+ uint64_t period,
+ uint64_t slice,
+ uint16_t soft);
int xc_sedf_domain_get(xc_interface *xch,
uint32_t domid,
- uint64_t* period, uint64_t *slice,
- uint64_t *latency, uint16_t *extratime,
- uint16_t *weight);
+ uint64_t *period,
+ uint64_t *slice,
+ uint16_t *soft);
int xc_sched_credit_domain_set(xc_interface *xch,
uint32_t domid,
diff --git a/tools/python/xen/lowlevel/xc/xc.c b/tools/python/xen/lowlevel/xc/xc.c
index cb34446..bf10165 100644
--- a/tools/python/xen/lowlevel/xc/xc.c
+++ b/tools/python/xen/lowlevel/xc/xc.c
@@ -1471,17 +1471,17 @@ static PyObject *pyxc_sedf_domain_set(XcObject *self,
PyObject *kwds)
{
uint32_t domid;
- uint64_t period, slice, latency;
- uint16_t extratime, weight;
- static char *kwd_list[] = { "domid", "period", "slice",
- "latency", "extratime", "weight",NULL };
+ uint64_t period, slice;
+ uint16_t soft;
+
+ static char *kwd_list[] = { "domid", "period", "slice", "soft",NULL };
- if( !PyArg_ParseTupleAndKeywords(args, kwds, "iLLLhh", kwd_list,
- &domid, &period, &slice,
- &latency, &extratime, &weight) )
+ if( !PyArg_ParseTupleAndKeywords(args, kwds, "iLLi", kwd_list,
+ &domid, &period, &slice, &soft) )
+
return NULL;
if ( xc_sedf_domain_set(self->xc_handle, domid, period,
- slice, latency, extratime,weight) != 0 )
+ slice, soft) != 0 )
return pyxc_error_to_exception(self->xc_handle);
Py_INCREF(zero);
@@ -1491,23 +1491,21 @@ static PyObject *pyxc_sedf_domain_set(XcObject *self,
static PyObject *pyxc_sedf_domain_get(XcObject *self, PyObject *args)
{
uint32_t domid;
- uint64_t period, slice,latency;
- uint16_t weight, extratime;
+ uint64_t period, slice;
+ uint16_t soft;
if(!PyArg_ParseTuple(args, "i", &domid))
return NULL;
if (xc_sedf_domain_get(self->xc_handle, domid, &period,
- &slice,&latency,&extratime,&weight))
+ &slice, &soft))
return pyxc_error_to_exception(self->xc_handle);
- return Py_BuildValue("{s:i,s:L,s:L,s:L,s:i,s:i}",
+ return Py_BuildValue("{s:i,s:L,s:L,s:i}",
"domid", domid,
"period", period,
"slice", slice,
- "latency", latency,
- "extratime", extratime,
- "weight", weight);
+ "soft", soft);
}
static PyObject *pyxc_shadow_control(PyObject *self,
@@ -2544,8 +2542,7 @@ static PyMethodDef pyxc_methods[] = {
" dom [int]: domain to set\n"
" period [long]: domain's scheduling period\n"
" slice [long]: domain's slice per period\n"
- " latency [long]: domain's wakeup latency hint\n"
- " extratime [int]: domain aware of extratime?\n"
+ " soft [int]: domain is a soft task?\n"
"Returns: [int] 0 on success; -1 on error.\n" },
{ "sedf_domain_get",
@@ -2558,8 +2555,7 @@ static PyMethodDef pyxc_methods[] = {
" domain [int]: domain ID\n"
" period [long]: scheduler period\n"
" slice [long]: CPU reservation per period\n"
- " latency [long]: domain's wakeup latency hint\n"
- " extratime [int]: domain aware of extratime?\n"},
+ " soft [int]: domain is a soft task?\n"},
{ "sched_credit_domain_set",
(PyCFunction)pyxc_sched_credit_domain_set,
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 565fa4c..6e143d3 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -331,9 +331,7 @@ struct xen_domctl_scheduler_op {
struct xen_domctl_sched_sedf {
uint64_aligned_t period;
uint64_aligned_t slice;
- uint64_aligned_t latency;
- uint32_t extratime;
- uint32_t weight;
+ uint32_t soft;
} sedf;
struct xen_domctl_sched_credit {
uint16_t weight;
--
1.7.9.5
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [RFC PATCH v2 5/7] Add cbs parameter support and removed sedf parameters with a LIBXL_API_VERSION gate from libxl.
2014-07-09 20:55 [RFC PATCH v2 0/7] Repurpose SEDF Scheduler for Real-time Use Josh Whitehead
` (3 preceding siblings ...)
2014-07-09 20:55 ` [RFC PATCH v2 4/7] Add cbs parameter support and removed sedf parameters from libxc Josh Whitehead
@ 2014-07-09 20:55 ` Josh Whitehead
2014-07-10 14:09 ` Ian Campbell
2014-07-10 14:26 ` Dario Faggioli
2014-07-09 20:55 ` [RFC PATCH v2 6/7] Changed slice to budget in libxc for the sedf scheduler Josh Whitehead
` (2 subsequent siblings)
7 siblings, 2 replies; 30+ messages in thread
From: Josh Whitehead @ 2014-07-09 20:55 UTC (permalink / raw)
To: Xen-devel
Cc: Ian Campbell, Stefano Stabellini, George Dunlap, Dario Faggioli,
Ian Jackson, Robert VanVossen, Nathan Studer, Josh Whitehead
From: Robbie VanVossen <robert.vanvossen@dornerworks.com>
We made an attempt at doing the versioning gate in this file. Please let us
know if further changes are needed, and where they are needed to properly guard
the API.
Signed-off-by: Robert VanVossen <Robert.VanVossen@dornerworks.com>
Signed-off-by: Nathan Studer <nate.studer@gmail.com>
---
docs/man/xl.cfg.pod.5 | 9 ++----
tools/libxl/libxl.c | 29 +++++++++++++++++++
tools/libxl/libxl.h | 1 +
tools/libxl/libxl_create.c | 61 ---------------------------------------
tools/libxl/libxl_types.idl | 1 +
tools/libxl/xl_cmdimpl.c | 66 +++++++++++++++++++++++++++++++++++++++++++
tools/libxl/xl_cmdtable.c | 5 ++++
7 files changed, 104 insertions(+), 68 deletions(-)
diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
index a94d037..5c55298 100644
--- a/docs/man/xl.cfg.pod.5
+++ b/docs/man/xl.cfg.pod.5
@@ -212,14 +212,9 @@ The normal EDF scheduling usage in nanoseconds. it defines the time
a domain get every period time.
Honoured by the sedf scheduler.
-=item B<latency=N>
+=item B<soft=BOOLEAN>
-Scaled period if domain is doing heavy I/O.
-Honoured by the sedf scheduler.
-
-=item B<extratime=BOOLEAN>
-
-Flag for allowing domain to run in extra time.
+Flag for setting a domain or VCPU to run as a soft task.
Honoured by the sedf scheduler.
=back
diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index 900b8d4..ca8c1c5 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -4932,13 +4932,21 @@ static int sched_sedf_domain_get(libxl__gc *gc, uint32_t domid,
{
uint64_t period;
uint64_t slice;
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
uint64_t latency;
uint16_t extratime;
uint16_t weight;
+#else
+ uint16_t soft;
+#endif
int rc;
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
rc = xc_sedf_domain_get(CTX->xch, domid, &period, &slice, &latency,
&extratime, &weight);
+#else
+ rc = xc_sedf_domain_get(CTX->xch, domid, &period, &slice, &soft);
+#endif
if (rc != 0) {
LOGE(ERROR, "getting domain sched sedf");
return ERROR_FAIL;
@@ -4948,9 +4956,13 @@ static int sched_sedf_domain_get(libxl__gc *gc, uint32_t domid,
scinfo->sched = LIBXL_SCHEDULER_SEDF;
scinfo->period = period / 1000000;
scinfo->slice = slice / 1000000;
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
scinfo->latency = latency / 1000000;
scinfo->extratime = extratime;
scinfo->weight = weight;
+#else
+ scinfo->soft = soft;
+#endif
return 0;
}
@@ -4960,14 +4972,22 @@ static int sched_sedf_domain_set(libxl__gc *gc, uint32_t domid,
{
uint64_t period;
uint64_t slice;
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
uint64_t latency;
uint16_t extratime;
uint16_t weight;
+#else
+ uint16_t soft;
+#endif
int ret;
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
ret = xc_sedf_domain_get(CTX->xch, domid, &period, &slice, &latency,
&extratime, &weight);
+#else
+ ret = xc_sedf_domain_get(CTX->xch, domid, &period, &slice, &soft);
+#endif
if (ret != 0) {
LOGE(ERROR, "getting domain sched sedf");
return ERROR_FAIL;
@@ -4977,15 +4997,24 @@ static int sched_sedf_domain_set(libxl__gc *gc, uint32_t domid,
period = (uint64_t)scinfo->period * 1000000;
if (scinfo->slice != LIBXL_DOMAIN_SCHED_PARAM_SLICE_DEFAULT)
slice = (uint64_t)scinfo->slice * 1000000;
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
if (scinfo->latency != LIBXL_DOMAIN_SCHED_PARAM_LATENCY_DEFAULT)
latency = (uint64_t)scinfo->latency * 1000000;
if (scinfo->extratime != LIBXL_DOMAIN_SCHED_PARAM_EXTRATIME_DEFAULT)
extratime = scinfo->extratime;
if (scinfo->weight != LIBXL_DOMAIN_SCHED_PARAM_WEIGHT_DEFAULT)
weight = scinfo->weight;
+#else
+ if (scinfo->soft != LIBXL_DOMAIN_SCHED_PARAM_SOFT_DEFAULT)
+ soft = scinfo->soft;
+#endif
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
ret = xc_sedf_domain_set(CTX->xch, domid, period, slice, latency,
extratime, weight);
+#else
+ ret = xc_sedf_domain_set(CTX->xch, domid, period, slice, soft);
+#endif
if ( ret < 0 ) {
LOGE(ERROR, "setting domain sched sedf");
return ERROR_FAIL;
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 80947c3..eacd8f6 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -1122,6 +1122,7 @@ int libxl_sched_credit_params_set(libxl_ctx *ctx, uint32_t poolid,
#define LIBXL_DOMAIN_SCHED_PARAM_SLICE_DEFAULT -1
#define LIBXL_DOMAIN_SCHED_PARAM_LATENCY_DEFAULT -1
#define LIBXL_DOMAIN_SCHED_PARAM_EXTRATIME_DEFAULT -1
+#define LIBXL_DOMAIN_SCHED_PARAM_SOFT_DEFAULT -1
int libxl_domain_sched_params_get(libxl_ctx *ctx, uint32_t domid,
libxl_domain_sched_params *params);
diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index d015cf4..83b593b 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -44,61 +44,6 @@ int libxl__domain_create_info_setdefault(libxl__gc *gc,
return 0;
}
-static int sched_params_valid(libxl__gc *gc,
- uint32_t domid, libxl_domain_sched_params *scp)
-{
- int has_weight = scp->weight != LIBXL_DOMAIN_SCHED_PARAM_WEIGHT_DEFAULT;
- int has_period = scp->period != LIBXL_DOMAIN_SCHED_PARAM_PERIOD_DEFAULT;
- int has_slice = scp->slice != LIBXL_DOMAIN_SCHED_PARAM_SLICE_DEFAULT;
- int has_extratime =
- scp->extratime != LIBXL_DOMAIN_SCHED_PARAM_EXTRATIME_DEFAULT;
-
- /* The sedf scheduler needs some more consistency checking */
- if (libxl__domain_scheduler(gc, domid) == LIBXL_SCHEDULER_SEDF) {
- if (has_weight && (has_period || has_slice))
- return 0;
- /* If you want a real-time domain, with its own period and
- * slice, please, do provide both! */
- if (has_period != has_slice)
- return 0;
-
- /*
- * Idea is, if we specify a weight, then both period and
- * slice has to be zero. OTOH, if we do specify a period and
- * slice, it is weight that should be zeroed. See
- * docs/misc/sedf_scheduler_mini-HOWTO.txt for more details
- * on the meaningful combinations and their meanings.
- */
- if (has_weight) {
- scp->slice = 0;
- scp->period = 0;
- }
- else if (!has_period) {
- /* No weight nor slice/period means best effort. Parameters needs
- * some mangling in order to properly ask for that, though. */
-
- /*
- * Providing no weight does not make any sense if we do not allow
- * the domain to run in extra time. On the other hand, if we have
- * extra time, weight will be ignored (and zeroed) by Xen, but it
- * can't be zero here, or the call for setting the scheduling
- * parameters will fail. So, avoid the latter by setting a random
- * weight (namely, 1), as it will be ignored anyway.
- */
-
- /* We can setup a proper best effort domain (extra time only)
- * iff we either already have or are asking for some extra time. */
- scp->weight = has_extratime ? scp->extratime : 1;
- scp->period = 0;
- } else {
- /* Real-time domain: will get slice CPU time over every period */
- scp->weight = 0;
- }
- }
-
- return 1;
-}
-
int libxl__domain_build_info_setdefault(libxl__gc *gc,
libxl_domain_build_info *b_info)
{
@@ -760,12 +705,6 @@ static void initiate_domain_create(libxl__egc *egc,
ret = libxl__domain_build_info_setdefault(gc, &d_config->b_info);
if (ret) goto error_out;
- if (!sched_params_valid(gc, domid, &d_config->b_info.sched_params)) {
- LOG(ERROR, "Invalid scheduling parameters\n");
- ret = ERROR_INVAL;
- goto error_out;
- }
-
for (i = 0; i < d_config->num_disks; i++) {
ret = libxl__device_disk_setdefault(gc, &d_config->disks[i]);
if (ret) goto error_out;
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index 52f1aa9..d02380e 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -294,6 +294,7 @@ libxl_domain_sched_params = Struct("domain_sched_params",[
("slice", integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_SLICE_DEFAULT'}),
("latency", integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_LATENCY_DEFAULT'}),
("extratime", integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_EXTRATIME_DEFAULT'}),
+ ("soft", integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_SOFT_DEFAULT'}),
])
libxl_domain_build_info = Struct("domain_build_info",[
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 5195914..1f6f04b 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -838,10 +838,15 @@ static void parse_config_data(const char *config_source,
b_info->sched_params.period = l;
if (!xlu_cfg_get_long (config, "slice", &l, 0))
b_info->sched_params.slice = l;
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
if (!xlu_cfg_get_long (config, "latency", &l, 0))
b_info->sched_params.latency = l;
if (!xlu_cfg_get_long (config, "extratime", &l, 0))
b_info->sched_params.extratime = l;
+#else
+ if (!xlu_cfg_get_long (config, "soft", &l, 0))
+ b_info->sched_params.soft = l;
+#endif
if (!xlu_cfg_get_long (config, "vcpus", &l, 0)) {
b_info->max_vcpus = l;
@@ -5181,14 +5186,20 @@ static int sched_sedf_domain_output(
int rc;
if (domid < 0) {
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
printf("%-33s %4s %6s %-6s %7s %5s %6s\n", "Name", "ID", "Period",
"Slice", "Latency", "Extra", "Weight");
+#else
+ printf("%-33s %4s %6s %-6s %5s\n", "Name", "ID", "Period",
+ "Slice", "Soft");
+#endif
return 0;
}
rc = sched_domain_get(LIBXL_SCHEDULER_SEDF, domid, &scinfo);
if (rc)
return rc;
domname = libxl_domid_to_name(ctx, domid);
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
printf("%-33s %4d %6d %6d %7d %5d %6d\n",
domname,
domid,
@@ -5197,6 +5208,14 @@ static int sched_sedf_domain_output(
scinfo.latency,
scinfo.extratime,
scinfo.weight);
+#else
+ printf("%-33s %4d %6d %6d %5d\n",
+ domname,
+ domid,
+ scinfo.period,
+ scinfo.slice,
+ scinfo.soft);
+#endif
free(domname);
libxl_domain_sched_params_dispose(&scinfo);
return 0;
@@ -5466,22 +5485,34 @@ int main_sched_sedf(int argc, char **argv)
const char *cpupool = NULL;
int period = 0, opt_p = 0;
int slice = 0, opt_s = 0;
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
int latency = 0, opt_l = 0;
int extra = 0, opt_e = 0;
int weight = 0, opt_w = 0;
+#else
+ int soft = 0, opt_t = 0;
+#endif
int opt, rc;
static struct option opts[] = {
{"period", 1, 0, 'p'},
{"slice", 1, 0, 's'},
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
{"latency", 1, 0, 'l'},
{"extra", 1, 0, 'e'},
{"weight", 1, 0, 'w'},
+#else
+ {"soft", 1, 0, 't'},
+#endif
{"cpupool", 1, 0, 'c'},
COMMON_LONG_OPTS,
{0, 0, 0, 0}
};
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
SWITCH_FOREACH_OPT(opt, "d:p:s:l:e:w:c:h", opts, "sched-sedf", 0) {
+#else
+ SWITCH_FOREACH_OPT(opt, "d:p:s:t:c:h", opts, "sched-sedf", 0) {
+#endif
case 'd':
dom = optarg;
break;
@@ -5493,6 +5524,7 @@ int main_sched_sedf(int argc, char **argv)
slice = strtol(optarg, NULL, 10);
opt_s = 1;
break;
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
case 'l':
latency = strtol(optarg, NULL, 10);
opt_l = 1;
@@ -5505,24 +5537,41 @@ int main_sched_sedf(int argc, char **argv)
weight = strtol(optarg, NULL, 10);
opt_w = 1;
break;
+#else
+ case 't':
+ soft = strtol(optarg, NULL, 10);
+ opt_t = 1;
+ break;
+#endif
case 'c':
cpupool = optarg;
break;
}
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
if (cpupool && (dom || opt_p || opt_s || opt_l || opt_e || opt_w)) {
+#else
+ if (cpupool && (dom || opt_p || opt_s || opt_t)) {
+#endif
fprintf(stderr, "Specifying a cpupool is not allowed with other "
"options.\n");
return 1;
}
+
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
if (!dom && (opt_p || opt_s || opt_l || opt_e || opt_w)) {
+#else
+ if (!dom && (opt_p || opt_s || opt_t)) {
+#endif
fprintf(stderr, "Must specify a domain.\n");
return 1;
}
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION > 0x040400
if (opt_w && (opt_p || opt_s)) {
fprintf(stderr, "Specifying a weight AND period or slice is not "
"allowed.\n");
}
+#endif
if (!dom) { /* list all domain's credit scheduler info */
return -sched_domain_output(LIBXL_SCHEDULER_SEDF,
@@ -5532,7 +5581,11 @@ int main_sched_sedf(int argc, char **argv)
} else {
uint32_t domid = find_domain(dom);
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
if (!opt_p && !opt_s && !opt_l && !opt_e && !opt_w) {
+#else
+ if (!opt_p && !opt_s) {
+#endif
/* output sedf scheduler info */
sched_sedf_domain_output(-1);
return -sched_sedf_domain_output(domid);
@@ -5541,6 +5594,8 @@ int main_sched_sedf(int argc, char **argv)
libxl_domain_sched_params_init(&scinfo);
scinfo.sched = LIBXL_SCHEDULER_SEDF;
+
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
if (opt_p) {
scinfo.period = period;
scinfo.weight = 0;
@@ -5558,6 +5613,17 @@ int main_sched_sedf(int argc, char **argv)
scinfo.period = 0;
scinfo.slice = 0;
}
+#else
+ if (opt_p) {
+ scinfo.period = period;
+ }
+ if (opt_s) {
+ scinfo.slice = slice;
+ }
+ if (opt_t) {
+ scinfo.soft = soft;
+ }
+#endif
rc = sched_domain_set(domid, &scinfo);
libxl_domain_sched_params_dispose(&scinfo);
if (rc)
diff --git a/tools/libxl/xl_cmdtable.c b/tools/libxl/xl_cmdtable.c
index 4279b9f..7f0aca4 100644
--- a/tools/libxl/xl_cmdtable.c
+++ b/tools/libxl/xl_cmdtable.c
@@ -269,12 +269,17 @@ struct cmd_spec cmd_table[] = {
"-p MS, --period=MS Relative deadline(ms)\n"
"-s MS, --slice=MS Worst-case execution time(ms).\n"
" (slice < period)\n"
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
"-l MS, --latency=MS Scaled period (ms) when domain\n"
" performs heavy I/O\n"
"-e FLAG, --extra=FLAG Flag (0 or 1) controls if domain\n"
" can run in extra time\n"
"-w FLOAT, --weight=FLOAT CPU Period/slice (do not set with\n"
" --period/--slice)\n"
+#else
+ "-t FLAG, --soft=FLAG Flag (0 or 1) controls if domain\n"
+ " can run as a soft task\n"
+#endif
"-c CPUPOOL, --cpupool=CPUPOOL Restrict output to CPUPOOL"
},
{ "domid",
--
1.7.9.5
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [RFC PATCH v2 6/7] Changed slice to budget in libxc for the sedf scheduler
2014-07-09 20:55 [RFC PATCH v2 0/7] Repurpose SEDF Scheduler for Real-time Use Josh Whitehead
` (4 preceding siblings ...)
2014-07-09 20:55 ` [RFC PATCH v2 5/7] Add cbs parameter support and removed sedf parameters with a LIBXL_API_VERSION gate from libxl Josh Whitehead
@ 2014-07-09 20:55 ` Josh Whitehead
2014-07-10 14:15 ` Ian Campbell
2014-07-10 23:11 ` Dario Faggioli
2014-07-09 20:55 ` [RFC PATCH v2 7/7] Changed slice to budget in libxl " Josh Whitehead
2014-07-10 14:43 ` [RFC PATCH v2 0/7] Repurpose SEDF Scheduler for Real-time Use Ian Campbell
7 siblings, 2 replies; 30+ messages in thread
From: Josh Whitehead @ 2014-07-09 20:55 UTC (permalink / raw)
To: Xen-devel
Cc: Ian Campbell, Stefano Stabellini, George Dunlap, Dario Faggioli,
Ian Jackson, Robert VanVossen, Nathan Studer, Josh Whitehead
From: Robbie VanVossen <robert.vanvossen@dornerworks.com>
Signed-off-by: Robert VanVossen <Robert.VanVossen@dornerworks.com>
Signed-off-by: Nathan Studer <nate.studer@gmail.com>
---
tools/libxc/xc_sedf.c | 8 ++++----
tools/libxc/xenctrl.h | 4 ++--
tools/python/xen/lowlevel/xc/xc.c | 20 ++++++++++----------
xen/include/public/domctl.h | 2 +-
4 files changed, 17 insertions(+), 17 deletions(-)
diff --git a/tools/libxc/xc_sedf.c b/tools/libxc/xc_sedf.c
index 81ff133..ab67546 100644
--- a/tools/libxc/xc_sedf.c
+++ b/tools/libxc/xc_sedf.c
@@ -28,7 +28,7 @@ int xc_sedf_domain_set(
xc_interface *xch,
uint32_t domid,
uint64_t period,
- uint64_t slice,
+ uint64_t budget,
uint16_t soft)
{
DECLARE_DOMCTL;
@@ -40,7 +40,7 @@ int xc_sedf_domain_set(
domctl.u.scheduler_op.cmd = XEN_DOMCTL_SCHEDOP_putinfo;
p->period = period;
- p->slice = slice;
+ p->budget = budget;
p->soft = soft;
return do_domctl(xch, &domctl);
}
@@ -49,7 +49,7 @@ int xc_sedf_domain_get(
xc_interface *xch,
uint32_t domid,
uint64_t *period,
- uint64_t *slice,
+ uint64_t *budget,
uint16_t *soft)
{
DECLARE_DOMCTL;
@@ -64,7 +64,7 @@ int xc_sedf_domain_get(
ret = do_domctl(xch, &domctl);
*period = p->period;
- *slice = p->slice;
+ *budget = p->budget;
*soft = p->soft;
return ret;
}
diff --git a/tools/libxc/xenctrl.h b/tools/libxc/xenctrl.h
index d5cfdb8..4677b58 100644
--- a/tools/libxc/xenctrl.h
+++ b/tools/libxc/xenctrl.h
@@ -766,13 +766,13 @@ int xc_shadow_control(xc_interface *xch,
int xc_sedf_domain_set(xc_interface *xch,
uint32_t domid,
uint64_t period,
- uint64_t slice,
+ uint64_t budget,
uint16_t soft);
int xc_sedf_domain_get(xc_interface *xch,
uint32_t domid,
uint64_t *period,
- uint64_t *slice,
+ uint64_t *budget,
uint16_t *soft);
int xc_sched_credit_domain_set(xc_interface *xch,
diff --git a/tools/python/xen/lowlevel/xc/xc.c b/tools/python/xen/lowlevel/xc/xc.c
index bf10165..ffe4a38 100644
--- a/tools/python/xen/lowlevel/xc/xc.c
+++ b/tools/python/xen/lowlevel/xc/xc.c
@@ -1471,17 +1471,17 @@ static PyObject *pyxc_sedf_domain_set(XcObject *self,
PyObject *kwds)
{
uint32_t domid;
- uint64_t period, slice;
+ uint64_t period, budget;
uint16_t soft;
- static char *kwd_list[] = { "domid", "period", "slice", "soft",NULL };
+ static char *kwd_list[] = { "domid", "period", "budget", "soft",NULL };
if( !PyArg_ParseTupleAndKeywords(args, kwds, "iLLi", kwd_list,
- &domid, &period, &slice, &soft) )
+ &domid, &period, &budget, &soft) )
return NULL;
if ( xc_sedf_domain_set(self->xc_handle, domid, period,
- slice, soft) != 0 )
+ budget, soft) != 0 )
return pyxc_error_to_exception(self->xc_handle);
Py_INCREF(zero);
@@ -1491,20 +1491,20 @@ static PyObject *pyxc_sedf_domain_set(XcObject *self,
static PyObject *pyxc_sedf_domain_get(XcObject *self, PyObject *args)
{
uint32_t domid;
- uint64_t period, slice;
+ uint64_t period, budget;
uint16_t soft;
if(!PyArg_ParseTuple(args, "i", &domid))
return NULL;
if (xc_sedf_domain_get(self->xc_handle, domid, &period,
- &slice, &soft))
+ &budget, &soft))
return pyxc_error_to_exception(self->xc_handle);
return Py_BuildValue("{s:i,s:L,s:L,s:i}",
- "domid", domid,
+ "domid", domid,
"period", period,
- "slice", slice,
+ "budget", budget,
"soft", soft);
}
@@ -2541,7 +2541,7 @@ static PyMethodDef pyxc_methods[] = {
"Set the scheduling parameters for a domain when running with Atropos.\n"
" dom [int]: domain to set\n"
" period [long]: domain's scheduling period\n"
- " slice [long]: domain's slice per period\n"
+ " budget [long]: domain's slice per period\n"
" soft [int]: domain is a soft task?\n"
"Returns: [int] 0 on success; -1 on error.\n" },
@@ -2554,7 +2554,7 @@ static PyMethodDef pyxc_methods[] = {
"Returns: [dict]\n"
" domain [int]: domain ID\n"
" period [long]: scheduler period\n"
- " slice [long]: CPU reservation per period\n"
+ " budget [long]: CPU reservation per period\n"
" soft [int]: domain is a soft task?\n"},
{ "sched_credit_domain_set",
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 6e143d3..0e8d14a 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -330,7 +330,7 @@ struct xen_domctl_scheduler_op {
union {
struct xen_domctl_sched_sedf {
uint64_aligned_t period;
- uint64_aligned_t slice;
+ uint64_aligned_t budget;
uint32_t soft;
} sedf;
struct xen_domctl_sched_credit {
--
1.7.9.5
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [RFC PATCH v2 7/7] Changed slice to budget in libxl for the sedf scheduler
2014-07-09 20:55 [RFC PATCH v2 0/7] Repurpose SEDF Scheduler for Real-time Use Josh Whitehead
` (5 preceding siblings ...)
2014-07-09 20:55 ` [RFC PATCH v2 6/7] Changed slice to budget in libxc for the sedf scheduler Josh Whitehead
@ 2014-07-09 20:55 ` Josh Whitehead
2014-07-10 14:43 ` [RFC PATCH v2 0/7] Repurpose SEDF Scheduler for Real-time Use Ian Campbell
7 siblings, 0 replies; 30+ messages in thread
From: Josh Whitehead @ 2014-07-09 20:55 UTC (permalink / raw)
To: Xen-devel
Cc: Ian Campbell, Stefano Stabellini, George Dunlap, Dario Faggioli,
Ian Jackson, Robert VanVossen, Nathan Studer, Josh Whitehead
From: Robbie VanVossen <robert.vanvossen@dornerworks.com>
Signed-off-by: Robert VanVossen <Robert.VanVossen@dornerworks.com>
Signed-off-by: Nathan Studer <nate.studer@gmail.com>
---
docs/man/xl.cfg.pod.5 | 4 ++--
tools/libxl/libxl.c | 19 +++++++++++-------
tools/libxl/libxl.h | 1 +
tools/libxl/libxl_types.idl | 1 +
tools/libxl/xl_cmdimpl.c | 46 ++++++++++++++++++++++++-------------------
tools/libxl/xl_cmdtable.c | 6 ++++--
6 files changed, 46 insertions(+), 31 deletions(-)
diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
index 5c55298..4bd0230 100644
--- a/docs/man/xl.cfg.pod.5
+++ b/docs/man/xl.cfg.pod.5
@@ -203,10 +203,10 @@ your BIOS.
=item B<period=NANOSECONDS>
The normal EDF scheduling usage in nanoseconds. This means every period
-the domain gets cpu time defined in slice.
+the domain gets cpu time defined in budget.
Honoured by the sedf scheduler.
-=item B<slice=NANOSECONDS>
+=item B<budget=NANOSECONDS>
The normal EDF scheduling usage in nanoseconds. it defines the time
a domain get every period time.
diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index ca8c1c5..75ea31c 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -4931,12 +4931,13 @@ static int sched_sedf_domain_get(libxl__gc *gc, uint32_t domid,
libxl_domain_sched_params *scinfo)
{
uint64_t period;
- uint64_t slice;
#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
+ uint64_t slice;
uint64_t latency;
uint16_t extratime;
uint16_t weight;
#else
+ uint64_t budget;
uint16_t soft;
#endif
int rc;
@@ -4945,7 +4946,7 @@ static int sched_sedf_domain_get(libxl__gc *gc, uint32_t domid,
rc = xc_sedf_domain_get(CTX->xch, domid, &period, &slice, &latency,
&extratime, &weight);
#else
- rc = xc_sedf_domain_get(CTX->xch, domid, &period, &slice, &soft);
+ rc = xc_sedf_domain_get(CTX->xch, domid, &period, &budget, &soft);
#endif
if (rc != 0) {
LOGE(ERROR, "getting domain sched sedf");
@@ -4955,12 +4956,13 @@ static int sched_sedf_domain_get(libxl__gc *gc, uint32_t domid,
libxl_domain_sched_params_init(scinfo);
scinfo->sched = LIBXL_SCHEDULER_SEDF;
scinfo->period = period / 1000000;
- scinfo->slice = slice / 1000000;
#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
+ scinfo->slice = slice / 1000000;
scinfo->latency = latency / 1000000;
scinfo->extratime = extratime;
scinfo->weight = weight;
#else
+ scinfo->budget = budget / 1000000;
scinfo->soft = soft;
#endif
@@ -4971,12 +4973,13 @@ static int sched_sedf_domain_set(libxl__gc *gc, uint32_t domid,
const libxl_domain_sched_params *scinfo)
{
uint64_t period;
- uint64_t slice;
#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
+ uint64_t slice;
uint64_t latency;
uint16_t extratime;
uint16_t weight;
#else
+ uint64_t budget;
uint16_t soft;
#endif
@@ -4986,7 +4989,7 @@ static int sched_sedf_domain_set(libxl__gc *gc, uint32_t domid,
ret = xc_sedf_domain_get(CTX->xch, domid, &period, &slice, &latency,
&extratime, &weight);
#else
- ret = xc_sedf_domain_get(CTX->xch, domid, &period, &slice, &soft);
+ ret = xc_sedf_domain_get(CTX->xch, domid, &period, &budget, &soft);
#endif
if (ret != 0) {
LOGE(ERROR, "getting domain sched sedf");
@@ -4995,9 +4998,9 @@ static int sched_sedf_domain_set(libxl__gc *gc, uint32_t domid,
if (scinfo->period != LIBXL_DOMAIN_SCHED_PARAM_PERIOD_DEFAULT)
period = (uint64_t)scinfo->period * 1000000;
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
if (scinfo->slice != LIBXL_DOMAIN_SCHED_PARAM_SLICE_DEFAULT)
slice = (uint64_t)scinfo->slice * 1000000;
-#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
if (scinfo->latency != LIBXL_DOMAIN_SCHED_PARAM_LATENCY_DEFAULT)
latency = (uint64_t)scinfo->latency * 1000000;
if (scinfo->extratime != LIBXL_DOMAIN_SCHED_PARAM_EXTRATIME_DEFAULT)
@@ -5005,6 +5008,8 @@ static int sched_sedf_domain_set(libxl__gc *gc, uint32_t domid,
if (scinfo->weight != LIBXL_DOMAIN_SCHED_PARAM_WEIGHT_DEFAULT)
weight = scinfo->weight;
#else
+ if (scinfo->budget != LIBXL_DOMAIN_SCHED_PARAM_BUDGET_DEFAULT)
+ budget = (uint64_t)scinfo->budget * 1000000;
if (scinfo->soft != LIBXL_DOMAIN_SCHED_PARAM_SOFT_DEFAULT)
soft = scinfo->soft;
#endif
@@ -5013,7 +5018,7 @@ static int sched_sedf_domain_set(libxl__gc *gc, uint32_t domid,
ret = xc_sedf_domain_set(CTX->xch, domid, period, slice, latency,
extratime, weight);
#else
- ret = xc_sedf_domain_set(CTX->xch, domid, period, slice, soft);
+ ret = xc_sedf_domain_set(CTX->xch, domid, period, budget, soft);
#endif
if ( ret < 0 ) {
LOGE(ERROR, "setting domain sched sedf");
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index eacd8f6..3c44aeb 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -1122,6 +1122,7 @@ int libxl_sched_credit_params_set(libxl_ctx *ctx, uint32_t poolid,
#define LIBXL_DOMAIN_SCHED_PARAM_SLICE_DEFAULT -1
#define LIBXL_DOMAIN_SCHED_PARAM_LATENCY_DEFAULT -1
#define LIBXL_DOMAIN_SCHED_PARAM_EXTRATIME_DEFAULT -1
+#define LIBXL_DOMAIN_SCHED_PARAM_BUDGET_DEFAULT -1
#define LIBXL_DOMAIN_SCHED_PARAM_SOFT_DEFAULT -1
int libxl_domain_sched_params_get(libxl_ctx *ctx, uint32_t domid,
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index d02380e..d8aefe0 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -294,6 +294,7 @@ libxl_domain_sched_params = Struct("domain_sched_params",[
("slice", integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_SLICE_DEFAULT'}),
("latency", integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_LATENCY_DEFAULT'}),
("extratime", integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_EXTRATIME_DEFAULT'}),
+ ("budget", integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_BUDGET_DEFAULT'}),
("soft", integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_SOFT_DEFAULT'}),
])
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 1f6f04b..e7362a6 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -836,14 +836,16 @@ static void parse_config_data(const char *config_source,
b_info->sched_params.cap = l;
if (!xlu_cfg_get_long (config, "period", &l, 0))
b_info->sched_params.period = l;
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
if (!xlu_cfg_get_long (config, "slice", &l, 0))
b_info->sched_params.slice = l;
-#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
if (!xlu_cfg_get_long (config, "latency", &l, 0))
b_info->sched_params.latency = l;
if (!xlu_cfg_get_long (config, "extratime", &l, 0))
b_info->sched_params.extratime = l;
#else
+ if (!xlu_cfg_get_long (config, "budget", &l, 0))
+ b_info->sched_params.budget = l;
if (!xlu_cfg_get_long (config, "soft", &l, 0))
b_info->sched_params.soft = l;
#endif
@@ -5191,7 +5193,7 @@ static int sched_sedf_domain_output(
"Slice", "Latency", "Extra", "Weight");
#else
printf("%-33s %4s %6s %-6s %5s\n", "Name", "ID", "Period",
- "Slice", "Soft");
+ "Budget", "Soft");
#endif
return 0;
}
@@ -5213,7 +5215,7 @@ static int sched_sedf_domain_output(
domname,
domid,
scinfo.period,
- scinfo.slice,
+ scinfo.budget,
scinfo.soft);
#endif
free(domname);
@@ -5484,24 +5486,26 @@ int main_sched_sedf(int argc, char **argv)
const char *dom = NULL;
const char *cpupool = NULL;
int period = 0, opt_p = 0;
- int slice = 0, opt_s = 0;
#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
+ int slice = 0, opt_s = 0;
int latency = 0, opt_l = 0;
int extra = 0, opt_e = 0;
int weight = 0, opt_w = 0;
#else
- int soft = 0, opt_t = 0;
+ int budget = 0, opt_b = 0;
+ int soft = 0, opt_s = 0;
#endif
int opt, rc;
static struct option opts[] = {
{"period", 1, 0, 'p'},
- {"slice", 1, 0, 's'},
#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
+ {"slice", 1, 0, 's'},
{"latency", 1, 0, 'l'},
{"extra", 1, 0, 'e'},
{"weight", 1, 0, 'w'},
#else
- {"soft", 1, 0, 't'},
+ {"budget", 1, 0, 'b'},
+ {"soft", 1, 0, 's'},
#endif
{"cpupool", 1, 0, 'c'},
COMMON_LONG_OPTS,
@@ -5511,7 +5515,7 @@ int main_sched_sedf(int argc, char **argv)
#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
SWITCH_FOREACH_OPT(opt, "d:p:s:l:e:w:c:h", opts, "sched-sedf", 0) {
#else
- SWITCH_FOREACH_OPT(opt, "d:p:s:t:c:h", opts, "sched-sedf", 0) {
+ SWITCH_FOREACH_OPT(opt, "d:p:b:s:c:h", opts, "sched-sedf", 0) {
#endif
case 'd':
dom = optarg;
@@ -5520,11 +5524,11 @@ int main_sched_sedf(int argc, char **argv)
period = strtol(optarg, NULL, 10);
opt_p = 1;
break;
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
case 's':
slice = strtol(optarg, NULL, 10);
opt_s = 1;
break;
-#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
case 'l':
latency = strtol(optarg, NULL, 10);
opt_l = 1;
@@ -5538,9 +5542,13 @@ int main_sched_sedf(int argc, char **argv)
opt_w = 1;
break;
#else
- case 't':
+ case 'b':
+ budget = strtol(optarg, NULL, 10);
+ opt_b = 1;
+ break;
+ case 's':
soft = strtol(optarg, NULL, 10);
- opt_t = 1;
+ opt_s = 1;
break;
#endif
case 'c':
@@ -5551,22 +5559,21 @@ int main_sched_sedf(int argc, char **argv)
#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
if (cpupool && (dom || opt_p || opt_s || opt_l || opt_e || opt_w)) {
#else
- if (cpupool && (dom || opt_p || opt_s || opt_t)) {
+ if (cpupool && (dom || opt_p || opt_b || opt_s)) {
#endif
fprintf(stderr, "Specifying a cpupool is not allowed with other "
"options.\n");
return 1;
}
-
#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
if (!dom && (opt_p || opt_s || opt_l || opt_e || opt_w)) {
#else
- if (!dom && (opt_p || opt_s || opt_t)) {
+ if (!dom && (opt_p || opt_b || opt_s)) {
#endif
fprintf(stderr, "Must specify a domain.\n");
return 1;
}
-#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION > 0x040400
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
if (opt_w && (opt_p || opt_s)) {
fprintf(stderr, "Specifying a weight AND period or slice is not "
"allowed.\n");
@@ -5584,7 +5591,7 @@ int main_sched_sedf(int argc, char **argv)
#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
if (!opt_p && !opt_s && !opt_l && !opt_e && !opt_w) {
#else
- if (!opt_p && !opt_s) {
+ if (!opt_p && !opt_b) {
#endif
/* output sedf scheduler info */
sched_sedf_domain_output(-1);
@@ -5594,7 +5601,6 @@ int main_sched_sedf(int argc, char **argv)
libxl_domain_sched_params_init(&scinfo);
scinfo.sched = LIBXL_SCHEDULER_SEDF;
-
#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
if (opt_p) {
scinfo.period = period;
@@ -5617,10 +5623,10 @@ int main_sched_sedf(int argc, char **argv)
if (opt_p) {
scinfo.period = period;
}
- if (opt_s) {
- scinfo.slice = slice;
+ if (opt_b) {
+ scinfo.budget = budget;
}
- if (opt_t) {
+ if (opt_s) {
scinfo.soft = soft;
}
#endif
diff --git a/tools/libxl/xl_cmdtable.c b/tools/libxl/xl_cmdtable.c
index 7f0aca4..526fecb 100644
--- a/tools/libxl/xl_cmdtable.c
+++ b/tools/libxl/xl_cmdtable.c
@@ -267,9 +267,9 @@ struct cmd_spec cmd_table[] = {
"[options]",
"-d DOMAIN, --domain=DOMAIN Domain to modify\n"
"-p MS, --period=MS Relative deadline(ms)\n"
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
"-s MS, --slice=MS Worst-case execution time(ms).\n"
" (slice < period)\n"
-#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
"-l MS, --latency=MS Scaled period (ms) when domain\n"
" performs heavy I/O\n"
"-e FLAG, --extra=FLAG Flag (0 or 1) controls if domain\n"
@@ -277,7 +277,9 @@ struct cmd_spec cmd_table[] = {
"-w FLOAT, --weight=FLOAT CPU Period/slice (do not set with\n"
" --period/--slice)\n"
#else
- "-t FLAG, --soft=FLAG Flag (0 or 1) controls if domain\n"
+ "-b MS, --budget=MS Constant bandwidth server budget(ms).\n"
+ " (budget < period)\n"
+ "-s FLAG, --soft=FLAG Flag (0 or 1) controls if domain\n"
" can run as a soft task\n"
#endif
"-c CPUPOOL, --cpupool=CPUPOOL Restrict output to CPUPOOL"
--
1.7.9.5
^ permalink raw reply related [flat|nested] 30+ messages in thread
* Re: [RFC PATCH v2 5/7] Add cbs parameter support and removed sedf parameters with a LIBXL_API_VERSION gate from libxl.
2014-07-09 20:55 ` [RFC PATCH v2 5/7] Add cbs parameter support and removed sedf parameters with a LIBXL_API_VERSION gate from libxl Josh Whitehead
@ 2014-07-10 14:09 ` Ian Campbell
2014-07-10 14:55 ` Ian Jackson
2014-07-10 23:02 ` Dario Faggioli
2014-07-10 14:26 ` Dario Faggioli
1 sibling, 2 replies; 30+ messages in thread
From: Ian Campbell @ 2014-07-10 14:09 UTC (permalink / raw)
To: Josh Whitehead
Cc: Stefano Stabellini, George Dunlap, Dario Faggioli, Ian Jackson,
Robert VanVossen, Xen-devel, Nathan Studer
On Wed, 2014-07-09 at 16:55 -0400, Josh Whitehead wrote:
> From: Robbie VanVossen <robert.vanvossen@dornerworks.com>
>
> We made an attempt at doing the versioning gate in this file. Please let us
> know if further changes are needed, and where they are needed to properly guard
> the API.
I previously said that I thought it was fine to remove the obsolete SEDF
stuff from the libxl API.
However I mentioned this to the other maintainers and they disagreed
with me. The feeling is that an old application which used the old SEDF
parameters should continue to build but that trying to use those
parameters should result in an error. I'm sorry for steering you wrong
here.
However looking at the diff of libxl_types.idl it seems like you haven't
removed anything from the API struct so perhaps this is already close to
ok. I think all you need to do is arrange to return an error when
someone tries to use the old parameters.
> diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
> index 900b8d4..ca8c1c5 100644
> --- a/tools/libxl/libxl.c
> +++ b/tools/libxl/libxl.c
> @@ -4932,13 +4932,21 @@ static int sched_sedf_domain_get(libxl__gc *gc, uint32_t domid,
> {
> uint64_t period;
> uint64_t slice;
> +#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
There shouldn't be any use of LIBXL_API_VERSION in libxl*.c files.
The libxl code always implement the latest interface. The purpose of
LIBXL_API_VERSION is so that in libxl.h we can provide shims to bridge
the gap between the version that the application has requested and the
latest implementation.
There will only be one libxl.so binary, but it should be possible to
compile and link an old application using an old LIBXL_API_VERSION
against the libxl.h header and have it still work, without heeding to
change the library.
See for example the handling of libxl_set_vcpuaffinity. In the 4.5 cycle
this gained a new parameter (a soft affinity bitmap). The compat shim is
just:
#if defined (LIBXL_API_VERSION) && LIBXL_API_VERSION < 0x040500
#define libxl_set_vcpuaffinity(ctx, domid, vcpuid, map) \
libxl_set_vcpuaffinity((ctx), (domid), (vcpuid), (map), NULL)
passing NULL for the new param. The implementation of
libxl_set_vcpuaffinity has to handle NULL for this parameter (which it
did anyway) bit it is not (and cannot be) affected by LIBXL_API_VERSION.
I think you can for the most part just remove any code which is behind a
#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400 and
instead add checks like
if (foo->slice != LIBXL_DOMAIN_SCHED_PARAM_SLICE_DEFAULT)
return ERROR_INVAL; /* deprecated option used */
at the entry points.
Ian (J) -- is this the sort of thing you meant?
> diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> index 52f1aa9..d02380e 100644
> --- a/tools/libxl/libxl_types.idl
> +++ b/tools/libxl/libxl_types.idl
> @@ -294,6 +294,7 @@ libxl_domain_sched_params = Struct("domain_sched_params",[
> ("slice", integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_SLICE_DEFAULT'}),
> ("latency", integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_LATENCY_DEFAULT'}),
> ("extratime", integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_EXTRATIME_DEFAULT'}),
> + ("soft", integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_SOFT_DEFAULT'}),
> ])
Please could you also add some comments to indicate which fields are no
longer supported.
Please also add a #define LIBXL_HAVE_* to indicate the presence of this
new field (and by implication the unavailability/uselessness of the old
ones). There are some examples in the header already.
> libxl_domain_build_info = Struct("domain_build_info",[
> diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
> index 5195914..1f6f04b 100644
> --- a/tools/libxl/xl_cmdimpl.c
> +++ b/tools/libxl/xl_cmdimpl.c
> @@ -838,10 +838,15 @@ static void parse_config_data(const char *config_source,
> b_info->sched_params.period = l;
> if (!xlu_cfg_get_long (config, "slice", &l, 0))
> b_info->sched_params.slice = l;
> +#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400
Since xl is intree it doesn't need to use LIBXL_API_VERSION, it can just
use the new interface. So on the xl side you should strip out everything
behind LIBXL_API_VERSION <= 0x040400.
The purpose of LIBXL_API_VERSION is that the application sets it while
including libxl.h and then proceeds to use that version's interface
without the need for all these ifdefs on the application side, so what
you've done is backwards.
Ian.
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH v2 6/7] Changed slice to budget in libxc for the sedf scheduler
2014-07-09 20:55 ` [RFC PATCH v2 6/7] Changed slice to budget in libxc for the sedf scheduler Josh Whitehead
@ 2014-07-10 14:15 ` Ian Campbell
2014-07-10 23:11 ` Dario Faggioli
1 sibling, 0 replies; 30+ messages in thread
From: Ian Campbell @ 2014-07-10 14:15 UTC (permalink / raw)
To: Josh Whitehead
Cc: Stefano Stabellini, George Dunlap, Dario Faggioli, Ian Jackson,
Robert VanVossen, Xen-devel, Nathan Studer
On Wed, 2014-07-09 at 16:55 -0400, Josh Whitehead wrote:
> From: Robbie VanVossen <robert.vanvossen@dornerworks.com>
>
> Signed-off-by: Robert VanVossen <Robert.VanVossen@dornerworks.com>
> Signed-off-by: Nathan Studer <nate.studer@gmail.com>
>
> ---
> tools/libxc/xc_sedf.c | 8 ++++----
> tools/libxc/xenctrl.h | 4 ++--
> tools/python/xen/lowlevel/xc/xc.c | 20 ++++++++++----------
> xen/include/public/domctl.h | 2 +-
> 4 files changed, 17 insertions(+), 17 deletions(-)
>
> diff --git a/tools/libxc/xc_sedf.c b/tools/libxc/xc_sedf.c
> index 81ff133..ab67546 100644
> --- a/tools/libxc/xc_sedf.c
> +++ b/tools/libxc/xc_sedf.c
> @@ -28,7 +28,7 @@ int xc_sedf_domain_set(
> xc_interface *xch,
> uint32_t domid,
> uint64_t period,
> - uint64_t slice,
> + uint64_t budget,
Isn't the going to break the libxl behaviour between this patch and the
next one?
I think it is unavoidable to change this interface and the libxl one at
the same time.
If there is sensible default budget then perhaps you could change libxl
to use that unconditionally in this patch and then do all the plumbing
out to the libxl API and into xl in the next patch.
Ian.
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH v2 5/7] Add cbs parameter support and removed sedf parameters with a LIBXL_API_VERSION gate from libxl.
2014-07-09 20:55 ` [RFC PATCH v2 5/7] Add cbs parameter support and removed sedf parameters with a LIBXL_API_VERSION gate from libxl Josh Whitehead
2014-07-10 14:09 ` Ian Campbell
@ 2014-07-10 14:26 ` Dario Faggioli
1 sibling, 0 replies; 30+ messages in thread
From: Dario Faggioli @ 2014-07-10 14:26 UTC (permalink / raw)
To: Josh Whitehead
Cc: Ian Campbell, Stefano Stabellini, George Dunlap, Ian Jackson,
Robert VanVossen, Xen-devel, Nathan Studer
[-- Attachment #1.1: Type: text/plain, Size: 1161 bytes --]
On mer, 2014-07-09 at 16:55 -0400, Josh Whitehead wrote:
> From: Robbie VanVossen <robert.vanvossen@dornerworks.com>
>
> We made an attempt at doing the versioning gate in this file. Please let us
> know if further changes are needed, and where they are needed to properly guard
> the API.
>
So, since you're adding soft to libxl_domain_sched_params, so you need
to add something like:
/*
* Some comments...
*/
#define LIBXL_HAVE_DOMAIN_SCHED_PARAMS_SOFT 1
in libxl.h.
Look at LIBXL_HAVE_DOMAIN_NODEAFFINITY,
LIBXL_HAVE_VCPUINFO_SOFT_AFFINITY for inspiration. :-)
If it were me, I think I'd include _SEDF_ in the name as well:
#define LIBXL_HAVE_DOMAIN_SCHED_SEDF_PARAMS_SOFT 1
or:
#define LIBXL_HAVE_DOMAIN_SCHEDPARAM_SEDF_SOFT 1
And I now see that Ian said plenty already in his email about
LIBXL_API_VERSION. I'll continue there...
Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH v2 0/7] Repurpose SEDF Scheduler for Real-time Use
2014-07-09 20:55 [RFC PATCH v2 0/7] Repurpose SEDF Scheduler for Real-time Use Josh Whitehead
` (6 preceding siblings ...)
2014-07-09 20:55 ` [RFC PATCH v2 7/7] Changed slice to budget in libxl " Josh Whitehead
@ 2014-07-10 14:43 ` Ian Campbell
2014-07-11 5:01 ` Dario Faggioli
7 siblings, 1 reply; 30+ messages in thread
From: Ian Campbell @ 2014-07-10 14:43 UTC (permalink / raw)
To: Josh Whitehead
Cc: Stefano Stabellini, George Dunlap, Dario Faggioli, Ian Jackson,
Robert VanVossen, Xen-devel, Nathan Studer
On Wed, 2014-07-09 at 16:55 -0400, Josh Whitehead wrote:
> Finally, the amount of name changes has been significantly reduced,
> the most prominent being to keep the "sedf" name and not switch to a
> new name as was also decided through previous discussion.
This is unfortunate since renaming would have meant a new member of the
libxl_scheduler enum which would have made the API
deprecation/compatibility stuff much simpler -- since you could just
return ERROR_INVAL (or ERROR_SEDF_WENT_AWAY ;-)) whenever you saw the
SEDF enum value, rather than having to check each field individually.
Ian.
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH v2 5/7] Add cbs parameter support and removed sedf parameters with a LIBXL_API_VERSION gate from libxl.
2014-07-10 14:09 ` Ian Campbell
@ 2014-07-10 14:55 ` Ian Jackson
2014-07-10 23:02 ` Dario Faggioli
1 sibling, 0 replies; 30+ messages in thread
From: Ian Jackson @ 2014-07-10 14:55 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, George Dunlap, Dario Faggioli,
Robert VanVossen, Xen-devel, Nathan Studer, Josh Whitehead
Ian Campbell writes ("Re: [RFC PATCH v2 5/7] Add cbs parameter support and removed sedf parameters with a LIBXL_API_VERSION gate from libxl."):
> I previously said that I thought it was fine to remove the obsolete SEDF
> stuff from the libxl API.
>
> However I mentioned this to the other maintainers and they disagreed
> with me. The feeling is that an old application which used the old SEDF
> parameters should continue to build but that trying to use those
> parameters should result in an error. [...]
Yes.
> I think you can for the most part just remove any code which is behind a
> #if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION <= 0x040400 and
> instead add checks like
> if (foo->slice != LIBXL_DOMAIN_SCHED_PARAM_SLICE_DEFAULT)
> return ERROR_INVAL; /* deprecated option used */
> at the entry points.
>
> Ian (J) -- is this the sort of thing you meant?
Yes. But I agree with what you said later about it perhaps being
easier if we give the new scheduler a new name (at the libxl API
layer, at least).
Ian.
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH v2 5/7] Add cbs parameter support and removed sedf parameters with a LIBXL_API_VERSION gate from libxl.
2014-07-10 14:09 ` Ian Campbell
2014-07-10 14:55 ` Ian Jackson
@ 2014-07-10 23:02 ` Dario Faggioli
2014-07-11 11:12 ` Ian Campbell
1 sibling, 1 reply; 30+ messages in thread
From: Dario Faggioli @ 2014-07-10 23:02 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, George Dunlap, Ian Jackson, Robert VanVossen,
Xen-devel, Nathan Studer, Josh Whitehead
[-- Attachment #1.1: Type: text/plain, Size: 1377 bytes --]
On gio, 2014-07-10 at 15:09 +0100, Ian Campbell wrote:
> On Wed, 2014-07-09 at 16:55 -0400, Josh Whitehead wrote:
> > From: Robbie VanVossen <robert.vanvossen@dornerworks.com>
> >
> > We made an attempt at doing the versioning gate in this file. Please let us
> > know if further changes are needed, and where they are needed to properly guard
> > the API.
>
> I previously said that I thought it was fine to remove the obsolete SEDF
> stuff from the libxl API.
>
> However I mentioned this to the other maintainers and they disagreed
> with me. The feeling is that an old application which used the old SEDF
> parameters should continue to build but that trying to use those
> parameters should result in an error.
>
Indeed. So, as far as this series is concerned, it looks like either the
following solutions:
- introduce a new scheduler, with a new name, and deprecate SEDF all
together
- keep the name and the full existing interface, but warn the user when
she touches one of the params that are no longer in use in the
implementation
would be ok.
Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH v2 6/7] Changed slice to budget in libxc for the sedf scheduler
2014-07-09 20:55 ` [RFC PATCH v2 6/7] Changed slice to budget in libxc for the sedf scheduler Josh Whitehead
2014-07-10 14:15 ` Ian Campbell
@ 2014-07-10 23:11 ` Dario Faggioli
2014-07-11 11:11 ` Ian Campbell
1 sibling, 1 reply; 30+ messages in thread
From: Dario Faggioli @ 2014-07-10 23:11 UTC (permalink / raw)
To: Josh Whitehead
Cc: Ian Campbell, Stefano Stabellini, George Dunlap, Ian Jackson,
Robert VanVossen, Xen-devel, Nathan Studer
[-- Attachment #1.1: Type: text/plain, Size: 1575 bytes --]
On mer, 2014-07-09 at 16:55 -0400, Josh Whitehead wrote:
> From: Robbie VanVossen <robert.vanvossen@dornerworks.com>
>
> Signed-off-by: Robert VanVossen <Robert.VanVossen@dornerworks.com>
> Signed-off-by: Nathan Studer <nate.studer@gmail.com>
>
> ---
> tools/libxc/xc_sedf.c | 8 ++++----
> tools/libxc/xenctrl.h | 4 ++--
> tools/python/xen/lowlevel/xc/xc.c | 20 ++++++++++----------
> xen/include/public/domctl.h | 2 +-
> 4 files changed, 17 insertions(+), 17 deletions(-)
>
> diff --git a/tools/libxc/xc_sedf.c b/tools/libxc/xc_sedf.c
> index 81ff133..ab67546 100644
> --- a/tools/libxc/xc_sedf.c
> +++ b/tools/libxc/xc_sedf.c
> @@ -28,7 +28,7 @@ int xc_sedf_domain_set(
> xc_interface *xch,
> uint32_t domid,
> uint64_t period,
> - uint64_t slice,
> + uint64_t budget,
>
Does this really matter?
'budget' is certainly the most used term in real-time literature and
research, yes, but slice is not that terrible after all. I think it does
get the point across about how one should interpret and use the
parameter itself.
If we were doing a new scheduler, I think I also would pick budget as
the name for this param. If we are to modify SEDF, I honestly think we
can live with 'slice'.
Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH v2 4/7] Add cbs parameter support and removed sedf parameters from libxc
2014-07-09 20:55 ` [RFC PATCH v2 4/7] Add cbs parameter support and removed sedf parameters from libxc Josh Whitehead
@ 2014-07-10 23:17 ` Dario Faggioli
0 siblings, 0 replies; 30+ messages in thread
From: Dario Faggioli @ 2014-07-10 23:17 UTC (permalink / raw)
To: Josh Whitehead
Cc: Ian Campbell, Stefano Stabellini, George Dunlap, Ian Jackson,
Robert VanVossen, Xen-devel, Nathan Studer
[-- Attachment #1.1: Type: text/plain, Size: 647 bytes --]
On mer, 2014-07-09 at 16:55 -0400, Josh Whitehead wrote:
> From: Robbie VanVossen <robert.vanvossen@dornerworks.com>
>
> This file may also need something similar to the LIBXL_API_VERSION gate.
>
Not really. In fact, libxc interface is not to be considered stable by
any means, and it's then fine to change it in build and/or run time
incompatible ways.
Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH v2 2/7] Fixed formatting and misleading comments/variables. Added comments and renamed variables to accurately reflect modern terminology
2014-07-09 20:55 ` [RFC PATCH v2 2/7] Fixed formatting and misleading comments/variables. Added comments and renamed variables to accurately reflect modern terminology Josh Whitehead
@ 2014-07-11 3:59 ` Dario Faggioli
0 siblings, 0 replies; 30+ messages in thread
From: Dario Faggioli @ 2014-07-11 3:59 UTC (permalink / raw)
To: Josh Whitehead
Cc: Ian Campbell, Stefano Stabellini, George Dunlap, Ian Jackson,
Robert VanVossen, Xen-devel, Nathan Studer
[-- Attachment #1.1: Type: text/plain, Size: 9836 bytes --]
On mer, 2014-07-09 at 16:55 -0400, Josh Whitehead wrote:
> Due to the age of the scheduler there were many incorrect/misleading comments
> and variable names, the bulk of which centered around the fact that "VCPU" and
> "Domain" used to be synonymous. Therefore a large portion of these modifcations
> involve simply changing a variable "d" to a "v" or a "dom" to "vcpu" so that the
> comments and variable names are accurate to what's being used.
>
Indeed!
> A few other name changes were also made, the most significant being the change
> from "slice" to "budget" to better reflect modern terminology used in real-time
> algorithms such as CBS and deferrable server.
>
BTW, I said that, at libxl and libxc level (but I mostly was referring
to libxl) it's fine to keep slice as the name of this param, at least
for now.
I confirm that that's what I think, for libxl. Right here, deep down
inside the hypervisor, I agree about the slice-->budget transition.
> --- a/xen/common/sched_sedf.c
> +++ b/xen/common/sched_sedf.c
> @@ -28,14 +49,13 @@
> #define SEDF_ASLEEP (16)
>
This one here alone, and define to 16, is kind of ugly, but I shall say
that when reviewing the previous patch...
> #define DEFAULT_PERIOD (MILLISECS(20))
> -#define DEFAULT_SLICE (MILLISECS(10))
> +#define DEFAULT_BUDGET (MILLISECS(10))
>
> #define PERIOD_MAX MILLISECS(10000) /* 10s */
> #define PERIOD_MIN (MICROSECS(10)) /* 10us */
> -#define SLICE_MIN (MICROSECS(5)) /* 5us */
> +#define BUDGET_MIN (MICROSECS(5)) /* 5us */
>
How about we make the name of the scheduler part of the name of these
macros. I.e., SEDF_DEFAULT_BUDGET and SEDF_MIN_BUDGET (or
SEDF_BUDGET_MIN, but I like it worse) ?
It's certainly not necessary, but it's what other schedulers do, and it
may come handy when grep-ping, etc.
> -#define IMPLY(a, b) (!(a) || (b))
> -#define EQ(a, b) ((!!(a)) == (!!(b)))
> +#define EQ(_A, _B) ((!!(_A)) == (!!(_B)))
>
Do we really need to keep this? Not such a big deal I guess, but I truly
tryly truly dislike it! :-(
> struct sedf_dom_info {
> @@ -52,16 +72,16 @@ struct sedf_vcpu_info {
> struct list_head list;
>
> /* Parameters for EDF */
> - s_time_t period; /* = relative deadline */
> - s_time_t slice; /* = worst case execution time */
> + s_time_t period; /* = Server scheduling period */
>
I'd say "VCPUs scheduling period". I know 'Server' is the correct term,
but I suspect it will just be confusing people...
> #define SEDF_PRIV(_ops) \
> ((struct sedf_priv_info *)((_ops)->sched_data))
> -#define EDOM_INFO(d) ((struct sedf_vcpu_info *)((d)->sched_priv))
> -#define CPU_INFO(cpu) \
> - ((struct sedf_cpu_info *)per_cpu(schedule_data, cpu).sched_priv)
> -#define LIST(d) (&EDOM_INFO(d)->list)
> -#define RUNQ(cpu) (&CPU_INFO(cpu)->runnableq)
> -#define WAITQ(cpu) (&CPU_INFO(cpu)->waitq)
> -#define IDLETASK(cpu) (idle_vcpu[cpu])
> +#define SEDF_VCPU(_vcpu) ((struct sedf_vcpu_info *)((_vcpu)->sched_priv))
> +#define SEDF_PCPU(_cpu) \
> + ((struct sedf_cpu_info *)per_cpu(schedule_data, _cpu).sched_priv)
> +#define LIST(_vcpu) (&SEDF_VCPU(_vcpu)->list)
> +#define RUNQ(_cpu) (&SEDF_PCPU(_cpu)->runnableq)
> +#define WAITQ(_cpu) (&SEDF_PCPU(_cpu)->waitq)
> +#define IDLETASK(_cpu) (idle_vcpu[_cpu])
>
While at this, can you put down a comment on what LIST, RUNQ and WAITQ
are? RUNQ is pretty obvious, the others, not so much.
> #define PERIOD_BEGIN(inf) ((inf)->deadl_abs - (inf)->period)
>
> -#define DIV_UP(x,y) (((x) + (y) - 1) / y)
> +#define DIV_UP(_X, _Y) (((_X) + (_Y) - 1) / _Y)
>
Does this makes much difference?
> /*
> - * Adds a domain to the queue of processes which wait for the beginning of the
> - * next period; this list is therefore sortet by this time, which is simply
> + * Adds a vcpu to the queue of processes which wait for the beginning of the
of vcpus ?
> + * next period; this list is therefore sorted by this time, which is simply
> * absol. deadline - period.
> */
> -DOMAIN_COMPARER(waitq, list, PERIOD_BEGIN(d1), PERIOD_BEGIN(d2));
> +VCPU_COMPARER(waitq, list, PERIOD_BEGIN(v1), PERIOD_BEGIN(v2));
> static inline void __add_to_waitqueue_sort(struct vcpu *v)
> {
> ASSERT(!__task_on_queue(v));
> @@ -156,12 +176,12 @@ static inline void __add_to_waitqueue_sort(struct vcpu *v)
> }
>
> /*
> - * Adds a domain to the queue of processes which have started their current
> + * Adds a vcpu to the queue of processes which have started their current
>
same here.
> * period and are runnable (i.e. not blocked, dieing,...). The first element
> * on this list is running on the processor, if the list is empty the idle
> * task will run. As we are implementing EDF, this list is sorted by deadlines.
> */
> @@ -187,19 +207,19 @@ static void *sedf_alloc_vdata(const struct scheduler *ops, struct vcpu *v, void
>
> inf->vcpu = v;
>
> - inf->deadl_abs = 0;
> - inf->status = SEDF_ASLEEP;
> + inf->deadl_abs = 0;
> + inf->status = SEDF_ASLEEP;
>
> if (v->domain->domain_id == 0)
> {
> - /* Domain 0, needs a slice to boot the machine */
> - inf->period = DEFAULT_PERIOD;
> - inf->slice = DEFAULT_SLICE;
> + /* Domain 0, needs a budget to boot the machine */
> + inf->period = DEFAULT_PERIOD;
> + inf->budget = DEFAULT_BUDGET;
> }
> else
> {
> - inf->period = DEFAULT_PERIOD;
> - inf->slice = 0;
> + inf->period = DEFAULT_PERIOD;
> + inf->budget = 0;
> }
>
So, by default, Dom0 gets DEFAULT_BUDGET, other domains get 0. Does this
mean they can't even start executing until someone changes that
manually, by modifying the parameters and giving them some budget?
> static struct task_slice sedf_do_schedule(
> const struct scheduler *ops, s_time_t now, bool_t tasklet_work_scheduled)
> @@ -401,18 +421,18 @@ static struct task_slice sedf_do_schedule(
> int cpu = smp_processor_id();
> struct list_head *runq = RUNQ(cpu);
> struct list_head *waitq = WAITQ(cpu);
> - struct sedf_vcpu_info *inf = EDOM_INFO(current);
> + struct sedf_vcpu_info *inf = SEDF_VCPU(current);
> struct sedf_vcpu_info *runinf, *waitinf;
> struct task_slice ret;
>
> SCHED_STAT_CRANK(schedule);
>
> - /* Idle tasks don't need any of the following stuf */
> + /* Idle tasks don't need any of the following stuff */
Idle vcpu
It's again not a big deal, but it's more consistent.
> /*
> - * This function wakes up a domain, i.e. moves them into the waitqueue
> - * things to mention are: admission control is taking place nowhere at
> - * the moment, so we can't be sure, whether it is safe to wake the domain
> - * up at all.
>
I am all in favor of the removal of this complex and broken logic for
handling wake-ups.
However, the chunk of commit above --which is pretty much unrelated to
wake-ups-- seems something useful to have somewhere, perhaps at the
beginning of the file.
Anyway, whether you want to use this chunk or not, what I have in mind
is it'd be worth mentioning that the scheduler does not (by design)
perform any admission control at all.
What do you think?
> -static void sedf_wake(const struct scheduler *ops, struct vcpu *d)
> +/*
> + * This function wakes up a vcpu, i.e. moves them into the appropriate queue
> + *
> + * When a blocked vcpu unblocks, it is allowed to start execution at
> + * the beginning of the next complete period
> + * (D..deadline, R..running, B..blocking/sleeping, U..unblocking/waking up
> + *
> + * DRRB_____D__U_____DRRRRR___D________ ...
> + *
> + * - This causes the vcpu to miss a period (and a deadlline)
> + * - Doesn't disturb the schedule at all
> + * - Deadlines keep occuring isochronous
> + */
>
Oh, ok. So, between the various special cases, you decided to pick
number 1, calle "Very conservative". If it were me, I would probably
have picked the one called "Part 2b" (i.e., the one that resets the
deadline).
Well, I don't think this matter much anyway, since you're changing it in
the next patch, I'm sure. :-)
> +static void sedf_wake(const struct scheduler *ops, struct vcpu *v)
> {
> s_time_t now = NOW();
> - struct sedf_vcpu_info* inf = EDOM_INFO(d);
> + struct sedf_vcpu_info* inf = SEDF_VCPU(v);
>
> - if ( unlikely(is_idle_vcpu(d)) )
> + if ( unlikely(is_idle_vcpu(v)) )
> return;
>
> - if ( unlikely(__task_on_queue(d)) )
> + if ( unlikely(__task_on_queue(v)) )
> return;
>
> - ASSERT(!sedf_runnable(d));
> + ASSERT(!sedf_runnable(v));
> inf->status &= ~SEDF_ASLEEP;
>
> if ( unlikely(inf->deadl_abs == 0) )
> {
> /* Initial setup of the deadline */
> - inf->deadl_abs = now + inf->slice;
> + inf->deadl_abs = now + inf->budget;
> }
>
> #ifdef SEDF_STATS
> @@ -627,14 +595,14 @@ static void sedf_wake(const struct scheduler *ops, struct vcpu *d)
> }
> else
> {
> - /* Long unblocking */
> + /* Long unblocking, someone is going to miss their deadline. */
>
Sorry? Someone who?
> inf->long_block_tot++;
> }
Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH v2 0/7] Repurpose SEDF Scheduler for Real-time Use
2014-07-10 14:43 ` [RFC PATCH v2 0/7] Repurpose SEDF Scheduler for Real-time Use Ian Campbell
@ 2014-07-11 5:01 ` Dario Faggioli
2014-07-11 11:15 ` Ian Campbell
2014-07-11 20:02 ` Konrad Rzeszutek Wilk
0 siblings, 2 replies; 30+ messages in thread
From: Dario Faggioli @ 2014-07-11 5:01 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, George Dunlap, Ian Jackson, Robert VanVossen,
Xen-devel, Nathan Studer, Josh Whitehead
[-- Attachment #1.1: Type: text/plain, Size: 7400 bytes --]
On gio, 2014-07-10 at 15:43 +0100, Ian Campbell wrote:
> On Wed, 2014-07-09 at 16:55 -0400, Josh Whitehead wrote:
> > Finally, the amount of name changes has been significantly reduced,
> > the most prominent being to keep the "sedf" name and not switch to a
> > new name as was also decided through previous discussion.
>
> This is unfortunate since renaming would have meant a new member of the
> libxl_scheduler enum which would have made the API
> deprecation/compatibility stuff much simpler -- since you could just
> return ERROR_INVAL (or ERROR_SEDF_WENT_AWAY ;-)) whenever you saw the
> SEDF enum value, rather than having to check each field individually.
>
*** Short (well, sort of!) Answer ***
Personally, I would prefer to keep SEDF alive at least for a few
versions (warning the user about some params being ignored, etc.).
However, whatever route we take, concentrating on this series, I don't
think the renaming+SEDF deprecation should happen until proper SMP
support is implemented, and probably also not until support for per-VCPU
scheduling parameters (quite important for an advanced real-time
scheduling solution) is there. It's fine for an RFC to be incomplete, so
I'm not complaining or saying this can't be a step in the right
direction.
However, the biggest problem of SEDF is its poor support for SMP, and
lack of capability for setting scheduling parameters on a per-VCPU
basis, which this series (rightfully) does not address.
Imagine if, after proper review and cleanups, the set of changes from
this series goes in, and no other follow up series makes it by feature
freeze, 4.5 will have:
- SCHED_SEDF deprecated, because it was lacking SMP support and support
for per-VCPU scheduling parameters
- SCHED_CBS, _lacking_ SMP support and support for per-VCPU scheduling
parameters
Not ideal. :-/
[That, of course, does not mean that I won't be reviewing the series
itself. On the contrary, I'm down to it, especially patches 1, 2 and 3.]
So, to summarize, I'm ok with whatever route we decide to take, included
full deprecation and renaming, but let's make it worth it!
If wanting to do that, Josh and Robert, you can well assume that the
unnecessary parameters and the messy code currently in sched_sedf.c
related to them can go away (either being ignored by libxl, or because
this will be a new scheduler), and so it's fine for you to kill them in
one of your patches. This is something I was not so sure myself, so,
thanks for bringing the discussion up on it with this series! :-)
Finally, I also apologize if my comments were not explicitly along this
line during v1 review... it just took a bit of thinking to wrap my head
around what the best way forward could be here. :-(
Dario
*** Long (Really Long!!) Answer ***
The _core_ of the contribution of this series, as of now, is not a new
scheduling policy. It is a slight (although, to me, very relevant)
modification to SEDF internals in order to fix a few problems the
scheduler has.
In fact, I like the fact that the subject itself is 'Repurpose SEDF
Scheduler'. :-)
The problems SEDF has are:
1. it has really really really poor SMP support
2. it does not allow to specify scheduling parameters on a per-VCPU
basis, but only on a domain basis. This is fine for general purpose
schedulers, but can be quite important in real-time workloads
3. it tries to be both a real-time and a general purpose scheduler
which is:
a) very hard, and often means adding a lot of complexity for
nothing, since you're not going to be that good at both anyway...
and this is one of the cases;
b) not necessary any longer, now that we have good general purpose
schedulers, like credit and (hopefully ready soon) credit2;
4. it deals with VCPU wake-ups in an hacky, unnecessarily complex and,
if you ask me, incorrect way.
Note that, in order to achieve 3, not only a lot of complexity (hacks!)
were introduced inside sched_sedf.c in Xen, it is also the interface
that contains parameters that are not useful for a real-time scheduler
which only aims at doing its own job well, i.e.,
_being_a_good_real-time_scheduler_. :-D
The issues are, in my opinion, sort-of in priority order, with 3 and 4
being related and almost equally important. 4 is probably the easiest,
as it only involves changes in xen/common/sched_sedf.c, while 3 is not
hard to do per se, but has interface implications (i.e., exactly what
we're discussing here).
* Just to make sure everyone understand, even if not into SEDF and RT
* scheduling, this patch series, as it stands now, fixes 4, and tries
* to deal with 2.
*
* That's why I'm saying "let's discuss renaming later".
In the long run, what we want is a good, SMP capable, real-time
scheduler. This good SMP capable real-time scheduler won't ever need
some of the parameters that the SEDF interface includes or, if we make
it digest them, semantic and behavior will be different from now anyway.
As far as I can tell, this can be implemented:
A) modifying SEDF, i.e., actually modifying sched_sedf.c, fixing the
problems above
B) introducing something completely new (sched_rt.c ?), which leaves
open the question of what to do with sched_sedf.c and
{LIBXL,XEN}_SCHEDULER_SEDF as a scheduling policy.
If it were me doing it, I probably would have tried to modify SEDF, in a
very similar way to what this series does, and then, probably, even in
the long run, I'd have kept the name and deprecated (or changed the
meaning, and warning the user about it) some of the params. And, in
fact, Josh's work springs from a conversation we had about this a while
back, right Josh? :-)
In fact, I don't like the option of getting rid of SEDF all together and
all of a sudden, even given the issues it has. I think that would not be
in line with what we've done up to now, in terms of API stability/not
breaking things for users (see xm-->xl migr.), and I don't think it's
necessary. Fixing it internally and warning the user that a few
parameters are now being ignored seems a good solution to me, at least
for a few versions.
And even if we decide that SEDF has to go, in favor of SCHED_CBS,
SCHED_RT, SCHED_<whatever>, I personally would _NOT_ do that until such
a new scheduler:
- is a good and SMP capable real-time scheduler
- has an interface we're happy with, and can commit to keep it stable
This series is an RFC, so it's ok for it not to be complete and, since
it tries to fix 2 of the 4 issues I identified myself, I certainly am
not saying it's not a step in (one of the possible) right direction. For
sure, thanks to the discussion we've had as a consequence of this
series, we now know that deprecating SEDF is --either sooner or later--
an option... probably even the best option!
So go ahead with the items you have in your 'FUTURE DEVELOPMENT' section
already, and we'll kill SEDF as soon as it will be worth it. I'll let
you have my comments on patches 1, 2 and 3 ASAP.
Thanks and Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH v2 1/7] Removed all code from sedf not needed for basic EDF functionality
2014-07-09 20:55 ` [RFC PATCH v2 1/7] Removed all code from sedf not needed for basic EDF functionality Josh Whitehead
@ 2014-07-11 7:05 ` Dario Faggioli
2014-07-11 19:54 ` Konrad Rzeszutek Wilk
0 siblings, 1 reply; 30+ messages in thread
From: Dario Faggioli @ 2014-07-11 7:05 UTC (permalink / raw)
To: Josh Whitehead
Cc: Ian Campbell, Stefano Stabellini, George Dunlap, Ian Jackson,
Robert VanVossen, Xen-devel, Nathan Studer
[-- Attachment #1.1: Type: text/plain, Size: 1158 bytes --]
On mer, 2014-07-09 at 16:55 -0400, Josh Whitehead wrote:
> A large amount of code had been added to the sedf scheduler over the years that
> had made it everything but a "simple" EDF scheduler.
>
Indeed! :-)
> --- a/xen/common/sched_sedf.c
> +++ b/xen/common/sched_sedf.c
> -#define EXTRA_NONE (0)
> -#define EXTRA_AWARE (1)
> -#define EXTRA_RUN_PEN (2)
> -#define EXTRA_RUN_UTIL (4)
> -#define EXTRA_WANT_PEN_Q (8)
> -#define EXTRA_PEN_Q (0)
> -#define EXTRA_UTIL_Q (1)
> #define SEDF_ASLEEP (16)
>
As said when reviewing other patches in the series, do not limit to
removing the flags, do also "renumber" the remaining one, instead of
living it #defined to 16.
I can clearly see someone looking at the code rather puzzled about why,
since there is only one flag, it is associated to the 5th bit! :-P
The rest of this patch looks fine to me.
Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH v2 3/7] Added constant bandwidth server functionality to sedf scheduler
2014-07-09 20:55 ` [RFC PATCH v2 3/7] Added constant bandwidth server functionality to sedf scheduler Josh Whitehead
@ 2014-07-11 9:37 ` Dario Faggioli
0 siblings, 0 replies; 30+ messages in thread
From: Dario Faggioli @ 2014-07-11 9:37 UTC (permalink / raw)
To: Josh Whitehead
Cc: Ian Campbell, Stefano Stabellini, George Dunlap, Ian Jackson,
Robert VanVossen, Xen-devel, Nathan Studer
[-- Attachment #1.1: Type: text/plain, Size: 15646 bytes --]
On mer, 2014-07-09 at 16:55 -0400, Josh Whitehead wrote:
> This patch adds the pieces needed for the CBS functionality.
>
So, first of all, this is absolutely my favorite resource-reservation
based real-time scheduling algorithm... so glad you chose it! :-P
> This includes a
> "soft" flag to indicate whether a domain should be handled in a pure EDF manner
> or handled by the CBS algorithm, the CBS algorithm itself to the wake function,
> and few other bits and comments to track scheduler statistics, set parameters,
> and document the functionality of the code itself.
>
Indeed, it's something pretty easy to implement, on top of a decent
EDF-looking scheduler.
> --- a/xen/common/sched_sedf.c
> +++ b/xen/common/sched_sedf.c
> /******************************************************************************
> - * Simple EDF Scheduler for xen
> + * Simple EDF Scheduler for Xen
> *
This line changed already in patch 2.
> * Permission is hereby granted, free of charge, to any person obtaining a copy
> * of this software and associated documentation files (the "Software"), to
> @@ -46,6 +46,7 @@
> #define CHECK(_p) ((void)0)
> #endif
>
> +#define SEDF_SOFT_TASK (1)
> #define SEDF_ASLEEP (16)
>
And this is why I was saying it was ugly and not very practical to leave
SEDF_ASLEEP alone and #define to 16. Now you have two status flag, and
instead of bits 1 and 2, they're represented by bits 1 and 5... Weird,
isn't it?
> #define DEFAULT_PERIOD (MILLISECS(20))
> @@ -73,7 +74,8 @@ struct sedf_vcpu_info {
>
> /* Parameters for EDF */
> s_time_t period; /* = Server scheduling period */
> - s_time_t budget; /* = Guarenteed minimum CPU time per period */
> + s_time_t budget; /* = Guarenteed minimum CPU time per period */
>
And this one too.
Just decide how do you want them to look, and make that happen once and
for ever! :-P :-P
> + /* Note: Server bandwidth = (budget / period) */
>
Again, I'd avoid the word server. 'VCPU bandwidth' sounds just fine.
> @@ -208,6 +215,7 @@ static void *sedf_alloc_vdata(const struct scheduler *ops, struct vcpu *v, void
> inf->vcpu = v;
>
> inf->deadl_abs = 0;
> + inf->cputime = 0;
> inf->status = SEDF_ASLEEP;
>
inf is xzalloc-ed above, so I don't think we need to zero the fields
explicitly. I see this is probably an attempt to achieve more simmetry
and clarity, given the already existing explicit zeroing of deadl_abs.
I don't have a strong opinion. It's certainly inconsistent and
odd-looking to have only one, so I'd either =0 both (as above) or none.
However, whatever path you choose to go, I don't think either this or
the removal of deadl_abs=0 belongs to this patch.
It's probably best to put this hunk in one of the cleanup two cleanup
patches before this.
> @@ -306,29 +314,64 @@ static void desched_edf_vcpu(s_time_t now, struct vcpu *v)
> /* Update the vcpu's cputime */
> inf->cputime += now - inf->sched_start_abs;
>
> - /* Scheduling decisions which don't remove the running vcpu from
> - * the runq */
> + /* If running vcpu has budget remaining, return */
> if ( (inf->cputime < inf->budget) && sedf_runnable(v) )
> return;
>
>
I happen to like the original comment better. Well, actually, I'd change
it a bit myself, but please, stick to something that provides some
insights about "the semantic" of what's happening.
I mean, the fact that you return if the condition is true is already
evident from the code, no much point in saying it in a comment. :-)
OTOH, it may help the reader to tell him what's the consequence of such
return, i.e., "if the vcpu still has budget left, leave it alone on the
RunQ" (or something like that).
> + /* Current vcpu has consumed its budget for this period */
> __del_from_queue(v);
>
> - /*
> - * Manage bookkeeping (i.e. calculate next deadline, memorise
> - * overrun-time of budget) of finished vcpus.
> - */
> +#ifdef SEDF_STATS
> + /* Manage deadline misses */
> + if ( unlikely(inf->deadl_abs < now) )
> + {
> + inf->miss_tot++;
> + inf->miss_time += inf->cputime;
>
Two questions:
1) you're checking whether a deadline miss happened only in case there
has been a budget overrun, right? So, why? Can't a vcpu overcome its
deadline even without overrunning (e.g., in case the system is
overloaded to more than 100%, i UP case, utilization)?
2) By miss_time you mean by how much the deadline has been missed (in
literature, that's called tardiness), right? If yes, what makes you
sure that the miss time is inf->cputime?
> + }
> +#endif
> +
>
These #ifdef are really really ugly. I think using some helpers, and
their empty stubs in case SEDF_STATS is not defined, would make it all
look a lot better.
E.g., have a look at what's done in sched_credit.c
However, this can well happen in another unrelated series.
> + /* Manage overruns */
> if ( inf->cputime >= inf->budget )
> {
> inf->cputime -= inf->budget;
>
> /* Set next deadline */
> inf->deadl_abs += inf->period;
> +
> + /* Ensure that the cputime is always less than budget */
> + if ( unlikely(inf->cputime > inf->budget) )
> + {
> +#ifdef SEDF_STATS
> + inf->over_tot++;
>
Why this is incremented only here? Or, I guess, what's the intended
meaning and usage of over_tot?
I'm asking because it looks like you want to count the number of
overruns. However, it seems to me that putting this here, you are
counting the occurrences of the vcpu having executed more than twice its
budget (you do the ++ only if cputime-budget>budget ==>
cputime>2*budget)... Is that what you want?
> + inf->over_time += inf->cputime;
> +#endif
> +
> + /* Make up for the overage by pushing the deadline
> + into the future; ensure one domain doesn't consistently
> + consume extra time by overrunning its budget */
>
Style of the comment.
> + inf->deadl_abs += ((inf->cputime / inf->budget)
> + * inf->period);
> + inf->cputime -= (inf->cputime / inf->budget) * inf->budget;
> + }
> +
Honestly, the whole "if ( inf->cputime >= inf->budget ) {}" block looks
a bit clumsy to me. How about just something like this:
while ( inf->cputime >= inf->budget )
{
inf->over_tot++; // if you want to count *all* overruns,
// otherwise, skip the ++ for 1st iteration
inf->cputime -= inf->budget;
inf->deadl_abs += inf->period;
}
> + /* Ensure that the start of the next period is in the future */
> + if ( unlikely(PERIOD_BEGIN(inf) < now) )
> + inf->deadl_abs +=
> + (DIV_UP(now - PERIOD_BEGIN(inf),
> + inf->period)) * inf->period;
>
Well, I agree that checking that the new period begins in future is
required. However, if at this point that is not true, we really are
lagging quite a bit!
On what to do, I'm not sure whether I'd just push the deadline ahead, as
you're doing, or go for a full reset (e.g., deadl_abs=NOW()+period).
I'm fine with the pushing ahead, if you like it better, but I'd consider
printing a warning somewhere, as this really should not happen.
> }
>
> /* Add a runnable vcpu to the appropriate queue */
> if ( sedf_runnable(v) )
> {
> - __add_to_waitqueue_sort(v);
> + if( sedf_soft(v) )
> + {
> + __add_to_runqueue_sort(v);
> + }
> + else
> + {
> + __add_to_waitqueue_sort(v);
> + }
> }
>
> ASSERT(EQ(sedf_runnable(v), __task_on_queue(v)));
>
As I said in another mail, let's kill EQ() macro too.
ASSERT(sedf_runnable(v) == __task_on_queue(v));
looks perfectly fine to me.
Of course, do it in the appropriate patch (not here).
> @@ -418,9 +461,9 @@ static void sedf_deinit(const struct scheduler *ops)
> static struct task_slice sedf_do_schedule(
> const struct scheduler *ops, s_time_t now, bool_t tasklet_work_scheduled)
> {
> - int cpu = smp_processor_id();
> - struct list_head *runq = RUNQ(cpu);
> - struct list_head *waitq = WAITQ(cpu);
> + int cpu = smp_processor_id();
> + struct list_head *runq = RUNQ(cpu);
> + struct list_head *waitq = WAITQ(cpu);
> struct sedf_vcpu_info *inf = SEDF_VCPU(current);
> struct sedf_vcpu_info *runinf, *waitinf;
> struct task_slice ret;
>
I may be wrong, but I feel like this hunk is something that would be
better fixed in patch 2 than in this one, isn't it?
> @@ -469,7 +512,7 @@ static struct task_slice sedf_do_schedule(
> waitinf = list_entry(waitq->next,
> struct sedf_vcpu_info, list);
> /*
> - * Rerun scheduler, when scheduled vcpu consumes
> + * Rerun scheduler when scheduled vcpu consumes
>
And this one too.
> * its budget or the first vcpu from the waitqueue
> * gets ready.
> */
> @@ -490,7 +533,7 @@ static struct task_slice sedf_do_schedule(
> }
>
> /*
> - * TODO: Do something USEFUL when this happens and find out, why it
> + * TODO: Do something USEFUL when this happens and find out why it
>
And also this one.
> * still can happen!!!
> */
> if ( ret.time < 0)
> @@ -554,15 +597,21 @@ static inline int should_switch(struct vcpu *cur,
> /*
> * This function wakes up a vcpu, i.e. moves them into the appropriate queue
> *
> - * When a blocked vcpu unblocks, it is allowed to start execution at
> - * the beginning of the next complete period
> - * (D..deadline, R..running, B..blocking/sleeping, U..unblocking/waking up
> + * For Hard Real-Time vcpus (soft = 0):
> + * -When a blocked vcpu unblocks, it is allowed to start execution at
> + * the beginning of the next complete period
> + * (D..deadline, R..running, B..blocking/sleeping, U..unblocking/waking up
> + *
> + * DRRB_____D__U_____DRRRRR___D________ ...
> + *
> + * -This causes the vcpu to miss a period (and a deadlline)
> + * -Doesn't disturb the schedule at all
> + * -Deadlines keep occuring isochronous
>
So, now I'm curious, why did you pick this variant, instead of the
'let's reset everything at wake-up time' one? You think keeping deadline
isochronous is important, to the point that is ok to postpone execution
by one period?
(Note that I'm not saying that is not the case... I'm genuinely
asking :-) )
> - if ( now < inf->deadl_abs )
> - {
> - /* Short blocking */
> - inf->short_block_tot++;
> - }
> - else
> + if ( sedf_soft(v) )
> {
> - /* Long unblocking, someone is going to miss their deadline. */
> - inf->long_block_tot++;
> + /*
> + * Apply CBS rule
> + * Where:
> + * c == Remaining server budget == (inf->budget - cpu_time)
> + * d == Server (vcpu) deadline == inf->deadl_abs
> + * r == Wake-up time of vcpu == now
> + * U == Server (vcpu) bandwidth == (inf->budget / inf->period)
> + *
> + * if c>=(d-r)*U --->
> + * (inf->budget - cputime) >=
> + * (inf->deadl_abs - now) * (inf->period / inf->period)
> + *
> + * If true, push deadline back by one period and refresh budget, else
> + * use current budget and deadline.
> + *
> + * Note: The 'if' statement below is equivalent to the above comments;
> + * it has been simplified to avoid the division operator
> + */
> + if ((inf->budget - inf->cputime) * inf->period >=
> + (inf->deadl_abs - now) * inf->budget)
> + {
> + /* Push back deadline by one period */
> + inf->deadl_abs += inf->period;
> + inf->cputime = 0;
> + }
> +
This looks ok.
> + /*
> + * In CBS we don't care if the period has begun,
> + * the task doesn't have to wait for its period
> + * because it'll never request more than its budget
> + * for any given period.
> + */
> + __add_to_runqueue_sort(v);
>
Well, I'm not sure I understand the comment or, if I understand it, that
I agree with it. It's not that the vcpu won't request more than its
budget. A vcpu may well try to execute for 100% of the time... What
happens in that case is that the budgetting mechanism of the algorithm
kicks in and slow it down, by pushing its deadline away (and hence
lowering its priority).
But perhaps you were trying to say something different?
> }
> + else {
> + /* Task is a hard task, treat accordingly */
> +#ifdef SEDF_STATS
> + if ( now < inf->deadl_abs )
> + {
> + /* Short blocking */
> + inf->short_block_tot++;
> + }
> + else
> + {
> + /* Long unblocking, someone is going to miss their deadline. */
> + inf->long_block_tot++;
>
Again, I'm not sure I understand who 'someone' should be. It looks to me
that it is right the vcpu that is unblocking that risks to miss its
deadline, since it will have to skip a period.
It should not be any of the other vcpus that miss its own deadline,
because of this vcpu unblocking, or we'd be breaking the CPU isolation
feature that this kind of real-time scheduling is supposed to provide...
Is that what you meant?
> + }
> +#endif
>
I think this could have been inside #ifdef SEDF_STATS from the
beginning, which would mean do part of this in patch 2.
> - if ( PERIOD_BEGIN(inf) > now )
> - __add_to_waitqueue_sort(v);
> - else
> - __add_to_runqueue_sort(v);
> + if ( PERIOD_BEGIN(inf) > now )
> + __add_to_waitqueue_sort(v);
> + else
> + __add_to_runqueue_sort(v);
> + }
>
> #ifdef SEDF_STATS
> /* Do some statistics here... */
> @@ -626,7 +716,7 @@ static void sedf_wake(const struct scheduler *ops, struct vcpu *v)
> cpu_raise_softirq(v->processor, SCHEDULE_SOFTIRQ);
> }
>
> -/* Print a lot of useful information about a vcpus in the system */
> +/* Print a lot of useful information about a vcpu in the system */
>
ditto (do this in patch 2).
> @@ -760,8 +868,9 @@ static int sedf_adjust(const struct scheduler *ops, struct domain *d, struct xen
> goto out;
> }
>
> - op->u.sedf.period = SEDF_VCPU(d->vcpu[0])->period;
> - op->u.sedf.slice = SEDF_VCPU(d->vcpu[0])->budget;
> + op->u.sedf.period = SEDF_VCPU(d->vcpu[0])->period;
> + op->u.sedf.budget = SEDF_VCPU(d->vcpu[0])->budget;
> + op->u.sedf.soft = sedf_soft(d->vcpu[0]);
>
And again (i.e., do this in patch 2?).
> }
>
Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH v2 6/7] Changed slice to budget in libxc for the sedf scheduler
2014-07-10 23:11 ` Dario Faggioli
@ 2014-07-11 11:11 ` Ian Campbell
2014-07-11 13:51 ` Dario Faggioli
0 siblings, 1 reply; 30+ messages in thread
From: Ian Campbell @ 2014-07-11 11:11 UTC (permalink / raw)
To: Dario Faggioli
Cc: Stefano Stabellini, George Dunlap, Ian Jackson, Robert VanVossen,
Xen-devel, Nathan Studer, Josh Whitehead
On Fri, 2014-07-11 at 01:11 +0200, Dario Faggioli wrote:
> On mer, 2014-07-09 at 16:55 -0400, Josh Whitehead wrote:
> > From: Robbie VanVossen <robert.vanvossen@dornerworks.com>
> >
> > Signed-off-by: Robert VanVossen <Robert.VanVossen@dornerworks.com>
> > Signed-off-by: Nathan Studer <nate.studer@gmail.com>
> >
> > ---
> > tools/libxc/xc_sedf.c | 8 ++++----
> > tools/libxc/xenctrl.h | 4 ++--
> > tools/python/xen/lowlevel/xc/xc.c | 20 ++++++++++----------
> > xen/include/public/domctl.h | 2 +-
> > 4 files changed, 17 insertions(+), 17 deletions(-)
> >
> > diff --git a/tools/libxc/xc_sedf.c b/tools/libxc/xc_sedf.c
> > index 81ff133..ab67546 100644
> > --- a/tools/libxc/xc_sedf.c
> > +++ b/tools/libxc/xc_sedf.c
> > @@ -28,7 +28,7 @@ int xc_sedf_domain_set(
> > xc_interface *xch,
> > uint32_t domid,
> > uint64_t period,
> > - uint64_t slice,
> > + uint64_t budget,
> >
> Does this really matter?
>
> 'budget' is certainly the most used term in real-time literature and
> research, yes, but slice is not that terrible after all. I think it does
> get the point across about how one should interpret and use the
> parameter itself.
>
> If we were doing a new scheduler, I think I also would pick budget as
> the name for this param. If we are to modify SEDF, I honestly think we
> can live with 'slice'.
Have the semantics changed though? If yes then a new name is preferable.
Ian.
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH v2 5/7] Add cbs parameter support and removed sedf parameters with a LIBXL_API_VERSION gate from libxl.
2014-07-10 23:02 ` Dario Faggioli
@ 2014-07-11 11:12 ` Ian Campbell
0 siblings, 0 replies; 30+ messages in thread
From: Ian Campbell @ 2014-07-11 11:12 UTC (permalink / raw)
To: Dario Faggioli
Cc: Stefano Stabellini, George Dunlap, Ian Jackson, Robert VanVossen,
Xen-devel, Nathan Studer, Josh Whitehead
On Fri, 2014-07-11 at 01:02 +0200, Dario Faggioli wrote:
> On gio, 2014-07-10 at 15:09 +0100, Ian Campbell wrote:
> > On Wed, 2014-07-09 at 16:55 -0400, Josh Whitehead wrote:
> > > From: Robbie VanVossen <robert.vanvossen@dornerworks.com>
> > >
> > > We made an attempt at doing the versioning gate in this file. Please let us
> > > know if further changes are needed, and where they are needed to properly guard
> > > the API.
> >
> > I previously said that I thought it was fine to remove the obsolete SEDF
> > stuff from the libxl API.
> >
> > However I mentioned this to the other maintainers and they disagreed
> > with me. The feeling is that an old application which used the old SEDF
> > parameters should continue to build but that trying to use those
> > parameters should result in an error.
> >
> Indeed. So, as far as this series is concerned, it looks like either the
> following solutions:
> - introduce a new scheduler, with a new name, and deprecate SEDF all
> together
> - keep the name and the full existing interface, but warn the user when
> she touches one of the params that are no longer in use in the
> implementation
> would be ok.
Personally I think the former is preferable from both an implementation
point of view and from the pov of explaining it to users etc.
If the variations to the interface were very minor maybe the second
approach would work, but it looks to me like there is pretty much a
rewrite going on.
Ian.
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH v2 0/7] Repurpose SEDF Scheduler for Real-time Use
2014-07-11 5:01 ` Dario Faggioli
@ 2014-07-11 11:15 ` Ian Campbell
2014-07-11 13:35 ` Dario Faggioli
2014-07-11 20:02 ` Konrad Rzeszutek Wilk
1 sibling, 1 reply; 30+ messages in thread
From: Ian Campbell @ 2014-07-11 11:15 UTC (permalink / raw)
To: Dario Faggioli
Cc: Stefano Stabellini, George Dunlap, Ian Jackson, Robert VanVossen,
Xen-devel, Nathan Studer, Josh Whitehead
On Fri, 2014-07-11 at 07:01 +0200, Dario Faggioli wrote:
> On gio, 2014-07-10 at 15:43 +0100, Ian Campbell wrote:
> > On Wed, 2014-07-09 at 16:55 -0400, Josh Whitehead wrote:
> > > Finally, the amount of name changes has been significantly reduced,
> > > the most prominent being to keep the "sedf" name and not switch to a
> > > new name as was also decided through previous discussion.
> >
> > This is unfortunate since renaming would have meant a new member of the
> > libxl_scheduler enum which would have made the API
> > deprecation/compatibility stuff much simpler -- since you could just
> > return ERROR_INVAL (or ERROR_SEDF_WENT_AWAY ;-)) whenever you saw the
> > SEDF enum value, rather than having to check each field individually.
> >
> *** Short (well, sort of!) Answer ***
> Personally, I would prefer to keep SEDF alive at least for a few
> versions
As it currently is or somehow partially modified?
> (warning the user about some params being ignored, etc.).
>
> However, whatever route we take, concentrating on this series, I don't
> think the renaming+SEDF deprecation should happen until proper SMP
> support is implemented, and probably also not until support for per-VCPU
> scheduling parameters (quite important for an advanced real-time
> scheduling solution) is there. It's fine for an RFC to be incomplete, so
> I'm not complaining or saying this can't be a step in the right
> direction.
>
> However, the biggest problem of SEDF is its poor support for SMP, and
> lack of capability for setting scheduling parameters on a per-VCPU
> basis, which this series (rightfully) does not address.
> Imagine if, after proper review and cleanups, the set of changes from
> this series goes in, and no other follow up series makes it by feature
> freeze, 4.5 will have:
> - SCHED_SEDF deprecated, because it was lacking SMP support and support
> for per-VCPU scheduling parameters
> - SCHED_CBS, _lacking_ SMP support and support for per-VCPU scheduling
> parameters
> Not ideal. :-/
If people aren't happy with that situation then I suppose we wouldn't
accept the series until CBS was more complete.
And this seems to me to be completely orthogonal to whether the approach
is to rename or rewrite.
> [That, of course, does not mean that I won't be reviewing the series
> itself. On the contrary, I'm down to it, especially patches 1, 2 and 3.]
>
> So, to summarize, I'm ok with whatever route we decide to take, included
> full deprecation and renaming, but let's make it worth it!
> If wanting to do that, Josh and Robert, you can well assume that the
> unnecessary parameters and the messy code currently in sched_sedf.c
> related to them can go away (either being ignored by libxl, or because
> this will be a new scheduler), and so it's fine for you to kill them in
> one of your patches. This is something I was not so sure myself, so,
> thanks for bringing the discussion up on it with this series! :-)
If we are going to deprecate sedf then we should leave it alone until
the point where it is removed, not mess around cleaning it up.
> Finally, I also apologize if my comments were not explicitly along this
> line during v1 review... it just took a bit of thinking to wrap my head
> around what the best way forward could be here. :-(
>
> Dario
>
> *** Long (Really Long!!) Answer ***
> The _core_ of the contribution of this series, as of now, is not a new
> scheduling policy. It is a slight (although, to me, very relevant)
> modification to SEDF internals in order to fix a few problems the
> scheduler has.
>
> In fact, I like the fact that the subject itself is 'Repurpose SEDF
> Scheduler'. :-)
>
> The problems SEDF has are:
> 1. it has really really really poor SMP support
> 2. it does not allow to specify scheduling parameters on a per-VCPU
> basis, but only on a domain basis. This is fine for general purpose
> schedulers, but can be quite important in real-time workloads
> 3. it tries to be both a real-time and a general purpose scheduler
> which is:
> a) very hard, and often means adding a lot of complexity for
> nothing, since you're not going to be that good at both anyway...
> and this is one of the cases;
> b) not necessary any longer, now that we have good general purpose
> schedulers, like credit and (hopefully ready soon) credit2;
> 4. it deals with VCPU wake-ups in an hacky, unnecessarily complex and,
> if you ask me, incorrect way.
>
> Note that, in order to achieve 3, not only a lot of complexity (hacks!)
> were introduced inside sched_sedf.c in Xen, it is also the interface
> that contains parameters that are not useful for a real-time scheduler
> which only aims at doing its own job well, i.e.,
> _being_a_good_real-time_scheduler_. :-D
>
> The issues are, in my opinion, sort-of in priority order, with 3 and 4
> being related and almost equally important. 4 is probably the easiest,
> as it only involves changes in xen/common/sched_sedf.c, while 3 is not
> hard to do per se, but has interface implications (i.e., exactly what
> we're discussing here).
>
> * Just to make sure everyone understand, even if not into SEDF and RT
> * scheduling, this patch series, as it stands now, fixes 4, and tries
> * to deal with 2.
> *
> * That's why I'm saying "let's discuss renaming later".
>
> In the long run, what we want is a good, SMP capable, real-time
> scheduler. This good SMP capable real-time scheduler won't ever need
> some of the parameters that the SEDF interface includes or, if we make
> it digest them, semantic and behavior will be different from now anyway.
>
> As far as I can tell, this can be implemented:
> A) modifying SEDF, i.e., actually modifying sched_sedf.c, fixing the
> problems above
> B) introducing something completely new (sched_rt.c ?), which leaves
> open the question of what to do with sched_sedf.c and
> {LIBXL,XEN}_SCHEDULER_SEDF as a scheduling policy.
>
> If it were me doing it, I probably would have tried to modify SEDF, in a
> very similar way to what this series does, and then, probably, even in
> the long run, I'd have kept the name and deprecated (or changed the
> meaning, and warning the user about it) some of the params. And, in
> fact, Josh's work springs from a conversation we had about this a while
> back, right Josh? :-)
>
> In fact, I don't like the option of getting rid of SEDF all together and
> all of a sudden, even given the issues it has. I think that would not be
> in line with what we've done up to now, in terms of API stability/not
> breaking things for users (see xm-->xl migr.), and I don't think it's
> necessary. Fixing it internally and warning the user that a few
> parameters are now being ignored seems a good solution to me, at least
> for a few versions.
>
> And even if we decide that SEDF has to go, in favor of SCHED_CBS,
> SCHED_RT, SCHED_<whatever>, I personally would _NOT_ do that until such
> a new scheduler:
> - is a good and SMP capable real-time scheduler
> - has an interface we're happy with, and can commit to keep it stable
>
> This series is an RFC, so it's ok for it not to be complete and, since
> it tries to fix 2 of the 4 issues I identified myself, I certainly am
> not saying it's not a step in (one of the possible) right direction. For
> sure, thanks to the discussion we've had as a consequence of this
> series, we now know that deprecating SEDF is --either sooner or later--
> an option... probably even the best option!
>
> So go ahead with the items you have in your 'FUTURE DEVELOPMENT' section
> already, and we'll kill SEDF as soon as it will be worth it. I'll let
> you have my comments on patches 1, 2 and 3 ASAP.
>
> Thanks and Regards,
> Dario
>
>
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH v2 0/7] Repurpose SEDF Scheduler for Real-time Use
2014-07-11 11:15 ` Ian Campbell
@ 2014-07-11 13:35 ` Dario Faggioli
2014-07-11 13:50 ` Ian Campbell
0 siblings, 1 reply; 30+ messages in thread
From: Dario Faggioli @ 2014-07-11 13:35 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, George Dunlap, Ian Jackson, Robert VanVossen,
Xen-devel, Nathan Studer, Josh Whitehead
[-- Attachment #1.1: Type: text/plain, Size: 2141 bytes --]
On ven, 2014-07-11 at 12:15 +0100, Ian Campbell wrote:
> On Fri, 2014-07-11 at 07:01 +0200, Dario Faggioli wrote:
> > *** Short (well, sort of!) Answer ***
> > Personally, I would prefer to keep SEDF alive at least for a few
> > versions
>
> As it currently is or somehow partially modified?
>
As it is we can leave it in place for how long we want, I guess, but
it's useless. My point was that, in a partially modified form, it can
serve as a transition to a nnew scheduler or even live "forever".
Anyway, I'm ok with deprecation, it that looks the best option.
> > However, the biggest problem of SEDF is its poor support for SMP, and
> > lack of capability for setting scheduling parameters on a per-VCPU
> > basis, which this series (rightfully) does not address.
> > Imagine if, after proper review and cleanups, the set of changes from
> > this series goes in, and no other follow up series makes it by feature
> > freeze, 4.5 will have:
> > - SCHED_SEDF deprecated, because it was lacking SMP support and support
> > for per-VCPU scheduling parameters
> > - SCHED_CBS, _lacking_ SMP support and support for per-VCPU scheduling
> > parameters
> > Not ideal. :-/
>
> If people aren't happy with that situation then I suppose we wouldn't
> accept the series until CBS was more complete.
>
Exactly my point. Thanks for summarizing. :-D
> And this seems to me to be completely orthogonal to whether the approach
> is to rename or rewrite.
>
Indeed it is, and that's why I provided a full review to the core part
of this patch series already.
That being said, I felt the need of going a bit farther than that, and
provide some more info on what a series like this should include, in
order to make sense in a non RFC status, or at least my take at this.
That is what this mail was all about. :-)
Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH v2 0/7] Repurpose SEDF Scheduler for Real-time Use
2014-07-11 13:35 ` Dario Faggioli
@ 2014-07-11 13:50 ` Ian Campbell
2014-07-11 13:58 ` Dario Faggioli
0 siblings, 1 reply; 30+ messages in thread
From: Ian Campbell @ 2014-07-11 13:50 UTC (permalink / raw)
To: Dario Faggioli
Cc: Stefano Stabellini, George Dunlap, Ian Jackson, Robert VanVossen,
Xen-devel, Nathan Studer, Josh Whitehead
On Fri, 2014-07-11 at 15:35 +0200, Dario Faggioli wrote:
> On ven, 2014-07-11 at 12:15 +0100, Ian Campbell wrote:
> > On Fri, 2014-07-11 at 07:01 +0200, Dario Faggioli wrote:
> > > *** Short (well, sort of!) Answer ***
> > > Personally, I would prefer to keep SEDF alive at least for a few
> > > versions
> >
> > As it currently is or somehow partially modified?
> >
> As it is we can leave it in place for how long we want, I guess, but
> it's useless. My point was that, in a partially modified form, it can
> serve as a transition to a nnew scheduler or even live "forever".
I think try to use it to serve as some sort of transitional staging
ground is both confusing and pointless. At least it is proving to be
confusing at the toolstack level.
If it is so broken as to be useless then lets just remove it. If it is
broken but still potentially useful under limited circumstances then
lets just leave it completely alone until the replacement is ready.
Ian.
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH v2 6/7] Changed slice to budget in libxc for the sedf scheduler
2014-07-11 11:11 ` Ian Campbell
@ 2014-07-11 13:51 ` Dario Faggioli
0 siblings, 0 replies; 30+ messages in thread
From: Dario Faggioli @ 2014-07-11 13:51 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, George Dunlap, Ian Jackson, Robert VanVossen,
Xen-devel, Nathan Studer, Josh Whitehead
[-- Attachment #1.1: Type: text/plain, Size: 1203 bytes --]
On ven, 2014-07-11 at 12:11 +0100, Ian Campbell wrote:
> On Fri, 2014-07-11 at 01:11 +0200, Dario Faggioli wrote:
> > 'budget' is certainly the most used term in real-time literature and
> > research, yes, but slice is not that terrible after all. I think it does
> > get the point across about how one should interpret and use the
> > parameter itself.
> >
> > If we were doing a new scheduler, I think I also would pick budget as
> > the name for this param. If we are to modify SEDF, I honestly think we
> > can live with 'slice'.
>
> Have the semantics changed though? If yes then a new name is preferable.
>
They're basically synonyms, and the new algorithm uses this budget
exactly as current SEDF use slice, no semantic change.
And in fact this patch basically is s/slice/budget/g all over the file.
Budget is more widely used and recognized in the real-time literature,
but that's it.
Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH v2 0/7] Repurpose SEDF Scheduler for Real-time Use
2014-07-11 13:50 ` Ian Campbell
@ 2014-07-11 13:58 ` Dario Faggioli
0 siblings, 0 replies; 30+ messages in thread
From: Dario Faggioli @ 2014-07-11 13:58 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, George Dunlap, Ian Jackson, Robert VanVossen,
Xen-devel, Nathan Studer, Josh Whitehead
[-- Attachment #1.1: Type: text/plain, Size: 1444 bytes --]
On ven, 2014-07-11 at 14:50 +0100, Ian Campbell wrote:
> On Fri, 2014-07-11 at 15:35 +0200, Dario Faggioli wrote:
> > On ven, 2014-07-11 at 12:15 +0100, Ian Campbell wrote:
> > > On Fri, 2014-07-11 at 07:01 +0200, Dario Faggioli wrote:
> > > > *** Short (well, sort of!) Answer ***
> > > > Personally, I would prefer to keep SEDF alive at least for a few
> > > > versions
> > >
> > > As it currently is or somehow partially modified?
> > >
> > As it is we can leave it in place for how long we want, I guess, but
> > it's useless. My point was that, in a partially modified form, it can
> > serve as a transition to a nnew scheduler or even live "forever".
>
> I think try to use it to serve as some sort of transitional staging
> ground is both confusing and pointless. At least it is proving to be
> confusing at the toolstack level.
>
Fair enough.
> If it is so broken as to be useless then lets just remove it. If it is
> broken but still potentially useful under limited circumstances then
> lets just leave it completely alone until the replacement is ready.
>
I'm ok with the latter (leave alone until replacement ready).
Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH v2 1/7] Removed all code from sedf not needed for basic EDF functionality
2014-07-11 7:05 ` Dario Faggioli
@ 2014-07-11 19:54 ` Konrad Rzeszutek Wilk
2014-07-14 16:06 ` Dario Faggioli
0 siblings, 1 reply; 30+ messages in thread
From: Konrad Rzeszutek Wilk @ 2014-07-11 19:54 UTC (permalink / raw)
To: Dario Faggioli
Cc: Ian Campbell, Stefano Stabellini, George Dunlap, Ian Jackson,
Robert VanVossen, Xen-devel, Nathan Studer, Josh Whitehead
On Fri, Jul 11, 2014 at 09:05:27AM +0200, Dario Faggioli wrote:
> On mer, 2014-07-09 at 16:55 -0400, Josh Whitehead wrote:
> > A large amount of code had been added to the sedf scheduler over the years that
> > had made it everything but a "simple" EDF scheduler.
> >
> Indeed! :-)
>
> > --- a/xen/common/sched_sedf.c
> > +++ b/xen/common/sched_sedf.c
>
> > -#define EXTRA_NONE (0)
> > -#define EXTRA_AWARE (1)
> > -#define EXTRA_RUN_PEN (2)
> > -#define EXTRA_RUN_UTIL (4)
> > -#define EXTRA_WANT_PEN_Q (8)
> > -#define EXTRA_PEN_Q (0)
> > -#define EXTRA_UTIL_Q (1)
> > #define SEDF_ASLEEP (16)
> >
> As said when reviewing other patches in the series, do not limit to
> removing the flags, do also "renumber" the remaining one, instead of
> living it #defined to 16.
>
> I can clearly see someone looking at the code rather puzzled about why,
> since there is only one flag, it is associated to the 5th bit! :-P
>
> The rest of this patch looks fine to me.
Is that an 'Reviewed-by' or 'Acked-by' ?
Thank you.
>
> Regards,
> Dario
>
> --
> <<This happens because I choose it to happen!>> (Raistlin Majere)
> -----------------------------------------------------------------
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli
> Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH v2 0/7] Repurpose SEDF Scheduler for Real-time Use
2014-07-11 5:01 ` Dario Faggioli
2014-07-11 11:15 ` Ian Campbell
@ 2014-07-11 20:02 ` Konrad Rzeszutek Wilk
1 sibling, 0 replies; 30+ messages in thread
From: Konrad Rzeszutek Wilk @ 2014-07-11 20:02 UTC (permalink / raw)
To: Dario Faggioli
Cc: Ian Campbell, Stefano Stabellini, George Dunlap, Ian Jackson,
Robert VanVossen, Xen-devel, Nathan Studer, Josh Whitehead
> However, the biggest problem of SEDF is its poor support for SMP, and
> lack of capability for setting scheduling parameters on a per-VCPU
> basis, which this series (rightfully) does not address.
> Imagine if, after proper review and cleanups, the set of changes from
> this series goes in, and no other follow up series makes it by feature
> freeze, 4.5 will have:
> - SCHED_SEDF deprecated, because it was lacking SMP support and support
> for per-VCPU scheduling parameters
> - SCHED_CBS, _lacking_ SMP support and support for per-VCPU scheduling
> parameters
> Not ideal. :-/
(With release manager hat on)
Actually I would think it is OK. In terms of X feature count at this
point of time (T) and on 4.5 feature freeze day (T-RC0) if the number of
features is greater or equal we are OK.
What is bad if the amount of features on T-RC0 is less than now (so say
X-1 features). That is not the case.
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH v2 1/7] Removed all code from sedf not needed for basic EDF functionality
2014-07-11 19:54 ` Konrad Rzeszutek Wilk
@ 2014-07-14 16:06 ` Dario Faggioli
0 siblings, 0 replies; 30+ messages in thread
From: Dario Faggioli @ 2014-07-14 16:06 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk
Cc: Ian Campbell, Stefano Stabellini, George Dunlap, Ian Jackson,
Robert VanVossen, Xen-devel, Nathan Studer, Josh Whitehead
[-- Attachment #1.1: Type: text/plain, Size: 1639 bytes --]
On ven, 2014-07-11 at 15:54 -0400, Konrad Rzeszutek Wilk wrote:
> On Fri, Jul 11, 2014 at 09:05:27AM +0200, Dario Faggioli wrote:
> > On mer, 2014-07-09 at 16:55 -0400, Josh Whitehead wrote:
> > > A large amount of code had been added to the sedf scheduler over the years that
> > > had made it everything but a "simple" EDF scheduler.
> > >
> > Indeed! :-)
> >
> > > --- a/xen/common/sched_sedf.c
> > > +++ b/xen/common/sched_sedf.c
> >
> > > -#define EXTRA_NONE (0)
> > > -#define EXTRA_AWARE (1)
> > > -#define EXTRA_RUN_PEN (2)
> > > -#define EXTRA_RUN_UTIL (4)
> > > -#define EXTRA_WANT_PEN_Q (8)
> > > -#define EXTRA_PEN_Q (0)
> > > -#define EXTRA_UTIL_Q (1)
> > > #define SEDF_ASLEEP (16)
> > >
> > As said when reviewing other patches in the series, do not limit to
> > removing the flags, do also "renumber" the remaining one, instead of
> > living it #defined to 16.
> >
> > I can clearly see someone looking at the code rather puzzled about why,
> > since there is only one flag, it is associated to the 5th bit! :-P
> >
> > The rest of this patch looks fine to me.
>
> Is that an 'Reviewed-by' or 'Acked-by' ?
>
It's not, actually. I'll say it explicitly when that will be the case.
BTW, given all the discussion about renaming/new file, etc., even if it
were, it'd be pretty pointless at this stage, I think.
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 30+ messages in thread
end of thread, other threads:[~2014-07-14 16:06 UTC | newest]
Thread overview: 30+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-07-09 20:55 [RFC PATCH v2 0/7] Repurpose SEDF Scheduler for Real-time Use Josh Whitehead
2014-07-09 20:55 ` [RFC PATCH v2 1/7] Removed all code from sedf not needed for basic EDF functionality Josh Whitehead
2014-07-11 7:05 ` Dario Faggioli
2014-07-11 19:54 ` Konrad Rzeszutek Wilk
2014-07-14 16:06 ` Dario Faggioli
2014-07-09 20:55 ` [RFC PATCH v2 2/7] Fixed formatting and misleading comments/variables. Added comments and renamed variables to accurately reflect modern terminology Josh Whitehead
2014-07-11 3:59 ` Dario Faggioli
2014-07-09 20:55 ` [RFC PATCH v2 3/7] Added constant bandwidth server functionality to sedf scheduler Josh Whitehead
2014-07-11 9:37 ` Dario Faggioli
2014-07-09 20:55 ` [RFC PATCH v2 4/7] Add cbs parameter support and removed sedf parameters from libxc Josh Whitehead
2014-07-10 23:17 ` Dario Faggioli
2014-07-09 20:55 ` [RFC PATCH v2 5/7] Add cbs parameter support and removed sedf parameters with a LIBXL_API_VERSION gate from libxl Josh Whitehead
2014-07-10 14:09 ` Ian Campbell
2014-07-10 14:55 ` Ian Jackson
2014-07-10 23:02 ` Dario Faggioli
2014-07-11 11:12 ` Ian Campbell
2014-07-10 14:26 ` Dario Faggioli
2014-07-09 20:55 ` [RFC PATCH v2 6/7] Changed slice to budget in libxc for the sedf scheduler Josh Whitehead
2014-07-10 14:15 ` Ian Campbell
2014-07-10 23:11 ` Dario Faggioli
2014-07-11 11:11 ` Ian Campbell
2014-07-11 13:51 ` Dario Faggioli
2014-07-09 20:55 ` [RFC PATCH v2 7/7] Changed slice to budget in libxl " Josh Whitehead
2014-07-10 14:43 ` [RFC PATCH v2 0/7] Repurpose SEDF Scheduler for Real-time Use Ian Campbell
2014-07-11 5:01 ` Dario Faggioli
2014-07-11 11:15 ` Ian Campbell
2014-07-11 13:35 ` Dario Faggioli
2014-07-11 13:50 ` Ian Campbell
2014-07-11 13:58 ` Dario Faggioli
2014-07-11 20:02 ` Konrad Rzeszutek Wilk
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.