Re: [RFC][PATCH 1/7] sched: Replace post_schedule with a balance callback list

[Kamalesh Babulal <kamalesh@linux.vnet.ibm.com>]

* Peter Zijlstra <peterz@infradead.org> [2015-06-01 15:58:19]:

[...]

> --- a/kernel/sched/deadline.c
> +++ b/kernel/sched/deadline.c
> @@ -213,9 +213,16 @@ static inline bool need_pull_dl_task(str
>  	return dl_task(prev);
>  }
>  
> -static inline void set_post_schedule(struct rq *rq)
> +static DEFINE_PER_CPU(struct callback_head, dl_balance_head);
> +
> +static void push_dl_tasks(struct rq *);
> +
> +static inline void queue_push_tasks(struct rq *rq)
>  {
> -	rq->post_schedule = has_pushable_dl_tasks(rq);
> +	if (!has_pushable_dl_tasks(rq))
> +		return;
> +
> +	queue_balance_callback(rq, &per_cpu(dl_balance_head, rq->cpu), push_dl_tasks);
>  }

When compiling with CONFIG_SMP=n, following build warning is triggered:
  CC      kernel/sched/deadline.o
kernel/sched/deadline.c: In function ‘pick_next_task_dl’:
kernel/sched/deadline.c:1136:2: error: implicit declaration of function ‘queue_push_tasks’ [-Werror=implicit-function-declaration]
  queue_push_tasks(rq);
  ^
cc1: some warnings being treated as errors

set_post_schedule() exist for CONFIG_SMP=n case, which was not modified to
queue_push_tasks().

[...]
> --- a/kernel/sched/sched.h
> +++ b/kernel/sched/sched.h
> @@ -624,9 +624,10 @@ struct rq {
>  	unsigned long cpu_capacity;
>  	unsigned long cpu_capacity_orig;
>  
> +	struct callback_head *balance_callback;
> +
>  	unsigned char idle_balance;
>  	/* For active balancing */
> -	int post_schedule;
>  	int active_balance;
>  	int push_cpu;
>  	struct cpu_stop_work active_balance_work;
> @@ -695,6 +696,21 @@ struct rq {
>  #endif
>  };
>  
> +static inline void
> +queue_balance_callback(struct rq *rq,
> +		       struct callback_head *head,
> +		       void (*func)(struct rq *rq))
> +{
> +	lockdep_assert_held(&rq->lock);
> +
> +	if (unlikely(head->next))
> +		return;
> +
> +	head->func = (void (*)(struct callback_head *))func;
> +	head->next = rq->balance_callback;
> +	rq->balance_callback = head;
> +}
> +

While compiling with CONFIG_SMP=n, another build error is seen:

In file included from kernel/sched/core.c:86:0:
kernel/sched/sched.h: In function ‘queue_balance_callback’:
kernel/sched/sched.h:710:17: error: ‘struct rq’ has no member named ‘balance_callback’
  head->next = rq->balance_callback;
                 ^
kernel/sched/sched.h:711:4: error: ‘struct rq’ has no member named ‘balance_callback’
  rq->balance_callback = head;
    ^

Guarding queue_balance_callback() with #ifdef CONFIG_SMP fixes
the issue, as all of the call sites are also with #ifdef  CONFIG_SMP

Thanks,
Kamalesh