RE: hotplug support for arch/arc/plat-eznps platform

["Ofer Levi(SW)" <oferle@mellanox.com>]

Well, this definitely have pleased the little toy :)
Thank you. I really appreciate your time and effort.

If I may, one more newbie question. What do I need to do for the two patches to find 
their way into formal kernel code?

Thanks
-Ofer



> -----Original Message-----
> From: Peter Zijlstra [mailto:peterz@infradead.org]
> Sent: Wednesday, August 9, 2017 6:35 PM
> To: Ofer Levi(SW) <oferle@mellanox.com>
> Cc: rusty@rustcorp.com.au; mingo@redhat.com;
> Vineet.Gupta1@synopsys.com; linux-kernel@vger.kernel.org; Tejun Heo
> <tj@kernel.org>
> Subject: Re: hotplug support for arch/arc/plat-eznps platform
> 
> On Wed, Aug 09, 2017 at 03:19:02PM +0000, Ofer Levi(SW) wrote:
> > I appreciate your effort and detailed reply, however I'm still
> > experiencing  performance hit at partition_sched_domains(). It seems the
> issue is due to the large magnitude of cpus.
> > I used he suggested method 2, patched in the diffs and used the
> > command line switch isolcpus to kill load-balancing.
> > It did save few hundredth of a sec per cpu. When I limited number of
> > available cpus (using present and possible cpus ) to 48, it did reduced
> dramatically this function execution time:
> >
> > With 4K available cpus :
> > [   48.890000] ## CPU16 LIVE ##: Executing Code...
> > [   48.910000] partition_sched_domains start
> > [   49.360000] partition_sched_domains end
> >
> > With 48 available cpus:
> > [   36.950000] ## CPU16 LIVE ##: Executing Code...
> > [   36.950000] partition_sched_domains start
> > [   36.960000] partition_sched_domains end
> >
> > Note that I currently use kernel version: 4.8.0.17.0600.00.0000, if this has
> any influence.
> > Would appreciate your thoughts.
> >
> 
> Does something like this cure things? It seems we're doing a possible_cpus
> iteration for sysctl cruft, and that will most certainly hurt on your little toy :-)
> 
> Not sure what the more generic solution to that would be, but the below
> avoids it for isolcpus.
> 
> ---
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -85,6 +85,7 @@ int sysctl_sched_rt_runtime = 950000;
> 
>  /* CPUs with isolated domains */
>  cpumask_var_t cpu_isolated_map;
> +cpumask_var_t non_isolated_cpus;
> 
>  /*
>   * __task_rq_lock - lock the rq @p resides on.
> @@ -5685,8 +5686,6 @@ static inline void sched_init_smt(void)
> 
>  void __init sched_init_smp(void)
>  {
> -	cpumask_var_t non_isolated_cpus;
> -
>  	alloc_cpumask_var(&non_isolated_cpus, GFP_KERNEL);
> 
>  	sched_init_numa();
> @@ -5697,17 +5696,17 @@ void __init sched_init_smp(void)
>  	 * happen.
>  	 */
>  	mutex_lock(&sched_domains_mutex);
> -	sched_init_domains(cpu_active_mask);
>  	cpumask_andnot(non_isolated_cpus, cpu_possible_mask,
> cpu_isolated_map);
>  	if (cpumask_empty(non_isolated_cpus))
>  		cpumask_set_cpu(smp_processor_id(), non_isolated_cpus);
> +
> +	sched_init_domains(cpu_active_mask);
>  	mutex_unlock(&sched_domains_mutex);
> 
>  	/* Move init over to a non-isolated CPU */
>  	if (set_cpus_allowed_ptr(current, non_isolated_cpus) < 0)
>  		BUG();
>  	sched_init_granularity();
> -	free_cpumask_var(non_isolated_cpus);
> 
>  	init_sched_rt_class();
>  	init_sched_dl_class();
> --- a/kernel/sched/debug.c
> +++ b/kernel/sched/debug.c
> @@ -327,6 +327,8 @@ static struct ctl_table *sd_alloc_ctl_cp
>  	return table;
>  }
> 
> +extern cpumask_var_t non_isolated_cpus;
> +
>  static struct ctl_table_header *sd_sysctl_header;  void
> register_sched_domain_sysctl(void)
>  {
> @@ -340,7 +342,7 @@ void register_sched_domain_sysctl(void)
>  	if (entry == NULL)
>  		return;
> 
> -	for_each_possible_cpu(i) {
> +	for_each_cpu(i, non_isolated_cpus) {
>  		snprintf(buf, 32, "cpu%d", i);
>  		entry->procname = kstrdup(buf, GFP_KERNEL);
>  		entry->mode = 0555;