Re: [2/2] sched/fair: Fix O(# total cgroups) in load balance path

From: Peter Zijlstra <peterz@infradead.org>
To: Tejun Heo <tj@kernel.org>
Cc: "Ingo Molnar" <mingo@redhat.com>,
	"“linux-kernel@vger.kernel.org”" <linux-kernel@vger.kernel.org>,
	"Linus Torvalds" <torvalds@linux-foundation.org>,
	"Mike Galbraith" <efault@gmx.de>, "Paul Turner" <pjt@google.com>,
	"Chris Mason" <clm@fb.com>,
	"“kernel-team@fb.com”" <kernel-team@fb.com>
Subject: Re: [2/2] sched/fair: Fix O(# total cgroups) in load balance path
Date: Mon, 1 May 2017 18:11:58 +0200	[thread overview]
Message-ID: <20170501161158.tgw3rko72aziygjx@hirez.programming.kicks-ass.net> (raw)
In-Reply-To: <20170426004350.GB3222@wtj.duckdns.org>

On Tue, Apr 25, 2017 at 05:43:50PM -0700, Tejun Heo wrote:
> @@ -7007,6 +7008,14 @@ static void update_blocked_averages(int
>  		se = cfs_rq->tg->se[cpu];
>  		if (se && !skip_blocked_update(se))
>  			update_load_avg(se, 0);
> +
> +		/*
> +		 * There can be a lot of idle CPU cgroups.  Don't let fully
> +		 * decayed cfs_rqs linger on the list.
> +		 */
> +		if (!cfs_rq->load.weight && !cfs_rq->avg.load_sum &&
> +		    !cfs_rq->avg.util_sum && !cfs_rq->runnable_load_sum)
> +			list_del_leaf_cfs_rq(cfs_rq);
>  	}
>  	rq_unlock_irqrestore(rq, &rf);
>  }

Right this is a 'known' issue and we recently talked about this.

I think you got the condition right, we want to wait for all the stuff
to be decayed out before taking it off the list.

The only 'problem', which Vincent mentioned in that other thread, is that
NOHZ idle doesn't guarantee decay -- then again, you don't want to go
wake a CPU just to decay this crud either. And if we're idle, the list
being long doesn't matter either.