Re: [PATCH 5/5] sched/fair: Skip LLC nohz logic for asymmetric systems

[Valentin Schneider <valentin.schneider@arm.com>]

On 07/02/2019 09:56, Peter Zijlstra wrote:
[...]
>> I'm afraid I don't follow - we don't lose a balance opportunity with the
>> below change (compared to the original patch), do we?
> 
> What if each big/little cluster would have multiple cache domains? Would
> we not want to spread the cache usage inside the big/little resp. ?
> 

Ah I see - somewhat, yes. And we kindof already hit that issue - on
regular big.LITTLE each "cluster" (big & LITTLE) will have their own L2,
but there's no L3 to share. That means

- sd_llc == MC
- sd_asym_cpucapacity == DIE

(which I believe is what you meant by sd_llc_shared < sd_asym_capacity)

However the LLC nohz kick condition is way too broad and will keep kicking
if we just have 2 busy LITTLEs with 1 task each, which could be a correct
placement if they were just running some small background task.

I'd argue that since we don't have SMT, <=1 task per CPU means we can't do
anything better to spread cache use within the "cluster", and we have to
rely on the other conditions (mostly rq->nr_running and misfit) to move
things around when we really need to.

The extra kicks are there on non asymmetric topologies as well, so we could
imagine an additional gate for the LLC kick that would look like:

  /* If SMT, <=1 task per CPU can be suboptimal
   * If we have a parent (and thus a sibling), we may need to move some tasks to
   * our sibling.
   * If we have asymmetry, asymmetric placement can be fine so just rely on the
   * other kick conditions
   */
  if (static_key_likely(&sched_smt_present) ||
      (!static_key_unlikely(&sched_asym_cpucapacity) && sd->parent)) {
	  // LLC kick stuff here
  }

but I'd argue the extra nohz kicks have more impact on asymmetric systems.