From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51394C433DB for ; Tue, 16 Mar 2021 17:32:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1B42D64F17 for ; Tue, 16 Mar 2021 17:32:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234193AbhCPRcA (ORCPT ); Tue, 16 Mar 2021 13:32:00 -0400 Received: from foss.arm.com ([217.140.110.172]:52540 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231637AbhCPRbm (ORCPT ); Tue, 16 Mar 2021 13:31:42 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3918831B; Tue, 16 Mar 2021 10:31:39 -0700 (PDT) Received: from e113632-lin (e113632-lin.cambridge.arm.com [10.1.194.46]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id BD80B3F70D; Tue, 16 Mar 2021 10:31:37 -0700 (PDT) From: Valentin Schneider To: Dietmar Eggemann , linux-kernel@vger.kernel.org Cc: Lingutla Chandrasekhar , Peter Zijlstra , Ingo Molnar , Vincent Guittot , Morten Rasmussen , Qais Yousef , Quentin Perret , Pavan Kondeti , Rik van Riel Subject: Re: [PATCH v3 1/7] sched/fair: Ignore percpu threads for imbalance pulls In-Reply-To: References: <20210311120527.167870-1-valentin.schneider@arm.com> <20210311120527.167870-2-valentin.schneider@arm.com> Date: Tue, 16 Mar 2021 17:31:32 +0000 Message-ID: <87pmzzronf.mognet@arm.com> MIME-Version: 1.0 Content-Type: text/plain Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 16/03/21 16:49, Dietmar Eggemann wrote: > On 11/03/2021 13:05, Valentin Schneider wrote: >> From: Lingutla Chandrasekhar >> >> In load balancing, when balancing group is unable to pull task >> due to ->cpus_ptr constraints from busy group, then it sets >> LBF_SOME_PINNED to lb env flags, as a consequence, sgc->imbalance >> is set for its parent domain level. which makes the group >> classified as imbalance to get help from another balancing cpu. >> >> Consider a 4-CPU big.LITTLE system with CPUs 0-1 as LITTLEs and > > Does it have to be a big.LITTLE system? I assume this issue also happens > on an SMP system. > Aye, though the consequences are "worse" on asym CPU capacity systems. >> CPUs 2-3 as Bigs with below scenario: >> - CPU0 doing newly_idle balancing >> - CPU1 running percpu kworker and RT task (small tasks) > > What's the role of the small RT task here in the story? > I don't think it matters much here. >> - CPU2 running 2 big tasks >> - CPU3 running 1 medium task >> >> While CPU0 is doing newly_idle load balance at MC level, it fails to >> pull percpu kworker from CPU1 and sets LBF_SOME_PINNED to lb env flag >> and set sgc->imbalance at DIE level domain. As LBF_ALL_PINNED not cleared, >> it tries to redo the balancing by clearing CPU1 in env cpus, but it don't >> find other busiest_group, so CPU0 stops balacing at MC level without >> clearing 'sgc->imbalance' and restart the load balacing at DIE level. >> >> And CPU0 (balancing cpu) finds LITTLE's group as busiest_group with group >> type as imbalance, and Bigs that classified the level below imbalance type >> would be ignored to pick as busiest, and the balancing would be aborted >> without pulling any tasks (by the time, CPU1 might not have running tasks). >> >> It is suboptimal decision to classify the group as imbalance due to >> percpu threads. So don't use LBF_SOME_PINNED for per cpu threads. > > This sentence mentioned per-cpu threads (and so does the patch name) but > the implementation (only) deals with per-cpu kernel threads. IMHO, it > would be good to align this. > Tell you what, I'll go for: 1) how can pcpu kthreads cause LBF_SOME_PINNED 2) why we may not want this, but still ignore !kthread pcpu tasks 3) why this is even more important for big.LITTLE