From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38243C433EF for ; Sun, 27 Feb 2022 08:08:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230126AbiB0II5 (ORCPT ); Sun, 27 Feb 2022 03:08:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58036 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229769AbiB0IIy (ORCPT ); Sun, 27 Feb 2022 03:08:54 -0500 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DCAE01FCF9 for ; Sun, 27 Feb 2022 00:08:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1645949297; x=1677485297; h=subject:to:cc:references:from:message-id:date: mime-version:in-reply-to:content-transfer-encoding; bh=iYYkwecdmiiQWNKWked1nOEeOcTZu4U9RJ0ovrzCVKU=; b=K6+c0MbOGiHnqaheEn+nUMX8oovXfJfAUthzmJOZCE6AxC/uW1o/Ff5j ys7jJurWS8TC/ehNKq8wiKJrkD4vMjyRQg0eTgHZbss5kpgWMWRar4aQd qr9qnYAEJZUOF4yCpT4PdQjPcIL8IbyMeEOAgLMIVDRp0H586SCDLQ6xB TMteM0t6JWJ67RzsWq3NLzJ2dRLd7kIfvLrReptfmQ8picZAtDBXik4i/ fwOTr0F1RLCjRR8YAXXoh53SAmuxALrY5eq+35LRHf8FQQ6SAWvsdiDAV qW64JyozBZvu1n3rUstItrH3mCAVnJyuRs62LKXbVuXFk5UPeO+kcKFLJ A==; X-IronPort-AV: E=McAfee;i="6200,9189,10270"; a="277358582" X-IronPort-AV: E=Sophos;i="5.90,140,1643702400"; d="scan'208";a="277358582" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Feb 2022 00:08:17 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,140,1643702400"; d="scan'208";a="534071864" Received: from aubrey-app.sh.intel.com (HELO [10.239.53.25]) ([10.239.53.25]) by orsmga007.jf.intel.com with ESMTP; 27 Feb 2022 00:08:13 -0800 Subject: Re: [RFC PATCH 1/5] sched/fair: record overloaded cpus To: "Gautham R. Shenoy" , Abel Wu Cc: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Daniel Bristot de Oliveira , linux-kernel@vger.kernel.org, srikar@linux.vnet.ibm.com, aubrey.li@intel.com References: <20220217154403.6497-1-wuyun.abel@bytedance.com> <20220217154403.6497-2-wuyun.abel@bytedance.com> From: Aubrey Li Message-ID: Date: Sun, 27 Feb 2022 16:08:12 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2/24/22 3:10 PM, Gautham R. Shenoy wrote: > Hello Abel, > > (+ Aubrey Li, Srikar) > > On Thu, Feb 17, 2022 at 11:43:57PM +0800, Abel Wu wrote: >> An CFS runqueue is considered overloaded when there are >> more than one pullable non-idle tasks on it (since sched- >> idle cpus are treated as idle cpus). And idle tasks are >> counted towards rq->cfs.idle_h_nr_running, that is either >> assigned SCHED_IDLE policy or placed under idle cgroups. >> >> The overloaded cfs rqs can cause performance issues to >> both task types: >> >> - for latency critical tasks like SCHED_NORMAL, >> time of waiting in the rq will increase and >> result in higher pct99 latency, and >> >> - batch tasks may not be able to make full use >> of cpu capacity if sched-idle rq exists, thus >> presents poorer throughput. >> >> The mask of overloaded cpus is updated in periodic tick >> and the idle path at the LLC domain basis. This cpumask >> will also be used in SIS as a filter, improving idle cpu >> searching. > > This is an interesting approach to minimise the tail latencies by > keeping track of the overloaded cpus in the LLC so that > idle/sched-idle CPUs can pull from them. This approach contrasts with the > following approaches that were previously tried : > > 1. Maintain the idle cpumask at the LLC level by Aubrey Li > https://lore.kernel.org/all/1615872606-56087-1-git-send-email-aubrey.li@intel.com/ > > 2. Maintain the identity of the idle core itself at the LLC level, by Srikar : > https://lore.kernel.org/lkml/20210513074027.543926-3-srikar@linux.vnet.ibm.com/ > > There have been concerns in the past about having to update the shared > mask/counter at regular intervals. Srikar, Aubrey any thoughts on this > ? > https://lkml.org/lkml/2022/2/7/1129