From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 454DBC38142 for ; Fri, 27 Jan 2023 16:32:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234951AbjA0Qc5 (ORCPT ); Fri, 27 Jan 2023 11:32:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60084 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234340AbjA0Qcx (ORCPT ); Fri, 27 Jan 2023 11:32:53 -0500 Received: from smtp-fw-80006.amazon.com (smtp-fw-80006.amazon.com [99.78.197.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8E276A5E1 for ; Fri, 27 Jan 2023 08:32:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.de; i=@amazon.de; q=dns/txt; s=amazon201209; t=1674837172; x=1706373172; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=6N0PUtuDcZ/wsatcTeReyNRAyc+sA3MLDZQY1xgdJ4I=; b=Td5Hz11uHHspWh9vLr5JHC1yHCjmhCg/XTaXVW2dJXlr2V7GRbJCGd88 EUrkIuyX4o7gxFmTAp4OspTVPcBSi7WBDCtUGsnLjt++aiFMGHctITP5J Sg+sYthO3h/DGQCGunivg5Cy+VGZivNt2FuBQjQXczHg3LnSNrfkfvphX g=; X-IronPort-AV: E=Sophos;i="5.97,251,1669075200"; d="scan'208";a="175779145" Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO email-inbound-relay-iad-1a-m6i4x-366646a6.us-east-1.amazon.com) ([10.25.36.210]) by smtp-border-fw-80006.pdx80.corp.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jan 2023 16:32:40 +0000 Received: from EX13MTAUEE002.ant.amazon.com (iad12-ws-svc-p26-lb9-vlan2.iad.amazon.com [10.40.163.34]) by email-inbound-relay-iad-1a-m6i4x-366646a6.us-east-1.amazon.com (Postfix) with ESMTPS id 9B299A35C0; Fri, 27 Jan 2023 16:32:35 +0000 (UTC) Received: from EX19D018UEC004.ant.amazon.com (10.252.135.223) by EX13MTAUEE002.ant.amazon.com (10.43.62.24) with Microsoft SMTP Server (TLS) id 15.0.1497.45; Fri, 27 Jan 2023 16:32:33 +0000 Received: from EX13MTAUEE002.ant.amazon.com (10.43.62.24) by EX19D018UEC004.ant.amazon.com (10.252.135.223) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.7; Fri, 27 Jan 2023 16:32:33 +0000 Received: from u40bc5e070a0153.ant.amazon.com (10.1.213.9) by mail-relay.amazon.com (10.43.62.224) with Microsoft SMTP Server (TLS) id 15.0.1497.45 via Frontend Transport; Fri, 27 Jan 2023 16:32:31 +0000 From: Roman Kagan To: CC: Daniel Bristot de Oliveira , Ben Segall , Zhang Qiao , Ingo Molnar , Waiman Long , Dietmar Eggemann , Peter Zijlstra , "Valentin Schneider" , Steven Rostedt , Vincent Guittot , Mel Gorman , Juri Lelli Subject: [PATCH] sched/fair: sanitize vruntime of entity being placed Date: Fri, 27 Jan 2023 17:32:30 +0100 Message-ID: <20230127163230.3339408-1-rkagan@amazon.de> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Zhang Qiao When a scheduling entity is placed onto cfs_rq, its vruntime is pulled to the base level (around cfs_rq->min_vruntime), so that the entity doesn't gain extra boost when placed backwards. However, if the entity being placed wasn't executed for a long time, its vruntime may get too far behind (e.g. while cfs_rq was executing a low-weight hog), which can inverse the vruntime comparison due to s64 overflow. This results in the entity being placed with its original vruntime way forwards, so that it will effectively never get to the cpu. To prevent that, ignore the vruntime of the entity being placed if it didn't execute for much longer than the characteristic sheduler time scale. [rkagan: formatted, adjusted commit log, comments, cutoff value] Co-developed-by: Roman Kagan Signed-off-by: Roman Kagan --- @zhangqiao22, I took the liberty to put you as the author of the patch, as this is essentially what you posted for discussion, with minor tweaks. Please stamp with your s-o-b if you're ok with it. kernel/sched/fair.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 0f8736991427..d6cf131ebb0b 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -4656,6 +4656,7 @@ static void place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial) { u64 vruntime = cfs_rq->min_vruntime; + u64 sleep_time; /* * The 'current' period is already promised to the current tasks, @@ -4685,8 +4686,18 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial) vruntime -= thresh; } - /* ensure we never gain time by being placed backwards. */ - se->vruntime = max_vruntime(se->vruntime, vruntime); + /* + * Pull vruntime of the entity being placed to the base level of + * cfs_rq, to prevent boosting it if placed backwards. If the entity + * slept for a long time, don't even try to compare its vruntime with + * the base as it may be too far off and the comparison may get + * inversed due to s64 overflow. + */ + sleep_time = rq_clock_task(rq_of(cfs_rq)) - se->exec_start; + if ((s64)sleep_time > 60 * NSEC_PER_SEC) + se->vruntime = vruntime; + else + se->vruntime = max_vruntime(se->vruntime, vruntime); } static void check_enqueue_throttle(struct cfs_rq *cfs_rq); -- 2.34.1 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879