From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.4 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB72DC4363A for ; Tue, 27 Oct 2020 03:17:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7CC712072D for ; Tue, 27 Oct 2020 03:17:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2504517AbgJ0DRP (ORCPT ); Mon, 26 Oct 2020 23:17:15 -0400 Received: from mga07.intel.com ([134.134.136.100]:45687 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2442515AbgJ0DRP (ORCPT ); Mon, 26 Oct 2020 23:17:15 -0400 IronPort-SDR: xZxNBoxQEC9cVmxyt2e22kdUlrErOq/ZCiX+oIGe5IbVAauibxHcmle0u1aoBW0Zv8xEofi4/V JfZ87dsx0cUw== X-IronPort-AV: E=McAfee;i="6000,8403,9786"; a="232208385" X-IronPort-AV: E=Sophos;i="5.77,422,1596524400"; d="scan'208";a="232208385" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Oct 2020 20:17:13 -0700 IronPort-SDR: UmTG3wTrOWapmJw0go0yrBJY7ZFiP2bFASJRNgqwUXoOL59MoDTzftmnvQCCFBb01ADME4nDVd SHgs6zZ+Oj7A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,422,1596524400"; d="scan'208";a="361226808" Received: from cli6-desk1.ccr.corp.intel.com (HELO [10.239.161.135]) ([10.239.161.135]) by orsmga007.jf.intel.com with ESMTP; 26 Oct 2020 20:17:04 -0700 Subject: Re: [PATCH v8 -tip 02/26] sched: Introduce sched_class::pick_task() To: Peter Zijlstra , Vineeth Pillai Cc: Joel Fernandes , Nishanth Aravamudan , Julien Desfossez , Tim Chen , Aaron Lu , Aubrey Li , Thomas Glexiner , LKML , Ingo Molnar , Linus Torvalds , Frederic Weisbecker , Kees Cook , Greg Kerr , Phil Auld , Valentin Schneider , Mel Gorman , Pawan Gupta , Paolo Bonzini , vineeth@bitbyteword.org, Chen Yu , Christian Brauner , Agata Gruza , Antonio Gomez Iglesias , graf@amazon.com, konrad.wilk@oracle.com, Dario Faggioli , Paul Turner , Steven Rostedt , Patrick Bellasi , =?UTF-8?B?YmVuYmppYW5nKOiSi+W9qik=?= , Alexandre Chartre , James.Bottomley@hansenpartnership.com, OWeisse@umich.edu, Dhaval Giani , Junaid Shahid , Jesse Barnes , "Hyser,Chris" , "Paul E. McKenney" , Tim Chen , "Ning, Hongyu" References: <20201020014336.2076526-1-joel@joelfernandes.org> <20201020014336.2076526-3-joel@joelfernandes.org> <8ea1aa61-4a1c-2687-9f15-1062d37606c7@linux.intel.com> <20201023214702.GA3603399@google.com> <4241e5ac-ecdf-8634-fa0d-dd6759e477e1@linux.microsoft.com> <8230ada7-839f-2335-9a55-b09f6a813e91@linux.microsoft.com> <20201026090131.GE2628@hirez.programming.kicks-ass.net> From: "Li, Aubrey" Message-ID: <3c6e2d04-ca47-299c-1131-31f003909875@linux.intel.com> Date: Tue, 27 Oct 2020 11:17:03 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.9.0 MIME-Version: 1.0 In-Reply-To: <20201026090131.GE2628@hirez.programming.kicks-ass.net> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2020/10/26 17:01, Peter Zijlstra wrote: > On Sat, Oct 24, 2020 at 08:27:16AM -0400, Vineeth Pillai wrote: >> >> >> On 10/24/20 7:10 AM, Vineeth Pillai wrote: >>> >>> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c >>> index 93a3b874077d..4cae5ac48b60 100644 >>> --- a/kernel/sched/fair.c >>> +++ b/kernel/sched/fair.c >>> @@ -4428,12 +4428,14 @@ pick_next_entity(struct cfs_rq *cfs_rq, struct >>> sched_entity *curr) >>>                         se = second; >>>         } >>> >>> -       if (cfs_rq->next && wakeup_preempt_entity(cfs_rq->next, left) < >>> 1) { >>> +       if (left && cfs_rq->next && >>> +                       wakeup_preempt_entity(cfs_rq->next, left) < 1) { >>>                 /* >>>                  * Someone really wants this to run. If it's not unfair, >>> run it. >>>                  */ >>>                 se = cfs_rq->next; >>> -       } else if (cfs_rq->last && wakeup_preempt_entity(cfs_rq->last, >>> left) < 1) { >>> +       } else if (left && cfs_rq->last && >>> +                       wakeup_preempt_entity(cfs_rq->last, left) < 1) { >>>                 /* >>>                  * Prefer last buddy, try to return the CPU to a >>> preempted task. >>> >>> >>> There reason for left being NULL needs to be investigated. This was >>> there from v1 and we did not yet get to it. I shall try to debug later >>> this week. >> >> Thinking more about it and looking at the crash, I think that >> 'left == NULL' can happen in pick_next_entity for core scheduling. >> If a cfs_rq has only one task that is running, then it will be >> dequeued and 'left = __pick_first_entity()' will be NULL as the >> cfs_rq will be empty. This would not happen outside of coresched >> because we never call pick_tack() before put_prev_task() which >> will enqueue the task back. >> >> With core scheduling, a cpu can call pick_task() for its sibling while >> the sibling is still running the active task and put_prev_task has yet >> not been called. This can result in 'left == NULL'. > > Quite correct. Hurmph.. the reason we do this is because... we do the > update_curr() the wrong way around. And I can't seem to remember why we > do that (it was in my original patches). > > Something like so seems the obvious thing to do, but I can't seem to > remember why we're not doing it :-( > > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @@ -6950,15 +6950,10 @@ static struct task_struct *pick_task_fai > do { > struct sched_entity *curr = cfs_rq->curr; > > - se = pick_next_entity(cfs_rq, NULL); > + if (curr && curr->on_rq) > + update_curr(cfs_rq); > > - if (curr) { > - if (se && curr->on_rq) > - update_curr(cfs_rq); > - > - if (!se || entity_before(curr, se)) > - se = curr; > - } > + se = pick_next_entity(cfs_rq, curr); > > cfs_rq = group_cfs_rq(se); > } while (cfs_rq); > This patch works too for my benchmark, thanks Peter!