From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6844CC433ED for ; Fri, 21 May 2021 13:12:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4B46B61244 for ; Fri, 21 May 2021 13:12:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233462AbhEUNOH (ORCPT ); Fri, 21 May 2021 09:14:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52228 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230408AbhEUNN4 (ORCPT ); Fri, 21 May 2021 09:13:56 -0400 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 745F3C061763 for ; Fri, 21 May 2021 06:12:33 -0700 (PDT) Received: by mail-wr1-x435.google.com with SMTP id d11so21013620wrw.8 for ; Fri, 21 May 2021 06:12:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=mAjk0S+UpNUqwhT8eJhAogvnWkfFNLPiDvekTv19GD0=; b=VsyUFe5OmlUqsGOd3dQwNCB0z0RSAQOBSADnYOEe8sMCMdFU3L7qSg9IxKF2qSqHAC cYlQoy1oWU4026exxAWAFhiuSoMQIp4wjir4+69ySxMrkpOHcWRuqrzz+Cqk8ucWqqNY a4Qugx3ZA31d6VPsCz7gxT73DESb551AVp6NEneY7m3ku41ubtKfDUNjplg4o9gt0QLf X/CnuF6kYo8x47LPlUposST5uxaGygy9i60H5dRvmXGrRttHzeMG8Q9ZtxR2s1E8P39/ pFq4lPXvNgN2bHellK1jfvRWOTN90ZEA5Z8WCPslXN8SaFD2DpeMdDB81p50mMu+TcaC XJIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=mAjk0S+UpNUqwhT8eJhAogvnWkfFNLPiDvekTv19GD0=; b=K8ffca1Itw3S+idnnXflnvdWj3R20AO86aDzZTC1kIQ7sXk/R6mqWRjG1jYeb2yNHP BCCG2GW1u5PiAUqf0P9RUOPmrnOjVQKEz9d5UR8fV991VxGZQnypU62u56H1ofAXZ/4v DB6/AaDASSV9LSHQGt0PtJLGHnRnJotUwsN74MX2E4F1St9D10dasJtdQvgRDCS5L8Iu 9vSqmpBBpriW4fjYTe8sUTSKHxnrcCYKbsQ+GG7udKjyopa3AdbsKdZDyHyRdVBkOzjy 2R+Jseu5J8ffx9t269IL+hsKpeKYMuW+k2mDT5XPge2cUd7kZk/TJ0xy3TKQuLE2FhwQ CSYQ== X-Gm-Message-State: AOAM533tDo3OF+vvJISKCuInzDIU6ylMKnTXd+ehQwoKly/Fut6c7QXt g/2JlA7Rk29aoWkLhaoqqe+EZg== X-Google-Smtp-Source: ABdhPJxwcRmT9rcPrheowVpO7PWRSo/GBx/N8FRmPc1y1zTxqeO9zVUG3ETKQYsgE+dKpqkobaTcxw== X-Received: by 2002:a05:6000:4d:: with SMTP id k13mr9790235wrx.98.1621602751913; Fri, 21 May 2021 06:12:31 -0700 (PDT) Received: from google.com (105.168.195.35.bc.googleusercontent.com. [35.195.168.105]) by smtp.gmail.com with ESMTPSA id l18sm2067425wrt.97.2021.05.21.06.12.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 May 2021 06:12:31 -0700 (PDT) Date: Fri, 21 May 2021 13:12:28 +0000 From: Quentin Perret To: Daniel Bristot de Oliveira Cc: Will Deacon , Juri Lelli , linux-arm-kernel@lists.infradead.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Tejun Heo , Johannes Weiner , Ingo Molnar , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: Re: [PATCH v6 13/21] sched: Admit forcefully-affined tasks into SCHED_DEADLINE Message-ID: References: <20210520101640.GA10065@willie-the-truck> <20210520180138.GA10523@willie-the-truck> <20210521103724.GA11680@willie-the-truck> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Friday 21 May 2021 at 15:00:42 (+0200), Daniel Bristot de Oliveira wrote: > On 5/21/21 12:37 PM, Will Deacon wrote: > > Interesting, thanks. Thinking about this some more, it strikes me that with > > these silly asymmetric systems there could be an interesting additional > > problem with hotplug and deadline tasks. Imagine the following sequence of > > events: > > > > 1. All online CPUs are 32-bit-capable > > 2. sched_setattr() admits a 32-bit deadline task > > 3. A 64-bit-only CPU is onlined > > At the point 3, the global scheduler assumption is broken. For instance, in a > system with four CPUs and five ready 32-bit-capable tasks, when the fifth CPU as > added, the working conserving rule is violated because the five highest priority > thread are not running (only four are) :-(. > > So, at this point, for us to keep to the current behavior, the addition should > be.. blocked? :-(( > > > 4. Some of the 32-bit-capable CPUs are offlined > > Assuming that point 3 does not exist (i.e., all CPUs are 32-bit-capable). At > this point, we will have an increase in the pressure on the 32-bit-capable CPUs. > > This can also create bad effects for 64-bit tasks, as the "contended" 32-bit > tasks will still be "queued" in a future time where they were supposed to be > done (leaving time for the 64-bit tasks). > > > I wonder if we can get into a situation where we think we have enough > > bandwidth available, but in reality the 32-bit task is in trouble because > > it can't make use of the 64-bit-only CPU. > > I would have to think more, but there might be a case where this contended > 32-bit tasks could cause deadline misses for the 64-bit too. > > > If so, then it seems to me that admission control is really just > > "best-effort" for 32-bit deadline tasks on these systems because it's based > > on a snapshot in time of the available resources. > > The admission test as is now is "best-effort" in the sense that it allows a > workload higher than it could handle (it is necessary, but not sufficient AC). > But it should not be considered "best-effort" because of violations in the > working conserving property as a result of arbitrary affinities among tasks. > Overall, we have been trying to close any "exception left" to this later case. > > I know, it is a complex situation, I am just trying to illustrate our concerns, > because, in the near future we might have a scheduler that handles arbitrary > affinity correctly. But that might require us to stick to an AC. The AC is > something precious for us. FWIW, I agree with the above. As pointed out in another reply, this looks like an existing bug and there is nothing specific to 32bits tasks here. But I don't think the existence of this bug should be a reason for breaking AC even more than it is. So it still feels cleaner to block execve() into 32bit if AC is on until we have proper support for affinities in DL. And if folks want to use 32bit DL tasks on these systems they'll have to disable AC, but at least they know what they are getting ... Thanks, Quentin From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5AA0CC433B4 for ; Fri, 21 May 2021 13:15:08 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D753E60FEF for ; Fri, 21 May 2021 13:15:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D753E60FEF Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=HXI/7eDN5b72tOsxGjF0LLHoRR6a6djemrDTdAeJLY0=; b=IrfDOS3KDP7YaGtkHt+L7DdXHx 2ITtNAS+n4pG0mDmV/TWPuO2yMKy17GSzW34xMY+v+QjkfLrt4KmXzbT2rL95ULUDc0HDk2965R48 1nnVhAMeIgokvfEKgGtFlK3zU5W2ZURKgNoZE9BwXTvN6odDYUPkHdRRW7BqiFL4rkbe4DnFmuQj4 LNfh+hjVYMxOF8/y46Q0ntz6inU5Q9kQVcnPXsodDbnD3Ogi2IdurHY26zavimUzb8AGXosC1LrHi z+FljXOz5E7A8xeKyEzjtLN3a5UqbS9Ck9rLAJ21rTtXHw1XhHTYFtndwAOrghyBEdYt6A5W+z/90 rP84toXA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lk4xG-005SIT-F5; Fri, 21 May 2021 13:12:42 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lk4xD-005SHL-Eh for linux-arm-kernel@desiato.infradead.org; Fri, 21 May 2021 13:12:39 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=In-Reply-To:Content-Type:MIME-Version :References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=mAjk0S+UpNUqwhT8eJhAogvnWkfFNLPiDvekTv19GD0=; b=rLHlMItPx0NNBjmubqPW/oynPs KAw7UWjrV7fVKZNzgS+0QW1Sts8CKkRXgf0EJmTx+05aCpvzayod2QjH0ty59vFBUIAIGccci2EFA TnQ65RREk48cDKJZ87Rq3glJo9/Pz+WcUsHydaWUk8EmLH3WSVdG9YnZAJgFWLOXS83fgxOIWQvs1 rUt8DDYRAv/kY+HZLQOAZvXSTse5LBnv+u/8v2UPvBjwUU+LHtAKbJD/xQ+m5hNlL2bERHZirq/+/ EoT1GGFFOF4Dt8zlFZS88BQtoZWMXI3O27A2iLw29fq80cN72WZfAQfhVjRFbGpNhjsRfbMrRdbbN 1SjsC89Q==; Received: from mail-wr1-x430.google.com ([2a00:1450:4864:20::430]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lk4x7-00H7pv-L0 for linux-arm-kernel@lists.infradead.org; Fri, 21 May 2021 13:12:37 +0000 Received: by mail-wr1-x430.google.com with SMTP id n2so21121423wrm.0 for ; Fri, 21 May 2021 06:12:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=mAjk0S+UpNUqwhT8eJhAogvnWkfFNLPiDvekTv19GD0=; b=VsyUFe5OmlUqsGOd3dQwNCB0z0RSAQOBSADnYOEe8sMCMdFU3L7qSg9IxKF2qSqHAC cYlQoy1oWU4026exxAWAFhiuSoMQIp4wjir4+69ySxMrkpOHcWRuqrzz+Cqk8ucWqqNY a4Qugx3ZA31d6VPsCz7gxT73DESb551AVp6NEneY7m3ku41ubtKfDUNjplg4o9gt0QLf X/CnuF6kYo8x47LPlUposST5uxaGygy9i60H5dRvmXGrRttHzeMG8Q9ZtxR2s1E8P39/ pFq4lPXvNgN2bHellK1jfvRWOTN90ZEA5Z8WCPslXN8SaFD2DpeMdDB81p50mMu+TcaC XJIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=mAjk0S+UpNUqwhT8eJhAogvnWkfFNLPiDvekTv19GD0=; b=E6/1mwPLErz2pYdQatnDrH9gWt7d0Xdwk05Bt0kUWMr7EuF38f9A9TQK4rm5EC7UR6 eOw/VaZi4jTmGeFs76JN7agBmTggVbMWYoo0PGuolAs0W2lbPj7mn5W83ZEeHNHvGK8c kj+mxGMGiWewjWsXgYAP+AFwGI6SbLwttvPNCdwB/d6BKeusy442g+m0x1movY+Oujo2 kndV2LG/1xQep7EkrXGWi84FP2hzr3huLp0iSuCDyn6HvYOfeU1o24Jscw/QF15rPeLV +Zt5Lof5k7xsI49Ha8y4duPEP8Xs1hgfsQD5/k+wY2p5j96YB8OgHQ33csW2gwCC1pNq Usuw== X-Gm-Message-State: AOAM5301Ir9jlAkhn5ZU1kft+PZPg8gtZQMrMDPmnhGgPmP9rcrLw669 b2Un2b2qVGlSJj5XpAOUzqY9Dl0M577lRQ== X-Google-Smtp-Source: ABdhPJxwcRmT9rcPrheowVpO7PWRSo/GBx/N8FRmPc1y1zTxqeO9zVUG3ETKQYsgE+dKpqkobaTcxw== X-Received: by 2002:a05:6000:4d:: with SMTP id k13mr9790235wrx.98.1621602751913; Fri, 21 May 2021 06:12:31 -0700 (PDT) Received: from google.com (105.168.195.35.bc.googleusercontent.com. [35.195.168.105]) by smtp.gmail.com with ESMTPSA id l18sm2067425wrt.97.2021.05.21.06.12.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 May 2021 06:12:31 -0700 (PDT) Date: Fri, 21 May 2021 13:12:28 +0000 From: Quentin Perret To: Daniel Bristot de Oliveira Cc: Will Deacon , Juri Lelli , linux-arm-kernel@lists.infradead.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Tejun Heo , Johannes Weiner , Ingo Molnar , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: Re: [PATCH v6 13/21] sched: Admit forcefully-affined tasks into SCHED_DEADLINE Message-ID: References: <20210520101640.GA10065@willie-the-truck> <20210520180138.GA10523@willie-the-truck> <20210521103724.GA11680@willie-the-truck> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210521_061233_700679_C0033B04 X-CRM114-Status: GOOD ( 34.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Friday 21 May 2021 at 15:00:42 (+0200), Daniel Bristot de Oliveira wrote: > On 5/21/21 12:37 PM, Will Deacon wrote: > > Interesting, thanks. Thinking about this some more, it strikes me that with > > these silly asymmetric systems there could be an interesting additional > > problem with hotplug and deadline tasks. Imagine the following sequence of > > events: > > > > 1. All online CPUs are 32-bit-capable > > 2. sched_setattr() admits a 32-bit deadline task > > 3. A 64-bit-only CPU is onlined > > At the point 3, the global scheduler assumption is broken. For instance, in a > system with four CPUs and five ready 32-bit-capable tasks, when the fifth CPU as > added, the working conserving rule is violated because the five highest priority > thread are not running (only four are) :-(. > > So, at this point, for us to keep to the current behavior, the addition should > be.. blocked? :-(( > > > 4. Some of the 32-bit-capable CPUs are offlined > > Assuming that point 3 does not exist (i.e., all CPUs are 32-bit-capable). At > this point, we will have an increase in the pressure on the 32-bit-capable CPUs. > > This can also create bad effects for 64-bit tasks, as the "contended" 32-bit > tasks will still be "queued" in a future time where they were supposed to be > done (leaving time for the 64-bit tasks). > > > I wonder if we can get into a situation where we think we have enough > > bandwidth available, but in reality the 32-bit task is in trouble because > > it can't make use of the 64-bit-only CPU. > > I would have to think more, but there might be a case where this contended > 32-bit tasks could cause deadline misses for the 64-bit too. > > > If so, then it seems to me that admission control is really just > > "best-effort" for 32-bit deadline tasks on these systems because it's based > > on a snapshot in time of the available resources. > > The admission test as is now is "best-effort" in the sense that it allows a > workload higher than it could handle (it is necessary, but not sufficient AC). > But it should not be considered "best-effort" because of violations in the > working conserving property as a result of arbitrary affinities among tasks. > Overall, we have been trying to close any "exception left" to this later case. > > I know, it is a complex situation, I am just trying to illustrate our concerns, > because, in the near future we might have a scheduler that handles arbitrary > affinity correctly. But that might require us to stick to an AC. The AC is > something precious for us. FWIW, I agree with the above. As pointed out in another reply, this looks like an existing bug and there is nothing specific to 32bits tasks here. But I don't think the existence of this bug should be a reason for breaking AC even more than it is. So it still feels cleaner to block execve() into 32bit if AC is on until we have proper support for affinities in DL. And if folks want to use 32bit DL tasks on these systems they'll have to disable AC, but at least they know what they are getting ... Thanks, Quentin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel