From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE77FC3279D for ; Tue, 6 Oct 2020 21:41:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 863BF20B1F for ; Tue, 6 Oct 2020 21:41:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602020479; bh=9wLduu0HibYTvQ1PxET1bwvOHmceqeDnpLGd3eg73zU=; h=Date:From:To:Cc:Subject:References:In-Reply-To:List-ID:From; b=xySsdPL++6oihnsSAZrS6WQEdhxspcQndG4OFxWu1Eg1hE9/RQpOffG2ATbQvu2r0 RXJQMuXT+32zOaRv5uPmbF9SDeucclZQ0gc+AcRZBnd+3V53w25/i+28Ci6VA+5hxh 9dq9mfQeGPLPDLBfdbBTJAGaVMaPMLhuDVgX09Lg= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727335AbgJFVlS (ORCPT ); Tue, 6 Oct 2020 17:41:18 -0400 Received: from mail.kernel.org ([198.145.29.99]:40950 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725947AbgJFVlS (ORCPT ); Tue, 6 Oct 2020 17:41:18 -0400 Received: from localhost (cha74-h07-176-172-165-181.dsl.sta.abo.bbox.fr [176.172.165.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id A895B208B6; Tue, 6 Oct 2020 21:41:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602020477; bh=9wLduu0HibYTvQ1PxET1bwvOHmceqeDnpLGd3eg73zU=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=2gCpOuwlb9OCrTs8XdwbsjCC4BukoUfz0Mvac17ZIGhzrZJLjfCkNYUOHOHUoqqri OIOiw9PbygkXW73CcQougUiK/NjNN+08X/TVwhkLEOiN9KU6jcrRLND5yoQwFZvivV kkF0BPaHRf4ooYAz0mS3oP3jWuSjTIeE9AhIWZDA= Date: Tue, 6 Oct 2020 23:41:13 +0200 From: Frederic Weisbecker To: Alex Belits Cc: "mingo@kernel.org" , "davem@davemloft.net" , "linux-api@vger.kernel.org" , "rostedt@goodmis.org" , "peterz@infradead.org" , "linux-arch@vger.kernel.org" , "catalin.marinas@arm.com" , "tglx@linutronix.de" , "will@kernel.org" , Prasun Kapoor , "linux-kernel@vger.kernel.org" , "netdev@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" Subject: Re: [EXT] Re: [PATCH v4 10/13] task_isolation: don't interrupt CPUs with tick_nohz_full_kick_cpu() Message-ID: <20201006214113.GA38684@lothringen> References: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> <5acf7502c071c0d1365ba5e5940e003a7da6521f.camel@marvell.com> <20201001144454.GB6595@lothringen> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sun, Oct 04, 2020 at 03:22:09PM +0000, Alex Belits wrote: > > On Thu, 2020-10-01 at 16:44 +0200, Frederic Weisbecker wrote: > > > @@ -268,7 +269,8 @@ static void tick_nohz_full_kick(void) > > > */ > > > void tick_nohz_full_kick_cpu(int cpu) > > > { > > > - if (!tick_nohz_full_cpu(cpu)) > > > + smp_rmb(); > > > > What is it ordering? > > ll_isol_flags will be read in task_isolation_on_cpu(), that accrss > should be ordered against writing in > task_isolation_kernel_enter(), fast_task_isolation_cpu_cleanup() > and task_isolation_start(). > > Since task_isolation_on_cpu() is often called for multiple CPUs in a > sequence, it would be wasteful to include a barrier inside it. Then I think you meant a full barrier: smp_mb() > > > > + if (!tick_nohz_full_cpu(cpu) || task_isolation_on_cpu(cpu)) > > > return; > > > > You can't simply ignore an IPI. There is always a reason for a > > nohz_full CPU > > to be kicked. Something triggered a tick dependency. It can be posix > > cpu timers > > for example, or anything. > > I realize that this is unusual, however the idea is that while the task > is running in isolated mode in userspace, we assume that from this CPUs > point of view whatever is happening in kernel, can wait until CPU is > back in kernel and when it first enters kernel from this mode, it > should "catch up" with everything that happened in its absence. > task_isolation_kernel_enter() is supposed to do that, so by the time > anything should be done involving the rest of the kernel, CPU is back > to normal. You can't assume that. If something needs the tick, this can't wait. If the user did something wrong, such as setting a posix cpu timer to an isolated task, that's his fault and the kernel has to stick with correctness and kick that task out of isolation mode. > > It is application's responsibility to avoid triggering things that > break its isolation Precisely. > so the application assumes that everything that > involves entering kernel will not be available while it is isolated. We can't do things that way and just ignore IPIs. You need to solve the source of the noise, not the symptoms. Thanks. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89C7BC4363D for ; Tue, 6 Oct 2020 21:42:52 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 14DD6206BE for ; Tue, 6 Oct 2020 21:42:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="G3FPvNFN"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="2gCpOuwl" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 14DD6206BE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=OXZ1CLEAWIWSOQPe/qFCSenIy4BhTFwz5qjmTZXsV3Q=; b=G3FPvNFNjC0HPDTiGmuzcszK4 HmA7E46r5kMiXzPX7WnZrXFJROOH/s88f7pqv9pHQABG9TCwTVxwWaHv/WRrciV4C8aqJBkroR8m0 sYc8qapEmDIfWQkpiO21Fay2feJQusOypOEzlbgxYG2AP0vfQw1u68+O5Wmc8mkLR9oiO80WLqhma zBpbWG1znTItOCqmPwnyPOZoF0jYMueQS21bhhz1I6HvoZFxiPHgBSl9il7AOcg5Udlhyu5i2P9zl ONlVDAoImAaFOgepu9NRhHP5WMuzWaC6c4XfbOiQGQuN2UgltgWlmljs2/npxsytgWd76pF0HCWyr siuIC26Xw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kPui0-00075u-HD; Tue, 06 Oct 2020 21:41:20 +0000 Received: from mail.kernel.org ([198.145.29.99]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kPuhy-000750-CE for linux-arm-kernel@lists.infradead.org; Tue, 06 Oct 2020 21:41:19 +0000 Received: from localhost (cha74-h07-176-172-165-181.dsl.sta.abo.bbox.fr [176.172.165.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id A895B208B6; Tue, 6 Oct 2020 21:41:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602020477; bh=9wLduu0HibYTvQ1PxET1bwvOHmceqeDnpLGd3eg73zU=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=2gCpOuwlb9OCrTs8XdwbsjCC4BukoUfz0Mvac17ZIGhzrZJLjfCkNYUOHOHUoqqri OIOiw9PbygkXW73CcQougUiK/NjNN+08X/TVwhkLEOiN9KU6jcrRLND5yoQwFZvivV kkF0BPaHRf4ooYAz0mS3oP3jWuSjTIeE9AhIWZDA= Date: Tue, 6 Oct 2020 23:41:13 +0200 From: Frederic Weisbecker To: Alex Belits Subject: Re: [EXT] Re: [PATCH v4 10/13] task_isolation: don't interrupt CPUs with tick_nohz_full_kick_cpu() Message-ID: <20201006214113.GA38684@lothringen> References: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> <5acf7502c071c0d1365ba5e5940e003a7da6521f.camel@marvell.com> <20201001144454.GB6595@lothringen> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201006_174118_635372_7456D01E X-CRM114-Status: GOOD ( 29.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "linux-arch@vger.kernel.org" , "peterz@infradead.org" , "linux-api@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "rostedt@goodmis.org" , "davem@davemloft.net" , "netdev@vger.kernel.org" , "catalin.marinas@arm.com" , Prasun Kapoor , "tglx@linutronix.de" , "will@kernel.org" , "mingo@kernel.org" , "linux-arm-kernel@lists.infradead.org" Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Sun, Oct 04, 2020 at 03:22:09PM +0000, Alex Belits wrote: > > On Thu, 2020-10-01 at 16:44 +0200, Frederic Weisbecker wrote: > > > @@ -268,7 +269,8 @@ static void tick_nohz_full_kick(void) > > > */ > > > void tick_nohz_full_kick_cpu(int cpu) > > > { > > > - if (!tick_nohz_full_cpu(cpu)) > > > + smp_rmb(); > > > > What is it ordering? > > ll_isol_flags will be read in task_isolation_on_cpu(), that accrss > should be ordered against writing in > task_isolation_kernel_enter(), fast_task_isolation_cpu_cleanup() > and task_isolation_start(). > > Since task_isolation_on_cpu() is often called for multiple CPUs in a > sequence, it would be wasteful to include a barrier inside it. Then I think you meant a full barrier: smp_mb() > > > > + if (!tick_nohz_full_cpu(cpu) || task_isolation_on_cpu(cpu)) > > > return; > > > > You can't simply ignore an IPI. There is always a reason for a > > nohz_full CPU > > to be kicked. Something triggered a tick dependency. It can be posix > > cpu timers > > for example, or anything. > > I realize that this is unusual, however the idea is that while the task > is running in isolated mode in userspace, we assume that from this CPUs > point of view whatever is happening in kernel, can wait until CPU is > back in kernel and when it first enters kernel from this mode, it > should "catch up" with everything that happened in its absence. > task_isolation_kernel_enter() is supposed to do that, so by the time > anything should be done involving the rest of the kernel, CPU is back > to normal. You can't assume that. If something needs the tick, this can't wait. If the user did something wrong, such as setting a posix cpu timer to an isolated task, that's his fault and the kernel has to stick with correctness and kick that task out of isolation mode. > > It is application's responsibility to avoid triggering things that > break its isolation Precisely. > so the application assumes that everything that > involves entering kernel will not be available while it is isolated. We can't do things that way and just ignore IPIs. You need to solve the source of the noise, not the symptoms. Thanks. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel