From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0CCFBC47096 for ; Thu, 3 Jun 2021 11:36:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E5C38613E9 for ; Thu, 3 Jun 2021 11:36:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230033AbhFCLhz (ORCPT ); Thu, 3 Jun 2021 07:37:55 -0400 Received: from mail.kernel.org ([198.145.29.99]:37892 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229818AbhFCLhw (ORCPT ); Thu, 3 Jun 2021 07:37:52 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id CCF37613E7; Thu, 3 Jun 2021 11:36:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1622720168; bh=C9DIzOphc6fr5cipQlhio6wxZ4sUPuZ30lMxnkXl5Zo=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=fz0CjwTisYMYUbWkaIXzpxZbgQn4ts9xE44dNWjcSFJzrNicz+zKibeah6MY/sIBq FrXC6uzanqvQXWP/INu3jhmPEpJHSBWyC/nrmX4qXxfgEtvYwibg+bt1Z9f0Fzlcif 3DOicRZL1uus5oUTTP5FrlXBArMImqiV4Ohb8zb1FOB8GAFIzpHCoEfDtql5qsxM7s wfiANDoD9vVykwpWlEFWlwbJQs4HNuHZClZacw0gftegHMeC70oHX3xlP31TDAPaL7 nmZwnN8AINC12/aSmrsM6HQwF7UEEz1QMlnlaPDol9wosiiDVrE2NNnkga0v6O/iEo nbvg5GDTnO+mQ== Date: Thu, 3 Jun 2021 12:36:01 +0100 From: Will Deacon To: Peter Zijlstra Cc: linux-arm-kernel@lists.infradead.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , Dietmar Eggemann , Daniel Bristot de Oliveira , kernel-team@android.com, Oleg Nesterov Subject: Re: [RFC][PATCH] freezer,sched: Rewrite core freezer logic Message-ID: <20210603113600.GA592@willie-the-truck> References: <20210525151432.16875-1-will@kernel.org> <20210525151432.16875-17-will@kernel.org> <20210602125452.GG30593@willie-the-truck> <20210603105856.GB32641@willie-the-truck> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Jun 03, 2021 at 01:26:06PM +0200, Peter Zijlstra wrote: > On Thu, Jun 03, 2021 at 11:58:56AM +0100, Will Deacon wrote: > > On Thu, Jun 03, 2021 at 12:35:22PM +0200, Peter Zijlstra wrote: > > > On Wed, Jun 02, 2021 at 01:54:53PM +0100, Will Deacon wrote: > > > > > > @@ -116,20 +173,8 @@ bool freeze_task(struct task_struct *p) > > > > > { > > > > > unsigned long flags; > > > > > > > > > > spin_lock_irqsave(&freezer_lock, flags); > > > > > + if (!freezing(p) || frozen(p) || __freeze_task(p)) { > > > > > spin_unlock_irqrestore(&freezer_lock, flags); > > > > > return false; > > > > > } > > > > > > > > I've been trying to figure out how this serialises with ttwu(), given that > > > > frozen(p) will go and read p->state. I suppose it works out because only the > > > > freezer can wake up tasks from the FROZEN state, but it feels a bit brittle. > > > > > > p->pi_lock; both ttwu() and __freeze_task() (which is essentially a > > > variant of set_special_state()) take ->pi_lock. I'll put in a comment. > > > > The part I struggled with was freeze_task(), which doesn't take ->pi_lock > > yet calls frozen(p). > > Ah, I can't read... I assumed you were asking about __freeze_task(). > > So frozen(p) checks for p->state == TASK_FROZEN (and complicated), which > is a stable state. Once you're frozen you stay frozen until thaw, which > is after freezing per construction. > > The tricky bit is __freeze_task(), that does take pi_lock. It checks for > FREEZABLE and if set, changes to FROZEN. And this does in fact race with > ttwu() and relies on pi_lock to serialize. > > A concurrent wakeup (from a non-frozen task) can try and wake the task > we're trying to freeze. If we win, ttwu will see FROZEN and ignore, if > ttwu() wins, we don't see FREEZABLE and do another round of freezing. Good, thanks. That's where I'd ended up. It means that nobody else better try waking up FROZEN tasks! > > > > > @@ -137,7 +182,7 @@ bool freeze_task(struct task_struct *p) > > > > > if (!(p->flags & PF_KTHREAD)) > > > > > fake_signal_wake_up(p); > > > > > else > > > > > - wake_up_state(p, TASK_INTERRUPTIBLE); > > > > > + wake_up_state(p, TASK_INTERRUPTIBLE); // TASK_NORMAL ?!? > > > > > > > > > > spin_unlock_irqrestore(&freezer_lock, flags); > > > > > return true; > > > > > @@ -148,8 +193,8 @@ void __thaw_task(struct task_struct *p) > > > > > unsigned long flags; > > > > > > > > > > spin_lock_irqsave(&freezer_lock, flags); > > > > > - if (frozen(p)) > > > > > - wake_up_process(p); > > > > > + WARN_ON_ONCE(freezing(p)); > > > > > + wake_up_state(p, TASK_FROZEN | TASK_NORMAL); > > > > > > > > Why do we need TASK_NORMAL here? > > > > > > It's a left-over from hacking, but I left it in because anything > > > TASK_NORMAL should be able to deal with spuriuos wakeups, something > > > try_to_freeze() now also relies on. > > > > I just worry that it might hide bugs if TASK_FROZEN is supposed to be > > sufficient, as it would imply that we have some unfrozen tasks kicking > > around. I dunno, maybe just a comment saying that everything _should_ be > > FROZEN at this point? > > I'll take it out. It really shouldn't matter. Perfect. Will From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2A3FC47082 for ; Thu, 3 Jun 2021 11:37:49 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 92F20613AC for ; Thu, 3 Jun 2021 11:37:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 92F20613AC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=IYMCDQgeFLBbjk3jHiPWzE//gbk4QUDxTZ7B34rVJUA=; b=iQFNdWaWfdOa1K IrcB2lFvv3P0FKkeMLCsQSqcRQJlIq8kEcI9iiA1xFz3oFkAQ66U/ClNtex1yB8fqCYoPA90JYhNl hVN+x2fCtO8HG20xNnivLy3MjlNU6y1R0Bz0625kDAE//vNL0AQdB/eSMGGoVzD9vjwXOSp1JRCuA kLO1nYYkuiOzcBWxCErUirBXIPeUkyJ3fAVsMFNSvGmIosl6+WuMrtg7CdoCxTkE5lpF/ueJoOFNO O/DFr2LJX7vC2onITA0ak/FotKD5/vZV0tN15/wqFZxjcjUol8FEEwXD0fCxtzWUGRpCWlrU2dB+H i8ROBl4FxnwHk3Rs22VA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lole2-008NO0-7L; Thu, 03 Jun 2021 11:36:14 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1loldw-008NMP-Cz for linux-arm-kernel@lists.infradead.org; Thu, 03 Jun 2021 11:36:12 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id CCF37613E7; Thu, 3 Jun 2021 11:36:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1622720168; bh=C9DIzOphc6fr5cipQlhio6wxZ4sUPuZ30lMxnkXl5Zo=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=fz0CjwTisYMYUbWkaIXzpxZbgQn4ts9xE44dNWjcSFJzrNicz+zKibeah6MY/sIBq FrXC6uzanqvQXWP/INu3jhmPEpJHSBWyC/nrmX4qXxfgEtvYwibg+bt1Z9f0Fzlcif 3DOicRZL1uus5oUTTP5FrlXBArMImqiV4Ohb8zb1FOB8GAFIzpHCoEfDtql5qsxM7s wfiANDoD9vVykwpWlEFWlwbJQs4HNuHZClZacw0gftegHMeC70oHX3xlP31TDAPaL7 nmZwnN8AINC12/aSmrsM6HQwF7UEEz1QMlnlaPDol9wosiiDVrE2NNnkga0v6O/iEo nbvg5GDTnO+mQ== Date: Thu, 3 Jun 2021 12:36:01 +0100 From: Will Deacon To: Peter Zijlstra Cc: linux-arm-kernel@lists.infradead.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , Dietmar Eggemann , Daniel Bristot de Oliveira , kernel-team@android.com, Oleg Nesterov Subject: Re: [RFC][PATCH] freezer,sched: Rewrite core freezer logic Message-ID: <20210603113600.GA592@willie-the-truck> References: <20210525151432.16875-1-will@kernel.org> <20210525151432.16875-17-will@kernel.org> <20210602125452.GG30593@willie-the-truck> <20210603105856.GB32641@willie-the-truck> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210603_043608_497813_927D9763 X-CRM114-Status: GOOD ( 33.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thu, Jun 03, 2021 at 01:26:06PM +0200, Peter Zijlstra wrote: > On Thu, Jun 03, 2021 at 11:58:56AM +0100, Will Deacon wrote: > > On Thu, Jun 03, 2021 at 12:35:22PM +0200, Peter Zijlstra wrote: > > > On Wed, Jun 02, 2021 at 01:54:53PM +0100, Will Deacon wrote: > > > > > > @@ -116,20 +173,8 @@ bool freeze_task(struct task_struct *p) > > > > > { > > > > > unsigned long flags; > > > > > > > > > > spin_lock_irqsave(&freezer_lock, flags); > > > > > + if (!freezing(p) || frozen(p) || __freeze_task(p)) { > > > > > spin_unlock_irqrestore(&freezer_lock, flags); > > > > > return false; > > > > > } > > > > > > > > I've been trying to figure out how this serialises with ttwu(), given that > > > > frozen(p) will go and read p->state. I suppose it works out because only the > > > > freezer can wake up tasks from the FROZEN state, but it feels a bit brittle. > > > > > > p->pi_lock; both ttwu() and __freeze_task() (which is essentially a > > > variant of set_special_state()) take ->pi_lock. I'll put in a comment. > > > > The part I struggled with was freeze_task(), which doesn't take ->pi_lock > > yet calls frozen(p). > > Ah, I can't read... I assumed you were asking about __freeze_task(). > > So frozen(p) checks for p->state == TASK_FROZEN (and complicated), which > is a stable state. Once you're frozen you stay frozen until thaw, which > is after freezing per construction. > > The tricky bit is __freeze_task(), that does take pi_lock. It checks for > FREEZABLE and if set, changes to FROZEN. And this does in fact race with > ttwu() and relies on pi_lock to serialize. > > A concurrent wakeup (from a non-frozen task) can try and wake the task > we're trying to freeze. If we win, ttwu will see FROZEN and ignore, if > ttwu() wins, we don't see FREEZABLE and do another round of freezing. Good, thanks. That's where I'd ended up. It means that nobody else better try waking up FROZEN tasks! > > > > > @@ -137,7 +182,7 @@ bool freeze_task(struct task_struct *p) > > > > > if (!(p->flags & PF_KTHREAD)) > > > > > fake_signal_wake_up(p); > > > > > else > > > > > - wake_up_state(p, TASK_INTERRUPTIBLE); > > > > > + wake_up_state(p, TASK_INTERRUPTIBLE); // TASK_NORMAL ?!? > > > > > > > > > > spin_unlock_irqrestore(&freezer_lock, flags); > > > > > return true; > > > > > @@ -148,8 +193,8 @@ void __thaw_task(struct task_struct *p) > > > > > unsigned long flags; > > > > > > > > > > spin_lock_irqsave(&freezer_lock, flags); > > > > > - if (frozen(p)) > > > > > - wake_up_process(p); > > > > > + WARN_ON_ONCE(freezing(p)); > > > > > + wake_up_state(p, TASK_FROZEN | TASK_NORMAL); > > > > > > > > Why do we need TASK_NORMAL here? > > > > > > It's a left-over from hacking, but I left it in because anything > > > TASK_NORMAL should be able to deal with spuriuos wakeups, something > > > try_to_freeze() now also relies on. > > > > I just worry that it might hide bugs if TASK_FROZEN is supposed to be > > sufficient, as it would imply that we have some unfrozen tasks kicking > > around. I dunno, maybe just a comment saying that everything _should_ be > > FROZEN at this point? > > I'll take it out. It really shouldn't matter. Perfect. Will _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel