From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94DC3C6FD1A for ; Tue, 7 Mar 2023 16:40:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E77C16B0074; Tue, 7 Mar 2023 11:40:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E27FB6B0075; Tue, 7 Mar 2023 11:40:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D158C280001; Tue, 7 Mar 2023 11:40:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C18E16B0074 for ; Tue, 7 Mar 2023 11:40:29 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 3CDC2AB406 for ; Tue, 7 Mar 2023 16:40:29 +0000 (UTC) X-FDA: 80542665378.27.F0D4F2F Received: from albert.telenet-ops.be (albert.telenet-ops.be [195.130.137.90]) by imf13.hostedemail.com (Postfix) with ESMTP id 5E80D2000A for ; Tue, 7 Mar 2023 16:40:26 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=none; spf=none (imf13.hostedemail.com: domain of geert@linux-m68k.org has no SPF policy when checking 195.130.137.90) smtp.mailfrom=geert@linux-m68k.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678207226; a=rsa-sha256; cv=none; b=KeOPcpxjDzCW/mMQ99Rg5Ci6/Ouj7BgYkiyh6bwIf7yVffAtMSQ6XvMiY5fhDSeKemIm0K FyX7u6ed1FSSUoliM+/uFV8inLXVJagX6JQBm6U5lQmiW7g+ZSJRiJFQq7zPpUdG9QA0G2 lNIRVfUFi57mR4YziWJufuF8zfchXY0= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=none; spf=none (imf13.hostedemail.com: domain of geert@linux-m68k.org has no SPF policy when checking 195.130.137.90) smtp.mailfrom=geert@linux-m68k.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678207226; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mQoPgOsOdMQXLuw3hPjudPYNz7wYr2Os5a513yqYCuI=; b=7qxtChsQBgk+DygYPbR1zNYYShl4ZZV9NpJU0nZpGBhmsmA6DgeEvdG1NpLX9vbQoh8Yuo N980mEpnVu5iQTD4at0SJA07bITj7hx4Q3y4ae5y6F8FxqldG3ECp5X0mf2mnwEEsBrRZd xVMyeOUAXVCkg5GJTKd4478zaLfkjNw= Received: from ramsan.of.borg ([IPv6:2a02:1810:ac12:ed50:614d:21b0:703:d0f9]) by albert.telenet-ops.be with bizsmtp id VUg82900A3mNwr406Ug80J; Tue, 07 Mar 2023 17:40:24 +0100 Received: from geert (helo=localhost) by ramsan.of.borg with local-esmtp (Exim 4.95) (envelope-from ) id 1pZaMC-00BCRZ-9r; Tue, 07 Mar 2023 17:40:08 +0100 Date: Tue, 7 Mar 2023 17:40:08 +0100 (CET) From: Geert Uytterhoeven To: Peter Zijlstra cc: Frederic Weisbecker , Guo Ren , "Rafael J. Wysocki" , Kajetan Puchalski , Tony Lindgren , Ulf Hansson , Ingo Molnar , linux@armlinux.org.uk, linux-imx@nxp.com, linux-kernel@vger.kernel.org, linux-omap@vger.kernel.org, linux-samsung-soc@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-pm@vger.kernel.org, linux-clk@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-tegra@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-renesas-soc@vger.kernel.org Subject: Re: [PATCH v3 07/51] cpuidle,psci: Push RCU-idle into driver In-Reply-To: <20230112195539.760296658@infradead.org> Message-ID: References: <20230112194314.845371875@infradead.org> <20230112195539.760296658@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Rspam-User: X-Rspamd-Queue-Id: 5E80D2000A X-Rspamd-Server: rspam01 X-Stat-Signature: kucwwa3kuy5jswfc6ut7qmaxbu815fmn X-HE-Tag: 1678207226-27337 X-HE-Meta: U2FsdGVkX1+i09ZO+/1vTcI0ee+30V/bU3isKYIjBZO41XL3eVhMwFwIlezpi+HQ04IL0dR3d+mz/RpS0jBec1VrElD3pDMgsQoIjr6USWZb6xooWQdOTf+TAZ3kcFmu4FUA0sMxx8WmI/JMdVBa+y2czlCxfIFCn01huKzOx/x0F+XFgHj+FQKdQK8S+ri7IRcVGt5ckncnoqDG7jMYgXdBR3dMGmZGFwtvIDrK8iwxWfjzEc6SlxCOiTDXHYr/Nzdt8oP3Xi+anFmrbU4KkRCedVBmbxXnjf4ufMMNL1WntMcrX203phjmcnPcfXeKhbRbgqgvApA0SSfXOAyU93BHAI2aI//bYAEGnwyUW/WS2A0cYgWZQuHWidKDV8mnKiWfJ/aPKwQvgaO1Rwcs6J5KpFRzC66f/Gm19z8h6ErOULuIBLXRMEzMbmLpFj5GThpq02pQ2WLVARs2Tx7GHRb3Yl2/ftDJq4xTzzUDmY2uZ4T3VTJImosyOxiffvw/TaG8aWksI51I7faaa8BlFZ84vd8xFjnhxN60jS/h1imEzdnRgeO74mYTKfwuBJVyBBdhylTyskwzfXqnRB4YxNO0VFj/bkhqpc6xDww4s30PfBmoXcmlj+Ink3LTu3zrfgvmVC3T2XMui6+MdPNzMk8D87DIRnQpBew9DNxOHWpqKT+nAthmFJPYO/Vk154FFIqC3bWt0AYn8srPi3xHZBi+BPHit7Wji4yRl7r/QzOgK1CzNxHElB7XFhvhq+gWc3ZcatzQtUYjfBmmNsuYVw1u1w0pPc9lYt+DLlj8sJnRRBQnLYMXjGeL/otkmN4g++tHlQvGzhsyzjtFXR3V995piPGYGAXL151ow+Nq7Vo4lDHfttMtDTiZeAukr2ODg+/QfymVJJQc0SFDNWDCtK8UWy4RrI2Tr4m2tV4ukaT7MLzkmyOyDPHMDqvlQQcrGQlV+CZYy8n8+NeXwVc yHggZghg ++kK28azbbYMS3sbxCofbt5myICKsCeKguwwkUH7bfa21H1KvXlcumiy3XrjkK2OxQFJqni7myv60UJGYLBUyCXKueQGzdkp3rnXK3ReWjaZeO9++lxfLYiepSSQVe4ldjLjl0Xd8NRsVJIZKuB22SxzcDvHIzyfjNeYHSbwepFt0xF2R3k8ibUWQio8Xwx00LGR66wgRyAJjWr7ue1LNuDzVMOiA2HDVG7JUy9ONw0j7N9W7XgoYCEMit+iLKT1cwb9gMuDJEh5zWF4Ixp7bOmZIQKNavJkz+EF80J2Hf+EXgxGO9jEzLiNK2+SIQiC+E27P8a22bv8AltVxzuggCIcrVd6HfM9KqyRSnrCQYO3zC4TLDaQ1smTAPWc5uUzcxmxr3GbMk+fHY0uM70vew6reyBAIKNBNJC3qQ7SxM3ROfcJUIBWDBYcr9xSbIKQcH18jS51PGvIWSf7stXmXwx1+6vbzOw4acf5ERCWY71RQyI0Z4z/yZbw578sMeo/jkths X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hoi Peter, (reduced the insane CC list) On Thu, 12 Jan 2023, Peter Zijlstra wrote: > Doing RCU-idle outside the driver, only to then temporarily enable it > again, at least twice, before going idle is daft. > > Notably once implicitly through the cpu_pm_*() calls and once > explicitly doing ct_irq_*_irqon(). > > Signed-off-by: Peter Zijlstra (Intel) > Reviewed-by: Frederic Weisbecker > Reviewed-by: Guo Ren > Acked-by: Rafael J. Wysocki > Tested-by: Kajetan Puchalski > Tested-by: Tony Lindgren > Tested-by: Ulf Hansson Thanks for your patch, which is now commit e038f7b8028a1d1b ("cpuidle, psci: Push RCU-idle into driver") in v6.3-rc1. I have bisected a PSCI checker regression on Renesas R-Car Gen3/4 SoCs to commit a01353cf1896ea5b ("cpuidle: Fix ct_idle_*() usage") (the 7 commits before that do not compile): psci_checker: PSCI checker started using 2 CPUs psci_checker: Starting hotplug tests psci_checker: Trying to turn off and on again all CPUs psci: CPU0 killed (polled 0 ms) Detected PIPT I-cache on CPU0 CPU0: Booted secondary processor 0x0000000000 [0x411fd073] psci_checker: Trying to turn off and on again group 0 (CPUs 0-1) psci: CPU0 killed (polled 0 ms) Detected PIPT I-cache on CPU0 CPU0: Booted secondary processor 0x0000000000 [0x411fd073] psci_checker: Hotplug tests passed OK psci_checker: Starting suspend tests (10 cycles per state) psci_checker: CPU 0 entering suspend cycles, states 1 through 1 psci_checker: CPU 1 entering suspend cycles, states 1 through 1 ------------[ cut here ]------------ WARNING: CPU: 1 PID: 177 at kernel/context_tracking.c:141 ct_kernel_exit.constprop.0+0xd8/0xf4 Modules linked in: CPU: 1 PID: 177 Comm: psci_suspend_te Not tainted 6.2.0-rc1-salvator-x-00052-ga01353cf1896 #1415 Hardware name: Renesas Salvator-X 2nd version board based on r8a77965 (DT) pstate: 604000c5 (nZCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--) pc : ct_kernel_exit.constprop.0+0xd8/0xf4 lr : ct_kernel_exit.constprop.0+0xc8/0xf4 sp : ffffffc00b73bd30 x29: ffffffc00b73bd30 x28: ffffff807fbadc90 x27: 0000000000000000 x26: 0000000000000000 x25: 0000000000000000 x24: 0000000000000000 x23: ffffff800981e140 x22: 0000000000000001 x21: 0000000000010000 x20: ffffffc0086be1d8 x19: ffffff807fbac070 x18: 0000000000000000 x17: ffffff80083d1000 x16: ffffffc00841fff8 x15: ffffffc00b73b990 x14: ffffffc00895be78 x13: 0000000000000001 x12: 0000000000000000 x11: 00000000000001aa x10: 00000000ffffffea x9 : 000000000000000f x8 : ffffffc00b73bb68 x7 : ffffffc00b73be18 x6 : ffffffc00815ff34 x5 : ffffffc00a6a0c30 x4 : ffffffc00801ce00 x3 : 0000000000000000 x2 : ffffffc008dc3070 x1 : ffffffc008dc3078 x0 : 0000000004208040 Call trace: ct_kernel_exit.constprop.0+0xd8/0xf4 ct_idle_enter+0x18/0x20 psci_enter_idle_state+0xa4/0xfc suspend_test_thread+0x238/0x2f0 kthread+0xd8/0xe8 ret_from_fork+0x10/0x20 irq event stamp: 0 hardirqs last enabled at (0): [<0000000000000000>] 0x0 hardirqs last disabled at (0): [] copy_process+0x608/0x13dc softirqs last enabled at (0): [] copy_process+0x608/0x13dc softirqs last disabled at (0): [<0000000000000000>] 0x0 ---[ end trace 0000000000000000 ]--- ------------[ cut here ]------------ WARNING: CPU: 1 PID: 177 at kernel/context_tracking.c:186 ct_kernel_enter.constprop.0+0x78/0xa4 Modules linked in: CPU: 1 PID: 177 Comm: psci_suspend_te Tainted: G W 6.2.0-rc1-salvator-x-00052-ga01353cf1896 #1415 Hardware name: Renesas Salvator-X 2nd version board based on r8a77965 (DT) pstate: 604000c5 (nZCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--) pc : ct_kernel_enter.constprop.0+0x78/0xa4 lr : ct_kernel_enter.constprop.0+0x68/0xa4 sp : ffffffc00b73bd30 x29: ffffffc00b73bd30 x28: ffffff807fbadc90 x27: 0000000000000000 x26: 0000000000000000 x25: 0000000000000000 x24: 0000000000000000 x23: ffffff800981e140 x22: 0000000000000001 x21: 00000000ffffffa1 x20: ffffffc0086be1d8 x19: 00000000000000c0 x18: 0000000000000000 x17: ffffff80083d1000 x16: ffffffc00841fff8 x15: ffffffc00b73b990 x14: ffffffc00895be78 x13: ffffff800e325180 x12: ffffffc076de9000 x11: 0000000034d4d91d x10: 0000000000000008 x9 : 0000000000001000 x8 : ffffffc008012800 x7 : 0000000000000000 x6 : ffffff807fbac070 x5 : ffffffc008dc3070 x4 : 0000000000000000 x3 : 000000000001a9fc x2 : 0000000000000003 x1 : ffffffc008dc3070 x0 : 0000000004208040 Call trace: ct_kernel_enter.constprop.0+0x78/0xa4 ct_idle_exit+0x18/0x38 psci_enter_idle_state+0xdc/0xfc suspend_test_thread+0x238/0x2f0 kthread+0xd8/0xe8 ret_from_fork+0x10/0x20 irq event stamp: 0 hardirqs last enabled at (0): [<0000000000000000>] 0x0 hardirqs last disabled at (0): [] copy_process+0x608/0x13dc softirqs last enabled at (0): [] copy_process+0x608/0x13dc softirqs last disabled at (0): [<0000000000000000>] 0x0 ---[ end trace 0000000000000000 ]--- psci_checker: Failed to suspend CPU 1: error -1 (requested state 1, cycle 0) psci_checker: CPU 0 suspend test results: success 0, shallow states 10, errors 0 mmcblk0rpmb: mmc0:0001 BGSD3R 4.00 MiB, chardev (243:0) psci_checker: CPU 1 suspend test results: success 0, shallow states 9, errors 1 psci_checker: 1 error(s) encountered in suspend tests psci_checker: PSCI checker completed > --- > drivers/cpuidle/cpuidle-psci.c | 9 +++++---- > 1 file changed, 5 insertions(+), 4 deletions(-) > > --- a/drivers/cpuidle/cpuidle-psci.c > +++ b/drivers/cpuidle/cpuidle-psci.c > @@ -69,12 +69,12 @@ static int __psci_enter_domain_idle_stat > return -1; > > /* Do runtime PM to manage a hierarchical CPU toplogy. */ > - ct_irq_enter_irqson(); > if (s2idle) > dev_pm_genpd_suspend(pd_dev); > else > pm_runtime_put_sync_suspend(pd_dev); > - ct_irq_exit_irqson(); > + > + ct_idle_enter(); > > state = psci_get_domain_state(); > if (!state) > @@ -82,12 +82,12 @@ static int __psci_enter_domain_idle_stat > > ret = psci_cpu_suspend_enter(state) ? -1 : idx; > > - ct_irq_enter_irqson(); > + ct_idle_exit(); > + > if (s2idle) > dev_pm_genpd_resume(pd_dev); > else > pm_runtime_get_sync(pd_dev); > - ct_irq_exit_irqson(); > > cpu_pm_exit(); > > @@ -240,6 +240,7 @@ static int psci_dt_cpu_init_topology(str > * of a shared state for the domain, assumes the domain states are all > * deeper states. > */ > + drv->states[state_count - 1].flags |= CPUIDLE_FLAG_RCU_IDLE; > drv->states[state_count - 1].enter = psci_enter_domain_idle_state; > drv->states[state_count - 1].enter_s2idle = psci_enter_s2idle_domain_idle_state; > psci_cpuidle_use_cpuhp = true; Gr{oetje,eeting}s, Geert -- Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds