From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20B01C3DA78 for ; Tue, 17 Jan 2023 15:35:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8FC386B007B; Tue, 17 Jan 2023 10:35:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 884766B007D; Tue, 17 Jan 2023 10:35:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6D7416B007E; Tue, 17 Jan 2023 10:35:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 5D20D6B007B for ; Tue, 17 Jan 2023 10:35:39 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 31AA316058A for ; Tue, 17 Jan 2023 15:35:39 +0000 (UTC) X-FDA: 80364690798.26.3212BAE Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf23.hostedemail.com (Postfix) with ESMTP id 9273F140013 for ; Tue, 17 Jan 2023 15:35:35 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of mark.rutland@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=mark.rutland@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1673969736; a=rsa-sha256; cv=none; b=jbmzq3gk/yBwiII0hzcbWZ9aQGFeg2R6uTwothAKZtUyqSYPv4tyYWFWq34qLlMV/MEOP/ +s67/gndXnvQnqcACnAhFz7Aloyn3GC2qdIwiVLTmMwfeUK7fz+wtepxkvKqvdunxYbOV4 MnLYp1K13VGX6mjWbUHBy8+v5ggsRxU= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of mark.rutland@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=mark.rutland@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1673969736; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zRMGEyi2M8CFImB7lq5BYYcUkmwgPVDf60Xa15DMOao=; b=PNhSpWR1CcbWwe41hvne3ZrMExdJ3FDvhL2DrF9vin+2WH+UUF3oHpu2nefbSiSR/sxYx4 xSwhWZY6Igg4ucrJDnoZuEVKFgf6jk/fGLVSWeNw6GrEkDZfOZarGUCfLMKBE3UOgmQqCP O0Qw3kAXbQwuFopdAX2/hSSw+bBleG0= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 94F4F12FC; Tue, 17 Jan 2023 07:36:16 -0800 (PST) Received: from FVFF77S0Q05N.cambridge.arm.com (FVFF77S0Q05N.cambridge.arm.com [10.1.31.153]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id ACA663F67D; Tue, 17 Jan 2023 07:35:17 -0800 (PST) Date: Tue, 17 Jan 2023 15:35:10 +0000 From: Mark Rutland To: Sudeep Holla Cc: Peter Zijlstra , richard.henderson@linaro.org, ink@jurassic.park.msu.ru, mattst88@gmail.com, vgupta@kernel.org, linux@armlinux.org.uk, nsekhar@ti.com, brgl@bgdev.pl, ulli.kroll@googlemail.com, linus.walleij@linaro.org, shawnguo@kernel.org, Sascha Hauer , kernel@pengutronix.de, festevam@gmail.com, linux-imx@nxp.com, tony@atomide.com, khilman@kernel.org, krzysztof.kozlowski@linaro.org, alim.akhtar@samsung.com, catalin.marinas@arm.com, will@kernel.org, guoren@kernel.org, bcain@quicinc.com, chenhuacai@kernel.org, kernel@xen0n.name, geert@linux-m68k.org, sammy@sammy.net, monstr@monstr.eu, tsbogend@alpha.franken.de, dinguyen@kernel.org, jonas@southpole.se, stefan.kristiansson@saunalahti.fi, shorne@gmail.com, James.Bottomley@hansenpartnership.com, deller@gmx.de, mpe@ellerman.id.au, npiggin@gmail.com, christophe.leroy@csgroup.eu, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, borntraeger@linux.ibm.com, svens@linux.ibm.com, ysato@users.sourceforge.jp, dalias@libc.org, davem@davemloft.net, richard@nod.at, anton.ivanov@cambridgegreys.com, johannes@sipsolutions.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, acme@kernel.org, alexander.shishkin@linux.intel.com, jolsa@kernel.org, namhyung@kernel.org, jgross@suse.com, srivatsa@csail.mit.edu, amakhalov@vmware.com, pv-drivers@vmware.com, boris.ostrovsky@oracle.com, chris@zankel.net, jcmvbkbc@gmail.com, rafael@kernel.org, lenb@kernel.org, pavel@ucw.cz, gregkh@linuxfoundation.org, mturquette@baylibre.com, sboyd@kernel.org, daniel.lezcano@linaro.org, lpieralisi@kernel.org, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, anup@brainfault.org, thierry.reding@gmail.com, jonathanh@nvidia.com, jacob.jun.pan@linux.intel.com, atishp@atishpatra.org, Arnd Bergmann , yury.norov@gmail.com, andriy.shevchenko@linux.intel.com, linux@rasmusvillemoes.dk, dennis@kernel.org, tj@kernel.org, cl@linux.com, rostedt@goodmis.org, mhiramat@kernel.org, frederic@kernel.org, paulmck@kernel.org, pmladek@suse.com, senozhatsky@chromium.org, john.ogness@linutronix.de, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, ryabinin.a.a@gmail.com, glider@google.com, andreyknvl@gmail.com, dvyukov@google.com, vincenzo.frascino@arm.com, Andrew Morton , jpoimboe@kernel.org, linux-alpha@vger.kernel.org, linux-kernel@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-omap@vger.kernel.org, linux-samsung-soc@vger.kernel.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-ia64@vger.kernel.org, loongarch@lists.linux.dev, linux-m68k@lists.linux-m68k.org, linux-mips@vger.kernel.org, openrisc@lists.librecores.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, linux-um@lists.infradead.org, linux-perf-users@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-xtensa@linux-xtensa.org, linux-acpi@vger.kernel.org, linux-pm@vger.kernel.org, linux-clk@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-tegra@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org, kasan-dev@googlegroups.com Subject: Re: [PATCH v3 00/51] cpuidle,rcu: Clean up the mess Message-ID: References: <20230112194314.845371875@infradead.org> <20230117142140.g423hxisv7djudof@bogus> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230117142140.g423hxisv7djudof@bogus> X-Rspam-User: X-Rspamd-Queue-Id: 9273F140013 X-Rspamd-Server: rspam01 X-Stat-Signature: 4ff6gcr5qz7xpq555e8j31sxdjonn4wi X-HE-Tag: 1673969735-722316 X-HE-Meta: U2FsdGVkX18LibOYVsGPpVPLMFpDB5SD6KQQckPSQx6Ic+eY8qj/UqSvIIAfGHCQMSn9EvwkeCwW2J72RtJgv9jcjxQxnfJeJ6xDxdBsz5aRdote0Z9fVb4dixrkTZLglajzu93mkWCCrN/snvSnaw6Q6eUKe5v2YpEXzou44Bk9CsodzuVExXUF6igntCBMegyZLHc6WzuxnfATibVR1dMgvwb65fZzyOAvks0YVxRomv2/ybCg6t/q0XiBLiqqTlbX6iMSaktJr0bMtxsC7nlK7i2ZW73U7LGDqD8epdGt6wVlmKM2ezbgJn2HF64V2mNhq4HqXmErRRiUdLxHexH3RGq2i3n6gVV1Rd0jhVXFvQmf7hMTxrwUU6KV+s/KHy7Mz+FFwX/XcVVSGmL24eMxItDzc+okVayAOetue8u5uov9GtXxTHftYjfWrO+Ewmnpb/afWZzAPafiMhvT6v1ZqwQrOYCuPGIaWRf41AU0troWPD3bGOeXPCQVwSqpwsj8QL7GOtpiHM5TUcoW5tg8VswY0bsvod+b/nn4IFC5UpqwowKCr8YuSLbXYQ7p12BODltCPBCi8bNS9vQvRAt4XkZ7OyIJbbXcOTcK6Hgtazy1PHxV0WpRv+Y882hI1BaaNPdfgb2bNJVjg6VcxK2deIYP8MmDQ2vYOUZnoxx+j7yIAlYYet5rmn28bt0GLW9guyc6QBwApPC5zmYzF3AgbAYo6Lo24QetWx6EEjMDoG/Ce2FPHN5IZJcdMu0qfmrMAg60BIxFtsUnF+9o2tjG0aK0OBgu1If2et2P6z6VUKUvDE3MYjs58YFCrmRGLkxYUGeND9tFS6ujEUT3xtq7sMIgScLVkq75+XASZADe/kOb6i5viAiCMHrBQ7h+oIYMWGIHJxkUiQJ+sI6CKa5nCf1lq9K14OwOwauLTbuRc3SSW8ULFy1J04O2GPl0CecmWfFCtH9kKoh6akE MpkLe4Mh 6b8BQuMLheAbOIEcWuW7NQxVWQzDzHF+Yy8VM0ueCWGerb7da8lkN27n4AJPeez6HyL5MxwF9lB6zPinwXbwtUUi0mgSmRRBh1LiKQQCmuirVffU6lkL14DaEbV1DWDGjbC0BObCBHv1gwvp5ubPE68tZx0HxIhHm1ByADYkn9v7KHXQWGLlOV0L/LHBAxHxM/d7qGg0UcKK7PB+8gUdfr8zoUBlSWBkmByMBPSgwnA420xZYbQoRyBj6z31CM2X5Ids1KOpctV4EoS6nLbLoyD/fnd+qQYgo+IA+vJfrnSVvZmYKLz8YCl4cmyxa+gqpPtq5+MiQXUSdKA26ec6excPyV7WF0z01aRvj7V5dNuIchm2lFZIwU+FkzDmu9XOo7Kljlcf8umkemapBhqfUUbvKBl+fFQpqh+urjMCwxCidREtmovgH3ZZ8wRC8RCKjp2ITdkq9wo2ynN+A4I47KCilA5ibnpGh3QF/DyIe0T9sLVDQ1WMgRULW3FG9NcKak27LVt7K0pEJND+g1iw/5FarHEu06NFD33Qt3iU1wZY+TPPkCFMsvypYtA/7QPFEKAlBxbkjkFpKhuCxAuqp+QJfoNbHBclJqELbrFPsu5h9aYLKntyzPMCnNn7O1kyhvMHRbYYXONpDUHYYmN98ZndKJcZEQZ0zMM2YZ36khDbx1ZWvhFBZnn8Xc6v5dJbpkeQbQ8kCB3DeBnxkSrKVP/td0BdPFwOoI6I+Gb9snCgluERIbM+qhvy5RFBR1ztacQKRqz0EfX8tZMIKseuEncECwY1lnKzTazy87+mq1A5lfwNHiQxRQwiT9pKzTsyvnqvzZd9QZidfuh8J+Y53XtZXC2sqKIufyRHcbehDYnSLHSKp+ZJUDevecpnomLAzBnTLWznmkJobMopXJ8W91+AwEX6BI8HoDbtS2MUsLXU3HcWabOtyBV0MWufUZrpj4xYHo4MFC8V4Bq19Os5m3cz0NLyq xMxcjWuZ DXR8UElaIPCQU5TTNpG2xnwm7QAxUmYeFHFPYLi2xfMfM6HskRYbiZIYJMV3vyBoB1OS4M/Z9C3gxaiO9t8A2LezEbBRaxeDGb3m/OFEDUGl7SyGHmMIrDty9biKcYyHmyKhp+ceZOZ2c4iue/g7ZoBtQWC6YBppHcniCPLm5BbN4o7KhQT552VqVqlxhF+w6UaOfuMb0UtFIhPsuGZQ9LWiEJK1zZB/gGPIKRN58rz56+KnHfOnQp+jTAj+n0m0gLX/WijLArVpArYv1z4MA6dIiWJ9QHXfiz/DG9cMY42OnQBRbsFNrDz/fppDPC95Nu0//UVnK/Yz5hOKRQ5XBMjEi3aLUmYfw5vBWqDzZHX4lhtjaVr0rK8XPpNynmvazkc4G5A92KqHNswGgd/AjPgl30ir5J8l08IDN5bikdi5/6siKQcrhWUhq5wM5+997nFbySP4M5qdOfwS5iITcLDJEdT6yCehIjF4x4PmbUH7Rp+RU89IGmG90NLx+OCWVxSMI0aSxaHM5UIbBNLK87RFMHnVytZS3J9Biy2mC7R3ChT3NnfW9BC1gZUwPenxuyS50V6FYdkoa29coj/KkGRFc0S8udX+bmPMgq8fT/ae7yCVxJJuiMX9HarqvskAGd/BRah/s0cXIIyhPdO/Rp0D5+B7+gDNKjKdUpXpQZZl/IU5Vrdx8pACtd9oLvNYKFji0u0KbPeJ2QsbWTcfs1QH62j9c48rbHVqubZGGd90pUd2rJDbaeZfGI3lWk41UEnvuWZoZKeirnSioxV2oAG5iuOCy5rEw4pUKfodjRjl+VhYrI+a0lPZeVVUCVUzz0F8V0y58X7BarKm/esDncDlVmQGbkXFUbyFJKHAxwAls2eiSkDygW+tjDR4AxxXsigA5QVxdx+eA1OXSDvizd6ZbBHOReIfCqXKTs09n5f1ZcRL9XGxmbj1f8Hn2oqnY/zhcbJ4KyDM1CUwM4Tj7Srxjcz5F C0DiaYWt mXQlADnhmlosDoi79P6ovugSJ39iJ+L7AqALSEI8FBQWw66uiT4Obv6ololR3yCl5iADwaWvstd8e6XmCpHROKmDKS3k03/5OHjwH7Yoa+qckf0CtpHDzjzHqIMTQZo5qC7Zf3zWyRFXwcPVn5BHHoMMiX/NwgZR38DMKLkiDd/Q5H+XzBUZwUIOn1qwQn/EX22DOjrevDs/NyCqXzRTN62OYXTyTw7eYtiVsdDV+W4PQx7ipV5m48pBRqrVXDED6E37d9CtSsYsOJpbN+bfWbpCHPuCpcSxFhHfBWKOfo3YAofSUjye7gSiQe77w3RL6FAkNoNCOb9m1lyUeRrnNoTIwL7ZOpwMjhTdhK13Tg3zf7yBuvnZMNyRnilBBW6NR4VkTKB+sabyWal6LhnDOs7z/JmF/rkT2+eJfc0XbmVyTCqGKWJUPszz52WiQAF+Xgm3qc41NY1GEFx2GlwtTY3Znt8tbk4VzG4Rh3aU60oR8lX/FlWmKVuUMu71NIc6OZbNgspgqYfHBj6N96AiBh8BfDQWvBXyNCwr7GfiqyWMvnZVFVjNgihLyc2T55oZ1e294Q7T73sMYdAyXrOKEKMJsgfky2B/whOC1t/iIaWvVhfVJjVMxsRjAwtHMQDC4ySwfmnPQLJ+pw9vm4xmWAg9/67aVGW4DgS4gAvJ7UuXJy66b+oP48upANcHjkWbBpxXHECO8Tc4cxfmq1OL2Vxl2PT2HuognmumZJoaQkeOOPHEP+BsxZoYRumS8VV5VS4Iqt4/WHOoCe61A1Pa1YabkXXr0HjYFgXQnjXSVhOBkmSXMn6WfN5jbu0CnZsbKSjWaJ5PrktI5MpA1YX9+f/DrkyWGOpU5QxQwovooda+3ZY2z8Sl4NQF1rtry8wTjUYw/Y3zGH6CrSJNZE3FTlEgklTZwCpMftm37mnfkONWMBV9RuKEgP15i45zTKvVswD3W+fRsxEJcN6jVFSDoe/l7Km3a yqLpM0iS sbuF0Fb7m2MIJB8aC6/12X0g6+v/0jSSZ/M3hkSAUtX7hUkZTY19fKagv/1dlMgwmy0tBTZ3WXxnRE892nNNEas9PfO6kGPIvESknzkW+euYRrIHDOseOCQzXfbU0C3X2+3caPCshhVg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000025, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Jan 17, 2023 at 02:21:40PM +0000, Sudeep Holla wrote: > On Tue, Jan 17, 2023 at 01:16:21PM +0000, Mark Rutland wrote: > > On Tue, Jan 17, 2023 at 11:26:29AM +0100, Peter Zijlstra wrote: > > > On Mon, Jan 16, 2023 at 04:59:04PM +0000, Mark Rutland wrote: > > > > > > > I'm sorry to have to bear some bad news on that front. :( > > > > > > Moo, something had to give.. > > > > > > > > > > IIUC what's happenign here is the PSCI cpuidle driver has entered idle and RCU > > > > is no longer watching when arm64's cpu_suspend() manipulates DAIF. Our > > > > local_daif_*() helpers poke lockdep and tracing, hence the call to > > > > trace_hardirqs_off() and the RCU usage. > > > > > > Right, strictly speaking not needed at this point, IRQs should have been > > > traced off a long time ago. > > > > True, but there are some other calls around here that *might* end up invoking > > RCU stuff (e.g. the MTE code). > > > > That all needs a noinstr cleanup too, which I'll sort out as a follow-up. > > > > > > I think we need RCU to be watching all the way down to cpu_suspend(), and it's > > > > cpu_suspend() that should actually enter/exit idle context. That and we need to > > > > make cpu_suspend() and the low-level PSCI invocation noinstr. > > > > > > > > I'm not sure whether 32-bit will have a similar issue or not. > > > > > > I'm not seeing 32bit or Risc-V have similar issues here, but who knows, > > > maybe I missed somsething. > > > > I reckon if they do, the core changes here give us the infrastructure to fix > > them if/when we get reports. > > > > > In any case, the below ought to cure the ARM64 case and remove that last > > > known RCU_NONIDLE() user as a bonus. > > > > The below works for me testing on a Juno R1 board with PSCI, using defconfig + > > CONFIG_PROVE_LOCKING=y + CONFIG_DEBUG_LOCKDEP=y + CONFIG_DEBUG_ATOMIC_SLEEP=y. > > I'm not sure how to test the LPI / FFH part, but it looks good to me. > > > > FWIW: > > > > Reviewed-by: Mark Rutland > > Tested-by: Mark Rutland > > > > Sudeep, would you be able to give the LPI/FFH side a spin with the kconfig > > options above? > > > > Not sure if I have messed up something in my mail setup, but I did reply > earlier. Sorry, that was my bad; I had been drafting my reply for a while and forgot to re-check prior to sending. > I did test both DT/cpuidle-psci driver and ACPI/LPI+FFH driver > with the fix Peter sent. I was seeing same splat as you in both DT and > ACPI boot which the patch fixed it. I used the same config as described by > you above. Perfect; thanks! Mark.