From: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org> To: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Cc: "Rafael J. Wysocki" <rafael@kernel.org>, Len Brown <len.brown@intel.com>, Pavel Machek <pavel@ucw.cz>, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, Kevin Hilman <khilman@kernel.org>, Ulf Hansson <ulf.hansson@linaro.org>, Daniel Lezcano <daniel.lezcano@linaro.org>, Lorenzo Pieralisi <lpieralisi@kernel.org>, Sudeep Holla <sudeep.holla@arm.com>, linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Adrien Thierry <athierry@redhat.com>, Brian Masney <bmasney@redhat.com>, linux-rt-users@vger.kernel.org Subject: Re: [PATCH v2 5/5] PM: domains: Do not call device_pm_check_callbacks() when holding genpd_lock() Date: Thu, 12 Jan 2023 12:37:26 +0100 [thread overview] Message-ID: <4562c5c8-e2bd-4ade-55df-dab07540ce19@linaro.org> (raw) In-Reply-To: <Y7/vdnD8qELX4A5m@linutronix.de> On 12/01/2023 12:31, Sebastian Andrzej Siewior wrote: > On 2022-12-19 16:15:03 [+0100], Krzysztof Kozlowski wrote: >> If PM domain on PREEMPT_RT is marked as GENPD_FLAG_RT_SAFE(), the >> genpd_lock() will be a raw spin lock, thus device_pm_check_callbacks() > > a raw_spinlock_t > >> must be called outside of the domain lock. > > Right. First the sleeping lock, followed by the spinning locks. This is > covered in > Documentation/locking/locktypes.rst > > at the end. I don't understand your comment. Do you expect me to change something? > >> This solves on PREEMPT_RT: > Yes but >> [ BUG: Invalid wait context ] > > This "Invalid wait context" should also trigger on !PREEMPT_RT with > CONFIG_PROVE_RAW_LOCK_NESTING. Could be, I just did not hit it. > >> 6.1.0-rt5-00325-g8a5f56bcfcca #8 Tainted: G W >> ----------------------------- >> swapper/0/1 is trying to lock: >> ffff76e045dec9a0 (&dev->power.lock){+.+.}-{3:3}, at: device_pm_check_callbacks+0x20/0xf0 >> other info that might help us debug this: >> context-{5:5} >> 3 locks held by swapper/0/1: >> #0: ffff76e045deb8e8 (&dev->mutex){....}-{4:4}, at: __device_attach+0x38/0x1c0 >> #1: ffffa92b81f825e0 (gpd_list_lock){+.+.}-{4:4}, at: __genpd_dev_pm_attach+0x7c/0x250 >> #2: ffff76e04105c7a0 (&genpd->rslock){....}-{2:2}, at: genpd_lock_rawspin+0x1c/0x30 >> stack backtrace: >> CPU: 5 PID: 1 Comm: swapper/0 Tainted: G W 6.1.0-rt5-00325-g8a5f56bcfcca #8 >> Hardware name: Qualcomm Technologies, Inc. Robotics RB5 (DT) >> Call trace: >> dump_backtrace.part.0+0xe0/0xf0 >> show_stack+0x18/0x40 >> dump_stack_lvl+0x8c/0xb8 >> dump_stack+0x18/0x34 >> __lock_acquire+0x938/0x2100 >> lock_acquire.part.0+0x104/0x28c >> lock_acquire+0x68/0x84 >> rt_spin_lock+0x40/0x100 >> device_pm_check_callbacks+0x20/0xf0 >> dev_pm_domain_set+0x54/0x64 >> genpd_add_device+0x258/0x340 >> __genpd_dev_pm_attach+0xa8/0x250 >> genpd_dev_pm_attach_by_id+0xc4/0x190 >> genpd_dev_pm_attach_by_name+0x3c/0x60 >> dev_pm_domain_attach_by_name+0x20/0x30 >> dt_idle_attach_cpu+0x24/0x90 >> psci_cpuidle_probe+0x300/0x4b0 >> platform_probe+0x68/0xe0 >> really_probe+0xbc/0x2dc >> __driver_probe_device+0x78/0xe0 >> driver_probe_device+0x3c/0x160 >> __device_attach_driver+0xb8/0x140 >> bus_for_each_drv+0x78/0xd0 >> __device_attach+0xa8/0x1c0 >> device_initial_probe+0x14/0x20 >> bus_probe_device+0x9c/0xa4 >> device_add+0x3b4/0x8dc >> platform_device_add+0x114/0x234 >> platform_device_register_full+0x108/0x1a4 >> psci_idle_init+0x6c/0xb0 >> do_one_initcall+0x74/0x450 >> kernel_init_freeable+0x2e0/0x350 >> kernel_init+0x24/0x130 >> ret_from_fork+0x10/0x20 > > I would prefer a description of the issue instead hacing this > backtrace. I'll extend the commit msg. > >> Cc: Adrien Thierry <athierry@redhat.com> >> Cc: Brian Masney <bmasney@redhat.com> >> Cc: linux-rt-users@vger.kernel.org >> Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org> >> --- >> drivers/base/power/domain.c | 6 +++++- >> 1 file changed, 5 insertions(+), 1 deletion(-) >> >> diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c >> index 4dfce1d476f4..db499ba40497 100644 >> --- a/drivers/base/power/domain.c >> +++ b/drivers/base/power/domain.c >> @@ -1666,10 +1666,14 @@ static int genpd_add_device(struct generic_pm_domain *genpd, struct device *dev, >> if (ret) >> goto out; >> >> + >> + /* PREEMPT_RT: Must be outside of genpd_lock */ > > Could this comment be rewritten if needed? > The callback, which acquires sleeping locks on PREEMPT_RT, can be moved > before genpd_lock() because it does not rely on this lock. It must be > moved because the latter may acquire spinning locks. Sure Best regards, Krzysztof _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
WARNING: multiple messages have this Message-ID (diff)
From: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org> To: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Cc: "Rafael J. Wysocki" <rafael@kernel.org>, Len Brown <len.brown@intel.com>, Pavel Machek <pavel@ucw.cz>, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, Kevin Hilman <khilman@kernel.org>, Ulf Hansson <ulf.hansson@linaro.org>, Daniel Lezcano <daniel.lezcano@linaro.org>, Lorenzo Pieralisi <lpieralisi@kernel.org>, Sudeep Holla <sudeep.holla@arm.com>, linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Adrien Thierry <athierry@redhat.com>, Brian Masney <bmasney@redhat.com>, linux-rt-users@vger.kernel.org Subject: Re: [PATCH v2 5/5] PM: domains: Do not call device_pm_check_callbacks() when holding genpd_lock() Date: Thu, 12 Jan 2023 12:37:26 +0100 [thread overview] Message-ID: <4562c5c8-e2bd-4ade-55df-dab07540ce19@linaro.org> (raw) In-Reply-To: <Y7/vdnD8qELX4A5m@linutronix.de> On 12/01/2023 12:31, Sebastian Andrzej Siewior wrote: > On 2022-12-19 16:15:03 [+0100], Krzysztof Kozlowski wrote: >> If PM domain on PREEMPT_RT is marked as GENPD_FLAG_RT_SAFE(), the >> genpd_lock() will be a raw spin lock, thus device_pm_check_callbacks() > > a raw_spinlock_t > >> must be called outside of the domain lock. > > Right. First the sleeping lock, followed by the spinning locks. This is > covered in > Documentation/locking/locktypes.rst > > at the end. I don't understand your comment. Do you expect me to change something? > >> This solves on PREEMPT_RT: > Yes but >> [ BUG: Invalid wait context ] > > This "Invalid wait context" should also trigger on !PREEMPT_RT with > CONFIG_PROVE_RAW_LOCK_NESTING. Could be, I just did not hit it. > >> 6.1.0-rt5-00325-g8a5f56bcfcca #8 Tainted: G W >> ----------------------------- >> swapper/0/1 is trying to lock: >> ffff76e045dec9a0 (&dev->power.lock){+.+.}-{3:3}, at: device_pm_check_callbacks+0x20/0xf0 >> other info that might help us debug this: >> context-{5:5} >> 3 locks held by swapper/0/1: >> #0: ffff76e045deb8e8 (&dev->mutex){....}-{4:4}, at: __device_attach+0x38/0x1c0 >> #1: ffffa92b81f825e0 (gpd_list_lock){+.+.}-{4:4}, at: __genpd_dev_pm_attach+0x7c/0x250 >> #2: ffff76e04105c7a0 (&genpd->rslock){....}-{2:2}, at: genpd_lock_rawspin+0x1c/0x30 >> stack backtrace: >> CPU: 5 PID: 1 Comm: swapper/0 Tainted: G W 6.1.0-rt5-00325-g8a5f56bcfcca #8 >> Hardware name: Qualcomm Technologies, Inc. Robotics RB5 (DT) >> Call trace: >> dump_backtrace.part.0+0xe0/0xf0 >> show_stack+0x18/0x40 >> dump_stack_lvl+0x8c/0xb8 >> dump_stack+0x18/0x34 >> __lock_acquire+0x938/0x2100 >> lock_acquire.part.0+0x104/0x28c >> lock_acquire+0x68/0x84 >> rt_spin_lock+0x40/0x100 >> device_pm_check_callbacks+0x20/0xf0 >> dev_pm_domain_set+0x54/0x64 >> genpd_add_device+0x258/0x340 >> __genpd_dev_pm_attach+0xa8/0x250 >> genpd_dev_pm_attach_by_id+0xc4/0x190 >> genpd_dev_pm_attach_by_name+0x3c/0x60 >> dev_pm_domain_attach_by_name+0x20/0x30 >> dt_idle_attach_cpu+0x24/0x90 >> psci_cpuidle_probe+0x300/0x4b0 >> platform_probe+0x68/0xe0 >> really_probe+0xbc/0x2dc >> __driver_probe_device+0x78/0xe0 >> driver_probe_device+0x3c/0x160 >> __device_attach_driver+0xb8/0x140 >> bus_for_each_drv+0x78/0xd0 >> __device_attach+0xa8/0x1c0 >> device_initial_probe+0x14/0x20 >> bus_probe_device+0x9c/0xa4 >> device_add+0x3b4/0x8dc >> platform_device_add+0x114/0x234 >> platform_device_register_full+0x108/0x1a4 >> psci_idle_init+0x6c/0xb0 >> do_one_initcall+0x74/0x450 >> kernel_init_freeable+0x2e0/0x350 >> kernel_init+0x24/0x130 >> ret_from_fork+0x10/0x20 > > I would prefer a description of the issue instead hacing this > backtrace. I'll extend the commit msg. > >> Cc: Adrien Thierry <athierry@redhat.com> >> Cc: Brian Masney <bmasney@redhat.com> >> Cc: linux-rt-users@vger.kernel.org >> Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org> >> --- >> drivers/base/power/domain.c | 6 +++++- >> 1 file changed, 5 insertions(+), 1 deletion(-) >> >> diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c >> index 4dfce1d476f4..db499ba40497 100644 >> --- a/drivers/base/power/domain.c >> +++ b/drivers/base/power/domain.c >> @@ -1666,10 +1666,14 @@ static int genpd_add_device(struct generic_pm_domain *genpd, struct device *dev, >> if (ret) >> goto out; >> >> + >> + /* PREEMPT_RT: Must be outside of genpd_lock */ > > Could this comment be rewritten if needed? > The callback, which acquires sleeping locks on PREEMPT_RT, can be moved > before genpd_lock() because it does not rely on this lock. It must be > moved because the latter may acquire spinning locks. Sure Best regards, Krzysztof
next prev parent reply other threads:[~2023-01-12 11:38 UTC|newest] Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-12-19 15:14 [PATCH v2 0/5] PM: Fixes for Realtime systems Krzysztof Kozlowski 2022-12-19 15:14 ` Krzysztof Kozlowski 2022-12-19 15:14 ` [PATCH v2 1/5] PM: domains: Add GENPD_FLAG_RT_SAFE for PREEMPT_RT Krzysztof Kozlowski 2022-12-19 15:14 ` Krzysztof Kozlowski 2023-01-04 15:45 ` Ulf Hansson 2023-01-04 15:45 ` Ulf Hansson 2023-01-06 14:52 ` Krzysztof Kozlowski 2023-01-06 14:52 ` Krzysztof Kozlowski 2023-01-12 10:36 ` Sebastian Andrzej Siewior 2023-01-12 10:36 ` Sebastian Andrzej Siewior 2023-01-12 11:27 ` Krzysztof Kozlowski 2023-01-12 11:27 ` Krzysztof Kozlowski 2023-01-12 10:32 ` Sebastian Andrzej Siewior 2023-01-12 10:32 ` Sebastian Andrzej Siewior 2023-01-12 11:31 ` Krzysztof Kozlowski 2023-01-12 11:31 ` Krzysztof Kozlowski 2022-12-19 15:15 ` [PATCH v2 2/5] cpuidle: psci: Mark as PREEMPT_RT safe Krzysztof Kozlowski 2022-12-19 15:15 ` Krzysztof Kozlowski 2023-01-12 11:00 ` Sebastian Andrzej Siewior 2023-01-12 11:00 ` Sebastian Andrzej Siewior 2023-01-12 11:32 ` Krzysztof Kozlowski 2023-01-12 11:32 ` Krzysztof Kozlowski 2023-01-17 15:27 ` Ulf Hansson 2023-01-17 15:27 ` Ulf Hansson 2023-01-19 15:40 ` Krzysztof Kozlowski 2023-01-19 15:40 ` Krzysztof Kozlowski 2023-01-19 17:06 ` Krzysztof Kozlowski 2023-01-19 17:06 ` Krzysztof Kozlowski 2022-12-19 15:15 ` [PATCH v2 3/5] cpuidle: psci: Do not suspend topology CPUs on PREEMPT_RT Krzysztof Kozlowski 2022-12-19 15:15 ` Krzysztof Kozlowski 2023-01-12 11:09 ` Sebastian Andrzej Siewior 2023-01-12 11:09 ` Sebastian Andrzej Siewior 2023-01-12 11:34 ` Krzysztof Kozlowski 2023-01-12 11:34 ` Krzysztof Kozlowski 2023-01-30 9:51 ` Sebastian Andrzej Siewior 2023-01-30 9:51 ` Sebastian Andrzej Siewior 2022-12-19 15:15 ` [PATCH v2 4/5] PM: Allow calling dev_pm_domain_set() with raw spinlock Krzysztof Kozlowski 2022-12-19 15:15 ` Krzysztof Kozlowski 2023-01-12 11:13 ` Sebastian Andrzej Siewior 2023-01-12 11:13 ` Sebastian Andrzej Siewior 2022-12-19 15:15 ` [PATCH v2 5/5] PM: domains: Do not call device_pm_check_callbacks() when holding genpd_lock() Krzysztof Kozlowski 2022-12-19 15:15 ` Krzysztof Kozlowski 2023-01-12 11:31 ` Sebastian Andrzej Siewior 2023-01-12 11:31 ` Sebastian Andrzej Siewior 2023-01-12 11:37 ` Krzysztof Kozlowski [this message] 2023-01-12 11:37 ` Krzysztof Kozlowski 2023-01-17 15:11 ` Ulf Hansson 2023-01-17 15:11 ` Ulf Hansson 2023-01-19 15:58 ` Krzysztof Kozlowski 2023-01-19 15:58 ` Krzysztof Kozlowski 2022-12-20 21:36 ` [PATCH v2 0/5] PM: Fixes for Realtime systems Adrien Thierry 2022-12-20 21:36 ` Adrien Thierry 2023-01-04 15:15 ` Ulf Hansson 2023-01-04 15:15 ` Ulf Hansson
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=4562c5c8-e2bd-4ade-55df-dab07540ce19@linaro.org \ --to=krzysztof.kozlowski@linaro.org \ --cc=athierry@redhat.com \ --cc=bigeasy@linutronix.de \ --cc=bmasney@redhat.com \ --cc=daniel.lezcano@linaro.org \ --cc=gregkh@linuxfoundation.org \ --cc=khilman@kernel.org \ --cc=len.brown@intel.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-pm@vger.kernel.org \ --cc=linux-rt-users@vger.kernel.org \ --cc=lpieralisi@kernel.org \ --cc=pavel@ucw.cz \ --cc=rafael@kernel.org \ --cc=sudeep.holla@arm.com \ --cc=ulf.hansson@linaro.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.