* [PATCH 0/3] PM: domains: Improvements for performance states in genpd @ 2021-09-02 10:16 Ulf Hansson 2021-09-02 10:16 ` [PATCH 1/3] PM: domains: Drop the performance state vote for a device at detach Ulf Hansson ` (2 more replies) 0 siblings, 3 replies; 29+ messages in thread From: Ulf Hansson @ 2021-09-02 10:16 UTC (permalink / raw) To: Rafael J . Wysocki, Viresh Kumar, Dmitry Osipenko, linux-pm Cc: Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, linux-arm-kernel, linux-kernel, Ulf Hansson This series intends to improve how genpd manages performance states votes while device are becoming attached/detached to it. More details are available in the commit messages for each patch. Ulf Hansson (3): PM: domains: Drop the performance state vote for a device at detach PM: domains: Restructure some code in __genpd_dev_pm_attach() PM: domains: Add a ->dev_get_performance_state() callback to genpd drivers/base/power/domain.c | 26 +++++++++++++++++--------- include/linux/pm_domain.h | 3 ++- 2 files changed, 19 insertions(+), 10 deletions(-) -- 2.25.1 ^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH 1/3] PM: domains: Drop the performance state vote for a device at detach 2021-09-02 10:16 [PATCH 0/3] PM: domains: Improvements for performance states in genpd Ulf Hansson @ 2021-09-02 10:16 ` Ulf Hansson 2021-09-03 6:01 ` Dmitry Osipenko 2021-09-02 10:16 ` [PATCH 2/3] PM: domains: Restructure some code in __genpd_dev_pm_attach() Ulf Hansson 2021-09-02 10:16 ` [PATCH 3/3] PM: domains: Add a ->dev_get_performance_state() callback to genpd Ulf Hansson 2 siblings, 1 reply; 29+ messages in thread From: Ulf Hansson @ 2021-09-02 10:16 UTC (permalink / raw) To: Rafael J . Wysocki, Viresh Kumar, Dmitry Osipenko, linux-pm Cc: Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, linux-arm-kernel, linux-kernel, Ulf Hansson When a device is detached from its genpd, genpd loses track of the device, including its performance state vote that may have been requested for it. Rather than relying on the consumer driver to drop the performance state vote for its device, let's do it internally in genpd when the device is getting detached. In this way, we makes sure that the aggregation of the votes in genpd becomes correct. Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> --- drivers/base/power/domain.c | 9 ++------- include/linux/pm_domain.h | 1 - 2 files changed, 2 insertions(+), 8 deletions(-) diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c index 5db704f02e71..278e040f607f 100644 --- a/drivers/base/power/domain.c +++ b/drivers/base/power/domain.c @@ -1665,6 +1665,8 @@ static int genpd_remove_device(struct generic_pm_domain *genpd, goto out; } + genpd_set_performance_state(dev, 0); + genpd->device_count--; genpd->max_off_time_changed = true; @@ -2604,12 +2606,6 @@ static void genpd_dev_pm_detach(struct device *dev, bool power_off) dev_dbg(dev, "removing from PM domain %s\n", pd->name); - /* Drop the default performance state */ - if (dev_gpd_data(dev)->default_pstate) { - dev_pm_genpd_set_performance_state(dev, 0); - dev_gpd_data(dev)->default_pstate = 0; - } - for (i = 1; i < GENPD_RETRY_MAX_MS; i <<= 1) { ret = genpd_remove_device(pd, dev); if (ret != -EAGAIN) @@ -2702,7 +2698,6 @@ static int __genpd_dev_pm_attach(struct device *dev, struct device *base_dev, ret = dev_pm_genpd_set_performance_state(dev, pstate); if (ret) goto err; - dev_gpd_data(dev)->default_pstate = pstate; } return 1; diff --git a/include/linux/pm_domain.h b/include/linux/pm_domain.h index 67017c9390c8..21a0577305ef 100644 --- a/include/linux/pm_domain.h +++ b/include/linux/pm_domain.h @@ -198,7 +198,6 @@ struct generic_pm_domain_data { struct notifier_block *power_nb; int cpu; unsigned int performance_state; - unsigned int default_pstate; unsigned int rpm_pstate; ktime_t next_wakeup; void *data; -- 2.25.1 ^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: [PATCH 1/3] PM: domains: Drop the performance state vote for a device at detach 2021-09-02 10:16 ` [PATCH 1/3] PM: domains: Drop the performance state vote for a device at detach Ulf Hansson @ 2021-09-03 6:01 ` Dmitry Osipenko 2021-09-03 8:22 ` Ulf Hansson 0 siblings, 1 reply; 29+ messages in thread From: Dmitry Osipenko @ 2021-09-03 6:01 UTC (permalink / raw) To: Ulf Hansson, Rafael J . Wysocki, Viresh Kumar, linux-pm Cc: Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, linux-arm-kernel, linux-kernel 02.09.2021 13:16, Ulf Hansson пишет: > When a device is detached from its genpd, genpd loses track of the device, > including its performance state vote that may have been requested for it. > > Rather than relying on the consumer driver to drop the performance state > vote for its device, let's do it internally in genpd when the device is > getting detached. In this way, we makes sure that the aggregation of the > votes in genpd becomes correct. This is a dangerous behaviour in a case where performance state represents voltage. If hardware is kept active on detachment, say it's always-on, then it may be a disaster to drop the voltage for the active hardware. It's safe to drop performance state only if you assume that there is a firmware behind kernel which has its own layer of performance management and it will prevent the disaster by saying 'nope, I'm not doing this'. The performance state should be persistent for a device and it should be controlled in a conjunction with runtime PM. If platform wants to drop performance state to zero on detachment, then this behaviour should be specific to that platform. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 1/3] PM: domains: Drop the performance state vote for a device at detach 2021-09-03 6:01 ` Dmitry Osipenko @ 2021-09-03 8:22 ` Ulf Hansson 2021-09-03 9:58 ` Dmitry Osipenko 0 siblings, 1 reply; 29+ messages in thread From: Ulf Hansson @ 2021-09-03 8:22 UTC (permalink / raw) To: Dmitry Osipenko Cc: Rafael J . Wysocki, Viresh Kumar, Linux PM, Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, Linux ARM, Linux Kernel Mailing List On Fri, 3 Sept 2021 at 08:01, Dmitry Osipenko <digetx@gmail.com> wrote: > > 02.09.2021 13:16, Ulf Hansson пишет: > > When a device is detached from its genpd, genpd loses track of the device, > > including its performance state vote that may have been requested for it. > > > > Rather than relying on the consumer driver to drop the performance state > > vote for its device, let's do it internally in genpd when the device is > > getting detached. In this way, we makes sure that the aggregation of the > > votes in genpd becomes correct. > > This is a dangerous behaviour in a case where performance state > represents voltage. If hardware is kept active on detachment, say it's > always-on, then it may be a disaster to drop the voltage for the active > hardware. > > It's safe to drop performance state only if you assume that there is a > firmware behind kernel which has its own layer of performance management > and it will prevent the disaster by saying 'nope, I'm not doing this'. > > The performance state should be persistent for a device and it should be > controlled in a conjunction with runtime PM. If platform wants to drop > performance state to zero on detachment, then this behaviour should be > specific to that platform. I understand your concern, but at this point, genpd can't help to fix this. Genpd has no information about the device, unless it's attached to it. For now and for these always on HWs, we simply need to make sure the device stays attached, in one way or the other. Kind regards Uffe ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 1/3] PM: domains: Drop the performance state vote for a device at detach 2021-09-03 8:22 ` Ulf Hansson @ 2021-09-03 9:58 ` Dmitry Osipenko 2021-09-03 14:03 ` Ulf Hansson 0 siblings, 1 reply; 29+ messages in thread From: Dmitry Osipenko @ 2021-09-03 9:58 UTC (permalink / raw) To: Ulf Hansson Cc: Rafael J . Wysocki, Viresh Kumar, Linux PM, Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, Linux ARM, Linux Kernel Mailing List 03.09.2021 11:22, Ulf Hansson пишет: > On Fri, 3 Sept 2021 at 08:01, Dmitry Osipenko <digetx@gmail.com> wrote: >> >> 02.09.2021 13:16, Ulf Hansson пишет: >>> When a device is detached from its genpd, genpd loses track of the device, >>> including its performance state vote that may have been requested for it. >>> >>> Rather than relying on the consumer driver to drop the performance state >>> vote for its device, let's do it internally in genpd when the device is >>> getting detached. In this way, we makes sure that the aggregation of the >>> votes in genpd becomes correct. >> >> This is a dangerous behaviour in a case where performance state >> represents voltage. If hardware is kept active on detachment, say it's >> always-on, then it may be a disaster to drop the voltage for the active >> hardware. >> >> It's safe to drop performance state only if you assume that there is a >> firmware behind kernel which has its own layer of performance management >> and it will prevent the disaster by saying 'nope, I'm not doing this'. >> >> The performance state should be persistent for a device and it should be >> controlled in a conjunction with runtime PM. If platform wants to drop >> performance state to zero on detachment, then this behaviour should be >> specific to that platform. > > I understand your concern, but at this point, genpd can't help to fix this. > > Genpd has no information about the device, unless it's attached to it. > For now and for these always on HWs, we simply need to make sure the > device stays attached, in one way or the other. This indeed requires to redesign GENPD to make it more coupled with a device, but this is not a real problem for any of the current API users AFAIK. Ideally the state should be persistent to make API more universal. Since for today we assume that device should be suspended at the time of the detachment (if the default OPP state isn't used), it may be better to add a noisy warning message if pstate!=0, keeping the state untouched if it's not zero. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 1/3] PM: domains: Drop the performance state vote for a device at detach 2021-09-03 9:58 ` Dmitry Osipenko @ 2021-09-03 14:03 ` Ulf Hansson 2021-09-05 8:26 ` Dmitry Osipenko 0 siblings, 1 reply; 29+ messages in thread From: Ulf Hansson @ 2021-09-03 14:03 UTC (permalink / raw) To: Dmitry Osipenko Cc: Rafael J . Wysocki, Viresh Kumar, Linux PM, Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, Linux ARM, Linux Kernel Mailing List On Fri, 3 Sept 2021 at 11:58, Dmitry Osipenko <digetx@gmail.com> wrote: > > 03.09.2021 11:22, Ulf Hansson пишет: > > On Fri, 3 Sept 2021 at 08:01, Dmitry Osipenko <digetx@gmail.com> wrote: > >> > >> 02.09.2021 13:16, Ulf Hansson пишет: > >>> When a device is detached from its genpd, genpd loses track of the device, > >>> including its performance state vote that may have been requested for it. > >>> > >>> Rather than relying on the consumer driver to drop the performance state > >>> vote for its device, let's do it internally in genpd when the device is > >>> getting detached. In this way, we makes sure that the aggregation of the > >>> votes in genpd becomes correct. > >> > >> This is a dangerous behaviour in a case where performance state > >> represents voltage. If hardware is kept active on detachment, say it's > >> always-on, then it may be a disaster to drop the voltage for the active > >> hardware. > >> > >> It's safe to drop performance state only if you assume that there is a > >> firmware behind kernel which has its own layer of performance management > >> and it will prevent the disaster by saying 'nope, I'm not doing this'. > >> > >> The performance state should be persistent for a device and it should be > >> controlled in a conjunction with runtime PM. If platform wants to drop > >> performance state to zero on detachment, then this behaviour should be > >> specific to that platform. > > > > I understand your concern, but at this point, genpd can't help to fix this. > > > > Genpd has no information about the device, unless it's attached to it. > > For now and for these always on HWs, we simply need to make sure the > > device stays attached, in one way or the other. > > This indeed requires to redesign GENPD to make it more coupled with a > device, but this is not a real problem for any of the current API users > AFAIK. Ideally the state should be persistent to make API more universal. Right. In fact this has been discussed in the past. In principle, the idea was to attach to genpd at device registration, rather than at driver probe. Although, this is not very easy to implement - and it seems like the churns to do, have not been really worth it. At least so far. > > Since for today we assume that device should be suspended at the time of > the detachment (if the default OPP state isn't used), it may be better > to add a noisy warning message if pstate!=0, keeping the state untouched > if it's not zero. That would just be very silly in my opinion. When the device is detached (suspended or not), it may cause it's PM domain to be powered off - and there is really nothing we can do about that from the genpd point of view. As stated, the only current short term solution is to avoid detaching the device. Anything else, would just be papering of the issue. Kind regards Uffe ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 1/3] PM: domains: Drop the performance state vote for a device at detach 2021-09-03 14:03 ` Ulf Hansson @ 2021-09-05 8:26 ` Dmitry Osipenko 2021-09-06 10:24 ` Ulf Hansson 0 siblings, 1 reply; 29+ messages in thread From: Dmitry Osipenko @ 2021-09-05 8:26 UTC (permalink / raw) To: Ulf Hansson Cc: Rafael J . Wysocki, Viresh Kumar, Linux PM, Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, Linux ARM, Linux Kernel Mailing List 03.09.2021 17:03, Ulf Hansson пишет: > On Fri, 3 Sept 2021 at 11:58, Dmitry Osipenko <digetx@gmail.com> wrote: >> >> 03.09.2021 11:22, Ulf Hansson пишет: >>> On Fri, 3 Sept 2021 at 08:01, Dmitry Osipenko <digetx@gmail.com> wrote: >>>> >>>> 02.09.2021 13:16, Ulf Hansson пишет: >>>>> When a device is detached from its genpd, genpd loses track of the device, >>>>> including its performance state vote that may have been requested for it. >>>>> >>>>> Rather than relying on the consumer driver to drop the performance state >>>>> vote for its device, let's do it internally in genpd when the device is >>>>> getting detached. In this way, we makes sure that the aggregation of the >>>>> votes in genpd becomes correct. >>>> >>>> This is a dangerous behaviour in a case where performance state >>>> represents voltage. If hardware is kept active on detachment, say it's >>>> always-on, then it may be a disaster to drop the voltage for the active >>>> hardware. >>>> >>>> It's safe to drop performance state only if you assume that there is a >>>> firmware behind kernel which has its own layer of performance management >>>> and it will prevent the disaster by saying 'nope, I'm not doing this'. >>>> >>>> The performance state should be persistent for a device and it should be >>>> controlled in a conjunction with runtime PM. If platform wants to drop >>>> performance state to zero on detachment, then this behaviour should be >>>> specific to that platform. >>> >>> I understand your concern, but at this point, genpd can't help to fix this. >>> >>> Genpd has no information about the device, unless it's attached to it. >>> For now and for these always on HWs, we simply need to make sure the >>> device stays attached, in one way or the other. >> >> This indeed requires to redesign GENPD to make it more coupled with a >> device, but this is not a real problem for any of the current API users >> AFAIK. Ideally the state should be persistent to make API more universal. > > Right. In fact this has been discussed in the past. In principle, the > idea was to attach to genpd at device registration, rather than at > driver probe. > > Although, this is not very easy to implement - and it seems like the > churns to do, have not been really worth it. At least so far. > >> >> Since for today we assume that device should be suspended at the time of >> the detachment (if the default OPP state isn't used), it may be better >> to add a noisy warning message if pstate!=0, keeping the state untouched >> if it's not zero. > > That would just be very silly in my opinion. > > When the device is detached (suspended or not), it may cause it's PM > domain to be powered off - and there is really nothing we can do about > that from the genpd point of view. > > As stated, the only current short term solution is to avoid detaching > the device. Anything else, would just be papering of the issue. What about to re-evaluate the performance state of the domain after detachment instead of setting the state to zero? This way PD driver may take an action on detachment if performance isn't zero, before hardware is crashed, for example it may emit a warning. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 1/3] PM: domains: Drop the performance state vote for a device at detach 2021-09-05 8:26 ` Dmitry Osipenko @ 2021-09-06 10:24 ` Ulf Hansson 2021-09-06 14:11 ` Dmitry Osipenko 0 siblings, 1 reply; 29+ messages in thread From: Ulf Hansson @ 2021-09-06 10:24 UTC (permalink / raw) To: Dmitry Osipenko Cc: Rafael J . Wysocki, Viresh Kumar, Linux PM, Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, Linux ARM, Linux Kernel Mailing List On Sun, 5 Sept 2021 at 10:26, Dmitry Osipenko <digetx@gmail.com> wrote: > > 03.09.2021 17:03, Ulf Hansson пишет: > > On Fri, 3 Sept 2021 at 11:58, Dmitry Osipenko <digetx@gmail.com> wrote: > >> > >> 03.09.2021 11:22, Ulf Hansson пишет: > >>> On Fri, 3 Sept 2021 at 08:01, Dmitry Osipenko <digetx@gmail.com> wrote: > >>>> > >>>> 02.09.2021 13:16, Ulf Hansson пишет: > >>>>> When a device is detached from its genpd, genpd loses track of the device, > >>>>> including its performance state vote that may have been requested for it. > >>>>> > >>>>> Rather than relying on the consumer driver to drop the performance state > >>>>> vote for its device, let's do it internally in genpd when the device is > >>>>> getting detached. In this way, we makes sure that the aggregation of the > >>>>> votes in genpd becomes correct. > >>>> > >>>> This is a dangerous behaviour in a case where performance state > >>>> represents voltage. If hardware is kept active on detachment, say it's > >>>> always-on, then it may be a disaster to drop the voltage for the active > >>>> hardware. > >>>> > >>>> It's safe to drop performance state only if you assume that there is a > >>>> firmware behind kernel which has its own layer of performance management > >>>> and it will prevent the disaster by saying 'nope, I'm not doing this'. > >>>> > >>>> The performance state should be persistent for a device and it should be > >>>> controlled in a conjunction with runtime PM. If platform wants to drop > >>>> performance state to zero on detachment, then this behaviour should be > >>>> specific to that platform. > >>> > >>> I understand your concern, but at this point, genpd can't help to fix this. > >>> > >>> Genpd has no information about the device, unless it's attached to it. > >>> For now and for these always on HWs, we simply need to make sure the > >>> device stays attached, in one way or the other. > >> > >> This indeed requires to redesign GENPD to make it more coupled with a > >> device, but this is not a real problem for any of the current API users > >> AFAIK. Ideally the state should be persistent to make API more universal. > > > > Right. In fact this has been discussed in the past. In principle, the > > idea was to attach to genpd at device registration, rather than at > > driver probe. > > > > Although, this is not very easy to implement - and it seems like the > > churns to do, have not been really worth it. At least so far. > > > >> > >> Since for today we assume that device should be suspended at the time of > >> the detachment (if the default OPP state isn't used), it may be better > >> to add a noisy warning message if pstate!=0, keeping the state untouched > >> if it's not zero. > > > > That would just be very silly in my opinion. > > > > When the device is detached (suspended or not), it may cause it's PM > > domain to be powered off - and there is really nothing we can do about > > that from the genpd point of view. > > > > As stated, the only current short term solution is to avoid detaching > > the device. Anything else, would just be papering of the issue. > > What about to re-evaluate the performance state of the domain after > detachment instead of setting the state to zero? I am not suggesting to set the performance state of the genpd to zero, but to drop a potential vote for a performance state for the *device* that is about to be detached. Calling genpd_set_performance_state(dev, 0), during detach will have the same effect as triggering a re-evaluation of the performance state for the genpd, but after the detach. > This way PD driver may > take an action on detachment if performance isn't zero, before hardware > is crashed, for example it may emit a warning. Not sure I got that. Exactly when do you want to emit a warning and for what reason? Do you want to add a check somewhere to see if 'gpd_data->performance_state' is non zero - and then print a warning? Kind regards Uffe ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 1/3] PM: domains: Drop the performance state vote for a device at detach 2021-09-06 10:24 ` Ulf Hansson @ 2021-09-06 14:11 ` Dmitry Osipenko 2021-09-06 17:34 ` Ulf Hansson 0 siblings, 1 reply; 29+ messages in thread From: Dmitry Osipenko @ 2021-09-06 14:11 UTC (permalink / raw) To: Ulf Hansson Cc: Rafael J . Wysocki, Viresh Kumar, Linux PM, Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, Linux ARM, Linux Kernel Mailing List 06.09.2021 13:24, Ulf Hansson пишет: > On Sun, 5 Sept 2021 at 10:26, Dmitry Osipenko <digetx@gmail.com> wrote: >> >> 03.09.2021 17:03, Ulf Hansson пишет: >>> On Fri, 3 Sept 2021 at 11:58, Dmitry Osipenko <digetx@gmail.com> wrote: >>>> >>>> 03.09.2021 11:22, Ulf Hansson пишет: >>>>> On Fri, 3 Sept 2021 at 08:01, Dmitry Osipenko <digetx@gmail.com> wrote: >>>>>> >>>>>> 02.09.2021 13:16, Ulf Hansson пишет: >>>>>>> When a device is detached from its genpd, genpd loses track of the device, >>>>>>> including its performance state vote that may have been requested for it. >>>>>>> >>>>>>> Rather than relying on the consumer driver to drop the performance state >>>>>>> vote for its device, let's do it internally in genpd when the device is >>>>>>> getting detached. In this way, we makes sure that the aggregation of the >>>>>>> votes in genpd becomes correct. >>>>>> >>>>>> This is a dangerous behaviour in a case where performance state >>>>>> represents voltage. If hardware is kept active on detachment, say it's >>>>>> always-on, then it may be a disaster to drop the voltage for the active >>>>>> hardware. >>>>>> >>>>>> It's safe to drop performance state only if you assume that there is a >>>>>> firmware behind kernel which has its own layer of performance management >>>>>> and it will prevent the disaster by saying 'nope, I'm not doing this'. >>>>>> >>>>>> The performance state should be persistent for a device and it should be >>>>>> controlled in a conjunction with runtime PM. If platform wants to drop >>>>>> performance state to zero on detachment, then this behaviour should be >>>>>> specific to that platform. >>>>> >>>>> I understand your concern, but at this point, genpd can't help to fix this. >>>>> >>>>> Genpd has no information about the device, unless it's attached to it. >>>>> For now and for these always on HWs, we simply need to make sure the >>>>> device stays attached, in one way or the other. >>>> >>>> This indeed requires to redesign GENPD to make it more coupled with a >>>> device, but this is not a real problem for any of the current API users >>>> AFAIK. Ideally the state should be persistent to make API more universal. >>> >>> Right. In fact this has been discussed in the past. In principle, the >>> idea was to attach to genpd at device registration, rather than at >>> driver probe. >>> >>> Although, this is not very easy to implement - and it seems like the >>> churns to do, have not been really worth it. At least so far. >>> >>>> >>>> Since for today we assume that device should be suspended at the time of >>>> the detachment (if the default OPP state isn't used), it may be better >>>> to add a noisy warning message if pstate!=0, keeping the state untouched >>>> if it's not zero. >>> >>> That would just be very silly in my opinion. >>> >>> When the device is detached (suspended or not), it may cause it's PM >>> domain to be powered off - and there is really nothing we can do about >>> that from the genpd point of view. >>> >>> As stated, the only current short term solution is to avoid detaching >>> the device. Anything else, would just be papering of the issue. >> >> What about to re-evaluate the performance state of the domain after >> detachment instead of setting the state to zero? > > I am not suggesting to set the performance state of the genpd to zero, > but to drop a potential vote for a performance state for the *device* > that is about to be detached. By removing the vote of the *device*, you will drop the performance state of the genpd. If device is active and it's wrong to drop its state, then you may cause the damage. > Calling genpd_set_performance_state(dev, 0), during detach will have > the same effect as triggering a re-evaluation of the performance state > for the genpd, but after the detach. Yes >> This way PD driver may >> take an action on detachment if performance isn't zero, before hardware >> is crashed, for example it may emit a warning. > > Not sure I got that. Exactly when do you want to emit a warning and > for what reason? > > Do you want to add a check somewhere to see if > 'gpd_data->performance_state' is non zero - and then print a warning? I want to check the 'gpd_data->performance_state' from the detachment callback and emit the warning + lock further performance changes in the PD driver since it's a error condition. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 1/3] PM: domains: Drop the performance state vote for a device at detach 2021-09-06 14:11 ` Dmitry Osipenko @ 2021-09-06 17:34 ` Ulf Hansson 2021-09-06 19:33 ` Dmitry Osipenko 0 siblings, 1 reply; 29+ messages in thread From: Ulf Hansson @ 2021-09-06 17:34 UTC (permalink / raw) To: Dmitry Osipenko Cc: Rafael J . Wysocki, Viresh Kumar, Linux PM, Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, Linux ARM, Linux Kernel Mailing List On Mon, 6 Sept 2021 at 16:11, Dmitry Osipenko <digetx@gmail.com> wrote: > > 06.09.2021 13:24, Ulf Hansson пишет: > > On Sun, 5 Sept 2021 at 10:26, Dmitry Osipenko <digetx@gmail.com> wrote: > >> > >> 03.09.2021 17:03, Ulf Hansson пишет: > >>> On Fri, 3 Sept 2021 at 11:58, Dmitry Osipenko <digetx@gmail.com> wrote: > >>>> > >>>> 03.09.2021 11:22, Ulf Hansson пишет: > >>>>> On Fri, 3 Sept 2021 at 08:01, Dmitry Osipenko <digetx@gmail.com> wrote: > >>>>>> > >>>>>> 02.09.2021 13:16, Ulf Hansson пишет: > >>>>>>> When a device is detached from its genpd, genpd loses track of the device, > >>>>>>> including its performance state vote that may have been requested for it. > >>>>>>> > >>>>>>> Rather than relying on the consumer driver to drop the performance state > >>>>>>> vote for its device, let's do it internally in genpd when the device is > >>>>>>> getting detached. In this way, we makes sure that the aggregation of the > >>>>>>> votes in genpd becomes correct. > >>>>>> > >>>>>> This is a dangerous behaviour in a case where performance state > >>>>>> represents voltage. If hardware is kept active on detachment, say it's > >>>>>> always-on, then it may be a disaster to drop the voltage for the active > >>>>>> hardware. > >>>>>> > >>>>>> It's safe to drop performance state only if you assume that there is a > >>>>>> firmware behind kernel which has its own layer of performance management > >>>>>> and it will prevent the disaster by saying 'nope, I'm not doing this'. > >>>>>> > >>>>>> The performance state should be persistent for a device and it should be > >>>>>> controlled in a conjunction with runtime PM. If platform wants to drop > >>>>>> performance state to zero on detachment, then this behaviour should be > >>>>>> specific to that platform. > >>>>> > >>>>> I understand your concern, but at this point, genpd can't help to fix this. > >>>>> > >>>>> Genpd has no information about the device, unless it's attached to it. > >>>>> For now and for these always on HWs, we simply need to make sure the > >>>>> device stays attached, in one way or the other. > >>>> > >>>> This indeed requires to redesign GENPD to make it more coupled with a > >>>> device, but this is not a real problem for any of the current API users > >>>> AFAIK. Ideally the state should be persistent to make API more universal. > >>> > >>> Right. In fact this has been discussed in the past. In principle, the > >>> idea was to attach to genpd at device registration, rather than at > >>> driver probe. > >>> > >>> Although, this is not very easy to implement - and it seems like the > >>> churns to do, have not been really worth it. At least so far. > >>> > >>>> > >>>> Since for today we assume that device should be suspended at the time of > >>>> the detachment (if the default OPP state isn't used), it may be better > >>>> to add a noisy warning message if pstate!=0, keeping the state untouched > >>>> if it's not zero. > >>> > >>> That would just be very silly in my opinion. > >>> > >>> When the device is detached (suspended or not), it may cause it's PM > >>> domain to be powered off - and there is really nothing we can do about > >>> that from the genpd point of view. > >>> > >>> As stated, the only current short term solution is to avoid detaching > >>> the device. Anything else, would just be papering of the issue. > >> > >> What about to re-evaluate the performance state of the domain after > >> detachment instead of setting the state to zero? > > > > I am not suggesting to set the performance state of the genpd to zero, > > but to drop a potential vote for a performance state for the *device* > > that is about to be detached. > > By removing the vote of the *device*, you will drop the performance > state of the genpd. If device is active and it's wrong to drop its > state, then you may cause the damage. > > > Calling genpd_set_performance_state(dev, 0), during detach will have > > the same effect as triggering a re-evaluation of the performance state > > for the genpd, but after the detach. > > Yes > > >> This way PD driver may > >> take an action on detachment if performance isn't zero, before hardware > >> is crashed, for example it may emit a warning. > > > > Not sure I got that. Exactly when do you want to emit a warning and > > for what reason? > > > > Do you want to add a check somewhere to see if > > 'gpd_data->performance_state' is non zero - and then print a warning? > > I want to check the 'gpd_data->performance_state' from the detachment > callback and emit the warning + lock further performance changes in the > PD driver since it's a error condition. Alright, so if I understand correctly, you intend to do the check for the "error condition" of the device in the genpd->detach_dev() callback? What exactly do you intend to do beyond this point, if you detect the "error condition"? Locking further changes of the performance state seems fragile too, especially if some other device/driver requires the performance state to be raised. It sounds like you simply need to call BUG_ON() then? Also note that a very similar problem exists, *before* the device gets attached in the first place. More precisely, nothing prevents the performance state from being set to a non-compatible value for an always-on HW/device that hasn't been attached yet. So maybe you need to set the maximum performance state at genpd initializations, then use the ->sync_state() callback to very that all consumers have been attached to the genpd provider, before allowing the state to be changed/lowered? Kind regards Uffe ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 1/3] PM: domains: Drop the performance state vote for a device at detach 2021-09-06 17:34 ` Ulf Hansson @ 2021-09-06 19:33 ` Dmitry Osipenko 2021-09-07 10:16 ` Ulf Hansson 0 siblings, 1 reply; 29+ messages in thread From: Dmitry Osipenko @ 2021-09-06 19:33 UTC (permalink / raw) To: Ulf Hansson Cc: Rafael J . Wysocki, Viresh Kumar, Linux PM, Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, Linux ARM, Linux Kernel Mailing List 06.09.2021 20:34, Ulf Hansson пишет: > On Mon, 6 Sept 2021 at 16:11, Dmitry Osipenko <digetx@gmail.com> wrote: >> >> 06.09.2021 13:24, Ulf Hansson пишет: >>> On Sun, 5 Sept 2021 at 10:26, Dmitry Osipenko <digetx@gmail.com> wrote: >>>> >>>> 03.09.2021 17:03, Ulf Hansson пишет: >>>>> On Fri, 3 Sept 2021 at 11:58, Dmitry Osipenko <digetx@gmail.com> wrote: >>>>>> >>>>>> 03.09.2021 11:22, Ulf Hansson пишет: >>>>>>> On Fri, 3 Sept 2021 at 08:01, Dmitry Osipenko <digetx@gmail.com> wrote: >>>>>>>> >>>>>>>> 02.09.2021 13:16, Ulf Hansson пишет: >>>>>>>>> When a device is detached from its genpd, genpd loses track of the device, >>>>>>>>> including its performance state vote that may have been requested for it. >>>>>>>>> >>>>>>>>> Rather than relying on the consumer driver to drop the performance state >>>>>>>>> vote for its device, let's do it internally in genpd when the device is >>>>>>>>> getting detached. In this way, we makes sure that the aggregation of the >>>>>>>>> votes in genpd becomes correct. >>>>>>>> >>>>>>>> This is a dangerous behaviour in a case where performance state >>>>>>>> represents voltage. If hardware is kept active on detachment, say it's >>>>>>>> always-on, then it may be a disaster to drop the voltage for the active >>>>>>>> hardware. >>>>>>>> >>>>>>>> It's safe to drop performance state only if you assume that there is a >>>>>>>> firmware behind kernel which has its own layer of performance management >>>>>>>> and it will prevent the disaster by saying 'nope, I'm not doing this'. >>>>>>>> >>>>>>>> The performance state should be persistent for a device and it should be >>>>>>>> controlled in a conjunction with runtime PM. If platform wants to drop >>>>>>>> performance state to zero on detachment, then this behaviour should be >>>>>>>> specific to that platform. >>>>>>> >>>>>>> I understand your concern, but at this point, genpd can't help to fix this. >>>>>>> >>>>>>> Genpd has no information about the device, unless it's attached to it. >>>>>>> For now and for these always on HWs, we simply need to make sure the >>>>>>> device stays attached, in one way or the other. >>>>>> >>>>>> This indeed requires to redesign GENPD to make it more coupled with a >>>>>> device, but this is not a real problem for any of the current API users >>>>>> AFAIK. Ideally the state should be persistent to make API more universal. >>>>> >>>>> Right. In fact this has been discussed in the past. In principle, the >>>>> idea was to attach to genpd at device registration, rather than at >>>>> driver probe. >>>>> >>>>> Although, this is not very easy to implement - and it seems like the >>>>> churns to do, have not been really worth it. At least so far. >>>>> >>>>>> >>>>>> Since for today we assume that device should be suspended at the time of >>>>>> the detachment (if the default OPP state isn't used), it may be better >>>>>> to add a noisy warning message if pstate!=0, keeping the state untouched >>>>>> if it's not zero. >>>>> >>>>> That would just be very silly in my opinion. >>>>> >>>>> When the device is detached (suspended or not), it may cause it's PM >>>>> domain to be powered off - and there is really nothing we can do about >>>>> that from the genpd point of view. >>>>> >>>>> As stated, the only current short term solution is to avoid detaching >>>>> the device. Anything else, would just be papering of the issue. >>>> >>>> What about to re-evaluate the performance state of the domain after >>>> detachment instead of setting the state to zero? >>> >>> I am not suggesting to set the performance state of the genpd to zero, >>> but to drop a potential vote for a performance state for the *device* >>> that is about to be detached. >> >> By removing the vote of the *device*, you will drop the performance >> state of the genpd. If device is active and it's wrong to drop its >> state, then you may cause the damage. >> >>> Calling genpd_set_performance_state(dev, 0), during detach will have >>> the same effect as triggering a re-evaluation of the performance state >>> for the genpd, but after the detach. >> >> Yes >> >>>> This way PD driver may >>>> take an action on detachment if performance isn't zero, before hardware >>>> is crashed, for example it may emit a warning. >>> >>> Not sure I got that. Exactly when do you want to emit a warning and >>> for what reason? >>> >>> Do you want to add a check somewhere to see if >>> 'gpd_data->performance_state' is non zero - and then print a warning? >> >> I want to check the 'gpd_data->performance_state' from the detachment >> callback and emit the warning + lock further performance changes in the >> PD driver since it's a error condition. > > Alright, so if I understand correctly, you intend to do the check for > the "error condition" of the device in the genpd->detach_dev() > callback? Yes > What exactly do you intend to do beyond this point, if you detect the > "error condition"? Locking further changes of the performance state > seems fragile too, especially if some other device/driver requires the > performance state to be raised. It sounds like you simply need to call > BUG_ON() then? I can lock it to high performance state. > Also note that a very similar problem exists, *before* the device gets > attached in the first place. More precisely, nothing prevents the > performance state from being set to a non-compatible value for an > always-on HW/device that hasn't been attached yet. So maybe you need > to set the maximum performance state at genpd initializations, then > use the ->sync_state() callback to very that all consumers have been > attached to the genpd provider, before allowing the state to be > changed/lowered? That is already done by the PD driver. https://elixir.bootlin.com/linux/latest/source/drivers/soc/tegra/pmc.c#L3790 ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 1/3] PM: domains: Drop the performance state vote for a device at detach 2021-09-06 19:33 ` Dmitry Osipenko @ 2021-09-07 10:16 ` Ulf Hansson 2021-09-09 13:48 ` Dmitry Osipenko 0 siblings, 1 reply; 29+ messages in thread From: Ulf Hansson @ 2021-09-07 10:16 UTC (permalink / raw) To: Dmitry Osipenko Cc: Rafael J . Wysocki, Viresh Kumar, Linux PM, Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, Linux ARM, Linux Kernel Mailing List On Mon, 6 Sept 2021 at 21:33, Dmitry Osipenko <digetx@gmail.com> wrote: > > 06.09.2021 20:34, Ulf Hansson пишет: > > On Mon, 6 Sept 2021 at 16:11, Dmitry Osipenko <digetx@gmail.com> wrote: > >> > >> 06.09.2021 13:24, Ulf Hansson пишет: > >>> On Sun, 5 Sept 2021 at 10:26, Dmitry Osipenko <digetx@gmail.com> wrote: > >>>> > >>>> 03.09.2021 17:03, Ulf Hansson пишет: > >>>>> On Fri, 3 Sept 2021 at 11:58, Dmitry Osipenko <digetx@gmail.com> wrote: > >>>>>> > >>>>>> 03.09.2021 11:22, Ulf Hansson пишет: > >>>>>>> On Fri, 3 Sept 2021 at 08:01, Dmitry Osipenko <digetx@gmail.com> wrote: > >>>>>>>> > >>>>>>>> 02.09.2021 13:16, Ulf Hansson пишет: > >>>>>>>>> When a device is detached from its genpd, genpd loses track of the device, > >>>>>>>>> including its performance state vote that may have been requested for it. > >>>>>>>>> > >>>>>>>>> Rather than relying on the consumer driver to drop the performance state > >>>>>>>>> vote for its device, let's do it internally in genpd when the device is > >>>>>>>>> getting detached. In this way, we makes sure that the aggregation of the > >>>>>>>>> votes in genpd becomes correct. > >>>>>>>> > >>>>>>>> This is a dangerous behaviour in a case where performance state > >>>>>>>> represents voltage. If hardware is kept active on detachment, say it's > >>>>>>>> always-on, then it may be a disaster to drop the voltage for the active > >>>>>>>> hardware. > >>>>>>>> > >>>>>>>> It's safe to drop performance state only if you assume that there is a > >>>>>>>> firmware behind kernel which has its own layer of performance management > >>>>>>>> and it will prevent the disaster by saying 'nope, I'm not doing this'. > >>>>>>>> > >>>>>>>> The performance state should be persistent for a device and it should be > >>>>>>>> controlled in a conjunction with runtime PM. If platform wants to drop > >>>>>>>> performance state to zero on detachment, then this behaviour should be > >>>>>>>> specific to that platform. > >>>>>>> > >>>>>>> I understand your concern, but at this point, genpd can't help to fix this. > >>>>>>> > >>>>>>> Genpd has no information about the device, unless it's attached to it. > >>>>>>> For now and for these always on HWs, we simply need to make sure the > >>>>>>> device stays attached, in one way or the other. > >>>>>> > >>>>>> This indeed requires to redesign GENPD to make it more coupled with a > >>>>>> device, but this is not a real problem for any of the current API users > >>>>>> AFAIK. Ideally the state should be persistent to make API more universal. > >>>>> > >>>>> Right. In fact this has been discussed in the past. In principle, the > >>>>> idea was to attach to genpd at device registration, rather than at > >>>>> driver probe. > >>>>> > >>>>> Although, this is not very easy to implement - and it seems like the > >>>>> churns to do, have not been really worth it. At least so far. > >>>>> > >>>>>> > >>>>>> Since for today we assume that device should be suspended at the time of > >>>>>> the detachment (if the default OPP state isn't used), it may be better > >>>>>> to add a noisy warning message if pstate!=0, keeping the state untouched > >>>>>> if it's not zero. > >>>>> > >>>>> That would just be very silly in my opinion. > >>>>> > >>>>> When the device is detached (suspended or not), it may cause it's PM > >>>>> domain to be powered off - and there is really nothing we can do about > >>>>> that from the genpd point of view. > >>>>> > >>>>> As stated, the only current short term solution is to avoid detaching > >>>>> the device. Anything else, would just be papering of the issue. > >>>> > >>>> What about to re-evaluate the performance state of the domain after > >>>> detachment instead of setting the state to zero? > >>> > >>> I am not suggesting to set the performance state of the genpd to zero, > >>> but to drop a potential vote for a performance state for the *device* > >>> that is about to be detached. > >> > >> By removing the vote of the *device*, you will drop the performance > >> state of the genpd. If device is active and it's wrong to drop its > >> state, then you may cause the damage. > >> > >>> Calling genpd_set_performance_state(dev, 0), during detach will have > >>> the same effect as triggering a re-evaluation of the performance state > >>> for the genpd, but after the detach. > >> > >> Yes > >> > >>>> This way PD driver may > >>>> take an action on detachment if performance isn't zero, before hardware > >>>> is crashed, for example it may emit a warning. > >>> > >>> Not sure I got that. Exactly when do you want to emit a warning and > >>> for what reason? > >>> > >>> Do you want to add a check somewhere to see if > >>> 'gpd_data->performance_state' is non zero - and then print a warning? > >> > >> I want to check the 'gpd_data->performance_state' from the detachment > >> callback and emit the warning + lock further performance changes in the > >> PD driver since it's a error condition. > > > > Alright, so if I understand correctly, you intend to do the check for > > the "error condition" of the device in the genpd->detach_dev() > > callback? > > Yes Okay. > > > What exactly do you intend to do beyond this point, if you detect the > > "error condition"? Locking further changes of the performance state > > seems fragile too, especially if some other device/driver requires the > > performance state to be raised. It sounds like you simply need to call > > BUG_ON() then? > > I can lock it to high performance state. Alright. > > > Also note that a very similar problem exists, *before* the device gets > > attached in the first place. More precisely, nothing prevents the > > performance state from being set to a non-compatible value for an > > always-on HW/device that hasn't been attached yet. So maybe you need > > to set the maximum performance state at genpd initializations, then > > use the ->sync_state() callback to very that all consumers have been > > attached to the genpd provider, before allowing the state to be > > changed/lowered? > > That is already done by the PD driver. > > https://elixir.bootlin.com/linux/latest/source/drivers/soc/tegra/pmc.c#L3790 Yes, I already knew that, but forgot it. :-) Thanks for the pointer. Let me rethink the approach. In a way, it kind of sounds like this is a generic problem - so perhaps we should think of adding a ->withdraw_sync_state() callback that can be assigned by provider drivers, to get informed when a consumer driver is getting unbinded. Kind regards Uffe ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 1/3] PM: domains: Drop the performance state vote for a device at detach 2021-09-07 10:16 ` Ulf Hansson @ 2021-09-09 13:48 ` Dmitry Osipenko 2021-09-09 14:45 ` Ulf Hansson 0 siblings, 1 reply; 29+ messages in thread From: Dmitry Osipenko @ 2021-09-09 13:48 UTC (permalink / raw) To: Ulf Hansson Cc: Rafael J . Wysocki, Viresh Kumar, Linux PM, Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, Linux ARM, Linux Kernel Mailing List 07.09.2021 13:16, Ulf Hansson пишет: ... >>> Also note that a very similar problem exists, *before* the device gets >>> attached in the first place. More precisely, nothing prevents the >>> performance state from being set to a non-compatible value for an >>> always-on HW/device that hasn't been attached yet. So maybe you need >>> to set the maximum performance state at genpd initializations, then >>> use the ->sync_state() callback to very that all consumers have been >>> attached to the genpd provider, before allowing the state to be >>> changed/lowered? >> >> That is already done by the PD driver. >> >> https://elixir.bootlin.com/linux/latest/source/drivers/soc/tegra/pmc.c#L3790 > > Yes, I already knew that, but forgot it. :-) Thanks for the pointer. > Let me rethink the approach. > > In a way, it kind of sounds like this is a generic problem - so > perhaps we should think of adding a ->withdraw_sync_state() callback > that can be assigned by provider drivers, to get informed when a > consumer driver is getting unbinded. Not sure, doesn't feel to me that this is necessary for today. A bit too cumbersome for a simple sanity-check, IMO. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 1/3] PM: domains: Drop the performance state vote for a device at detach 2021-09-09 13:48 ` Dmitry Osipenko @ 2021-09-09 14:45 ` Ulf Hansson 0 siblings, 0 replies; 29+ messages in thread From: Ulf Hansson @ 2021-09-09 14:45 UTC (permalink / raw) To: Dmitry Osipenko Cc: Rafael J . Wysocki, Viresh Kumar, Linux PM, Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, Linux ARM, Linux Kernel Mailing List On Thu, 9 Sept 2021 at 15:48, Dmitry Osipenko <digetx@gmail.com> wrote: > > 07.09.2021 13:16, Ulf Hansson пишет: > ... > >>> Also note that a very similar problem exists, *before* the device gets > >>> attached in the first place. More precisely, nothing prevents the > >>> performance state from being set to a non-compatible value for an > >>> always-on HW/device that hasn't been attached yet. So maybe you need > >>> to set the maximum performance state at genpd initializations, then > >>> use the ->sync_state() callback to very that all consumers have been > >>> attached to the genpd provider, before allowing the state to be > >>> changed/lowered? > >> > >> That is already done by the PD driver. > >> > >> https://elixir.bootlin.com/linux/latest/source/drivers/soc/tegra/pmc.c#L3790 > > > > Yes, I already knew that, but forgot it. :-) Thanks for the pointer. > > Let me rethink the approach. > > > > In a way, it kind of sounds like this is a generic problem - so > > perhaps we should think of adding a ->withdraw_sync_state() callback > > that can be assigned by provider drivers, to get informed when a > > consumer driver is getting unbinded. > > Not sure, doesn't feel to me that this is necessary for today. A bit too > cumbersome for a simple sanity-check, IMO. Maybe, but I can bring it up with the fw_devlinks people to see what they think. In any case, we should not move forward with $subject patch as is. Let me think about it. Kind regards Uffe ^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH 2/3] PM: domains: Restructure some code in __genpd_dev_pm_attach() 2021-09-02 10:16 [PATCH 0/3] PM: domains: Improvements for performance states in genpd Ulf Hansson 2021-09-02 10:16 ` [PATCH 1/3] PM: domains: Drop the performance state vote for a device at detach Ulf Hansson @ 2021-09-02 10:16 ` Ulf Hansson 2021-09-02 10:16 ` [PATCH 3/3] PM: domains: Add a ->dev_get_performance_state() callback to genpd Ulf Hansson 2 siblings, 0 replies; 29+ messages in thread From: Ulf Hansson @ 2021-09-02 10:16 UTC (permalink / raw) To: Rafael J . Wysocki, Viresh Kumar, Dmitry Osipenko, linux-pm Cc: Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, linux-arm-kernel, linux-kernel, Ulf Hansson To slightly improve readability of the code, but also to prepare for a subsequent change on top, let's move the code that calls of_get_required_opp_performance_state() into a new separate function. Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> --- drivers/base/power/domain.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c index 278e040f607f..800adf831cae 100644 --- a/drivers/base/power/domain.c +++ b/drivers/base/power/domain.c @@ -2640,6 +2640,17 @@ static void genpd_dev_pm_sync(struct device *dev) genpd_queue_power_off_work(pd); } +static int genpd_get_default_performance_state(struct device *dev, + unsigned int index) +{ + int pstate = of_get_required_opp_performance_state(dev->of_node, index); + + if (pstate == -ENODEV || pstate == -EOPNOTSUPP) + return 0; + + return pstate; +} + static int __genpd_dev_pm_attach(struct device *dev, struct device *base_dev, unsigned int index, bool power_on) { @@ -2690,8 +2701,8 @@ static int __genpd_dev_pm_attach(struct device *dev, struct device *base_dev, } /* Set the default performance state */ - pstate = of_get_required_opp_performance_state(dev->of_node, index); - if (pstate < 0 && pstate != -ENODEV && pstate != -EOPNOTSUPP) { + pstate = genpd_get_default_performance_state(dev, index); + if (pstate < 0) { ret = pstate; goto err; } else if (pstate > 0) { -- 2.25.1 ^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH 3/3] PM: domains: Add a ->dev_get_performance_state() callback to genpd 2021-09-02 10:16 [PATCH 0/3] PM: domains: Improvements for performance states in genpd Ulf Hansson 2021-09-02 10:16 ` [PATCH 1/3] PM: domains: Drop the performance state vote for a device at detach Ulf Hansson 2021-09-02 10:16 ` [PATCH 2/3] PM: domains: Restructure some code in __genpd_dev_pm_attach() Ulf Hansson @ 2021-09-02 10:16 ` Ulf Hansson 2021-09-03 6:00 ` Dmitry Osipenko 2 siblings, 1 reply; 29+ messages in thread From: Ulf Hansson @ 2021-09-02 10:16 UTC (permalink / raw) To: Rafael J . Wysocki, Viresh Kumar, Dmitry Osipenko, linux-pm Cc: Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, linux-arm-kernel, linux-kernel, Ulf Hansson Hardware may be preprogrammed to a specific performance state, which may not be zero initially during boot. This may lead to that genpd's current performance state becomes inconsistent with the state of the hardware. To deal with this, the driver for a device that is being attached to its genpd, need to request an initial performance state vote, which is typically done by calling some of the OPP APIs while probing. In some cases this would lead to boilerplate code in the drivers. Let's make it possible to avoid this, by adding a new optional callback to genpd and invoke it per device during the attach process. In this way, the genpd provider driver can inform genpd about the initial performance state that is needed for the device. Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> --- drivers/base/power/domain.c | 8 +++++--- include/linux/pm_domain.h | 2 ++ 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c index 800adf831cae..1a6f3538af8d 100644 --- a/drivers/base/power/domain.c +++ b/drivers/base/power/domain.c @@ -2640,13 +2640,15 @@ static void genpd_dev_pm_sync(struct device *dev) genpd_queue_power_off_work(pd); } -static int genpd_get_default_performance_state(struct device *dev, +static int genpd_get_default_performance_state(struct generic_pm_domain *genpd, + struct device *dev, unsigned int index) { int pstate = of_get_required_opp_performance_state(dev->of_node, index); if (pstate == -ENODEV || pstate == -EOPNOTSUPP) - return 0; + pstate = genpd->dev_get_performance_state ? + genpd->dev_get_performance_state(genpd, dev) : 0; return pstate; } @@ -2701,7 +2703,7 @@ static int __genpd_dev_pm_attach(struct device *dev, struct device *base_dev, } /* Set the default performance state */ - pstate = genpd_get_default_performance_state(dev, index); + pstate = genpd_get_default_performance_state(pd, dev, index); if (pstate < 0) { ret = pstate; goto err; diff --git a/include/linux/pm_domain.h b/include/linux/pm_domain.h index 21a0577305ef..da694489a05a 100644 --- a/include/linux/pm_domain.h +++ b/include/linux/pm_domain.h @@ -131,6 +131,8 @@ struct generic_pm_domain { struct opp_table *opp_table; /* OPP table of the genpd */ unsigned int (*opp_to_performance_state)(struct generic_pm_domain *genpd, struct dev_pm_opp *opp); + int (*dev_get_performance_state)(struct generic_pm_domain *genpd, + struct device *dev); int (*set_performance_state)(struct generic_pm_domain *genpd, unsigned int state); struct gpd_dev_ops dev_ops; -- 2.25.1 ^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: [PATCH 3/3] PM: domains: Add a ->dev_get_performance_state() callback to genpd 2021-09-02 10:16 ` [PATCH 3/3] PM: domains: Add a ->dev_get_performance_state() callback to genpd Ulf Hansson @ 2021-09-03 6:00 ` Dmitry Osipenko 2021-09-03 8:55 ` Ulf Hansson 0 siblings, 1 reply; 29+ messages in thread From: Dmitry Osipenko @ 2021-09-03 6:00 UTC (permalink / raw) To: Ulf Hansson, Rafael J . Wysocki, Viresh Kumar, linux-pm Cc: Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, linux-arm-kernel, linux-kernel 02.09.2021 13:16, Ulf Hansson пишет: > Hardware may be preprogrammed to a specific performance state, which may > not be zero initially during boot. This may lead to that genpd's current > performance state becomes inconsistent with the state of the hardware. To > deal with this, the driver for a device that is being attached to its > genpd, need to request an initial performance state vote, which is > typically done by calling some of the OPP APIs while probing. > > In some cases this would lead to boilerplate code in the drivers. Let's > make it possible to avoid this, by adding a new optional callback to genpd > and invoke it per device during the attach process. In this way, the genpd > provider driver can inform genpd about the initial performance state that > is needed for the device. > > Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> > --- > drivers/base/power/domain.c | 8 +++++--- > include/linux/pm_domain.h | 2 ++ > 2 files changed, 7 insertions(+), 3 deletions(-) > > diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c > index 800adf831cae..1a6f3538af8d 100644 > --- a/drivers/base/power/domain.c > +++ b/drivers/base/power/domain.c > @@ -2640,13 +2640,15 @@ static void genpd_dev_pm_sync(struct device *dev) > genpd_queue_power_off_work(pd); > } > > -static int genpd_get_default_performance_state(struct device *dev, > +static int genpd_get_default_performance_state(struct generic_pm_domain *genpd, > + struct device *dev, > unsigned int index) > { > int pstate = of_get_required_opp_performance_state(dev->of_node, index); > > if (pstate == -ENODEV || pstate == -EOPNOTSUPP) > - return 0; > + pstate = genpd->dev_get_performance_state ? > + genpd->dev_get_performance_state(genpd, dev) : 0; > > return pstate; > } > @@ -2701,7 +2703,7 @@ static int __genpd_dev_pm_attach(struct device *dev, struct device *base_dev, > } > > /* Set the default performance state */ > - pstate = genpd_get_default_performance_state(dev, index); > + pstate = genpd_get_default_performance_state(pd, dev, index); If base device is suspended, then its performance state is zero. When device will be rpm-resumed, then its performance should be set to the default state. You're setting performance state of the wrong device, it should be the base device and not the virtual domain device. These all is handled properly by my patch [1]. Hence it's complicated for the reason. [1] https://patchwork.ozlabs.org/project/linux-tegra/patch/20210831135450.26070-5-digetx@gmail.com/ ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 3/3] PM: domains: Add a ->dev_get_performance_state() callback to genpd 2021-09-03 6:00 ` Dmitry Osipenko @ 2021-09-03 8:55 ` Ulf Hansson 2021-09-03 10:06 ` Dmitry Osipenko 0 siblings, 1 reply; 29+ messages in thread From: Ulf Hansson @ 2021-09-03 8:55 UTC (permalink / raw) To: Dmitry Osipenko Cc: Rafael J . Wysocki, Viresh Kumar, Linux PM, Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, Linux ARM, Linux Kernel Mailing List On Fri, 3 Sept 2021 at 08:00, Dmitry Osipenko <digetx@gmail.com> wrote: > > 02.09.2021 13:16, Ulf Hansson пишет: > > Hardware may be preprogrammed to a specific performance state, which may > > not be zero initially during boot. This may lead to that genpd's current > > performance state becomes inconsistent with the state of the hardware. To > > deal with this, the driver for a device that is being attached to its > > genpd, need to request an initial performance state vote, which is > > typically done by calling some of the OPP APIs while probing. > > > > In some cases this would lead to boilerplate code in the drivers. Let's > > make it possible to avoid this, by adding a new optional callback to genpd > > and invoke it per device during the attach process. In this way, the genpd > > provider driver can inform genpd about the initial performance state that > > is needed for the device. > > > > Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> > > --- > > drivers/base/power/domain.c | 8 +++++--- > > include/linux/pm_domain.h | 2 ++ > > 2 files changed, 7 insertions(+), 3 deletions(-) > > > > diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c > > index 800adf831cae..1a6f3538af8d 100644 > > --- a/drivers/base/power/domain.c > > +++ b/drivers/base/power/domain.c > > @@ -2640,13 +2640,15 @@ static void genpd_dev_pm_sync(struct device *dev) > > genpd_queue_power_off_work(pd); > > } > > > > -static int genpd_get_default_performance_state(struct device *dev, > > +static int genpd_get_default_performance_state(struct generic_pm_domain *genpd, > > + struct device *dev, > > unsigned int index) > > { > > int pstate = of_get_required_opp_performance_state(dev->of_node, index); > > > > if (pstate == -ENODEV || pstate == -EOPNOTSUPP) > > - return 0; > > + pstate = genpd->dev_get_performance_state ? > > + genpd->dev_get_performance_state(genpd, dev) : 0; > > > > return pstate; > > } > > @@ -2701,7 +2703,7 @@ static int __genpd_dev_pm_attach(struct device *dev, struct device *base_dev, > > } > > > > /* Set the default performance state */ > > - pstate = genpd_get_default_performance_state(dev, index); > > + pstate = genpd_get_default_performance_state(pd, dev, index); > > If base device is suspended, then its performance state is zero. > > When device will be rpm-resumed, then its performance should be set to > the default state. > > You're setting performance state of the wrong device, it should be the > base device and not the virtual domain device. No I am not. :-) Let me elaborate. For the single PM domain case, 'dev' and 'base_dev' are pointing to the same device. So this works fine. For the multiple PM domain case or when attaching goes via genpd_dev_pm_attach_by_id(), 'dev' is the virtual device registered in genpd_dev_pm_attach_by_id(). In this case, it's 'dev' that is becoming attached to genpd and not the 'base_dev'. Note also that, runtime PM has not been enabled for 'dev' yet at this point and 'dev' has been assigned the same OF node as 'base_dev", to allow OF parsing to work as is for it. Moreover, to deal with runtime PM in the multiple PM domain case, the consumer driver should create a device link. Along the lines of this: device_link_add(base_dev, dev, DL_FLAG_PM_RUNTIME | DL_FLAG_STATELESS), thus assigning the virtual device ('dev') as the supplier for its consumer device ('base_dev'). > > These all is handled properly by my patch [1]. Hence it's complicated > for the reason. See above. It shouldn't have to be complicated. If it still is, there is something to fix for the multiple PM domain case. > > [1] > https://patchwork.ozlabs.org/project/linux-tegra/patch/20210831135450.26070-5-digetx@gmail.com/ Kind regards Uffe ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 3/3] PM: domains: Add a ->dev_get_performance_state() callback to genpd 2021-09-03 8:55 ` Ulf Hansson @ 2021-09-03 10:06 ` Dmitry Osipenko 2021-09-03 14:09 ` Ulf Hansson 0 siblings, 1 reply; 29+ messages in thread From: Dmitry Osipenko @ 2021-09-03 10:06 UTC (permalink / raw) To: Ulf Hansson Cc: Rafael J . Wysocki, Viresh Kumar, Linux PM, Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, Linux ARM, Linux Kernel Mailing List 03.09.2021 11:55, Ulf Hansson пишет: > On Fri, 3 Sept 2021 at 08:00, Dmitry Osipenko <digetx@gmail.com> wrote: >> >> 02.09.2021 13:16, Ulf Hansson пишет: >>> Hardware may be preprogrammed to a specific performance state, which may >>> not be zero initially during boot. This may lead to that genpd's current >>> performance state becomes inconsistent with the state of the hardware. To >>> deal with this, the driver for a device that is being attached to its >>> genpd, need to request an initial performance state vote, which is >>> typically done by calling some of the OPP APIs while probing. >>> >>> In some cases this would lead to boilerplate code in the drivers. Let's >>> make it possible to avoid this, by adding a new optional callback to genpd >>> and invoke it per device during the attach process. In this way, the genpd >>> provider driver can inform genpd about the initial performance state that >>> is needed for the device. >>> >>> Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> >>> --- >>> drivers/base/power/domain.c | 8 +++++--- >>> include/linux/pm_domain.h | 2 ++ >>> 2 files changed, 7 insertions(+), 3 deletions(-) >>> >>> diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c >>> index 800adf831cae..1a6f3538af8d 100644 >>> --- a/drivers/base/power/domain.c >>> +++ b/drivers/base/power/domain.c >>> @@ -2640,13 +2640,15 @@ static void genpd_dev_pm_sync(struct device *dev) >>> genpd_queue_power_off_work(pd); >>> } >>> >>> -static int genpd_get_default_performance_state(struct device *dev, >>> +static int genpd_get_default_performance_state(struct generic_pm_domain *genpd, >>> + struct device *dev, >>> unsigned int index) >>> { >>> int pstate = of_get_required_opp_performance_state(dev->of_node, index); >>> >>> if (pstate == -ENODEV || pstate == -EOPNOTSUPP) >>> - return 0; >>> + pstate = genpd->dev_get_performance_state ? >>> + genpd->dev_get_performance_state(genpd, dev) : 0; >>> >>> return pstate; >>> } >>> @@ -2701,7 +2703,7 @@ static int __genpd_dev_pm_attach(struct device *dev, struct device *base_dev, >>> } >>> >>> /* Set the default performance state */ >>> - pstate = genpd_get_default_performance_state(dev, index); >>> + pstate = genpd_get_default_performance_state(pd, dev, index); >> >> If base device is suspended, then its performance state is zero. >> >> When device will be rpm-resumed, then its performance should be set to >> the default state. >> You're setting performance state of the wrong device, it should be the Are you okay with my variant of handling the suspended device? >> base device and not the virtual domain device. > > No I am not. :-) Let me elaborate. > > For the single PM domain case, 'dev' and 'base_dev' are pointing to > the same device. So this works fine. > > For the multiple PM domain case or when attaching goes via > genpd_dev_pm_attach_by_id(), 'dev' is the virtual device registered in > genpd_dev_pm_attach_by_id(). In this case, it's 'dev' that is becoming > attached to genpd and not the 'base_dev'. Note also that, runtime PM > has not been enabled for 'dev' yet at this point and 'dev' has been > assigned the same OF node as 'base_dev", to allow OF parsing to work > as is for it. > > Moreover, to deal with runtime PM in the multiple PM domain case, the > consumer driver should create a device link. Along the lines of this: > device_link_add(base_dev, dev, DL_FLAG_PM_RUNTIME | > DL_FLAG_STATELESS), thus assigning the virtual device ('dev') as the > supplier for its consumer device ('base_dev'). > >> >> These all is handled properly by my patch [1]. Hence it's complicated >> for the reason. > > See above. It shouldn't have to be complicated. If it still is, there > is something to fix for the multiple PM domain case. >> [1] Alright, it actually works now on Tegra using the dev in the callback for the case of multiple domains, I re-checked it. Previously, when I tried that, there was a conflict in regards to OPP usage, I don't remember details anymore. Maybe the recent changes that were suggested by Viresh helped with that. So yes, there is no need to pass the base device anymore. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 3/3] PM: domains: Add a ->dev_get_performance_state() callback to genpd 2021-09-03 10:06 ` Dmitry Osipenko @ 2021-09-03 14:09 ` Ulf Hansson 2021-09-05 9:11 ` Dmitry Osipenko 0 siblings, 1 reply; 29+ messages in thread From: Ulf Hansson @ 2021-09-03 14:09 UTC (permalink / raw) To: Dmitry Osipenko Cc: Rafael J . Wysocki, Viresh Kumar, Linux PM, Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, Linux ARM, Linux Kernel Mailing List On Fri, 3 Sept 2021 at 12:06, Dmitry Osipenko <digetx@gmail.com> wrote: > > 03.09.2021 11:55, Ulf Hansson пишет: > > On Fri, 3 Sept 2021 at 08:00, Dmitry Osipenko <digetx@gmail.com> wrote: > >> > >> 02.09.2021 13:16, Ulf Hansson пишет: > >>> Hardware may be preprogrammed to a specific performance state, which may > >>> not be zero initially during boot. This may lead to that genpd's current > >>> performance state becomes inconsistent with the state of the hardware. To > >>> deal with this, the driver for a device that is being attached to its > >>> genpd, need to request an initial performance state vote, which is > >>> typically done by calling some of the OPP APIs while probing. > >>> > >>> In some cases this would lead to boilerplate code in the drivers. Let's > >>> make it possible to avoid this, by adding a new optional callback to genpd > >>> and invoke it per device during the attach process. In this way, the genpd > >>> provider driver can inform genpd about the initial performance state that > >>> is needed for the device. > >>> > >>> Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> > >>> --- > >>> drivers/base/power/domain.c | 8 +++++--- > >>> include/linux/pm_domain.h | 2 ++ > >>> 2 files changed, 7 insertions(+), 3 deletions(-) > >>> > >>> diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c > >>> index 800adf831cae..1a6f3538af8d 100644 > >>> --- a/drivers/base/power/domain.c > >>> +++ b/drivers/base/power/domain.c > >>> @@ -2640,13 +2640,15 @@ static void genpd_dev_pm_sync(struct device *dev) > >>> genpd_queue_power_off_work(pd); > >>> } > >>> > >>> -static int genpd_get_default_performance_state(struct device *dev, > >>> +static int genpd_get_default_performance_state(struct generic_pm_domain *genpd, > >>> + struct device *dev, > >>> unsigned int index) > >>> { > >>> int pstate = of_get_required_opp_performance_state(dev->of_node, index); > >>> > >>> if (pstate == -ENODEV || pstate == -EOPNOTSUPP) > >>> - return 0; > >>> + pstate = genpd->dev_get_performance_state ? > >>> + genpd->dev_get_performance_state(genpd, dev) : 0; > >>> > >>> return pstate; > >>> } > >>> @@ -2701,7 +2703,7 @@ static int __genpd_dev_pm_attach(struct device *dev, struct device *base_dev, > >>> } > >>> > >>> /* Set the default performance state */ > >>> - pstate = genpd_get_default_performance_state(dev, index); > >>> + pstate = genpd_get_default_performance_state(pd, dev, index); > >> > >> If base device is suspended, then its performance state is zero. > >> > >> When device will be rpm-resumed, then its performance should be set to > >> the default state. > >> You're setting performance state of the wrong device, it should be the > Are you okay with my variant of handling the suspended device? Not sure if you intended to post this line? In any case, I am happy to help and review to move things forward. > > > >> base device and not the virtual domain device. > > > > No I am not. :-) Let me elaborate. > > > > For the single PM domain case, 'dev' and 'base_dev' are pointing to > > the same device. So this works fine. > > > > For the multiple PM domain case or when attaching goes via > > genpd_dev_pm_attach_by_id(), 'dev' is the virtual device registered in > > genpd_dev_pm_attach_by_id(). In this case, it's 'dev' that is becoming > > attached to genpd and not the 'base_dev'. Note also that, runtime PM > > has not been enabled for 'dev' yet at this point and 'dev' has been > > assigned the same OF node as 'base_dev", to allow OF parsing to work > > as is for it. > > > > Moreover, to deal with runtime PM in the multiple PM domain case, the > > consumer driver should create a device link. Along the lines of this: > > device_link_add(base_dev, dev, DL_FLAG_PM_RUNTIME | > > DL_FLAG_STATELESS), thus assigning the virtual device ('dev') as the > > supplier for its consumer device ('base_dev'). > > > >> > >> These all is handled properly by my patch [1]. Hence it's complicated > >> for the reason. > > > > See above. It shouldn't have to be complicated. If it still is, there > > is something to fix for the multiple PM domain case. > >> [1] > Alright, it actually works now on Tegra using the dev in the callback > for the case of multiple domains, I re-checked it. Previously, when I > tried that, there was a conflict in regards to OPP usage, I don't > remember details anymore. Maybe the recent changes that were suggested > by Viresh helped with that. So yes, there is no need to pass the base > device anymore. Great! So, it seems like $subject patch should be a way forward for you then? BTW, I forgot to add your Suggested-by: tag for the patch, you certainly deserve at least that. Or perhaps you are fine with co-developed by tag? Kind regards Uffe ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 3/3] PM: domains: Add a ->dev_get_performance_state() callback to genpd 2021-09-03 14:09 ` Ulf Hansson @ 2021-09-05 9:11 ` Dmitry Osipenko 2021-09-06 10:53 ` Ulf Hansson 0 siblings, 1 reply; 29+ messages in thread From: Dmitry Osipenko @ 2021-09-05 9:11 UTC (permalink / raw) To: Ulf Hansson Cc: Rafael J . Wysocki, Viresh Kumar, Linux PM, Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, Linux ARM, Linux Kernel Mailing List 03.09.2021 17:09, Ulf Hansson пишет: > On Fri, 3 Sept 2021 at 12:06, Dmitry Osipenko <digetx@gmail.com> wrote: >> >> 03.09.2021 11:55, Ulf Hansson пишет: >>> On Fri, 3 Sept 2021 at 08:00, Dmitry Osipenko <digetx@gmail.com> wrote: >>>> >>>> 02.09.2021 13:16, Ulf Hansson пишет: >>>>> Hardware may be preprogrammed to a specific performance state, which may >>>>> not be zero initially during boot. This may lead to that genpd's current >>>>> performance state becomes inconsistent with the state of the hardware. To >>>>> deal with this, the driver for a device that is being attached to its >>>>> genpd, need to request an initial performance state vote, which is >>>>> typically done by calling some of the OPP APIs while probing. >>>>> >>>>> In some cases this would lead to boilerplate code in the drivers. Let's >>>>> make it possible to avoid this, by adding a new optional callback to genpd >>>>> and invoke it per device during the attach process. In this way, the genpd >>>>> provider driver can inform genpd about the initial performance state that >>>>> is needed for the device. >>>>> >>>>> Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> >>>>> --- >>>>> drivers/base/power/domain.c | 8 +++++--- >>>>> include/linux/pm_domain.h | 2 ++ >>>>> 2 files changed, 7 insertions(+), 3 deletions(-) >>>>> >>>>> diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c >>>>> index 800adf831cae..1a6f3538af8d 100644 >>>>> --- a/drivers/base/power/domain.c >>>>> +++ b/drivers/base/power/domain.c >>>>> @@ -2640,13 +2640,15 @@ static void genpd_dev_pm_sync(struct device *dev) >>>>> genpd_queue_power_off_work(pd); >>>>> } >>>>> >>>>> -static int genpd_get_default_performance_state(struct device *dev, >>>>> +static int genpd_get_default_performance_state(struct generic_pm_domain *genpd, >>>>> + struct device *dev, >>>>> unsigned int index) >>>>> { >>>>> int pstate = of_get_required_opp_performance_state(dev->of_node, index); >>>>> >>>>> if (pstate == -ENODEV || pstate == -EOPNOTSUPP) >>>>> - return 0; >>>>> + pstate = genpd->dev_get_performance_state ? >>>>> + genpd->dev_get_performance_state(genpd, dev) : 0; >>>>> >>>>> return pstate; >>>>> } >>>>> @@ -2701,7 +2703,7 @@ static int __genpd_dev_pm_attach(struct device *dev, struct device *base_dev, >>>>> } >>>>> >>>>> /* Set the default performance state */ >>>>> - pstate = genpd_get_default_performance_state(dev, index); >>>>> + pstate = genpd_get_default_performance_state(pd, dev, index); >>>> >>>> If base device is suspended, then its performance state is zero. >>>> >>>> When device will be rpm-resumed, then its performance should be set to >>>> the default state. >>>> You're setting performance state of the wrong device, it should be the >> Are you okay with my variant of handling the suspended device? > > Not sure if you intended to post this line? > > In any case, I am happy to help and review to move things forward. It's not clear to me whether you omitted handling the case of rpm-suspended device on purpose or not. I think it should be a part of this patch, but sounds like you want to work on it separately, correct? >>>> base device and not the virtual domain device. >>> >>> No I am not. :-) Let me elaborate. >>> >>> For the single PM domain case, 'dev' and 'base_dev' are pointing to >>> the same device. So this works fine. >>> >>> For the multiple PM domain case or when attaching goes via >>> genpd_dev_pm_attach_by_id(), 'dev' is the virtual device registered in >>> genpd_dev_pm_attach_by_id(). In this case, it's 'dev' that is becoming >>> attached to genpd and not the 'base_dev'. Note also that, runtime PM >>> has not been enabled for 'dev' yet at this point and 'dev' has been >>> assigned the same OF node as 'base_dev", to allow OF parsing to work >>> as is for it. >>> >>> Moreover, to deal with runtime PM in the multiple PM domain case, the >>> consumer driver should create a device link. Along the lines of this: >>> device_link_add(base_dev, dev, DL_FLAG_PM_RUNTIME | >>> DL_FLAG_STATELESS), thus assigning the virtual device ('dev') as the >>> supplier for its consumer device ('base_dev'). >>> >>>> >>>> These all is handled properly by my patch [1]. Hence it's complicated >>>> for the reason. >>> >>> See above. It shouldn't have to be complicated. If it still is, there >>> is something to fix for the multiple PM domain case. >>>> [1] >> Alright, it actually works now on Tegra using the dev in the callback >> for the case of multiple domains, I re-checked it. Previously, when I >> tried that, there was a conflict in regards to OPP usage, I don't >> remember details anymore. Maybe the recent changes that were suggested >> by Viresh helped with that. So yes, there is no need to pass the base >> device anymore. > > Great! So, it seems like $subject patch should be a way forward for you then? The current behaviour is incorrect for Tegra because it needs to set the rpm_pstate for rpm-suspended device, instead of bumping the state immediately. Power management is defeated without it on Tegra because SoC will start to consume extra power while device that needs this power is suspended. Otherwise $subject looks okay. > BTW, I forgot to add your Suggested-by: tag for the patch, you > certainly deserve at least that. Or perhaps you are fine with > co-developed by tag? Either is fine. Although, won't it be easier to keep these PD patches within the Tegra series since it depends on them? I can pick up the patches into the next version of Tegra series. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 3/3] PM: domains: Add a ->dev_get_performance_state() callback to genpd 2021-09-05 9:11 ` Dmitry Osipenko @ 2021-09-06 10:53 ` Ulf Hansson 2021-09-06 14:35 ` Dmitry Osipenko 0 siblings, 1 reply; 29+ messages in thread From: Ulf Hansson @ 2021-09-06 10:53 UTC (permalink / raw) To: Dmitry Osipenko Cc: Rafael J . Wysocki, Viresh Kumar, Linux PM, Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, Linux ARM, Linux Kernel Mailing List On Sun, 5 Sept 2021 at 11:11, Dmitry Osipenko <digetx@gmail.com> wrote: > > 03.09.2021 17:09, Ulf Hansson пишет: > > On Fri, 3 Sept 2021 at 12:06, Dmitry Osipenko <digetx@gmail.com> wrote: > >> > >> 03.09.2021 11:55, Ulf Hansson пишет: > >>> On Fri, 3 Sept 2021 at 08:00, Dmitry Osipenko <digetx@gmail.com> wrote: > >>>> > >>>> 02.09.2021 13:16, Ulf Hansson пишет: > >>>>> Hardware may be preprogrammed to a specific performance state, which may > >>>>> not be zero initially during boot. This may lead to that genpd's current > >>>>> performance state becomes inconsistent with the state of the hardware. To > >>>>> deal with this, the driver for a device that is being attached to its > >>>>> genpd, need to request an initial performance state vote, which is > >>>>> typically done by calling some of the OPP APIs while probing. > >>>>> > >>>>> In some cases this would lead to boilerplate code in the drivers. Let's > >>>>> make it possible to avoid this, by adding a new optional callback to genpd > >>>>> and invoke it per device during the attach process. In this way, the genpd > >>>>> provider driver can inform genpd about the initial performance state that > >>>>> is needed for the device. > >>>>> > >>>>> Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> > >>>>> --- > >>>>> drivers/base/power/domain.c | 8 +++++--- > >>>>> include/linux/pm_domain.h | 2 ++ > >>>>> 2 files changed, 7 insertions(+), 3 deletions(-) > >>>>> > >>>>> diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c > >>>>> index 800adf831cae..1a6f3538af8d 100644 > >>>>> --- a/drivers/base/power/domain.c > >>>>> +++ b/drivers/base/power/domain.c > >>>>> @@ -2640,13 +2640,15 @@ static void genpd_dev_pm_sync(struct device *dev) > >>>>> genpd_queue_power_off_work(pd); > >>>>> } > >>>>> > >>>>> -static int genpd_get_default_performance_state(struct device *dev, > >>>>> +static int genpd_get_default_performance_state(struct generic_pm_domain *genpd, > >>>>> + struct device *dev, > >>>>> unsigned int index) > >>>>> { > >>>>> int pstate = of_get_required_opp_performance_state(dev->of_node, index); > >>>>> > >>>>> if (pstate == -ENODEV || pstate == -EOPNOTSUPP) > >>>>> - return 0; > >>>>> + pstate = genpd->dev_get_performance_state ? > >>>>> + genpd->dev_get_performance_state(genpd, dev) : 0; > >>>>> > >>>>> return pstate; > >>>>> } > >>>>> @@ -2701,7 +2703,7 @@ static int __genpd_dev_pm_attach(struct device *dev, struct device *base_dev, > >>>>> } > >>>>> > >>>>> /* Set the default performance state */ > >>>>> - pstate = genpd_get_default_performance_state(dev, index); > >>>>> + pstate = genpd_get_default_performance_state(pd, dev, index); > >>>> > >>>> If base device is suspended, then its performance state is zero. > >>>> > >>>> When device will be rpm-resumed, then its performance should be set to > >>>> the default state. > >>>> You're setting performance state of the wrong device, it should be the > >> Are you okay with my variant of handling the suspended device? > > > > Not sure if you intended to post this line? > > > > In any case, I am happy to help and review to move things forward. > > It's not clear to me whether you omitted handling the case of > rpm-suspended device on purpose or not. I think it should be a part of > this patch, but sounds like you want to work on it separately, correct? I didn't omit the handling, but instead relied solely on the pm_runtime_suspended() check in dev_pm_genpd_set_performance_state(). > > >>>> base device and not the virtual domain device. > >>> > >>> No I am not. :-) Let me elaborate. > >>> > >>> For the single PM domain case, 'dev' and 'base_dev' are pointing to > >>> the same device. So this works fine. > >>> > >>> For the multiple PM domain case or when attaching goes via > >>> genpd_dev_pm_attach_by_id(), 'dev' is the virtual device registered in > >>> genpd_dev_pm_attach_by_id(). In this case, it's 'dev' that is becoming > >>> attached to genpd and not the 'base_dev'. Note also that, runtime PM > >>> has not been enabled for 'dev' yet at this point and 'dev' has been > >>> assigned the same OF node as 'base_dev", to allow OF parsing to work > >>> as is for it. > >>> > >>> Moreover, to deal with runtime PM in the multiple PM domain case, the > >>> consumer driver should create a device link. Along the lines of this: > >>> device_link_add(base_dev, dev, DL_FLAG_PM_RUNTIME | > >>> DL_FLAG_STATELESS), thus assigning the virtual device ('dev') as the > >>> supplier for its consumer device ('base_dev'). > >>> > >>>> > >>>> These all is handled properly by my patch [1]. Hence it's complicated > >>>> for the reason. > >>> > >>> See above. It shouldn't have to be complicated. If it still is, there > >>> is something to fix for the multiple PM domain case. > >>>> [1] > >> Alright, it actually works now on Tegra using the dev in the callback > >> for the case of multiple domains, I re-checked it. Previously, when I > >> tried that, there was a conflict in regards to OPP usage, I don't > >> remember details anymore. Maybe the recent changes that were suggested > >> by Viresh helped with that. So yes, there is no need to pass the base > >> device anymore. > > > > Great! So, it seems like $subject patch should be a way forward for you then? > > The current behaviour is incorrect for Tegra because it needs to set the > rpm_pstate for rpm-suspended device, instead of bumping the state > immediately. > > Power management is defeated without it on Tegra because SoC will start > to consume extra power while device that needs this power is suspended. Okay, I understand your concern. For devices that may remain runtime suspended when their consumer drivers probes them, the behaviour may be suboptimal. This because it could lead to having an active performance state vote for a runtime suspended device, at least until it gets runtime resumed and then runtime suspended again. This all boils down to how the consumer driver deploys support for runtime PM - and genpd doesn't know nor can control that. I wonder if we perhaps should just leave this as is then. In other words, rely on the consumer driver to vote for an initial performance state of the device during ->probe(). In this way, the consumer driver can decide what is the best thing to do, rather than letting genpd make guesses. Note that, comparing what we have done for power on/off during attach/probe. For the legacy case (the single PM domain case) we power on the PM domain. For the multiple PM domain case, we leave the PM domain as is. > > Otherwise $subject looks okay. > > > BTW, I forgot to add your Suggested-by: tag for the patch, you > > certainly deserve at least that. Or perhaps you are fine with > > co-developed by tag? > > Either is fine. Although, won't it be easier to keep these PD patches > within the Tegra series since it depends on them? I can pick up the > patches into the next version of Tegra series. Let's see. If we can agree on how to deal with this series, maybe we can ask Rafael to send another pull-request before rc1 is out, so you can base your series for Tegra more easily on top. Kind regards Uffe ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 3/3] PM: domains: Add a ->dev_get_performance_state() callback to genpd 2021-09-06 10:53 ` Ulf Hansson @ 2021-09-06 14:35 ` Dmitry Osipenko 2021-09-07 3:40 ` Viresh Kumar 2021-09-07 9:57 ` Ulf Hansson 0 siblings, 2 replies; 29+ messages in thread From: Dmitry Osipenko @ 2021-09-06 14:35 UTC (permalink / raw) To: Ulf Hansson, Viresh Kumar Cc: Rafael J . Wysocki, Linux PM, Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, Linux ARM, Linux Kernel Mailing List 06.09.2021 13:53, Ulf Hansson пишет: > On Sun, 5 Sept 2021 at 11:11, Dmitry Osipenko <digetx@gmail.com> wrote: >> >> 03.09.2021 17:09, Ulf Hansson пишет: >>> On Fri, 3 Sept 2021 at 12:06, Dmitry Osipenko <digetx@gmail.com> wrote: >>>> >>>> 03.09.2021 11:55, Ulf Hansson пишет: >>>>> On Fri, 3 Sept 2021 at 08:00, Dmitry Osipenko <digetx@gmail.com> wrote: >>>>>> >>>>>> 02.09.2021 13:16, Ulf Hansson пишет: >>>>>>> Hardware may be preprogrammed to a specific performance state, which may >>>>>>> not be zero initially during boot. This may lead to that genpd's current >>>>>>> performance state becomes inconsistent with the state of the hardware. To >>>>>>> deal with this, the driver for a device that is being attached to its >>>>>>> genpd, need to request an initial performance state vote, which is >>>>>>> typically done by calling some of the OPP APIs while probing. >>>>>>> >>>>>>> In some cases this would lead to boilerplate code in the drivers. Let's >>>>>>> make it possible to avoid this, by adding a new optional callback to genpd >>>>>>> and invoke it per device during the attach process. In this way, the genpd >>>>>>> provider driver can inform genpd about the initial performance state that >>>>>>> is needed for the device. >>>>>>> >>>>>>> Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> >>>>>>> --- >>>>>>> drivers/base/power/domain.c | 8 +++++--- >>>>>>> include/linux/pm_domain.h | 2 ++ >>>>>>> 2 files changed, 7 insertions(+), 3 deletions(-) >>>>>>> >>>>>>> diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c >>>>>>> index 800adf831cae..1a6f3538af8d 100644 >>>>>>> --- a/drivers/base/power/domain.c >>>>>>> +++ b/drivers/base/power/domain.c >>>>>>> @@ -2640,13 +2640,15 @@ static void genpd_dev_pm_sync(struct device *dev) >>>>>>> genpd_queue_power_off_work(pd); >>>>>>> } >>>>>>> >>>>>>> -static int genpd_get_default_performance_state(struct device *dev, >>>>>>> +static int genpd_get_default_performance_state(struct generic_pm_domain *genpd, >>>>>>> + struct device *dev, >>>>>>> unsigned int index) >>>>>>> { >>>>>>> int pstate = of_get_required_opp_performance_state(dev->of_node, index); >>>>>>> >>>>>>> if (pstate == -ENODEV || pstate == -EOPNOTSUPP) >>>>>>> - return 0; >>>>>>> + pstate = genpd->dev_get_performance_state ? >>>>>>> + genpd->dev_get_performance_state(genpd, dev) : 0; >>>>>>> >>>>>>> return pstate; >>>>>>> } >>>>>>> @@ -2701,7 +2703,7 @@ static int __genpd_dev_pm_attach(struct device *dev, struct device *base_dev, >>>>>>> } >>>>>>> >>>>>>> /* Set the default performance state */ >>>>>>> - pstate = genpd_get_default_performance_state(dev, index); >>>>>>> + pstate = genpd_get_default_performance_state(pd, dev, index); >>>>>> >>>>>> If base device is suspended, then its performance state is zero. >>>>>> >>>>>> When device will be rpm-resumed, then its performance should be set to >>>>>> the default state. >>>>>> You're setting performance state of the wrong device, it should be the >>>> Are you okay with my variant of handling the suspended device? >>> >>> Not sure if you intended to post this line? >>> >>> In any case, I am happy to help and review to move things forward. >> >> It's not clear to me whether you omitted handling the case of >> rpm-suspended device on purpose or not. I think it should be a part of >> this patch, but sounds like you want to work on it separately, correct? > > I didn't omit the handling, but instead relied solely on the > pm_runtime_suspended() check in dev_pm_genpd_set_performance_state(). It doesn't work as expected for Tegra because pm_runtime_suspended() returns false while RPM is disabled and it's normally disabled at the attachment time. >>>>>> base device and not the virtual domain device. >>>>> >>>>> No I am not. :-) Let me elaborate. >>>>> >>>>> For the single PM domain case, 'dev' and 'base_dev' are pointing to >>>>> the same device. So this works fine. >>>>> >>>>> For the multiple PM domain case or when attaching goes via >>>>> genpd_dev_pm_attach_by_id(), 'dev' is the virtual device registered in >>>>> genpd_dev_pm_attach_by_id(). In this case, it's 'dev' that is becoming >>>>> attached to genpd and not the 'base_dev'. Note also that, runtime PM >>>>> has not been enabled for 'dev' yet at this point and 'dev' has been >>>>> assigned the same OF node as 'base_dev", to allow OF parsing to work >>>>> as is for it. >>>>> >>>>> Moreover, to deal with runtime PM in the multiple PM domain case, the >>>>> consumer driver should create a device link. Along the lines of this: >>>>> device_link_add(base_dev, dev, DL_FLAG_PM_RUNTIME | >>>>> DL_FLAG_STATELESS), thus assigning the virtual device ('dev') as the >>>>> supplier for its consumer device ('base_dev'). >>>>> >>>>>> >>>>>> These all is handled properly by my patch [1]. Hence it's complicated >>>>>> for the reason. >>>>> >>>>> See above. It shouldn't have to be complicated. If it still is, there >>>>> is something to fix for the multiple PM domain case. >>>>>> [1] >>>> Alright, it actually works now on Tegra using the dev in the callback >>>> for the case of multiple domains, I re-checked it. Previously, when I >>>> tried that, there was a conflict in regards to OPP usage, I don't >>>> remember details anymore. Maybe the recent changes that were suggested >>>> by Viresh helped with that. So yes, there is no need to pass the base >>>> device anymore. >>> >>> Great! So, it seems like $subject patch should be a way forward for you then? >> >> The current behaviour is incorrect for Tegra because it needs to set the >> rpm_pstate for rpm-suspended device, instead of bumping the state >> immediately. >> >> Power management is defeated without it on Tegra because SoC will start >> to consume extra power while device that needs this power is suspended. > > Okay, I understand your concern. > > For devices that may remain runtime suspended when their consumer > drivers probes them, the behaviour may be suboptimal. This because it > could lead to having an active performance state vote for a runtime > suspended device, at least until it gets runtime resumed and then > runtime suspended again. > > This all boils down to how the consumer driver deploys support for > runtime PM - and genpd doesn't know nor can control that. Previously, I added the 'dev_suspended' argument to the dev_get_performance_state() callback to allow PD driver to decide whether state should applied immediately or on rpm-resume, but you asked to remove it because it didn't make sense to you [1]. [1] https://lore.kernel.org/linux-pm/CAPDyKFo=SFpm+uJYH4UDfKWLVnkP2cKkBcbOQeVhU5hRxHUMCw@mail.gmail.com/ Does it make sense now? > I wonder if we perhaps should just leave this as is then. In other > words, rely on the consumer driver to vote for an initial performance > state of the device during ->probe(). In this way, the consumer driver > can decide what is the best thing to do, rather than letting genpd > make guesses. The point of this series is to remove the boilerplate code from consumer drivers. I already implemented variant with the explicit state syncing done by consumer drivers, but Viresh suggested that it should be done by the PD driver, this is why we're discussing it all over again. We either need to add quirks to consumer drivers or make PD API more flexible. You're not in favor of extending the PD API. To me the variant with the PD API extension is a bit nicer since it removes the boilerplate code, but I also see why you don't like it. Viresh, are you okay with going back to the variant with the dev_pm_opp_sync() helper? > Note that, comparing what we have done for power on/off during > attach/probe. For the legacy case (the single PM domain case) we power > on the PM domain. For the multiple PM domain case, we leave the PM > domain as is. That is a similar situation here. If PD driver doesn't implement the new dev_get_performance_state() callback, then nothing is changed for the consumer drivers. >> Otherwise $subject looks okay. >> >>> BTW, I forgot to add your Suggested-by: tag for the patch, you >>> certainly deserve at least that. Or perhaps you are fine with >>> co-developed by tag? >> >> Either is fine. Although, won't it be easier to keep these PD patches >> within the Tegra series since it depends on them? I can pick up the >> patches into the next version of Tegra series. > > Let's see. If we can agree on how to deal with this series, maybe we > can ask Rafael to send another pull-request before rc1 is out, so you > can base your series for Tegra more easily on top. Alright ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 3/3] PM: domains: Add a ->dev_get_performance_state() callback to genpd 2021-09-06 14:35 ` Dmitry Osipenko @ 2021-09-07 3:40 ` Viresh Kumar 2021-09-07 8:10 ` Dmitry Osipenko 2021-09-07 9:57 ` Ulf Hansson 1 sibling, 1 reply; 29+ messages in thread From: Viresh Kumar @ 2021-09-07 3:40 UTC (permalink / raw) To: Dmitry Osipenko Cc: Ulf Hansson, Rafael J . Wysocki, Linux PM, Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, Linux ARM, Linux Kernel Mailing List On 06-09-21, 17:35, Dmitry Osipenko wrote: > Viresh, are you okay with going back to the variant with the > dev_pm_opp_sync() helper? I have missed a lot of stuff in between and wasn't following this carefully as I thought my half was resolved :) Can you describe what to propose to do again ? From what I remember, doing this one time from probe() is okay, doing it from suspend/resume, not so much. -- viresh ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 3/3] PM: domains: Add a ->dev_get_performance_state() callback to genpd 2021-09-07 3:40 ` Viresh Kumar @ 2021-09-07 8:10 ` Dmitry Osipenko 0 siblings, 0 replies; 29+ messages in thread From: Dmitry Osipenko @ 2021-09-07 8:10 UTC (permalink / raw) To: Viresh Kumar, Ulf Hansson Cc: Rafael J . Wysocki, Linux PM, Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, Linux ARM, Linux Kernel Mailing List 07.09.2021 06:40, Viresh Kumar пишет: > On 06-09-21, 17:35, Dmitry Osipenko wrote: >> Viresh, are you okay with going back to the variant with the >> dev_pm_opp_sync() helper? > > I have missed a lot of stuff in between and wasn't following this > carefully as I thought my half was resolved :) > > Can you describe what to propose to do again ? From what I remember, > doing this one time from probe() is okay, doing it from > suspend/resume, not so much. Hmm.. actually, it's not a problem to set up the performance state from probe() now with that recent change that was made to the PD core. [1] [1] https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?id=3c5a272202c28c1f9309566f206ba40787246149 And then we indeed don't need neither the dev_get_performance_state() callback, nor the dev_pm_opp_sync() helper. The devm_tegra_core_dev_init_opp_table() already supports performance state syncing, so I will just need to call it after the RPM setup made by consumer driver, allowing PD core to set the rpm_pstate. I already gave it a quick test and it works perfectly. Ulf, are you okay with abandoning the dev_get_performance_state() callback? We don't need it anymore. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 3/3] PM: domains: Add a ->dev_get_performance_state() callback to genpd 2021-09-06 14:35 ` Dmitry Osipenko 2021-09-07 3:40 ` Viresh Kumar @ 2021-09-07 9:57 ` Ulf Hansson 2021-09-09 13:48 ` Dmitry Osipenko 1 sibling, 1 reply; 29+ messages in thread From: Ulf Hansson @ 2021-09-07 9:57 UTC (permalink / raw) To: Dmitry Osipenko Cc: Viresh Kumar, Rafael J . Wysocki, Linux PM, Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, Linux ARM, Linux Kernel Mailing List On Mon, 6 Sept 2021 at 16:35, Dmitry Osipenko <digetx@gmail.com> wrote: > > 06.09.2021 13:53, Ulf Hansson пишет: > > On Sun, 5 Sept 2021 at 11:11, Dmitry Osipenko <digetx@gmail.com> wrote: > >> > >> 03.09.2021 17:09, Ulf Hansson пишет: > >>> On Fri, 3 Sept 2021 at 12:06, Dmitry Osipenko <digetx@gmail.com> wrote: > >>>> > >>>> 03.09.2021 11:55, Ulf Hansson пишет: > >>>>> On Fri, 3 Sept 2021 at 08:00, Dmitry Osipenko <digetx@gmail.com> wrote: > >>>>>> > >>>>>> 02.09.2021 13:16, Ulf Hansson пишет: > >>>>>>> Hardware may be preprogrammed to a specific performance state, which may > >>>>>>> not be zero initially during boot. This may lead to that genpd's current > >>>>>>> performance state becomes inconsistent with the state of the hardware. To > >>>>>>> deal with this, the driver for a device that is being attached to its > >>>>>>> genpd, need to request an initial performance state vote, which is > >>>>>>> typically done by calling some of the OPP APIs while probing. > >>>>>>> > >>>>>>> In some cases this would lead to boilerplate code in the drivers. Let's > >>>>>>> make it possible to avoid this, by adding a new optional callback to genpd > >>>>>>> and invoke it per device during the attach process. In this way, the genpd > >>>>>>> provider driver can inform genpd about the initial performance state that > >>>>>>> is needed for the device. > >>>>>>> > >>>>>>> Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> > >>>>>>> --- > >>>>>>> drivers/base/power/domain.c | 8 +++++--- > >>>>>>> include/linux/pm_domain.h | 2 ++ > >>>>>>> 2 files changed, 7 insertions(+), 3 deletions(-) > >>>>>>> > >>>>>>> diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c > >>>>>>> index 800adf831cae..1a6f3538af8d 100644 > >>>>>>> --- a/drivers/base/power/domain.c > >>>>>>> +++ b/drivers/base/power/domain.c > >>>>>>> @@ -2640,13 +2640,15 @@ static void genpd_dev_pm_sync(struct device *dev) > >>>>>>> genpd_queue_power_off_work(pd); > >>>>>>> } > >>>>>>> > >>>>>>> -static int genpd_get_default_performance_state(struct device *dev, > >>>>>>> +static int genpd_get_default_performance_state(struct generic_pm_domain *genpd, > >>>>>>> + struct device *dev, > >>>>>>> unsigned int index) > >>>>>>> { > >>>>>>> int pstate = of_get_required_opp_performance_state(dev->of_node, index); > >>>>>>> > >>>>>>> if (pstate == -ENODEV || pstate == -EOPNOTSUPP) > >>>>>>> - return 0; > >>>>>>> + pstate = genpd->dev_get_performance_state ? > >>>>>>> + genpd->dev_get_performance_state(genpd, dev) : 0; > >>>>>>> > >>>>>>> return pstate; > >>>>>>> } > >>>>>>> @@ -2701,7 +2703,7 @@ static int __genpd_dev_pm_attach(struct device *dev, struct device *base_dev, > >>>>>>> } > >>>>>>> > >>>>>>> /* Set the default performance state */ > >>>>>>> - pstate = genpd_get_default_performance_state(dev, index); > >>>>>>> + pstate = genpd_get_default_performance_state(pd, dev, index); > >>>>>> > >>>>>> If base device is suspended, then its performance state is zero. > >>>>>> > >>>>>> When device will be rpm-resumed, then its performance should be set to > >>>>>> the default state. > >>>>>> You're setting performance state of the wrong device, it should be the > >>>> Are you okay with my variant of handling the suspended device? > >>> > >>> Not sure if you intended to post this line? > >>> > >>> In any case, I am happy to help and review to move things forward. > >> > >> It's not clear to me whether you omitted handling the case of > >> rpm-suspended device on purpose or not. I think it should be a part of > >> this patch, but sounds like you want to work on it separately, correct? > > > > I didn't omit the handling, but instead relied solely on the > > pm_runtime_suspended() check in dev_pm_genpd_set_performance_state(). > > It doesn't work as expected for Tegra because pm_runtime_suspended() > returns false while RPM is disabled and it's normally disabled at the > attachment time. Runtime PM is in most cases (probably all) not enabled for the device when attaching. This isn't specific to Tegra, but a common behavior of how it works during attach. > > >>>>>> base device and not the virtual domain device. > >>>>> > >>>>> No I am not. :-) Let me elaborate. > >>>>> > >>>>> For the single PM domain case, 'dev' and 'base_dev' are pointing to > >>>>> the same device. So this works fine. > >>>>> > >>>>> For the multiple PM domain case or when attaching goes via > >>>>> genpd_dev_pm_attach_by_id(), 'dev' is the virtual device registered in > >>>>> genpd_dev_pm_attach_by_id(). In this case, it's 'dev' that is becoming > >>>>> attached to genpd and not the 'base_dev'. Note also that, runtime PM > >>>>> has not been enabled for 'dev' yet at this point and 'dev' has been > >>>>> assigned the same OF node as 'base_dev", to allow OF parsing to work > >>>>> as is for it. > >>>>> > >>>>> Moreover, to deal with runtime PM in the multiple PM domain case, the > >>>>> consumer driver should create a device link. Along the lines of this: > >>>>> device_link_add(base_dev, dev, DL_FLAG_PM_RUNTIME | > >>>>> DL_FLAG_STATELESS), thus assigning the virtual device ('dev') as the > >>>>> supplier for its consumer device ('base_dev'). > >>>>> > >>>>>> > >>>>>> These all is handled properly by my patch [1]. Hence it's complicated > >>>>>> for the reason. > >>>>> > >>>>> See above. It shouldn't have to be complicated. If it still is, there > >>>>> is something to fix for the multiple PM domain case. > >>>>>> [1] > >>>> Alright, it actually works now on Tegra using the dev in the callback > >>>> for the case of multiple domains, I re-checked it. Previously, when I > >>>> tried that, there was a conflict in regards to OPP usage, I don't > >>>> remember details anymore. Maybe the recent changes that were suggested > >>>> by Viresh helped with that. So yes, there is no need to pass the base > >>>> device anymore. > >>> > >>> Great! So, it seems like $subject patch should be a way forward for you then? > >> > >> The current behaviour is incorrect for Tegra because it needs to set the > >> rpm_pstate for rpm-suspended device, instead of bumping the state > >> immediately. > >> > >> Power management is defeated without it on Tegra because SoC will start > >> to consume extra power while device that needs this power is suspended. > > > > Okay, I understand your concern. > > > > For devices that may remain runtime suspended when their consumer > > drivers probes them, the behaviour may be suboptimal. This because it > > could lead to having an active performance state vote for a runtime > > suspended device, at least until it gets runtime resumed and then > > runtime suspended again. > > > > This all boils down to how the consumer driver deploys support for > > runtime PM - and genpd doesn't know nor can control that. > > Previously, I added the 'dev_suspended' argument to the > dev_get_performance_state() callback to allow PD driver to decide > whether state should applied immediately or on rpm-resume, but you asked > to remove it because it didn't make sense to you [1]. > > [1] > https://lore.kernel.org/linux-pm/CAPDyKFo=SFpm+uJYH4UDfKWLVnkP2cKkBcbOQeVhU5hRxHUMCw@mail.gmail > > Does it make sense now? Unfortunately, no, it still doesn't. Let me try to elaborate why below. > > > I wonder if we perhaps should just leave this as is then. In other > > words, rely on the consumer driver to vote for an initial performance > > state of the device during ->probe(). In this way, the consumer driver > > can decide what is the best thing to do, rather than letting genpd > > make guesses. > > The point of this series is to remove the boilerplate code from consumer > drivers. > > I already implemented variant with the explicit state syncing done by > consumer drivers, but Viresh suggested that it should be done by the PD > driver, this is why we're discussing it all over again. > > We either need to add quirks to consumer drivers or make PD API more > flexible. You're not in favor of extending the PD API. To me the variant > with the PD API extension is a bit nicer since it removes the > boilerplate code, but I also see why you don't like it. I don't mind extending the genpd API, but it needs to serve a good purpose. As I said earlier, genpd doesn't know nor can control how the consumer driver deploys runtime PM. Unfortunately, that also includes genpd providers, as the behavior isn't a platform or PM domain specific thing. This means genpd needs to be generic enough so it works for all cases. In the $subject patch, we rely on the pm_runtime_suspended() check in dev_pm_genpd_set_performance_state(), which should work for all cases, even if it may be sub-optimal for some scenarios. Note that, in the approach your suggested [1], pm_runtime_status_suspended() is used instead. This doesn't work when a consumer driver doesn't enable runtime PM - or calls pm_runtime_set_active() during ->probe(), because genpd_runtime_resume() won't be invoked to restore the gpd->rpm_state. That said, I wouldn't mind to simply skip adding the ->dev_get_performance_state() all together, if that is what you prefer? In this way, it becomes the responsibility for the consumer driver to do right thing, with the cost of some boilerplate code added in its ->probe() routine. > > Viresh, are you okay with going back to the variant with the > dev_pm_opp_sync() helper? Rather than trying this again, I would suggest you start by open coding these parts, for now. But I leave that to Viresh to decide. [...] Kind regards Uffe [1] [PATCH v10 4/8] PM: domains: Add dev_get_performance_state() callback ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 3/3] PM: domains: Add a ->dev_get_performance_state() callback to genpd 2021-09-07 9:57 ` Ulf Hansson @ 2021-09-09 13:48 ` Dmitry Osipenko 2021-09-09 14:39 ` Ulf Hansson 0 siblings, 1 reply; 29+ messages in thread From: Dmitry Osipenko @ 2021-09-09 13:48 UTC (permalink / raw) To: Ulf Hansson Cc: Viresh Kumar, Rafael J . Wysocki, Linux PM, Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, Linux ARM, Linux Kernel Mailing List 07.09.2021 12:57, Ulf Hansson пишет: > I don't mind extending the genpd API, but it needs to serve a good purpose. > > As I said earlier, genpd doesn't know nor can control how the consumer > driver deploys runtime PM. Unfortunately, that also includes genpd > providers, as the behavior isn't a platform or PM domain specific > thing. This means genpd needs to be generic enough so it works for all > cases. > > In the $subject patch, we rely on the pm_runtime_suspended() check in > dev_pm_genpd_set_performance_state(), which should work for all cases, > even if it may be sub-optimal for some scenarios. > > Note that, in the approach your suggested [1], > pm_runtime_status_suspended() is used instead. This doesn't work when > a consumer driver doesn't enable runtime PM - or calls > pm_runtime_set_active() during ->probe(), because > genpd_runtime_resume() won't be invoked to restore the gpd->rpm_state. > > That said, I wouldn't mind to simply skip adding the > ->dev_get_performance_state() all together, if that is what you > prefer? In this way, it becomes the responsibility for the consumer > driver to do right thing, with the cost of some boilerplate code added > in its ->probe() routine. Until a day ago, it wasn't clear to me that consumer drivers now can set up rpm_pstate during probe(), which is a cleaner solution that works well. So let's skip adding the questionable ->dev_get_performance_state(). The boilerplate code in the probe() is minimal in comparison to a previous variant with the state-syncing done by rpm-resume callbacks of consumer drivers, it's good enough. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 3/3] PM: domains: Add a ->dev_get_performance_state() callback to genpd 2021-09-09 13:48 ` Dmitry Osipenko @ 2021-09-09 14:39 ` Ulf Hansson 2021-09-10 11:24 ` Dmitry Osipenko 0 siblings, 1 reply; 29+ messages in thread From: Ulf Hansson @ 2021-09-09 14:39 UTC (permalink / raw) To: Dmitry Osipenko Cc: Viresh Kumar, Rafael J . Wysocki, Linux PM, Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, Linux ARM, Linux Kernel Mailing List On Thu, 9 Sept 2021 at 15:48, Dmitry Osipenko <digetx@gmail.com> wrote: > > 07.09.2021 12:57, Ulf Hansson пишет: > > I don't mind extending the genpd API, but it needs to serve a good purpose. > > > > As I said earlier, genpd doesn't know nor can control how the consumer > > driver deploys runtime PM. Unfortunately, that also includes genpd > > providers, as the behavior isn't a platform or PM domain specific > > thing. This means genpd needs to be generic enough so it works for all > > cases. > > > > In the $subject patch, we rely on the pm_runtime_suspended() check in > > dev_pm_genpd_set_performance_state(), which should work for all cases, > > even if it may be sub-optimal for some scenarios. > > > > Note that, in the approach your suggested [1], > > pm_runtime_status_suspended() is used instead. This doesn't work when > > a consumer driver doesn't enable runtime PM - or calls > > pm_runtime_set_active() during ->probe(), because > > genpd_runtime_resume() won't be invoked to restore the gpd->rpm_state. > > > > That said, I wouldn't mind to simply skip adding the > > ->dev_get_performance_state() all together, if that is what you > > prefer? In this way, it becomes the responsibility for the consumer > > driver to do right thing, with the cost of some boilerplate code added > > in its ->probe() routine. > > Until a day ago, it wasn't clear to me that consumer drivers now can set > up rpm_pstate during probe(), which is a cleaner solution that works > well. So let's skip adding the questionable ->dev_get_performance_state(). > > The boilerplate code in the probe() is minimal in comparison to a > previous variant with the state-syncing done by rpm-resume callbacks of > consumer drivers, it's good enough. Alright, that sounds good to me as well. I am happy to help with review of the consumer driver changes, just keep me posted. Thanks and kind regards Uffe ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 3/3] PM: domains: Add a ->dev_get_performance_state() callback to genpd 2021-09-09 14:39 ` Ulf Hansson @ 2021-09-10 11:24 ` Dmitry Osipenko 0 siblings, 0 replies; 29+ messages in thread From: Dmitry Osipenko @ 2021-09-10 11:24 UTC (permalink / raw) To: Ulf Hansson Cc: Viresh Kumar, Rafael J . Wysocki, Linux PM, Dmitry Baryshkov, Jonathan Hunter, Thierry Reding, Rajendra Nayak, Stephan Gerhold, Bjorn Andersson, Linux ARM, Linux Kernel Mailing List 09.09.2021 17:39, Ulf Hansson пишет: > On Thu, 9 Sept 2021 at 15:48, Dmitry Osipenko <digetx@gmail.com> wrote: >> >> 07.09.2021 12:57, Ulf Hansson пишет: >>> I don't mind extending the genpd API, but it needs to serve a good purpose. >>> >>> As I said earlier, genpd doesn't know nor can control how the consumer >>> driver deploys runtime PM. Unfortunately, that also includes genpd >>> providers, as the behavior isn't a platform or PM domain specific >>> thing. This means genpd needs to be generic enough so it works for all >>> cases. >>> >>> In the $subject patch, we rely on the pm_runtime_suspended() check in >>> dev_pm_genpd_set_performance_state(), which should work for all cases, >>> even if it may be sub-optimal for some scenarios. >>> >>> Note that, in the approach your suggested [1], >>> pm_runtime_status_suspended() is used instead. This doesn't work when >>> a consumer driver doesn't enable runtime PM - or calls >>> pm_runtime_set_active() during ->probe(), because >>> genpd_runtime_resume() won't be invoked to restore the gpd->rpm_state. >>> >>> That said, I wouldn't mind to simply skip adding the >>> ->dev_get_performance_state() all together, if that is what you >>> prefer? In this way, it becomes the responsibility for the consumer >>> driver to do right thing, with the cost of some boilerplate code added >>> in its ->probe() routine. >> >> Until a day ago, it wasn't clear to me that consumer drivers now can set >> up rpm_pstate during probe(), which is a cleaner solution that works >> well. So let's skip adding the questionable ->dev_get_performance_state(). >> >> The boilerplate code in the probe() is minimal in comparison to a >> previous variant with the state-syncing done by rpm-resume callbacks of >> consumer drivers, it's good enough. > > Alright, that sounds good to me as well. > > I am happy to help with review of the consumer driver changes, just > keep me posted. Thank you and Viresh for the help, very appreciate it! ^ permalink raw reply [flat|nested] 29+ messages in thread
end of thread, other threads:[~2021-09-10 11:24 UTC | newest] Thread overview: 29+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-09-02 10:16 [PATCH 0/3] PM: domains: Improvements for performance states in genpd Ulf Hansson 2021-09-02 10:16 ` [PATCH 1/3] PM: domains: Drop the performance state vote for a device at detach Ulf Hansson 2021-09-03 6:01 ` Dmitry Osipenko 2021-09-03 8:22 ` Ulf Hansson 2021-09-03 9:58 ` Dmitry Osipenko 2021-09-03 14:03 ` Ulf Hansson 2021-09-05 8:26 ` Dmitry Osipenko 2021-09-06 10:24 ` Ulf Hansson 2021-09-06 14:11 ` Dmitry Osipenko 2021-09-06 17:34 ` Ulf Hansson 2021-09-06 19:33 ` Dmitry Osipenko 2021-09-07 10:16 ` Ulf Hansson 2021-09-09 13:48 ` Dmitry Osipenko 2021-09-09 14:45 ` Ulf Hansson 2021-09-02 10:16 ` [PATCH 2/3] PM: domains: Restructure some code in __genpd_dev_pm_attach() Ulf Hansson 2021-09-02 10:16 ` [PATCH 3/3] PM: domains: Add a ->dev_get_performance_state() callback to genpd Ulf Hansson 2021-09-03 6:00 ` Dmitry Osipenko 2021-09-03 8:55 ` Ulf Hansson 2021-09-03 10:06 ` Dmitry Osipenko 2021-09-03 14:09 ` Ulf Hansson 2021-09-05 9:11 ` Dmitry Osipenko 2021-09-06 10:53 ` Ulf Hansson 2021-09-06 14:35 ` Dmitry Osipenko 2021-09-07 3:40 ` Viresh Kumar 2021-09-07 8:10 ` Dmitry Osipenko 2021-09-07 9:57 ` Ulf Hansson 2021-09-09 13:48 ` Dmitry Osipenko 2021-09-09 14:39 ` Ulf Hansson 2021-09-10 11:24 ` Dmitry Osipenko
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).