pm runtime and system suspend resume

pm runtime and system suspend resume

[chao xie]

hi
PM_RUNTIME provide a way for device driver do runtime PM. so for some
devices, they have some surrounded logic. For example, the device may
get clock from outside, or it need PHY support(USB is a example).
To get these dependency out of device driver, i define a struct
dev_pm_domain, and make dev.pm_domain point to it. So in the device
driver, when the hardware should be enabled, we can call
pm_runtime_get_sync while when the hardware is idle or does not work,
we can call pm_runtime_put_sync.
It seems work well, but i have question about the suspend/resume of
device. When the whole system will go to deep idle, and it will
suspend the devices. for the function do device suspend
__device_suspend, it will call pm_runtime_get_noresume(dev). As i
think it will make the device not do runtime suspend any more. Is that
correct?
There is the question, how device driver handle the logic surrounds
it? I want to add pm_runtime_put_sync in dev->driver->pm->suspend
function, and pm_runtime_get_sync in dev->driver->pm->resume. Because
__device_suspend increase the usage_count, pm_runtime_put_sync will
not do real work.
So is that right that i directly call pm_runtime_suspend in
dev->driver->pm->suspend and pm_runtime_resume in
dev->driver->pm->resume?
Thanks.

Re: pm runtime and system suspend resume

[Alan Stern]

On Tue, 19 Jun 2012, chao xie wrote:

> hi
> PM_RUNTIME provide a way for device driver do runtime PM. so for some
> devices, they have some surrounded logic. For example, the device may
> get clock from outside, or it need PHY support(USB is a example).
> To get these dependency out of device driver, i define a struct
> dev_pm_domain, and make dev.pm_domain point to it. So in the device
> driver, when the hardware should be enabled, we can call
> pm_runtime_get_sync while when the hardware is idle or does not work,
> we can call pm_runtime_put_sync.
> It seems work well, but i have question about the suspend/resume of
> device. When the whole system will go to deep idle, and it will
> suspend the devices. for the function do device suspend
> __device_suspend,

What source file is that function in?

>  it will call pm_runtime_get_noresume(dev). As i
> think it will make the device not do runtime suspend any more. Is that
> correct?

It sounds wrong.  Why would a suspend routine do that?

> There is the question, how device driver handle the logic surrounds
> it? I want to add pm_runtime_put_sync in dev->driver->pm->suspend
> function, and pm_runtime_get_sync in dev->driver->pm->resume.

You must not do that.  If you do, your driver will hang.

> Because
> __device_suspend increase the usage_count, pm_runtime_put_sync will
> not do real work.
> So is that right that i directly call pm_runtime_suspend in
> dev->driver->pm->suspend and pm_runtime_resume in
> dev->driver->pm->resume?

No, it works the other way around.  pm_runtime_suspend calls 
dev->driver->pm->suspend, and pm_runtime_resume calls 
dev->driver->pm->resume.

Alan Stern

Re: pm runtime and system suspend resume

[chao xie]

>> hi
>> PM_RUNTIME provide a way for device driver do runtime PM. so for some
>> devices, they have some surrounded logic. For example, the device may
>> get clock from outside, or it need PHY support(USB is a example).
>> To get these dependency out of device driver, i define a struct
>> dev_pm_domain, and make dev.pm_domain point to it. So in the device
>> driver, when the hardware should be enabled, we can call
>> pm_runtime_get_sync while when the hardware is idle or does not work,
>> we can call pm_runtime_put_sync.
>> It seems work well, but i have question about the suspend/resume of
>> device. When the whole system will go to deep idle, and it will
>> suspend the devices. for the function do device suspend
>> __device_suspend,
>
> What source file is that function in?
>

The function is located at drivers/base/power/main.c

>>  it will call pm_runtime_get_noresume(dev). As i
>> think it will make the device not do runtime suspend any more. Is that
>> correct?
>
> It sounds wrong.  Why would a suspend routine do that?
>

I do knot know, but the code is there.

>> There is the question, how device driver handle the logic surrounds
>> it? I want to add pm_runtime_put_sync in dev->driver->pm->suspend
>> function, and pm_runtime_get_sync in dev->driver->pm->resume.
>
> You must not do that.  If you do, your driver will hang.
>

why it will hang?

>> Because
>> __device_suspend increase the usage_count, pm_runtime_put_sync will
>> not do real work.
>> So is that right that i directly call pm_runtime_suspend in
>> dev->driver->pm->suspend and pm_runtime_resume in
>> dev->driver->pm->resume?
>
> No, it works the other way around.  pm_runtime_suspend calls
> dev->driver->pm->suspend, and pm_runtime_resume calls
> dev->driver->pm->resume.
>

I have checked the pm_runtime_suspend implementation. It will call
__pm_runtime_suspend without RPM_GET_PUT. It means that it will
directly call rpm_suspend. in this function, the pm_ops call sequence
is
if (dev->pm_domain)
               callback = dev->pm_domain->ops.runtime_suspend;
       else if (dev->type && dev->type->pm)
               callback = dev->type->pm->runtime_suspend;
       else if (dev->class && dev->class->pm)
               callback = dev->class->pm->runtime_suspend;
       else if (dev->bus && dev->bus->pm)
               callback = dev->bus->pm->runtime_suspend;
       else
               callback = NULL;

       if (!callback && dev->driver && dev->driver->pm)
               callback = dev->driver->pm->runtime_suspend;

It means that if dev->pm_domain is not NULL, it will not call
dev->driver->pm operations.

Finally the real question is who will deal with the dev->pm_domain?
when system go to sleep.
The driver will use pm_runtime_get/put to deal with dev->pm_domain
when it want to work or is in idle. Then when system go to sleep, the
driver need care about dev->pm_domain too.
Usally, dev->pm_domain is a easy way to handle the difference SOCs
with same device IP. For example, USB. Many vendor may have same usb
IP but difference phy and clock configuration. putting these stuff
into dev->pm_domain can make driver be shared by different SOCs.

> Alan Stern
>

Re: pm runtime and system suspend resume

[Alan Stern]

On Wed, 20 Jun 2012, chao xie wrote:

> >> It seems work well, but i have question about the suspend/resume of
> >> device. When the whole system will go to deep idle, and it will
> >> suspend the devices. for the function do device suspend
> >> __device_suspend,
> >
> > What source file is that function in?
> >
> 
> The function is located at drivers/base/power/main.c
> 
> >>  it will call pm_runtime_get_noresume(dev). As i
> >> think it will make the device not do runtime suspend any more. Is that
> >> correct?
> >
> > It sounds wrong.  Why would a suspend routine do that?
> >
> 
> I do knot know, but the code is there.

Okay, now I understand your question.  The problem is that system 
suspend uses a separate mechanism from runtime suspend, and we don't 
want the two things to happen at the same time.  Therefore 
__device_suspend does a get_noresume in order to prevent runtime 
suspend from occurring in the middle of a system suspend.

> >> There is the question, how device driver handle the logic surrounds
> >> it? I want to add pm_runtime_put_sync in dev->driver->pm->suspend
> >> function, and pm_runtime_get_sync in dev->driver->pm->resume.
> >
> > You must not do that.  If you do, your driver will hang.
> >
> 
> why it will hang?

Sorry, I misunderstood your question.  It will not hang.  But why do 
you want to add those calls?

> >> Because
> >> __device_suspend increase the usage_count, pm_runtime_put_sync will
> >> not do real work.
> >> So is that right that i directly call pm_runtime_suspend in
> >> dev->driver->pm->suspend and pm_runtime_resume in
> >> dev->driver->pm->resume?

No, don't call pm_runtime_suspend.  Just call your driver's
runtime_suspend routine directly.  Same for resume.

> Finally the real question is who will deal with the dev->pm_domain?
> when system go to sleep.

The __device_suspend routine in drivers/base/main.c uses 
dev->pm_domain.

> The driver will use pm_runtime_get/put to deal with dev->pm_domain
> when it want to work or is in idle. Then when system go to sleep, the
> driver need care about dev->pm_domain too.
> Usally, dev->pm_domain is a easy way to handle the difference SOCs
> with same device IP. For example, USB. Many vendor may have same usb
> IP but difference phy and clock configuration. putting these stuff
> into dev->pm_domain can make driver be shared by different SOCs.

Alan Stern

Re: pm runtime and system suspend resume

[chao xie]

2012/6/20 Alan Stern <stern@rowland.harvard.edu>:
> On Wed, 20 Jun 2012, chao xie wrote:
>
>> >> It seems work well, but i have question about the suspend/resume of
>> >> device. When the whole system will go to deep idle, and it will
>> >> suspend the devices. for the function do device suspend
>> >> __device_suspend,
>> >
>> > What source file is that function in?
>> >
>>
>> The function is located at drivers/base/power/main.c
>>
>> >>  it will call pm_runtime_get_noresume(dev). As i
>> >> think it will make the device not do runtime suspend any more. Is that
>> >> correct?
>> >
>> > It sounds wrong.  Why would a suspend routine do that?
>> >
>>
>> I do knot know, but the code is there.
>
> Okay, now I understand your question.  The problem is that system
> suspend uses a separate mechanism from runtime suspend, and we don't
> want the two things to happen at the same time.  Therefore
> __device_suspend does a get_noresume in order to prevent runtime
> suspend from occurring in the middle of a system suspend.
>
>> >> There is the question, how device driver handle the logic surrounds
>> >> it? I want to add pm_runtime_put_sync in dev->driver->pm->suspend
>> >> function, and pm_runtime_get_sync in dev->driver->pm->resume.
>> >
>> > You must not do that.  If you do, your driver will hang.
>> >
>>
>> why it will hang?
>
> Sorry, I misunderstood your question.  It will not hang.  But why do
> you want to add those calls?
>
>> >> Because
>> >> __device_suspend increase the usage_count, pm_runtime_put_sync will
>> >> not do real work.
>> >> So is that right that i directly call pm_runtime_suspend in
>> >> dev->driver->pm->suspend and pm_runtime_resume in
>> >> dev->driver->pm->resume?
>
> No, don't call pm_runtime_suspend.  Just call your driver's
> runtime_suspend routine directly.  Same for resume.
>
>> Finally the real question is who will deal with the dev->pm_domain?
>> when system go to sleep.
>
> The __device_suspend routine in drivers/base/main.c uses
> dev->pm_domain.
>

I check the code, __device_suspend will invoke dev->pm_domain->ops
first if dev->pm_domain is not NULL.
Taking "suspend" as esample, i can do the following things in the
dev->pm_domain->ops->suspend
1. invoke dev->driver->pm->suspend for suspending the device
2. do what we do in dev->pm_domain->ops->runtime_suspend for
suspending the surrounded logic, for example, shutdown the phy or
clocks.
So is above implementation fine?


>> The driver will use pm_runtime_get/put to deal with dev->pm_domain
>> when it want to work or is in idle. Then when system go to sleep, the
>> driver need care about dev->pm_domain too.
>> Usally, dev->pm_domain is a easy way to handle the difference SOCs
>> with same device IP. For example, USB. Many vendor may have same usb
>> IP but difference phy and clock configuration. putting these stuff
>> into dev->pm_domain can make driver be shared by different SOCs.
>
> Alan Stern
>

Re: pm runtime and system suspend resume

[Alan Stern]

On Thu, 21 Jun 2012, chao xie wrote:

> I check the code, __device_suspend will invoke dev->pm_domain->ops
> first if dev->pm_domain is not NULL.
> Taking "suspend" as esample, i can do the following things in the
> dev->pm_domain->ops->suspend
> 1. invoke dev->driver->pm->suspend for suspending the device
> 2. do what we do in dev->pm_domain->ops->runtime_suspend for
> suspending the surrounded logic, for example, shutdown the phy or
> clocks.
> So is above implementation fine?

Yes, that sounds like it will work.  Watch out for the case where your 
device is already runtime-suspended when a system suspend occurs.

Have you read section 6 in Documentation/power/runtime_pm.txt?  It is 
related to your question.

Alan Stern

Re: pm runtime and system suspend resume

[chao xie]

2012/6/21 Alan Stern <stern@rowland.harvard.edu>:
> On Thu, 21 Jun 2012, chao xie wrote:
>
>> I check the code, __device_suspend will invoke dev->pm_domain->ops
>> first if dev->pm_domain is not NULL.
>> Taking "suspend" as esample, i can do the following things in the
>> dev->pm_domain->ops->suspend
>> 1. invoke dev->driver->pm->suspend for suspending the device
>> 2. do what we do in dev->pm_domain->ops->runtime_suspend for
>> suspending the surrounded logic, for example, shutdown the phy or
>> clocks.
>> So is above implementation fine?
>
> Yes, that sounds like it will work.  Watch out for the case where your
> device is already runtime-suspended when a system suspend occurs.
>
> Have you read section 6 in Documentation/power/runtime_pm.txt?  It is
> related to your question.
>
I will read the document. Thanks very much.

> Alan Stern
>

Re: pm runtime and system suspend resume

[chao xie]

2012/6/22 chao xie <xiechao.linux@gmail.com>:
> 2012/6/21 Alan Stern <stern@rowland.harvard.edu>:
>> On Thu, 21 Jun 2012, chao xie wrote:
>>
>>> I check the code, __device_suspend will invoke dev->pm_domain->ops
>>> first if dev->pm_domain is not NULL.
>>> Taking "suspend" as esample, i can do the following things in the
>>> dev->pm_domain->ops->suspend
>>> 1. invoke dev->driver->pm->suspend for suspending the device
>>> 2. do what we do in dev->pm_domain->ops->runtime_suspend for
>>> suspending the surrounded logic, for example, shutdown the phy or
>>> clocks.
>>> So is above implementation fine?
>>
>> Yes, that sounds like it will work.  Watch out for the case where your
>> device is already runtime-suspended when a system suspend occurs.
>>
>> Have you read section 6 in Documentation/power/runtime_pm.txt?  It is
>> related to your question.
>>
> I will read the document. Thanks very much.
>
>> Alan Stern
>>

There is no more question about runtime_idle and runtime_suspend
as i understand that runtime_idle indicates that the device is idle,
clock may be off, no function there, runtime_suspend means that the
device my be power off, and making it rework may need some
initialization work.
So in the scenario, for USB, there are some clocks relates to it and
there is a phy attached to it.
When phy is powered off, it need to be initialized again, and it will
take some time.
So in the driver, when there is no action, we will call
pm_runtime_put, and it will shut off the clocks, for
pm_runtime_put_sync_suspend, it will shut off the phy to save more
power.
Then how do we call pm_runtime_put_sync_suspend if it already calls
pm_runtime_put? Do we need do call pm_runtime_get before call
pm_runtime_put_sync_suspend because the usage_count is not 0?

Re: pm runtime and system suspend resume

[Alan Stern]

On Mon, 25 Jun 2012, chao xie wrote:

> There is no more question about runtime_idle and runtime_suspend
> as i understand that runtime_idle indicates that the device is idle,
> clock may be off, no function there,

runtime_idle indicates the the device is idle and the usage counter is
0, right.  Your runtime_idle routine is allowed to call
pm_runtime_suspend, pm_runtime_autosuspend, or a similar function.

>  runtime_suspend means that the
> device my be power off, and making it rework may need some
> initialization work.
> So in the scenario, for USB, there are some clocks relates to it and
> there is a phy attached to it.
> When phy is powered off, it need to be initialized again, and it will
> take some time.
> So in the driver, when there is no action, we will call
> pm_runtime_put, and it will shut off the clocks, for
> pm_runtime_put_sync_suspend, it will shut off the phy to save more
> power.

I'm not sure what you mean.

There are only two differences between pm_runtime_put and 
pm_runtime_put_sync_suspend:

	pm_runtime_put calls pm_request_idle, which means your
	driver's runtime_idle routine gets called asynchronously;

	pm_runtime_put_sync_suspend pm_runtime_suspend, which means
	your driver's runtime_suspend routine gets called
	synchronously.

Your runtime_idle routine shouldn't turn off anything.  All it should 
do is call pm_runtime_suspend (or something similar).  Thus, in either 
case your driver's runtime_suspend routine will be called.  It can turn 
off both the clocks and the phy.

> Then how do we call pm_runtime_put_sync_suspend if it already calls
> pm_runtime_put? Do we need do call pm_runtime_get before call
> pm_runtime_put_sync_suspend because the usage_count is not 0?

You should always call pm_runtime_get (or one of its variants) before 
calling pm_runtime_put (or any of its variants).  Increments and 
decrements of the usage counter must balance.

Alan Stern

Re: pm runtime and system suspend resume

[chao xie]

2012/6/25 Alan Stern <stern@rowland.harvard.edu>:
> On Mon, 25 Jun 2012, chao xie wrote:
>
>> There is no more question about runtime_idle and runtime_suspend
>> as i understand that runtime_idle indicates that the device is idle,
>> clock may be off, no function there,
>
> runtime_idle indicates the the device is idle and the usage counter is
> 0, right.  Your runtime_idle routine is allowed to call
> pm_runtime_suspend, pm_runtime_autosuspend, or a similar function.
>
>>  runtime_suspend means that the
>> device my be power off, and making it rework may need some
>> initialization work.
>> So in the scenario, for USB, there are some clocks relates to it and
>> there is a phy attached to it.
>> When phy is powered off, it need to be initialized again, and it will
>> take some time.
>> So in the driver, when there is no action, we will call
>> pm_runtime_put, and it will shut off the clocks, for
>> pm_runtime_put_sync_suspend, it will shut off the phy to save more
>> power.
>
> I'm not sure what you mean.
>
> There are only two differences between pm_runtime_put and
> pm_runtime_put_sync_suspend:
>
>        pm_runtime_put calls pm_request_idle, which means your
>        driver's runtime_idle routine gets called asynchronously;
>
>        pm_runtime_put_sync_suspend pm_runtime_suspend, which means
>        your driver's runtime_suspend routine gets called
>        synchronously.
>
> Your runtime_idle routine shouldn't turn off anything.  All it should
> do is call pm_runtime_suspend (or something similar).  Thus, in either
> case your driver's runtime_suspend routine will be called.  It can turn
> off both the clocks and the phy.
>
If we want to separate clocks off and phy off, what should we do?
in fact, in the driver, at some time, it can shutdown clock, and at
some time, it can shut down phy too. So what i want to do it make
runtime_idle to shut down the clocks while in runtime_suspend to
shutdown the clocks and phy.
So when the driver is in idle, and i can call pm_runtime_put which
will call runtime_idle to shutdown the clocks, while when the driver
want to turn off phy, it means it will enter deeper idle, so i need
call pm_runtime_put_sync_suspend.
The question is if when driver calls pm_runtime_put_sync_suspend, the
driver has already called pm_runtime_put, the usaga_count is 0 now, so
driver has to call pm_runtime_get and then call
pm_runtime_put_sync_suspend.
PM runtime provide runtime_idle and runtime_suspend, but the device
status has only two kinds of status, active or no-active. It confuses
me. It means that the pm runtime can not support multiple level of
power mode for device. actually, i think it has provided two kinds of
callback runtime_idle and runtime_suspend, and it should maintain at
least 3 state, idle, suspend, active for the device.

>> Then how do we call pm_runtime_put_sync_suspend if it already calls
>> pm_runtime_put? Do we need do call pm_runtime_get before call
>> pm_runtime_put_sync_suspend because the usage_count is not 0?
>


> You should always call pm_runtime_get (or one of its variants) before
> calling pm_runtime_put (or any of its variants).  Increments and
> decrements of the usage counter must balance.
>
> Alan Stern
>

Re: pm runtime and system suspend resume

[Alan Stern]

On Tue, 26 Jun 2012, chao xie wrote:

> If we want to separate clocks off and phy off, what should we do?

Why do you want to separate them?

The Runtime PM framework doesn't have any notion of multiple low-power 
states.  It recognizes only two possibilities: active and suspended.

If turning the clocks on and off is very quick, you can simply have the
driver turn them on before each I/O operation and turn them off when
the operation is finished.

> in fact, in the driver, at some time, it can shutdown clock, and at
> some time, it can shut down phy too. So what i want to do it make
> runtime_idle to shut down the clocks while in runtime_suspend to
> shutdown the clocks and phy.

The runtime_idle routine is not supposed to turn anything off.  All it 
should do is decide whether or not to call pm_runtime_suspend.  The 
actual work of turning things on and off belongs in the driver's (or 
subsystem's or PM domain's) runtime_resume and runtime_suspend 
routines.

> So when the driver is in idle, and i can call pm_runtime_put which
> will call runtime_idle to shutdown the clocks, while when the driver
> want to turn off phy, it means it will enter deeper idle, so i need
> call pm_runtime_put_sync_suspend.

You can do that if you want, but it is not how the Runtime PM framework 
was intended to work.  Instead of calling pm_runtime_put, why not just 
turn off the clocks?

Or why not turn off the clocks at the same time as the phy?

> The question is if when driver calls pm_runtime_put_sync_suspend, the
> driver has already called pm_runtime_put, the usaga_count is 0 now, so
> driver has to call pm_runtime_get and then call
> pm_runtime_put_sync_suspend.

If you know the usage count is 0 already, you can just call
pm_runtime_suspend directly.

> PM runtime provide runtime_idle and runtime_suspend, but the device
> status has only two kinds of status, active or no-active. It confuses
> me.

Idle means "the device is active but the usage count is 0".  The 
purpose of the runtime_idle callback is to let your driver know that it 
can start a suspend if it wants to.

> It means that the pm runtime can not support multiple level of
> power mode for device.

That's right.

>  actually, i think it has provided two kinds of
> callback runtime_idle and runtime_suspend, and it should maintain at
> least 3 state, idle, suspend, active for the device.

No, that's not how it was designed.

Alan Stern