Problems with PM_FREEZE

Problems with PM_FREEZE

[Alan Stern]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1561 bytes --]

A problem with the PM_FREEZE state has surfaced recently.

It has to do with the device state recorded in the memory image.  When the 
image is made, devices are in the FREEZE state, and that's what gets 
recorded.  But then the image is written to disk and devices are put into 
SUSPEND.

Later on, when the system wakes up and the image is restored, drivers are 
asked to resume the devices.  The problem is that now the drivers think 
the devices are in FREEZE when in fact they are really in SUSPEND.  The 
difference is significant and it can cause errors in the resume procedure.

What's the correct way to handle this?  On possible solution would be to
let drivers know when a resume call is returning from a system sleep.  
When that happens drivers can safely assume that the devices really are
in SUSPEND, no matter what the recorded states say.


Here's another, slightly more far-fetched problem.  I don't know if this 
can come up in actual practice.

When a disk device is put in FREEZE, there may be pending write requests
in its I/O queue.  After the image has been made, the device is resumed in
order to write the image to a swap partition.  When that happens the I/O
queue will start running again, briefly.  Those pending writes will be 
carried out.

Then when the system wakes up, the pending requests will be restored along
with the rest of the image.  But they have already been recorded on the
disk.  I can imagine that with some kinds of filesystem this might lead to
an error (although I don't know how).


Any comments?

Alan Stern


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1241 bytes --]

Hi!

> A problem with the PM_FREEZE state has surfaced recently.
> 
> It has to do with the device state recorded in the memory image.  When the 
> image is made, devices are in the FREEZE state, and that's what gets 
> recorded.  But then the image is written to disk and devices are put into 
> SUSPEND.
> 
> Later on, when the system wakes up and the image is restored, drivers are 
> asked to resume the devices.  The problem is that now the drivers think 
> the devices are in FREEZE when in fact they are really in SUSPEND.  The 
> difference is significant and it can cause errors in the resume procedure.

No; devices are in FREEZE if their driver was in kernel, and in some kind
of power up state when not. Drivers should just handle both.

> Here's another, slightly more far-fetched problem.  I don't know if this 
> can come up in actual practice.
> 
> When a disk device is put in FREEZE, there may be pending write requests
> in its I/O queue.  After the image has been 
Can they? If so, we need to solve that somehow.

...it might happen with network devices, too. We would send packet twice.
We should limit queueing in drivers across FREEZE...
-- 
64 bytes from 195.113.31.123: icmp_seq=28 ttl=51 time=448769.1 ms         


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Alan Stern]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1955 bytes --]

On Tue, 27 Sep 2005, Pavel Machek wrote:

> Hi!
> 
> > A problem with the PM_FREEZE state has surfaced recently.
> > 
> > It has to do with the device state recorded in the memory image.  When the 
> > image is made, devices are in the FREEZE state, and that's what gets 
> > recorded.  But then the image is written to disk and devices are put into 
> > SUSPEND.
> > 
> > Later on, when the system wakes up and the image is restored, drivers are 
> > asked to resume the devices.  The problem is that now the drivers think 
> > the devices are in FREEZE when in fact they are really in SUSPEND.  The 
> > difference is significant and it can cause errors in the resume procedure.
> 
> No; devices are in FREEZE if their driver was in kernel, and in some kind
> of power up state when not. Drivers should just handle both.

For USB, that "some kind of power up state" will in fact be SUSPEND.

Drivers should just handle both -- how?  Do a full resume in either case?  
Doesn't that go against the whole idea of FREEZE, that it could be 
implemented without some of the overhead of SUSPEND?

Well, I guess the FREEZE transition could still avoid that overhead.  But
the RESUME-from-FREEZE transition ought to be able to avoid it also, and
you've just shown that it can't afford to.


> > Here's another, slightly more far-fetched problem.  I don't know if this 
> > can come up in actual practice.
> > 
> > When a disk device is put in FREEZE, there may be pending write requests
> > in its I/O queue.  After the image has been 
> Can they? If so, we need to solve that somehow.

I don't know.  It depends on how the disk driver implements FREEZE.  Does 
it drain the queue, or does it merely plug the queue?

> ...it might happen with network devices, too. We would send packet twice.
> We should limit queueing in drivers across FREEZE...

Exactly.  But remember that those queues _need_ to be used in order to 
write out the memory image.

Alan Stern


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1962 bytes --]

Hi!

> > > Later on, when the system wakes up and the image is restored, drivers are 
> > > asked to resume the devices.  The problem is that now the drivers think 
> > > the devices are in FREEZE when in fact they are really in SUSPEND.  The 
> > > difference is significant and it can cause errors in the resume procedure.
> > 
> > No; devices are in FREEZE if their driver was in kernel, and in some kind
> > of power up state when not. Drivers should just handle both.
> 
> For USB, that "some kind of power up state" will in fact be SUSPEND.
> 
> Drivers should just handle both -- how?  Do a full resume in either case?  
> Doesn't that go against the whole idea of FREEZE, that it could be 
> implemented without some of the overhead of SUSPEND?
> 
> Well, I guess the FREEZE transition could still avoid that overhead.  But
> the RESUME-from-FREEZE transition ought to be able to avoid it also, and
> you've just shown that it can't afford to.

We can add a hint to pm_message_t. But for "full resume", we really
don't know if devices are only FROZEN or if they need full reinit.

So you may get info "only frozen" and "maybe frozen, maybe it was
powered down"; if you think that helps the drivers, just add a field
to pm_message_t (flags), and add this as a first flag.

> > > Here's another, slightly more far-fetched problem.  I don't know if this 
> > > can come up in actual practice.
> > > 
> > > When a disk device is put in FREEZE, there may be pending write requests
> > > in its I/O queue.  After the image has been 
> > Can they? If so, we need to solve that somehow.
> 
> I don't know.  It depends on how the disk driver implements FREEZE.  Does 
> it drain the queue, or does it merely plug the queue?

...actually it does not matter for disk driver, because two writes
with same data are harmless. It may still matter for network (repeated
packets).
								Pavel
-- 
if you have sharp zaurus hardware you don't need... you know my address

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Alan Stern]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 497 bytes --]

On Tue, 27 Sep 2005, Pavel Machek wrote:

> We can add a hint to pm_message_t. But for "full resume", we really
> don't know if devices are only FROZEN or if they need full reinit.
> 
> So you may get info "only frozen" and "maybe frozen, maybe it was
> powered down"; if you think that helps the drivers, just add a field
> to pm_message_t (flags), and add this as a first flag.

Okay.  Can you tell me where is the code that resumes all the devices once
the memory image is loaded?

Alan Stern


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 816 bytes --]

Hi!

> > We can add a hint to pm_message_t. But for "full resume", we really
> > don't know if devices are only FROZEN or if they need full reinit.
> > 
> > So you may get info "only frozen" and "maybe frozen, maybe it was
> > powered down"; if you think that helps the drivers, just add a field
> > to pm_message_t (flags), and add this as a first flag.
> 
> Okay.  Can you tell me where is the code that resumes all the devices once
> the memory image is loaded?

kernel/power/swsusp.c swsusp_suspend() does device_power_up(). and
disk.c: pm_suspend_disk(). Use in_suspend() variable.

Ouch, but device_power_up and device_resume() do not take pm_message_t
parameter. They probably should. But the patch will be quite big.

								Pavel
-- 
if you have sharp zaurus hardware you don't need... you know my address

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Nigel Cunningham]

[-- Attachment #1: Type: text/plain, Size: 1922 bytes --]

Hi Alan.

On Wed, 2005-09-28 at 05:02, Alan Stern wrote:
> On Tue, 27 Sep 2005, Pavel Machek wrote:
> 
> > Hi!
> > 
> > > A problem with the PM_FREEZE state has surfaced recently.
> > > 
> > > It has to do with the device state recorded in the memory image.  When the 
> > > image is made, devices are in the FREEZE state, and that's what gets 
> > > recorded.  But then the image is written to disk and devices are put into 
> > > SUSPEND.
> > > 
> > > Later on, when the system wakes up and the image is restored, drivers are 
> > > asked to resume the devices.  The problem is that now the drivers think 
> > > the devices are in FREEZE when in fact they are really in SUSPEND.  The 
> > > difference is significant and it can cause errors in the resume procedure.
> > 
> > No; devices are in FREEZE if their driver was in kernel, and in some kind
> > of power up state when not. Drivers should just handle both.
> 
> For USB, that "some kind of power up state" will in fact be SUSPEND.

Excuse me taking a step back, but I think you guys might be solving a
problem that doesn't exist...

How do USB drivers get into a suspend state? At suspend time, before the
atomic copy is made they have been told to FREEZE. At resume time, prior
to the atomic restore, they have been told to FREEZE. The state they're
told to enter when we powerdown should be irrelevant, because after that
we have powered down the whole system and then powered it back up like
we would for a normal boot. The "some kind of powered up state" should
be ON, unless the usb drivers are managing to retain power and state
across a power down and reboot (I suppose you're going to tell me they
are!).

Regards,

Nigel
> 
> ______________________________________________________________________
> _______________________________________________
> linux-pm mailing list
> linux-pm@lists.osdl.org
> https://lists.osdl.org/mailman/listinfo/linux-pm
-- 



[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1175 bytes --]

Hi!

> > > > Later on, when the system wakes up and the image is restored, drivers are 
> > > > asked to resume the devices.  The problem is that now the drivers think 
> > > > the devices are in FREEZE when in fact they are really in SUSPEND.  The 
> > > > difference is significant and it can cause errors in the resume procedure.
> > > 
> > > No; devices are in FREEZE if their driver was in kernel, and in some kind
> > > of power up state when not. Drivers should just handle both.
> > 
> > For USB, that "some kind of power up state" will in fact be SUSPEND.
> 
> Excuse me taking a step back, but I think you guys might be solving a
> problem that doesn't exist...
> 
> How do USB drivers get into a suspend state? At suspend time, before the
> atomic copy is made they have been told to FREEZE. At resume time, prior
> to the atomic restore, they have been told to FREEZE. The state

They have been told to freeze _if you have usb built in_, and not
modular. Imagine kernel with usb as a module, doing resume from kernel
command line. usb will be in "just powered on" state.

								Pavel

-- 
if you have sharp zaurus hardware you don't need... you know my address

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Nigel Cunningham]

[-- Attachment #1: Type: text/plain, Size: 2076 bytes --]

Hi.

On Wed, 2005-09-28 at 08:01, Pavel Machek wrote:
> Hi!
> 
> > > > > Later on, when the system wakes up and the image is restored, drivers are 
> > > > > asked to resume the devices.  The problem is that now the drivers think 
> > > > > the devices are in FREEZE when in fact they are really in SUSPEND.  The 
> > > > > difference is significant and it can cause errors in the resume procedure.
> > > > 
> > > > No; devices are in FREEZE if their driver was in kernel, and in some kind
> > > > of power up state when not. Drivers should just handle both.
> > > 
> > > For USB, that "some kind of power up state" will in fact be SUSPEND.
> > 
> > Excuse me taking a step back, but I think you guys might be solving a
> > problem that doesn't exist...
> > 
> > How do USB drivers get into a suspend state? At suspend time, before the
> > atomic copy is made they have been told to FREEZE. At resume time, prior
> > to the atomic restore, they have been told to FREEZE. The state
> 
> They have been told to freeze _if you have usb built in_, and not
> modular. Imagine kernel with usb as a module, doing resume from kernel
> command line. usb will be in "just powered on" state.

Yes, that's true. If the usb modules are loaded when suspending and not
loaded when resuming or vice versa, you'll get inconsistencies:

State at suspend	State at resume		Image state

Built in		Built in		Freeze->Freeze
Loaded modules		Unloaded modules	Undefined->Freeze
Unloaded modules	Loaded modules		Freeze->Undefined

I guess there are three possible solutions:
1) Leave things as they are and say it is the user's problem if they
make the state inconsistent.
2) Keep knowledge of the device states across the atomic restore and use
that information in deciding what to do in device resume/powerup.
3) Make device drivers handle the situation properly without knowledge
of what state the hardware is really in (or check the real state - where
possible - and rely on that in deciding what to do).

2 seems to me to make for the most reliable solution.

Regards,

Nigel
> 								Pavel
-- 



[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Alan Stern]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2122 bytes --]

On Wed, 28 Sep 2005, Nigel Cunningham wrote:

> Yes, that's true. If the usb modules are loaded when suspending and not
> loaded when resuming or vice versa, you'll get inconsistencies:
> 
> State at suspend	State at resume		Image state
> 
> Built in		Built in		Freeze->Freeze
> Loaded modules	Unloaded modules	Undefined->Freeze
> Unloaded modules	Loaded modules		Freeze->Undefined

This table is misleading.  Better to describe it like this:

	If the image doesn't contain USB drivers, the device state
	doesn't matter.

	If the image does contain USB drivers and the boot kernel
	did not meddle with the device states, then the devices
	will be suspended even though the image thinks they are
	frozen.

	If the image does contain USB drivers and the boot kernel
	did meddle with the device states, then the devices probably
	will not be resumable by the image kernel.  They will have
	to be rediscovered.

> I guess there are three possible solutions:
> 1) Leave things as they are and say it is the user's problem if they
> make the state inconsistent.
> 2) Keep knowledge of the device states across the atomic restore and use
> that information in deciding what to do in device resume/powerup.
> 3) Make device drivers handle the situation properly without knowledge
> of what state the hardware is really in (or check the real state - where
> possible - and rely on that in deciding what to do).
> 
> 2 seems to me to make for the most reliable solution.

No.  The best answer is to

	(A) tell the boot kernel that the impending freeze is for a
	restore-from-disk, so that it can wipe out the state of any
	devices it has changed, and

	(B) tell the image kernel that it is resuming from disk, so
	that it can know that the devices are really suspended even
	though its internal records say they are frozen.

Better than (A) would be to tell the boot kernel that it _is_ only a boot 
kernel, so that its drivers will know not to mess up the state of any 
devices.  This would have the side effect of making it impossible to 
reload an image from a USB drive, but that's pretty much unavoidable 
anyway.

Alan Stern


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Nigel Cunningham]

[-- Attachment #1: Type: text/plain, Size: 2893 bytes --]

Hi Alan.

On Wed, 2005-09-28 at 12:58, Alan Stern wrote:
> On Wed, 28 Sep 2005, Nigel Cunningham wrote:
> 
> > Yes, that's true. If the usb modules are loaded when suspending and not
> > loaded when resuming or vice versa, you'll get inconsistencies:
> > 
> > State at suspend	State at resume		Image state
> > 
> > Built in		Built in		Freeze->Freeze
> > Loaded modules	Unloaded modules	Undefined->Freeze
> > Unloaded modules	Loaded modules		Freeze->Undefined
> 
> This table is misleading.  Better to describe it like this:
> 
> 	If the image doesn't contain USB drivers, the device state
> 	doesn't matter.
> 
> 	If the image does contain USB drivers and the boot kernel
> 	did not meddle with the device states, then the devices
> 	will be suspended even though the image thinks they are
> 	frozen.

So a power off or reboot doesn't reset the USB devices?

> 	If the image does contain USB drivers and the boot kernel
> 	did meddle with the device states, then the devices probably
> 	will not be resumable by the image kernel.  They will have
> 	to be rediscovered.

Even if frozen? They should end up in the same state. But then USB
suspend/resume hasn't worked reliably for me, so I'm still in
unload-usb-while-suspending mode.

> > I guess there are three possible solutions:
> > 1) Leave things as they are and say it is the user's problem if they
> > make the state inconsistent.
> > 2) Keep knowledge of the device states across the atomic restore and use
> > that information in deciding what to do in device resume/powerup.
> > 3) Make device drivers handle the situation properly without knowledge
> > of what state the hardware is really in (or check the real state - where
> > possible - and rely on that in deciding what to do).
> > 
> > 2 seems to me to make for the most reliable solution.
> 
> No.  The best answer is to
> 
> 	(A) tell the boot kernel that the impending freeze is for a
> 	restore-from-disk, so that it can wipe out the state of any
> 	devices it has changed, and
> 
> 	(B) tell the image kernel that it is resuming from disk, so
> 	that it can know that the devices are really suspended even
> 	though its internal records say they are frozen.

But if the drivers are loaded, we will tell them to freeze, so the state
will be frozen.

> Better than (A) would be to tell the boot kernel that it _is_ only a boot 
> kernel, so that its drivers will know not to mess up the state of any 
> devices.  This would have the side effect of making it impossible to 
> reload an image from a USB drive, but that's pretty much unavoidable 
> anyway.

I would like to be able to get suspend to and resuming from usb going at
some stage. No chance?

The problem with telling the kernel it is only a boot kernel is that we
don't know that until we look and see if there's an image, which may
involve running an initramfs/initrd first (encryption, eg).

Regards,

Nigel



[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 430 bytes --]

Hi!

> 	(B) tell the image kernel that it is resuming from disk, so
> 	that it can know that the devices are really suspended even
> 	though its internal records say they are frozen.

Feel free to submit a patch. We have talked about this before, and the
patch will be rather big, touching most of the kernel. You want it,
then you do it.
								Pavel

-- 
if you have sharp zaurus hardware you don't need... you know my address

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 707 bytes --]

Hi!

> I guess there are three possible solutions:
> 1) Leave things as they are and say it is the user's problem if they
> make the state inconsistent.
> 2) Keep knowledge of the device states across the atomic restore and use
> that information in deciding what to do in device resume/powerup.
> 3) Make device drivers handle the situation properly without knowledge
> of what state the hardware is really in (or check the real state - where
> possible - and rely on that in deciding what to do).
> 
> 2 seems to me to make for the most reliable solution.

I actually like 3 most -- it does not need core changes.

								Pavel
-- 
if you have sharp zaurus hardware you don't need... you know my address

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Rafael J. Wysocki]

[-- Attachment #1: Type: text/plain, Size: 978 bytes --]

Hi,

On Wednesday, 28 of September 2005 15:04, Pavel Machek wrote:
> Hi!
> 
> > I guess there are three possible solutions:
> > 1) Leave things as they are and say it is the user's problem if they
> > make the state inconsistent.
> > 2) Keep knowledge of the device states across the atomic restore and use
> > that information in deciding what to do in device resume/powerup.
> > 3) Make device drivers handle the situation properly without knowledge
> > of what state the hardware is really in (or check the real state - where
> > possible - and rely on that in deciding what to do).
> > 
> > 2 seems to me to make for the most reliable solution.
> 
> I actually like 3 most -- it does not need core changes.

IMO if the driver is a module it should not make any assumptions
on the state of the device when its resume routine is called.  Instead,
it should assume the device can be in an arbitrary state and proceed
in the safest way possible.  Which is 3.

Greetings,
Rafael

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[David Brownell]

[-- Attachment #1: Type: text/plain, Size: 2763 bytes --]

> > > > > Later on, when the system wakes up and the image is restored,
> > > > > drivers are asked to
> > > > > resume the devices.  The problem is that now the drivers think 
> > > > > the devices are in FREEZE when in fact they are really in SUSPEND.
> > > > > The difference is significant and it can cause errors in the
> > > > > resume procedure.
> > > > 
> > > > No; devices are in FREEZE if their driver was in kernel, and in some kind
> > > > of power up state when not. Drivers should just handle both.
> > > 
> > > For USB, that "some kind of power up state" will in fact be SUSPEND.
> > 
> > Excuse me taking a step back, but I think you guys might be solving a
> > problem that doesn't exist...

I see the problem as being the latest definition of FREEZE:

  FREEZE -- stop DMA and interrupts, and be prepared to reinit HW from
  scratch. That probably means ...

  SUSPEND -- like FREEZE, but also put hardware into low-power state.

Definitions like those are "probably" imprecise enough to be useless
to implementors.

And since drivers are free to put hardware into low power states any
time it's convenient to them, there is really no practical difference
between those two notions.

(This is different from earlier notions of FREEZE I remember being
discussed.  In particular the one that viewed it as just quiescing.)


> > How do USB drivers get into a suspend state?

There's driver state ... and there's device state.  For USB the two are
very cleanly separated, since drivers are bound to usb_interface objects,
while only usb_device objects (or root hubs) can have real power state.


> >	At suspend time, before the
> > atomic copy is made they have been told to FREEZE. At resume time, prior
> > to the atomic restore, they have been told to FREEZE. The state
>
> They have been told to freeze _if you have usb built in_, and not
> modular.

Pavel, there's no such "is it built in" logic I've seen anywhere in
the PM framework.  It may be appropriate to add some at some point;
but that'd notion would need to be decently thought through.


>	Imagine kernel with usb as a module, doing resume from kernel
> command line. usb will be in "just powered on" state.

Well, "from kernel command line" all but guarantees a reset has
been done, and so the USB devices will not be suspended.  Remember,
you may be typing that command line on a USB keyboard, which BIOS
had to re-initialize.  Linux and BIOS won't share USB device state.

Now if it's a real resume transition -- from suspend-to-RAM, or
something similar that doesn't load a new Linux kernel -- then
hardware can often maintain the USB suspend state while the
system is in its low power state.  The cost is normally 500 uA
per device, though some may consume a bit more.

- Dave


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[David Brownell]

[-- Attachment #1: Type: text/plain, Size: 2172 bytes --]

> > 	If the image doesn't contain USB drivers, the device state
> > 	doesn't matter.
> > 
> > 	If the image does contain USB drivers and the boot kernel
> > 	did not meddle with the device states, then the devices
> > 	will be suspended even though the image thinks they are
> > 	frozen.
>
> So a power off or reboot doesn't reset the USB devices?

Those would reset them, yes ... or more to the point, disconnect
them.  All the checkpoint/resume style PM scenarios should trigger
disconnection for every USB device.  (And in fact they do, with
the USB PM fixes upcoming for 2.6.15 ... that seems to have been
broken in the past few releases.)


> > 	If the image does contain USB drivers and the boot kernel
> > 	did meddle with the device states, then the devices probably
> > 	will not be resumable by the image kernel.  They will have
> > 	to be rediscovered.
>
> Even if frozen? They should end up in the same state.

No ... the fact that there **WAS** a boot kernel implies a reset,
hence disconnect.  The BIOS may have re-enumerated things and
changed device state.  (And of course the USER may have switched
cables around too...)


> > Better than (A) would be to tell the boot kernel that it _is_ only a boot 
> > kernel, so that its drivers will know not to mess up the state of any 
> > devices.  This would have the side effect of making it impossible to 
> > reload an image from a USB drive, but that's pretty much unavoidable 
> > anyway.

Actually, reloading an image from a USB drive should be easy;
I don't understand the difficulty.  These comments seem to be
circling around the fact that such "resume from swsusp" cases
are not real PM resumes.


> I would like to be able to get suspend to and resuming from usb going at
> some stage. No chance?

Have a look at the USB PM patches now in Greg's tree; I'll
be posting a note to this list about them soonish.

Briefly, status for me is that while I see resume from swsusp
snapshots working, there seems to be difficulty (a) getting
remote wakeup signals through ACPI after they wake the system
up, (b) often resuming from S1 or S3 in general.  None of those
issues appear to be from USB code.

- Dave


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Alan Stern]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 4843 bytes --]

[Replying to two people at once]

On Wed, 28 Sep 2005, Nigel Cunningham wrote:
On Wed, 28 Sep 2005, David Brownell wrote:

> > 	If the image does contain USB drivers and the boot kernel
> > 	did not meddle with the device states, then the devices
> > 	will be suspended even though the image thinks they are
> > 	frozen.
> 
> So a power off or reboot doesn't reset the USB devices?

> Those would reset them, yes ... or more to the point, disconnect
> them.  All the checkpoint/resume style PM scenarios should trigger
> disconnection for every USB device.  (And in fact they do, with
> the USB PM fixes upcoming for 2.6.15 ... that seems to have been
> broken in the past few releases.)

You need to state these things carefully.  A complete power-off does
reset/disconnect USB devices.  However, turning off your computer
might not do a complete power-off; it may well leave suspend current 
available.  If that's true, USB devices won't be disconnected.

Reboot is yet another hurdle.  If the BIOS takes control of a USB
controller before the OS can load, then all the devices attached to
it effectively get disconnected.  If the BIOS is smart enough not to
do that, and the boot kernel is smart enough not to initialize the
controllers, then the attached devices will remain safely suspended.


> > 	If the image does contain USB drivers and the boot kernel
> > 	did meddle with the device states, then the devices probably
> > 	will not be resumable by the image kernel.  They will have
> > 	to be rediscovered.
> 
> Even if frozen? They should end up in the same state. But then USB
> suspend/resume hasn't worked reliably for me, so I'm still in
> unload-usb-while-suspending mode.

> No ... the fact that there **WAS** a boot kernel implies a reset,
> hence disconnect.  The BIOS may have re-enumerated things and
> changed device state.  (And of course the USER may have switched
> cables around too...)

Like Dave says, the power level of the device isn't as important as the
state of the USB controller and the device's internal state.  If those
remain unchanged, we can recover the device no matter what its power
level is.

> > No.  The best answer is to
> > 
> > 	(A) tell the boot kernel that the impending freeze is for a
> > 	restore-from-disk, so that it can wipe out the state of any
> > 	devices it has changed, and
> > 
> > 	(B) tell the image kernel that it is resuming from disk, so
> > 	that it can know that the devices are really suspended even
> > 	though its internal records say they are frozen.
> 
> But if the drivers are loaded, we will tell them to freeze, so the state
> will be frozen.

I assume you mean "if the drivers are loaded in the boot kernel".  This
statement is misleading.  If the boot kernel is aware of the devices at
all then it has already reset the USB controller and the devices.  Hence
no matter what their power level is, they will not be resumable by the
image kernel.


> > Better than (A) would be to tell the boot kernel that it _is_ only a boot 
> > kernel, so that its drivers will know not to mess up the state of any 
> > devices.  This would have the side effect of making it impossible to 
> > reload an image from a USB drive, but that's pretty much unavoidable 
> > anyway.

> Actually, reloading an image from a USB drive should be easy;
> I don't understand the difficulty.  These comments seem to be
> circling around the fact that such "resume from swsusp" cases
> are not real PM resumes.

I should have said that it's not possible to load the image from a USB
device and still have USB devices be resumable by the image kernel.  If
the boot kernel initializes the USB controllers then it will destroy the
state preserved by the suspend.  If it doesn't initialize the controllers
then of course it can't restore the image from a USB drive.

> I would like to be able to get suspend to and resuming from usb going at
> some stage. No chance?

There's no way the boot kernel can preserve the state information,
because it doesn't know what that information is and so is forced to
destroy it during initialization.  There's no way, for example, for
the boot kernel to guarantee that the user hasn't swapped USB drives
while the system was asleep -- and the boot kernel will erase the
information the image kernel needs to detect such things.

> The problem with telling the kernel it is only a boot kernel is that we
> don't know that until we look and see if there's an image, which may
> involve running an initramfs/initrd first (encryption, eg).

Isn't there a command-line parameter that tells you whether or not to
try loading an image?  The problem is, what do you do if that
parameter is present but the image isn't.  Depending on how early you
learn this, you could decide not to be a boot kernel (i.e., be a
normal kernel) or you could force a reboot.

Alan Stern


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Alan Stern]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 838 bytes --]

On Wed, 28 Sep 2005, Rafael J. Wysocki wrote:

> IMO if the driver is a module it should not make any assumptions
> on the state of the device when its resume routine is called.  Instead,
> it should assume the device can be in an arbitrary state and proceed
> in the safest way possible.  Which is 3.

That's what it should do when resuming from disk.

But that's not what it should do when it's being resumed just after the
memory image was created, in order to write out the image.  In this case
the device is known to be in FREEZE, not SUSPEND, and to save time we
would like the driver not to go through a full resume procedure.

The problem is that currently the driver has no way to tell the difference
between the two types of resume.  What's needed is a way for driver to 
tell, and that can be added easily enough.

Alan Stern


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Rafael J. Wysocki]

Hi,

On Wednesday, 28 of September 2005 20:54, Alan Stern wrote:
> On Wed, 28 Sep 2005, Rafael J. Wysocki wrote:
> 
> > IMO if the driver is a module it should not make any assumptions
> > on the state of the device when its resume routine is called.  Instead,
> > it should assume the device can be in an arbitrary state and proceed
> > in the safest way possible.  Which is 3.
> 
> That's what it should do when resuming from disk.
> 
> But that's not what it should do when it's being resumed just after the
> memory image was created, in order to write out the image.  In this case
> the device is known to be in FREEZE, not SUSPEND, and to save time we
> would like the driver not to go through a full resume procedure.
> 
> The problem is that currently the driver has no way to tell the difference
> between the two types of resume.  What's needed is a way for driver to 
> tell, and that can be added easily enough.

Yes, I've been thinking about it for some time.  It seems to me we could do
this with the help of an additional field in pm_message_t.

Greetings,
Rafael

Re: Problems with PM_FREEZE

[Alan Stern]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1138 bytes --]

On Wed, 28 Sep 2005, Rafael J. Wysocki wrote:

> > That's what it should do when resuming from disk.
> > 
> > But that's not what it should do when it's being resumed just after the
> > memory image was created, in order to write out the image.  In this case
> > the device is known to be in FREEZE, not SUSPEND, and to save time we
> > would like the driver not to go through a full resume procedure.
> > 
> > The problem is that currently the driver has no way to tell the difference
> > between the two types of resume.  What's needed is a way for driver to 
> > tell, and that can be added easily enough.
> 
> Yes, I've been thinking about it for some time.  It seems to me we could do
> this with the help of an additional field in pm_message_t.

The problem that no pm_message_t is passed to resume routines.

Another solution would be to create a globally-available subroutine, 
something like:

	int resuming_from_disk(void)

which would return 1 during the image resume and 0 at other times.  It 
wouldn't hurt to have another routine available,

	int temporary_boot_kernel(void)

That one may be harder to arrange.

Alan Stern


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 786 bytes --]

Hi!

> > IMO if the driver is a module it should not make any assumptions
> > on the state of the device when its resume routine is called.  Instead,
> > it should assume the device can be in an arbitrary state and proceed
> > in the safest way possible.  Which is 3.
> 
> That's what it should do when resuming from disk.
> 
> But that's not what it should do when it's being resumed just after the
> memory image was created, in order to write out the image.  In this case
> the device is known to be in FREEZE, not SUSPEND, and to save time we
> would like the driver not to go through a full resume procedure.

Do you have any driver which can save some significant ammount of time
that way?
								Pavel
-- 
if you have sharp zaurus hardware you don't need... you know my address

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1209 bytes --]

On St 28-09-05 15:31:36, Alan Stern wrote:
> On Wed, 28 Sep 2005, Rafael J. Wysocki wrote:
> 
> > > That's what it should do when resuming from disk.
> > > 
> > > But that's not what it should do when it's being resumed just after the
> > > memory image was created, in order to write out the image.  In this case
> > > the device is known to be in FREEZE, not SUSPEND, and to save time we
> > > would like the driver not to go through a full resume procedure.
> > > 
> > > The problem is that currently the driver has no way to tell the difference
> > > between the two types of resume.  What's needed is a way for driver to 
> > > tell, and that can be added easily enough.
> > 
> > Yes, I've been thinking about it for some time.  It seems to me we could do
> > this with the help of an additional field in pm_message_t.
> 
> The problem that no pm_message_t is passed to resume routines.
> 
> Another solution would be to create a globally-available subroutine, 
> something like:
> 
> 	int resuming_from_disk(void)

No. We do not have freezing_devices() nor suspending_devices(), and we
should not have this one.
								Pavel

-- 
if you have sharp zaurus hardware you don't need... you know my address

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1224 bytes --]

On St 28-09-05 12:17:33, Alan Stern wrote:
> [Replying to two people at once]
> 
> On Wed, 28 Sep 2005, Nigel Cunningham wrote:
> On Wed, 28 Sep 2005, David Brownell wrote:
> 
> > > 	If the image does contain USB drivers and the boot kernel
> > > 	did not meddle with the device states, then the devices
> > > 	will be suspended even though the image thinks they are
> > > 	frozen.
> > 
> > So a power off or reboot doesn't reset the USB devices?
> 
> > Those would reset them, yes ... or more to the point, disconnect
> > them.  All the checkpoint/resume style PM scenarios should trigger
> > disconnection for every USB device.  (And in fact they do, with
> > the USB PM fixes upcoming for 2.6.15 ... that seems to have been
> > broken in the past few releases.)
> 
> You need to state these things carefully.  A complete power-off does
> reset/disconnect USB devices.  However, turning off your computer
> might not do a complete power-off; it may well leave suspend current 
> available.  If that's true, USB devices won't be disconnected.

Examples, please. Thinkpad X32 certainly does not provide power on USB
when turned off.
								Pavel

-- 
if you have sharp zaurus hardware you don't need... you know my address

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[David Brownell]

[-- Attachment #1: Type: text/plain, Size: 456 bytes --]

> > But that's not what it should do when it's being resumed just after the
> > memory image was created, in order to write out the image.  In this case
> > the device is known to be in FREEZE, not SUSPEND, and to save time we
> > would like the driver not to go through a full resume procedure.
>
> Do you have any driver which can save some significant ammount of time
> that way?

If there is no such driver, then there's no need for the FREEZE state.


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Alan Stern]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 576 bytes --]

On Wed, 28 Sep 2005, Pavel Machek wrote:

> > But that's not what it should do when it's being resumed just after the
> > memory image was created, in order to write out the image.  In this case
> > the device is known to be in FREEZE, not SUSPEND, and to save time we
> > would like the driver not to go through a full resume procedure.
> 
> Do you have any driver which can save some significant ammount of time
> that way?

All USB devices require several 10's of milliseconds to resume.  Whether 
that amount is "significant" is a matter of personal opinion.

Alan Stern


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 674 bytes --]

Hi!

> > > But that's not what it should do when it's being resumed just after the
> > > memory image was created, in order to write out the image.  In this case
> > > the device is known to be in FREEZE, not SUSPEND, and to save time we
> > > would like the driver not to go through a full resume procedure.
> >
> > Do you have any driver which can save some significant ammount of time
> > that way?
> 
> If there is no such driver, then there's no need for the FREEZE state.

Wrong. IDE can avoid spinning down disk when *entering* FREEZE
state. No difference during *exiting* it.
								Pavel
-- 
if you have sharp zaurus hardware you don't need... you know my address

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Alan Stern]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 483 bytes --]

On Wed, 28 Sep 2005, Pavel Machek wrote:

> > The problem that no pm_message_t is passed to resume routines.
> > 
> > Another solution would be to create a globally-available subroutine, 
> > something like:
> > 
> > 	int resuming_from_disk(void)
> 
> No. We do not have freezing_devices() nor suspending_devices(), and we
> should not have this one.

Well, it would be a lot easier than changing all the resume routines.

Do you have some reason for not allowing this?

Alan Stern


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Alan Stern]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 681 bytes --]

On Wed, 28 Sep 2005, Pavel Machek wrote:

> > You need to state these things carefully.  A complete power-off does
> > reset/disconnect USB devices.  However, turning off your computer
> > might not do a complete power-off; it may well leave suspend current 
> > available.  If that's true, USB devices won't be disconnected.
> 
> Examples, please. Thinkpad X32 certainly does not provide power on USB
> when turned off.

I don't have any examples.  However you're more likely to find suspend 
current available on a desktop system than on a laptop, I imagine.  Those 
big power supplies don't really turn off all the way unless you unplug 
them from the wall socket.

Alan Stern


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 900 bytes --]

On St 28-09-05 17:15:21, Alan Stern wrote:
> On Wed, 28 Sep 2005, Pavel Machek wrote:
> 
> > > You need to state these things carefully.  A complete power-off does
> > > reset/disconnect USB devices.  However, turning off your computer
> > > might not do a complete power-off; it may well leave suspend current 
> > > available.  If that's true, USB devices won't be disconnected.
> > 
> > Examples, please. Thinkpad X32 certainly does not provide power on USB
> > when turned off.
> 
> I don't have any examples.  However you're more likely to find suspend 
> current available on a desktop system than on a laptop, I imagine.  Those 
> big power supplies don't really turn off all the way unless you unplug 
> them from the wall socket.

USB takes enough power to be connected to 5V, not 5V-standby. Sorry.

								Pavel
-- 
if you have sharp zaurus hardware you don't need... you know my address

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 669 bytes --]

On St 28-09-05 17:13:44, Alan Stern wrote:
> On Wed, 28 Sep 2005, Pavel Machek wrote:
> 
> > > The problem that no pm_message_t is passed to resume routines.
> > > 
> > > Another solution would be to create a globally-available subroutine, 
> > > something like:
> > > 
> > > 	int resuming_from_disk(void)
> > 
> > No. We do not have freezing_devices() nor suspending_devices(), and we
> > should not have this one.
> 
> Well, it would be a lot easier than changing all the resume routines.
> 
> Do you have some reason for not allowing this?

Yes, it is ugly and inconsistent.
								Pavel

-- 
if you have sharp zaurus hardware you don't need... you know my address

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[David Brownell]

[-- Attachment #1: Type: text/plain, Size: 924 bytes --]

> > You need to state these things carefully.  A complete power-off does
> > reset/disconnect USB devices.  However, turning off your computer
> > might not do a complete power-off; it may well leave suspend current 
> > available.  If that's true, USB devices won't be disconnected.
>
> Examples, please. Thinkpad X32 certainly does not provide power on USB
> when turned off.

I think Alan's point applies more directly to things like ACPI S4
state than ACPI S5 (off) ... it's quite legit for USB controllers
to maintain VBUS power even after entering D3cold, which they'll
often do in S4 sleep.

EHCI for example is carefully specified in terms of what registers
are powered through PCI VAUX, and it's designed to support remote
wakeup from D3cold.  VAUX provides suspend current to the various
USB devices that are connected, which can initiate remote wakeup
by pulling D+ high for long enough (maybe 20 msec).

- Dave


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 745 bytes --]

On St 28-09-05 14:20:22, David Brownell wrote:
> > > You need to state these things carefully.  A complete power-off does
> > > reset/disconnect USB devices.  However, turning off your computer
                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

> > > might not do a complete power-off; it may well leave suspend current 
> > > available.  If that's true, USB devices won't be disconnected.
> >
> > Examples, please. Thinkpad X32 certainly does not provide power on USB
> > when turned off.
> 
> I think Alan's point applies more directly to things like ACPI S4
> state than ACPI S5 (off) ... it's quite legit for USB controllers

Eh?

							Pavel
-- 
if you have sharp zaurus hardware you don't need... you know my address

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Rafael J. Wysocki]

Hi,

On Wednesday, 28 of September 2005 21:31, Alan Stern wrote:
> On Wed, 28 Sep 2005, Rafael J. Wysocki wrote:
> 
> > > That's what it should do when resuming from disk.
> > > 
> > > But that's not what it should do when it's being resumed just after the
> > > memory image was created, in order to write out the image.  In this case
> > > the device is known to be in FREEZE, not SUSPEND, and to save time we
> > > would like the driver not to go through a full resume procedure.
> > > 
> > > The problem is that currently the driver has no way to tell the difference
> > > between the two types of resume.  What's needed is a way for driver to 
> > > tell, and that can be added easily enough.
> > 
> > Yes, I've been thinking about it for some time.  It seems to me we could do
> > this with the help of an additional field in pm_message_t.
> 
> The problem that no pm_message_t is passed to resume routines.

Yes, shame on me (I desperately need some sleep).

How about calling *_suspend() twice, first before the image is created
and second after it has been created but before *_resume() are called,
with the second call only telling the driver the next resume will be
resume-during-suspend? 

Greetings,
Rafael

Re: Problems with PM_FREEZE

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 732 bytes --]

Hi!

> > > Yes, I've been thinking about it for some time.  It seems to me we could do
> > > this with the help of an additional field in pm_message_t.
> > 
> > The problem that no pm_message_t is passed to resume routines.
> 
> Yes, shame on me (I desperately need some sleep).
> 
> How about calling *_suspend() twice, first before the image is created
> and second after it has been created but before *_resume() are called,
> with the second call only telling the driver the next resume will be
> resume-during-suspend? 

Could we do it properly and pass pm_message_t to resume()? Yes, it is
more work, but no, we don't need hacks there.
								Pavel
-- 
if you have sharp zaurus hardware you don't need... you know my address

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Rafael J. Wysocki]

On Thursday, 29 of September 2005 00:01, Pavel Machek wrote:
> Hi!
> 
> > > > Yes, I've been thinking about it for some time.  It seems to me we could do
> > > > this with the help of an additional field in pm_message_t.
> > > 
> > > The problem that no pm_message_t is passed to resume routines.
> > 
> > Yes, shame on me (I desperately need some sleep).
> > 
> > How about calling *_suspend() twice, first before the image is created
> > and second after it has been created but before *_resume() are called,
> > with the second call only telling the driver the next resume will be
> > resume-during-suspend? 
> 
> Could we do it properly and pass pm_message_t to resume()? Yes, it is
> more work, but no, we don't need hacks there.

Sure.  Passing pm_message_t to resume() is much better.

Greetings,
Rafael

Re: Problems with PM_FREEZE

[Alan Stern]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 835 bytes --]

Dave:

Pavel is obviously right that the clean solution is to add a pm_message_t 
argument to resume().

But there might be an easier approach that would help this particular
case.  You changed things so that for USB, PM_EVENT_FREEZE works like a
PM_EVENT_SUSPEND.  But once PM_EVENT_RUNTIME is defined and used by the
/sys/.../power/state attribute, why not do the opposite?  Make USB
PM_EVENT_SUSPEND work like a PM_EVENT_FREEZE -- don't actually suspend any
devices other than the root hub.

That way there would be no difficulty about devices being in the wrong
state when the resumed kernel tries to wake them up.  And it would reduce
the time required for the sleep transition.  Actual power consumption
would not be an issue; when the root hub suspends, all the devices below
it will automatically suspend as well.

Alan Stern


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[David Brownell]

[-- Attachment #1: Type: text/plain, Size: 2537 bytes --]

> Date: Thu, 29 Sep 2005 11:45:15 -0400 (EDT)
> From: Alan Stern <stern@rowland.harvard.edu>
>
> Dave:
>
> Pavel is obviously right that the clean solution is to add a pm_message_t 
> argument to resume().

I tend to disagree with that.  Changing every resume() method is
not "clean", and it's more obvious to me that the pm_message
semantics are (still) problematic.  Any argument to resume() would
be encouraging fragile "if (came_from(X)) { ... }" style logic.

Heck, there's still no way for drivers to know what the target
system state is ... and THAT is a core issue here.

During FREEZE, the target system state is "something snapshottable".
During SUSPEND, it's one of numerous variants of "low power" ... and
device drivers can only guess which one it'll be.  (Is it an ACPI
state?  S1, S3, S4?  Some non-ACPI platform state?)

Case in point:  in some system low power states, drivers need to turn
off certain clocks, and may not be able to support wakeup.  In others,
drivers leave those clocks on, and can support wakeup.  But there is
no way to figure out the target state given the pm_message ...


> But there might be an easier approach that would help this particular
> case.  You changed things so that for USB, PM_EVENT_FREEZE works like a
> PM_EVENT_SUSPEND.  But once PM_EVENT_RUNTIME is defined and used by the
> /sys/.../power/state attribute, why not do the opposite?  Make USB
> PM_EVENT_SUSPEND work like a PM_EVENT_FREEZE -- don't actually suspend any
> devices other than the root hub.

If the device states in question are real ones (and are for example
testable), I don't have an issue with defining them.

But defining more and more pseudo-states seems to me like trouble.
I've not seen a definition of PM_EVENT_RUNTIME, for one thing;


> That way there would be no difficulty about devices being in the wrong
> state when the resumed kernel tries to wake them up.  And it would reduce
> the time required for the sleep transition.  Actual power consumption
> would not be an issue; when the root hub suspends, all the devices below
> it will automatically suspend as well.

Power consumption WOULD be an issue.  Devices in USB_STATE_SUSPEND (a
real hardware state, not a pseudo-state!!) consume about 0.5 mA each.
Devices that are in other (wired) USB states consume about 100 mA each.
(Or for "high power" devices, 2.5 mA and 500 mA.)  

It's true that could shrink the time for sleep transitions, assuming
the devices weren't already asleep.  But I don't think that's a big
deal at this point.

- Dave


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1536 bytes --]

Hi!

> > Pavel is obviously right that the clean solution is to add a pm_message_t 
> > argument to resume().
> 
> I tend to disagree with that.  Changing every resume() method is
> not "clean", and it's more obvious to me that the pm_message
> semantics are (still) problematic.  Any argument to resume() would
> be encouraging fragile "if (came_from(X)) { ... }" style logic.

Alan wants to cut few miliseconds from suspend-to-disk. That's okay
with me, just add pm_message_t to resume(). If it is okay to use it in
specific driver is other question.

> Heck, there's still no way for drivers to know what the target
> system state is ... and THAT is a core issue here.
> 
> During FREEZE, the target system state is "something snapshottable".
> During SUSPEND, it's one of numerous variants of "low power" ... and
> device drivers can only guess which one it'll be.  (Is it an ACPI
> state?  S1, S3, S4?  Some non-ACPI platform state?)

Why does your driver need to know? Anyway, extending pm_message_t with
flags is okay with me.

> Case in point:  in some system low power states, drivers need to turn
> off certain clocks, and may not be able to support wakeup.  In others,
> drivers leave those clocks on, and can support wakeup.  But there is
> no way to figure out the target state given the pm_message ...

Eh? I do not see how knowing S1 vs. S3 vs. S4 help you here. It looks
more like "does user want to resume from that?" question.

								Pavel
-- 
if you have sharp zaurus hardware you don't need... you know my address

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Alan Stern]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 4167 bytes --]

On Thu, 29 Sep 2005, David Brownell wrote:

> > Pavel is obviously right that the clean solution is to add a pm_message_t 
> > argument to resume().
> 
> I tend to disagree with that.  Changing every resume() method is
> not "clean", and it's more obvious to me that the pm_message
> semantics are (still) problematic.  Any argument to resume() would
> be encouraging fragile "if (came_from(X)) { ... }" style logic.
> 
> Heck, there's still no way for drivers to know what the target
> system state is ... and THAT is a core issue here.
> 
> During FREEZE, the target system state is "something snapshottable".
> During SUSPEND, it's one of numerous variants of "low power" ... and
> device drivers can only guess which one it'll be.  (Is it an ACPI
> state?  S1, S3, S4?  Some non-ACPI platform state?)
> 
> Case in point:  in some system low power states, drivers need to turn
> off certain clocks, and may not be able to support wakeup.  In others,
> drivers leave those clocks on, and can support wakeup.  But there is
> no way to figure out the target state given the pm_message ...

Yes, those are valid points.  But they aren't what I want to discuss here.


> > But there might be an easier approach that would help this particular
> > case.  You changed things so that for USB, PM_EVENT_FREEZE works like a
> > PM_EVENT_SUSPEND.  But once PM_EVENT_RUNTIME is defined and used by the
> > /sys/.../power/state attribute, why not do the opposite?  Make USB
> > PM_EVENT_SUSPEND work like a PM_EVENT_FREEZE -- don't actually suspend any
> > devices other than the root hub.
> 
> If the device states in question are real ones (and are for example
> testable), I don't have an issue with defining them.
> 
> But defining more and more pseudo-states seems to me like trouble.
> I've not seen a definition of PM_EVENT_RUNTIME, for one thing;

It's not fully relevant to this discussion, but here it is anyway:

PM_EVENT_RUNTIME is (or rather, will be) the event value used for calls to
a driver's suspend or resume method that originated from the user writing
to /sys/.../dev/power/state.  It has no other meaning, but this clearly
implies that such calls fall into the "runtime PM" category rather than
"system PM".  Consequently when a driver receives such an event, it
mustn't assume that all its children have already been frozen/suspended or
that all its ancestors will be.

Passed along during a PM_EVENT_RUNTIME call, as part of pm_message_t, will 
be a const char * that points to the name of the state requested by the 
user.  More strongly, it will point to a canonical string containing that 
name, a string created by the driver or its bus.  So drivers will be able 
to use the pointer value itself as a selector for which state to go to.

(Ideally that same pointer could also be used to identify the target state
for a system suspend.  But that's something for the future...)


> > That way there would be no difficulty about devices being in the wrong
> > state when the resumed kernel tries to wake them up.  And it would reduce
> > the time required for the sleep transition.  Actual power consumption
> > would not be an issue; when the root hub suspends, all the devices below
> > it will automatically suspend as well.
> 
> Power consumption WOULD be an issue.  Devices in USB_STATE_SUSPEND (a
> real hardware state, not a pseudo-state!!) consume about 0.5 mA each.
> Devices that are in other (wired) USB states consume about 100 mA each.
> (Or for "high power" devices, 2.5 mA and 500 mA.)  

I meant that power consumption would be no more of an issue than it is 
now.  In case some people might worry that failing to suspend a USB device 
would leave it still at full power when the system goes to sleep, I wanted 
to reassure them: That won't happen.

> It's true that could shrink the time for sleep transitions, assuming
> the devices weren't already asleep.  But I don't think that's a big
> deal at this point.

Every little bit helps...

The main point of the suggestion is that it allows us to use the FREEZE
optimization without fear of a device's actual state not matching the
driver's records during a system resume.

Alan Stern


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[David Brownell]

[-- Attachment #1: Type: text/plain, Size: 2531 bytes --]

> > > Pavel is obviously right that the clean solution is to add a pm_message_t 
> > > argument to resume().
> > 
> > I tend to disagree with that.  Changing every resume() method is
> > not "clean", and it's more obvious to me that the pm_message
> > semantics are (still) problematic.  Any argument to resume() would
> > be encouraging fragile "if (came_from(X)) { ... }" style logic.
>
> Alan wants to cut few miliseconds from suspend-to-disk. That's okay
> with me, just add pm_message_t to resume(). If it is okay to use it in
> specific driver is other question.

Sorry; changing every driver in the whole tree is in my book
under "things to avoid unless there's a more significant win".


> > Heck, there's still no way for drivers to know what the target
> > system state is ... and THAT is a core issue here.
> > 
> > During FREEZE, the target system state is "something snapshottable".
> > During SUSPEND, it's one of numerous variants of "low power" ... and
> > device drivers can only guess which one it'll be.  (Is it an ACPI
> > state?  S1, S3, S4?  Some non-ACPI platform state?)
>
> Why does your driver need to know? Anyway, extending pm_message_t with
> flags is okay with me.

Well, I've always said that what we want _instead of_ a pm_message
is just an object describing the platform-specific target state.  
And clean definitions of those states.

I don't quite see how to map those to "flags".


> > Case in point:  in some system low power states, drivers need to turn
> > off certain clocks, and may not be able to support wakeup.  In others,
> > drivers leave those clocks on, and can support wakeup.  But there is
> > no way to figure out the target state given the pm_message ...
>
> Eh? I do not see how knowing S1 vs. S3 vs. S4 help you here. It looks
> more like "does user want to resume from that?" question.

The "user" will _always_ want to resume.  S1/S3/S4 is only an answer
for ACPI systems (that don't use swsusp/"shutdown"!), and ISTR "video
needs to use PCI_D2 in S1 and S3" was a common issue (since PCI_D3
tends to expect re-init, but Linux developers don't have chip specs
saying how to re-init).  But non-ACPI systems have similar issues.

I've given examples before with USB controllers, where the 48 MHz clock
must be turned off to enter certain system states, but not others;
and turning the clock off has _significant drawbacks_ that make it
sub-optimal to always do it.  (Drawbacks like resetting the controller,
which disconnects all devices, and not supporting wakeup.)

- Dave


[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problems with PM_FREEZE

[Rafael J. Wysocki]

[-- Attachment #1: Type: text/plain, Size: 1272 bytes --]

Hi,

On Thursday, 29 of September 2005 20:22, David Brownell wrote:
> > > > Pavel is obviously right that the clean solution is to add a pm_message_t 
> > > > argument to resume().
> > > 
> > > I tend to disagree with that.  Changing every resume() method is
> > > not "clean", and it's more obvious to me that the pm_message
> > > semantics are (still) problematic.  Any argument to resume() would
> > > be encouraging fragile "if (came_from(X)) { ... }" style logic.
> >
> > Alan wants to cut few miliseconds from suspend-to-disk. That's okay
> > with me, just add pm_message_t to resume(). If it is okay to use it in
> > specific driver is other question.
> 
> Sorry; changing every driver in the whole tree is in my book
> under "things to avoid unless there's a more significant win".

I think it's avoidable.

Instead of changing every single driver that registers _resume(), we can
introduce another resume template, something like

void resume_from(pm_message_t state)

where we will pass the state which we expect the device to be in before
resume.  Then we can call:
- resume_from() if the driver registers it, or
- resume() if the driver registers it, etc.
(ie. resume() can only be called if resume_from() is not registered).

Just an idea.

Greetings,
Rafael

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]