RE: Subtle semantic issue with sleep callbacks in drivers

RE: Subtle semantic issue with sleep callbacks in drivers

[Grover, Andrew]

> From: Alan Cox [mailto:alan@lxorguk.ukuu.org.uk] 
> On Llu, 2003-04-14 at 20:07, Grover, Andrew wrote:
> > All I am saying is that on Windows, the driver gets no help 
> from the 
> > BIOS, APM, or ACPI, but yet it restores the video to full working 
> > condition. I understand that this sounds complicated, but 
> since there 
> > is an implementation that already does this then I think we have to 
> > assume it's possible. :) Perhaps we should start with 
> older, simpler 
> > gfx hw, or maybe POST the bios, but only as an interim 
> solution until 
> > gfx drivers get better in this area.
> 
> You might be suprised how much BIOS help they get. Im not at 
> liberty to discuss details but at least two vendors jump into 
> bios space in their ACPI recovery routines.

Which strikes me as kind of silly since guess who called the ACPI resume
vector - the BIOS, so why didn't it do whatever stuff then? :) Anyways
it's not really relevant. The BIOS will never know about add-in cards,
and my contention is that even these can be woken up properly w/o bios
repost (after surmounting technical and potential lack-of-documentation
hurdles, which is why I'd think we would start with an old, ubiquitous,
thouroughly documented video card as our first guinea pig. Matrox
Millennium 2, perhaps?)

I'm not at the point where I can devote time to this yet, so please take
all this with a grain of salt.

Regards -- Andy

RE: Subtle semantic issue with sleep callbacks in drivers

[Patrick Mochel]

> Which strikes me as kind of silly since guess who called the ACPI resume
> vector - the BIOS, so why didn't it do whatever stuff then? :) Anyways
> it's not really relevant. The BIOS will never know about add-in cards,
> and my contention is that even these can be woken up properly w/o bios
> repost (after surmounting technical and potential lack-of-documentation
> hurdles, which is why I'd think we would start with an old, ubiquitous,
> thouroughly documented video card as our first guinea pig. Matrox
> Millennium 2, perhaps?)

I completely agree with Andy. We should not re-POST the video hardware, no
matter what. The idea behind ACPI is that the OS takes care of everything, 
including video save/restore. 

We may not have the documentation to properly do that for all hardware 
currently, but that is something that we have to suck up and deal with. 
For now, we go with hardware that we're able to handle. 

The drivers that cannot support reinitialization will not be able to 
support suspend-to-RAM. When we get to a point where it really becomes an 
issue (i.e. after we have decent working code), then we concentrate on 
getting the appropriate docuementation (or code itself, source or binary) 
to do it correctly. 

Trying to figure out if we need to POST or not for different hardware, 
based what the driver knows, is going to become quite a mess real fast. I 
don't want to deal with the pain, and would rather take the high ground, 
even if it means suffering in the short term. 


	-pat

RE: Subtle semantic issue with sleep callbacks in drivers

[Benjamin Herrenschmidt]

On Wed, 2003-04-16 at 20:39, Patrick Mochel wrote:

> I completely agree with Andy. We should not re-POST the video hardware, no
> matter what. The idea behind ACPI is that the OS takes care of everything, 
> including video save/restore. 

If I understand Alan properly, we don't always have choice, some BIOSes
will do it anyway...

> We may not have the documentation to properly do that for all hardware 
> currently, but that is something that we have to suck up and deal with. 
> For now, we go with hardware that we're able to handle. 
> 
> The drivers that cannot support reinitialization will not be able to 
> support suspend-to-RAM. When we get to a point where it really becomes an 
> issue (i.e. after we have decent working code), then we concentrate on 
> getting the appropriate docuementation (or code itself, source or binary) 
> to do it correctly. 

It is now already ! I don't think we will _ever_ get ATI and nVidia
provide enough documentation to POST all chip models (which isn't always
possible without knowledge of every single way the chip is wired on a
each board).

Currently, I cannot implement suspend-to-RAM on the latest PowerBooks
because of that (they use nVidia chip that are powered down and not just
unclocked unlike earlier ATI based models) because of that. Nor can I
implement it on any other "desktop" Mac.

> Trying to figure out if we need to POST or not for different hardware, 
> based what the driver knows, is going to become quite a mess real fast. I 
> don't want to deal with the pain, and would rather take the high ground, 
> even if it means suffering in the short term. 

When I finally have an implementation of an OF runtime so I can re-POST
the card, I could eventually have the driver itself call this to explicitely
ask for a re-POST...

Ben.

RE: Subtle semantic issue with sleep callbacks in drivers

[Alan Cox]

On Mer, 2003-04-16 at 19:39, Patrick Mochel wrote:
> I completely agree with Andy. We should not re-POST the video hardware, no
> matter what. The idea behind ACPI is that the OS takes care of everything, 
> including video save/restore. 

Outside of happyville ivory towers you probably have no choice. Only the
BIOS knows stuff like the RAM timings, and some windows drivers just use
the BIOS, others rely on being shipped compiled for the right variant of
card they came with.

We keep the BIOS segment data so you can re-post the card on the 16bit
resume path. Probably there should be a flag for relevant frame buffer
drivers to set that says "don't re-Post but call me later"

> We may not have the documentation to properly do that for all hardware 
> currently, but that is something that we have to suck up and deal with. 

> For now, we go with hardware that we're able to handle. 

Thats pretty close to "there isnt any"

> The drivers that cannot support reinitialization will not be able to 
> support suspend-to-RAM. When we get to a point where it really becomes an 
> issue (i.e. after we have decent working code), then we concentrate on 
> getting the appropriate docuementation (or code itself, source or binary) 
> to do it correctly. 

X people have been trying to do this for multihead setups for years.
You'll get Matrox, you'll get 3DFX voodoo1/2, you might get early SiS.
Intel - dunno, but the rest - no chance.

> Trying to figure out if we need to POST or not for different hardware, 
> based what the driver knows, is going to become quite a mess real fast. I 
> don't want to deal with the pain, and would rather take the high ground, 
> even if it means suffering in the short term. 

Short, long and forever. I agree we want drivers to be able to say "Don't POST
I can hack this one", however we need to accept the real world that most
hardware wants posting and that in many cases its what the windows driver
does anyway.

Re: Subtle semantic issue with sleep callbacks in drivers

[Jeff Garzik]

On Thu, Apr 17, 2003 at 02:35:50PM +0100, Alan Cox wrote:
> On Mer, 2003-04-16 at 19:39, Patrick Mochel wrote:
> > I completely agree with Andy. We should not re-POST the video hardware, no
> > matter what. The idea behind ACPI is that the OS takes care of everything, 
> > including video save/restore. 
> 
> Outside of happyville ivory towers you probably have no choice. Only the
> BIOS knows stuff like the RAM timings, and some windows drivers just use
> the BIOS, others rely on being shipped compiled for the right variant of
> card they came with.

You are exactly right.

The video BIOS on a card often contains information that is found
-nowhere- else.  Not in the chip docs.  Not in a device driver.
Such information can and does vary from board-to-board, such as RAM
timings, while the chip remains unchanged.

You mention "windows drivers" above... even some Linux X drivers
depend on video BIOS.  The S3 Savage XFree86 driver, for example,
uses video BIOS quite heavily unless you tell it not to (or are on
a platform that prevents such).

WRT save and restore, it is certainly possible without video re-POST...

However, support such will require a monumental effort of testing and
debugging for each video board.  This monumental effort _will_ include
XFree86 hacking and possibly the additional of some save-n-restore
video drivers, if we do not wish to simply require CONFIG_FBDEV if
CONFIG_SUSPEND is set.

Video re-POST is simply a Real Life(tm) shortcut to that monumental effort.

	Jeff, originally an fbdev hacker back in the day...

Re: Subtle semantic issue with sleep callbacks in drivers

[John Bradford]

> > Trying to figure out if we need to POST or not for different
> > hardware, based what the driver knows, is going to become quite a
> > mess real fast. I don't want to deal with the pain, and would
> > rather take the high ground, even if it means suffering in the
> > short term.
> 
> Short, long and forever. I agree we want drivers to be able to say
> "Don't POST I can hack this one", however we need to accept the real
> world that most hardware wants posting and that in many cases its
> what the windows driver does anyway.

One scenario we haven't covered is that we have a VGA card in a box,
which we don't care about being re-initialised after a suspend/resume.

A lot of cluster nodes have vga cards in them just so that they boot,
and you can plug a monitor and keyboard in to change BIOS settings.

The machines are never accessed except over the LAN in normal use, but
it's quite possible that you'd want to suspend the whole cluster
overnight, for example, or at least some nodes which were not in use,
and you wouldn't care about the VGA card being restored.

John.

Re: Subtle semantic issue with sleep callbacks in drivers

[Alan Cox]

On Iau, 2003-04-17 at 15:59, John Bradford wrote:
> The machines are never accessed except over the LAN in normal use, but
> it's quite possible that you'd want to suspend the whole cluster
> overnight, for example, or at least some nodes which were not in use,
> and you wouldn't care about the VGA card being restored.

Fine,. set the "dont post" flag, set the callback to a null function

Re: Subtle semantic issue with sleep callbacks in drivers

[John Bradford]

> The video BIOS on a card often contains information that is found
> -nowhere- else.  Not in the chip docs.  Not in a device driver.
> Such information can and does vary from board-to-board, such as RAM
> timings, while the chip remains unchanged.

Incidently, what happens if we:

* Suspend
* Swap VGA card with another one
* Restore

!?

John.

Re: Subtle semantic issue with sleep callbacks in drivers

[Jeff Garzik]

On Thu, Apr 17, 2003 at 04:09:25PM +0100, John Bradford wrote:
> > The video BIOS on a card often contains information that is found
> > -nowhere- else.  Not in the chip docs.  Not in a device driver.
> > Such information can and does vary from board-to-board, such as RAM
> > timings, while the chip remains unchanged.
> 
> Incidently, what happens if we:
> 
> * Suspend
> * Swap VGA card with another one
> * Restore

When it breaks, you get to keep both pieces.

That's a "Don't Do That" issue for any hardware between suspend
and resume.

	Jeff

Re: Subtle semantic issue with sleep callbacks in drivers

[John Bradford]

> > > The video BIOS on a card often contains information that is found
> > > -nowhere- else.  Not in the chip docs.  Not in a device driver.
> > > Such information can and does vary from board-to-board, such as RAM
> > > timings, while the chip remains unchanged.
> > 
> > Incidently, what happens if we:
> > 
> > * Suspend
> > * Swap VGA card with another one
> > * Restore
> 
> When it breaks, you get to keep both pieces.
> 
> That's a "Don't Do That" issue for any hardware between suspend
> and resume.

Hmm, well what about with a PCI hotswap capable board - presumably
then we could have the situation where a new VGA card appears that we
_have_ to POST?

John.

Re: Subtle semantic issue with sleep callbacks in drivers

[Jeff Garzik]

On Thu, Apr 17, 2003 at 04:47:45PM +0100, John Bradford wrote:
> Hmm, well what about with a PCI hotswap capable board - presumably
> then we could have the situation where a new VGA card appears that we
> _have_ to POST?

Then XFree86 will POST it.

The kernel really only cares about POST'ing the primary display, too.
Firmware typically completely disables, and does not POST, secondary
displays.  XFree86 is charged with the responsibility of POST'ing
secondary displays.

	Jeff

Re: Subtle semantic issue with sleep callbacks in drivers

[Alan Cox]

On Iau, 2003-04-17 at 16:56, Jeff Garzik wrote:
> On Thu, Apr 17, 2003 at 04:47:45PM +0100, John Bradford wrote:
> > Hmm, well what about with a PCI hotswap capable board - presumably
> > then we could have the situation where a new VGA card appears that we
> > _have_ to POST?
> 
> Then XFree86 will POST it.
> 
> The kernel really only cares about POST'ing the primary display, too.
> Firmware typically completely disables, and does not POST, secondary
> displays.  XFree86 is charged with the responsibility of POST'ing
> secondary displays.

XFree86 currently has to do some pray and hope stuff here. We cannot have
X and the kernel both controlling PCI allocations and config space at once.
X needs kernel interfaces to do the job right some day.

Re: Subtle semantic issue with sleep callbacks in drivers

[Greg KH]

On Thu, Apr 17, 2003 at 04:47:45PM +0100, John Bradford wrote:
> > > > The video BIOS on a card often contains information that is found
> > > > -nowhere- else.  Not in the chip docs.  Not in a device driver.
> > > > Such information can and does vary from board-to-board, such as RAM
> > > > timings, while the chip remains unchanged.
> > > 
> > > Incidently, what happens if we:
> > > 
> > > * Suspend
> > > * Swap VGA card with another one
> > > * Restore
> > 
> > When it breaks, you get to keep both pieces.
> > 
> > That's a "Don't Do That" issue for any hardware between suspend
> > and resume.
> 
> Hmm, well what about with a PCI hotswap capable board - presumably
> then we could have the situation where a new VGA card appears that we
> _have_ to POST?

PCI Hotplug does not support video cards for just this reason.

greg k-h

Re: Subtle semantic issue with sleep callbacks in drivers

[John Bradford]

> > > > > The video BIOS on a card often contains information that is found
> > > > > -nowhere- else.  Not in the chip docs.  Not in a device driver.
> > > > > Such information can and does vary from board-to-board, such as RAM
> > > > > timings, while the chip remains unchanged.
> > > > 
> > > > Incidently, what happens if we:
> > > > 
> > > > * Suspend
> > > > * Swap VGA card with another one
> > > > * Restore
> > > 
> > > When it breaks, you get to keep both pieces.
> > > 
> > > That's a "Don't Do That" issue for any hardware between suspend
> > > and resume.
> > 
> > Hmm, well what about with a PCI hotswap capable board - presumably
> > then we could have the situation where a new VGA card appears that we
> > _have_ to POST?
> 
> PCI Hotplug does not support video cards for just this reason.

Hmm, so is there no way at all we could support it?  I know it sounds
a bit pointless for a typical server or desktop machine, but it's
getting increasingly possible, practical, and attractive to use Linux
boxes more like minicomputers - dual headed VGA cards are cheap, and
USB allows you to connect practically as many keyboards and mice as
you want to.

I'm just thinking that at some point in the future, buying a new
machine could be less pratical than hot plugging in an additional VGA
card, keyboard, monitor and mouse, and editing a config file
somewhere.  With the new input layer, we are very close to this being
a reality.

John.

Re: Subtle semantic issue with sleep callbacks in drivers

[Russell King]

On Fri, Apr 18, 2003 at 12:37:54AM -0700, Greg KH wrote:
> PCI Hotplug does not support video cards for just this reason.

/me points at the Mobility Electronics EV1000 Cardbus-PCI widget
with a ATI Rage VGA device.  Ok, it's hot-pluggable, but it'd be
nice to work out a way to support it.

-- 
Russell King (rmk@arm.linux.org.uk)                The developer of ARM Linux
             http://www.arm.linux.org.uk/personal/aboutme.html

Re: Subtle semantic issue with sleep callbacks in drivers

[Alan Cox]

On Gwe, 2003-04-18 at 10:10, Russell King wrote:
> On Fri, Apr 18, 2003 at 12:37:54AM -0700, Greg KH wrote:
> > PCI Hotplug does not support video cards for just this reason.
> 
> /me points at the Mobility Electronics EV1000 Cardbus-PCI widget
> with a ATI Rage VGA device.  Ok, it's hot-pluggable, but it'd be
> nice to work out a way to support it.

2.4 PCI hotplug correctly supports switching frame buffer on a plug
event. Who broke it 8) ?

Re: Subtle semantic issue with sleep callbacks in drivers

[Alan Cox]

On Gwe, 2003-04-18 at 08:37, Greg KH wrote:
> > Hmm, well what about with a PCI hotswap capable board - presumably
> > then we could have the situation where a new VGA card appears that we
> > _have_ to POST?
> 
> PCI Hotplug does not support video cards for just this reason.

Crap reason. You let the /sbin/hotplug helper use LRMI to post the card. 
The technology side is not that hard except for the problems with X 
sticking its nose in places it shouldnt and the PCI user space access
from kernel being too weak for some stuff.

Re: Subtle semantic issue with sleep callbacks in drivers

[Pavel Machek]

Hi!

> > The video BIOS on a card often contains information that is found
> > -nowhere- else.  Not in the chip docs.  Not in a device driver.
> > Such information can and does vary from board-to-board, such as RAM
> > timings, while the chip remains unchanged.
> 
> Incidently, what happens if we:
> 
> * Suspend
> * Swap VGA card with another one
> * Restore

Thats equivalent to VGA hotplug, and is
not supported.

-- 
				Pavel
Written on sharp zaurus, because my Velo1 broke. If you have Velo you don't need...

Re: Subtle semantic issue with sleep callbacks in drivers

[Pavel Machek]

Hi!

> > What I mean here is that none of our drivers know how to bring 
> > back a chip as complicated as a radeon or a nvidia up from 
> > power off, this requires intimate knowledge of the chip 
> > internals, the way it's wired on a given board, etc...
> 
> All I am saying is that on Windows, the driver gets no help from the
> BIOS, APM, or ACPI, but yet it restores the video to full working
> condition. I understand that this sounds complicated, but since
> there is

It is not only complicated, we also lack docs neccessary to do that...

								Pavel
-- 
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]

Re: Subtle semantic issue with sleep callbacks in drivers

[Pavel Machek]

Hi!

> - On any machine, with suspend to disk, the slot is powered off (along
> with the entire host machine). But the machine will be rebooted and the
> device put back to "idle" state some way by the loader kernel, so this
> ends up a bit like the above case.

Actually as swsusp gets better it should power the device off then
on... OTOH that is very hard for video cards.

> So basically, the "state" parameter should encore not only what state
> we want to go to, but rather, what will happen to the slot:
> 
> - Nothing (it's entirely up to the driver to do it's own power
> management, that happens for some devices inside Apple ASIC), though in
> this case at least, those driver have control over the chip power, reset
> lines, etc...
> - Slot will be unclocked (it's up to the driver, it the chip supports
> static mode, to go to D2 or D3 if the driver can deal with it, though
> the system will do nothing to help the driver)
> - Slot will be powered off. This case should be broken up (via an
> additional flag passed to the driver ?) into 1) the system _will_
> re-POST the card before resume (BIOS/ACPI support, swsusp) or the
> system will NOT re-POST the card, the driver shall fail the sleep
> request if it can't do it by itself.

This is also possibility. Yes, this is probably good idea for at least
short term. Then we should make sure that "will re-post" is used less
and less and that it finally disappears.
								Pavel
-- 
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]

Re: Subtle semantic issue with sleep callbacks in drivers

[Pavel Machek]

Hi!

> > - On non-PPC machines, the slot will eventually go to D3, but 
> > the APM BIOS or ACPI will be able to re-POST the card 
> > properly on wakeup, so the driver only needs to restore the 
> > current display mode, at least I guess so since I don't know 
> > much about x86's. Similar will happen once I have an OF 
> > emulator ready on PPC to re-POST some cards, thus changing 
> > the previous example into this one. In this case, the driver 
> > can put the chip to D3 and can _accept_ the sleep request 
> > because it's explicitely told by the system (how ?) that the 
> > card will be re-POSTED prior to the
> > resume() callback.
> 
> Topic drift...
> 
> After asking around internally, it sounds like we should not be doing a
> video re-POST on wakeup. Windows only used to in order to workaround
> buggy video drivers, according to what I've heard.

We really should not be doing that, but we... kind of have to. Thats
why acpi_sleep=s3_bios exists. I really don't know how to work around
it.
								Pavel
-- 
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]

Re: Subtle semantic issue with sleep callbacks in drivers

[Benjamin Herrenschmidt]

On Wed, 2003-04-16 at 20:31, Patrick Mochel wrote:

> This is not necessarily a slot-by-slot question, but whether the entire 
> PCI/AGP buses will lose power during the sleep state, right? 

This is really per-slot. Especially on embedded or laptops, you really
don't know how each slot is wired regarding the power planes.

> There are a couple of things to note. 
> 
> This is only an issue when doing suspend-to-RAM. Suspend-to-disk, and
> power-on suspend will definitely lose power and definitely not lose any
> power, respectively. So, you need a mechanism to determine what state the 
> system is entering. 

Yes.

> Next, once you determine that we're entering suspend-to-RAM, you need to
> know if the buses will lose power. In order to have a generic suspend
> sequence, there must be a set of platform-specific methods to do all the
> fun platform things that must be done. In that object, we can easily add a
> flag that specifies whether or not the platform will lose power. This flag
> can be initialized based on platform knowledge on startup.

It's individual to each device though. And we must also have a flag the
platform can use to indicate it can/will re-POST the card (by re-running
the BIOS or whatever firmware)... 

> In short, there should be no problems. Hopefully, I should have something 
> within the week to review/test. (Yeah yeah, talk is cheap, but I'm getting 
> there).

Good :) I'm getting there too for the pmac implementation finally...

Ben.

Re: Subtle semantic issue with sleep callbacks in drivers

[Patrick Mochel]

> So basically, the "state" parameter should encore not only what state
> we want to go to, but rather, what will happen to the slot:
> 
> - Nothing (it's entirely up to the driver to do it's own power
> management, that happens for some devices inside Apple ASIC), though in
> this case at least, those driver have control over the chip power, reset
> lines, etc...
> - Slot will be unclocked (it's up to the driver, it the chip supports
> static mode, to go to D2 or D3 if the driver can deal with it, though
> the system will do nothing to help the driver)
> - Slot will be powered off. This case should be broken up (via an
> additional flag passed to the driver ?) into 1) the system _will_
> re-POST the card before resume (BIOS/ACPI support, swsusp) or the
> system will NOT re-POST the card, the driver shall fail the sleep
> request if it can't do it by itself.
>  - Embedded people may have even more weird cases ?

This is not necessarily a slot-by-slot question, but whether the entire 
PCI/AGP buses will lose power during the sleep state, right? 

There are a couple of things to note. 

This is only an issue when doing suspend-to-RAM. Suspend-to-disk, and
power-on suspend will definitely lose power and definitely not lose any
power, respectively. So, you need a mechanism to determine what state the 
system is entering. 

Next, once you determine that we're entering suspend-to-RAM, you need to
know if the buses will lose power. In order to have a generic suspend
sequence, there must be a set of platform-specific methods to do all the
fun platform things that must be done. In that object, we can easily add a
flag that specifies whether or not the platform will lose power. This flag
can be initialized based on platform knowledge on startup.

In short, there should be no problems. Hopefully, I should have something 
within the week to review/test. (Yeah yeah, talk is cheap, but I'm getting 
there).


	-pat

RE: Subtle semantic issue with sleep callbacks in drivers

[Alan Cox]

On Llu, 2003-04-14 at 20:07, Grover, Andrew wrote:
> All I am saying is that on Windows, the driver gets no help from the
> BIOS, APM, or ACPI, but yet it restores the video to full working
> condition. I understand that this sounds complicated, but since there is
> an implementation that already does this then I think we have to assume
> it's possible. :) Perhaps we should start with older, simpler gfx hw, or
> maybe POST the bios, but only as an interim solution until gfx drivers
> get better in this area.

You might be suprised how much BIOS help they get. Im not at liberty to
discuss details but at least two vendors jump into bios space in their
ACPI recovery routines.

Older APM has compiled in support for a load of common video card
variants and also uses int10 on the resume 16bit path.

RE: Subtle semantic issue with sleep callbacks in drivers

[Benjamin Herrenschmidt]

> All I am saying is that on Windows, the driver gets no help from the
> BIOS, APM, or ACPI, but yet it restores the video to full working
> condition. I understand that this sounds complicated, but since there is
> an implementation that already does this then I think we have to assume
> it's possible. :) Perhaps we should start with older, simpler gfx hw, or
> maybe POST the bios, but only as an interim solution until gfx drivers
> get better in this area.

It's definitely possible as MacOS does that as well. But I doubt
the card vendors will ever provide us with the necessary informations...

Ben.

RE: Subtle semantic issue with sleep callbacks in drivers

[Grover, Andrew]

> From: Benjamin Herrenschmidt [mailto:benh@kernel.crashing.org] 
> On Mon, 2003-04-14 at 19:09, Grover, Andrew wrote:
> > Ben obviously PPC is ahead of the pack on this stuff 
> (congrats) but I 
> > did just want to put forward the idea that when we're all done with 
> > this stuff on all archs, we will hopefully not be regularly 
> re-POSTing 
> > the video bios.
> 
> But how ? let's make clear what we call POST first ...
> 
> If the card is powered off, it must be POSTed to be brought 
> back to life. Either we do it by running the BIOS code 
> (probably what you are talking about and should be 
> deprecated), or the driver "knows" how to restore the chip 
> from power off. I don't know if APM/ACPI provides other ways, 
> I suspect the APM BIOS will re-POST the card by itself or 
> else, things wouldn't work today. I don't know about ACPI.
> 
> What I mean here is that none of our drivers know how to bring 
> back a chip as complicated as a radeon or a nvidia up from 
> power off, this requires intimate knowledge of the chip 
> internals, the way it's wired on a given board, etc...

All I am saying is that on Windows, the driver gets no help from the
BIOS, APM, or ACPI, but yet it restores the video to full working
condition. I understand that this sounds complicated, but since there is
an implementation that already does this then I think we have to assume
it's possible. :) Perhaps we should start with older, simpler gfx hw, or
maybe POST the bios, but only as an interim solution until gfx drivers
get better in this area.

Regards -- Andy

RE: Subtle semantic issue with sleep callbacks in drivers

[Benjamin Herrenschmidt]

On Mon, 2003-04-14 at 19:09, Grover, Andrew wrote:

> Topic drift...
> 
> After asking around internally, it sounds like we should not be doing a
> video re-POST on wakeup. Windows only used to in order to workaround
> buggy video drivers, according to what I've heard.
> 
> Ben obviously PPC is ahead of the pack on this stuff (congrats) but I
> did just want to put forward the idea that when we're all done with this
> stuff on all archs, we will hopefully not be regularly re-POSTing the
> video bios.

But how ? let's make clear what we call POST first ...

If the card is powered off, it must be POSTed to be brought back to
life. Either we do it by running the BIOS code (probably what you are
talking about and should be deprecated), or the driver "knows" how to
restore the chip from power off. I don't know if APM/ACPI provides
other ways, I suspect the APM BIOS will re-POST the card by itself
or else, things wouldn't work today. I don't know about ACPI.

What I mean here is that none of our drivers know how to bring 
back a chip as complicated as a radeon or a nvidia up from power off,
this requires intimate knowledge of the chip internals, the way it's
wired on a given board, etc...

On pmacs, the ROM does nothing for us. On wakeup from sleep, I basically
get control from the CPU right after the ROM figured out we are waking
up, that is a few instructions after the CPU itself is brought back from
power off, thus all HW is either just powered on by the PM microcontroller,
or in the state I left it when going to sleep.

So on machines where the PM microcontroller shuts off the PCI/AGP slots,
I must be able to fully restore PCI devices like video chips from their
power-on state, which I don't know how to do unless I'm the chip/board
manufacturer for a lot of "complicated" chips, like video ones...

The ability to re-run the BIOS (or open firmware) under some kind of
emulation or vm86 is a way to "work around" this problem.

The issue is that the driver must be told what is/canbe done so it
knows in what state it will get the chip back, and if it can resume
it at all or should reject sleep.

Ben.

RE: Subtle semantic issue with sleep callbacks in drivers

[Grover, Andrew]

> From: Benjamin Herrenschmidt [mailto:benh@kernel.crashing.org] 
> - On non-PPC machines, the slot will eventually go to D3, but 
> the APM BIOS or ACPI will be able to re-POST the card 
> properly on wakeup, so the driver only needs to restore the 
> current display mode, at least I guess so since I don't know 
> much about x86's. Similar will happen once I have an OF 
> emulator ready on PPC to re-POST some cards, thus changing 
> the previous example into this one. In this case, the driver 
> can put the chip to D3 and can _accept_ the sleep request 
> because it's explicitely told by the system (how ?) that the 
> card will be re-POSTED prior to the
> resume() callback.

Topic drift...

After asking around internally, it sounds like we should not be doing a
video re-POST on wakeup. Windows only used to in order to workaround
buggy video drivers, according to what I've heard.

Ben obviously PPC is ahead of the pack on this stuff (congrats) but I
did just want to put forward the idea that when we're all done with this
stuff on all archs, we will hopefully not be regularly re-POSTing the
video bios.

Regards -- Andy

Re: Subtle semantic issue with sleep callbacks in drivers

[Benjamin Herrenschmidt]

> I also noticed the IDE PM code is hopeless, I'll try to hack something
> that works at least as good than what I had in the pmac 2.4 code, and
> the fbdev PM code is almost unexisting (I need at least a way for the
> driver to tell the PM layer not to mess with the framebuffer once the
                     ^^
> driver is asleep). I'll hack something as well.

 ... to tell the fbdev layer ... you will have corrected yourself ;)

Ben.

Subtle semantic issue with sleep callbacks in drivers

[Benjamin Herrenschmidt]

Hi Patrick !

While implementing power management of PowerBooks based on
the new model (while porting my drivers at the same time), I
found a couple of problems that could be solved by better defining
the values of the "state" parameter passed to drivers. Or maybe it
is already, in which case, let me know :)

Basically, the driver need to know what kind of support is provided
by the motherboard for wake up and what will happen to the chip. The
typical example is the radeon driver which shows all cases:

- On mac laptops, the AGP slot is unclocked. That means the driver
has to go to D2 (for which I have full code from ATI). That is
independant from other things on the motherboard, some things are
powerd off, but not the AGP slot, so I need a way to affect the
"state" parameter on a per-slot basis.

- On non-laptops, the AGP slot is powered off. So I beleive I
basically need to tell the driver to go to D3 (or basically prepare
to re-POST the card on wakeup).
However, we can't do that. We don't have enough infos from ATI to
be able to do it right now, which is why I don't support suspend-to-RAM
on non-laptops pmacs. So here, the driver should fail the suspend
request if it's told, via the "state" parameter that the chip will be
powered off by the system

- On non-PPC machines, the slot will eventually go to D3, but the APM
BIOS or ACPI will be able to re-POST the card properly on wakeup, so the
driver only needs to restore the current display mode, at least I guess
so since I don't know much about x86's. Similar will happen once I have
an OF emulator ready on PPC to re-POST some cards, thus changing the
previous example into this one. In this case, the driver can put the
chip to D3 and can _accept_ the sleep request because it's explicitely
told by the system (how ?) that the card will be re-POSTED prior to the
resume() callback.

- On any machine, with suspend to disk, the slot is powered off (along
with the entire host machine). But the machine will be rebooted and the
device put back to "idle" state some way by the loader kernel, so this
ends up a bit like the above case.


So basically, the "state" parameter should encore not only what state
we want to go to, but rather, what will happen to the slot:

- Nothing (it's entirely up to the driver to do it's own power
management, that happens for some devices inside Apple ASIC), though in
this case at least, those driver have control over the chip power, reset
lines, etc...
- Slot will be unclocked (it's up to the driver, it the chip supports
static mode, to go to D2 or D3 if the driver can deal with it, though
the system will do nothing to help the driver)
- Slot will be powered off. This case should be broken up (via an
additional flag passed to the driver ?) into 1) the system _will_
re-POST the card before resume (BIOS/ACPI support, swsusp) or the
system will NOT re-POST the card, the driver shall fail the sleep
request if it can't do it by itself.
 - Embedded people may have even more weird cases ?

Any comments ?

I also noticed the IDE PM code is hopeless, I'll try to hack something
that works at least as good than what I had in the pmac 2.4 code, and
the fbdev PM code is almost unexisting (I need at least a way for the
driver to tell the PM layer not to mess with the framebuffer once the
driver is asleep). I'll hack something as well.

Regards
Ben.