RE: [PATCH] SCSI midlayer power management

RE: [PATCH] SCSI midlayer power management

[James.Smart]

Glad to see where this thread ended up...

Suspend, for scsi hosts, essentially has to end up being an unattach. You can't make assumptions on DMA use, etc based on just quiescing host i/o flow or no target for multi-initiators. There's too much variance in adapters design that depend on adapter-pull functionality - rings, ring pointers, host-based contexts, host-based firmware (scripts), and so on. There's also lots of potential asynchronous traffic that can occur outside of just host-requested io : for FC, there's ELS/BLS's; for iSCSI there's non-io PDU's (NOPs, AENs). Posted buffers to the adapter must be killed, and so on.  Ultimately, the scsi class drivers must quiesce the devices, then the LLDD's must "suspend", essentially saving software state and resetting hardware.

For Resume, given that the MMIO state isn't restored, and given the above argument of "detach" on suspend - the LLDD must essentially restart the hardware, reprogramming dma addresses, reposting buffers, reinstantiating contexts, etc.  As indicated, easier on some hardware, harder on others.

So, as Pavel indicates - powermanagement for the LLDD should look very similar to rmmod/insmod. The question usually comes down to whether the overhead of a suspend/resume, where the driver has to participate to save/restore software state, is worth the extra complexity vs. whether you are better off just having the LLDD fully detach/reattach. Latter usually wins as it supports hot-plug well, and moves the complexities out of each driver and into the infrastructure.

-- James S


> -----Original Message-----
> From: linux-scsi-owner@vger.kernel.org
> [mailto:linux-scsi-owner@vger.kernel.org]On Behalf Of Pavel Machek
> Sent: Thursday, August 12, 2004 4:49 PM
> To: James Bottomley
> Cc: Benjamin Herrenschmidt; Nathan Bryant; Linux SCSI Reflector; Linux
> Kernel list; Jeff Garzik
> Subject: Re: [PATCH] SCSI midlayer power management
> 
> 
> Hi!
> 
> > > Can't you simply reuse bootup code? It will no longer be __init,
> > > but it should make suspend/resume functions quite simple.
> > 
> > Unfortunately, no that simply.
> > 
> > Bootup is all about allocating these areas and initialising 
> the card. 
> > Resume will be about initialising the card knowing the 
> existing areas
> > (and the data about the existing areas will have to be part of our
> > persistent data on suspend).
> 
> You can simply free those areas during suspend, so that resume will be
> same as powerup....
> 
> Essentially if you can do insmod and rmmod, you can "simply" emulate
> rmmod during suspend and emulate "insmod" during resume...
> 
> > > > to pick three drivers to do this for, that would be 
> aic7xxx, aic79xx and
> > > > sym_2?
> > > 
> > > No idea, only SCSI host I owned was some 8-bit isa thing....
> > 
> > Well, someone who's interested needs to pick a driver.  It's usually
> > easier to persuade everyone to add the feature if there's 
> an example to
> > copy...
> 
> Make it aic7xxx then... Someone already worked on that one.
> 
> 								
> 	Pavel
> -- 
> People were complaining that M$ turns users into beta-testers...
> ...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!
> -
> To unsubscribe from this list: send the line "unsubscribe 
> linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH] SCSI midlayer power management

[Benjamin Herrenschmidt]

On Fri, 2004-08-13 at 06:31, James Bottomley wrote:
> On Thu, 2004-08-12 at 16:26, Pavel Machek wrote:
> > Yes.
> 
> Well, that makes the suspend and resume functions rather complex. 
> They're not going to be coded simply if we have to save and restore the
> register state of the cards and reinitialise them.  I assume if you had
> to pick three drivers to do this for, that would be aic7xxx, aic79xx and
> sym_2?

It's not simple for some chips, it's simple for others, in lots of
cases, it's just a matter of re-doing the driver init code though.

Ben.

Re: [PATCH] SCSI midlayer power management

[Benjamin Herrenschmidt]

On Fri, 2004-08-13 at 05:34, James Bottomley wrote:
> On Thu, 2004-08-12 at 15:11, Pavel Machek wrote:
> > Ok, but what happens on next resume? If coherent mbox is at exactly
> > same place at every boot I guess it could even work, but...
> 
> Er, well this is a huge problem then.  Even if DMA were stopped, the
> registers for all these locations need to be altered to change the
> location of the DMA mboxes.  This isn't just a SCSI problem, it's a
> general device problem (most devices having mboxes programmed by
> register).  If we can't rely on the resuming kernel setting up these
> registers for us to exactly what they were in the resume image, then
> we're in a bit of trouble.

Ugh ? What do you mean ? The suspend kernel will snapshot the kernel
memory at one point, after the device suspend call has been done. If
the device relies on some changes done after that, then it's broken
somewhat.

The resume will restore all memory, not MMIO of course, and it's up
to the driver to do whatever is necessary to restore operations
properly.

> Architecturally what you are trying to do is to re POST the SCSI card. 
> Except it's the kernel's job to POST it, so the kernel init code needs
> to be re-run.  I assume that's what the pci suspend/resume calls are
> supposed to do?

Yes.

The problem is the same with suspend-to-RAM basically.

Ben.

Re: [PATCH] SCSI midlayer power management

[Benjamin Herrenschmidt]

On Thu, 2004-08-12 at 22:48, James Bottomley wrote:
> On Wed, 2004-08-11 at 19:36, Benjamin Herrenschmidt wrote:
> > Some hosts will continuously DMA to memory iirc.. I remember having a
> > problem with 53c8xx on some macs when transitionning from MacOS to Linux
> > because of that.
> 
> I think you're thinking of the scripts engine?  on pre 53c875 chips,
> yes, this is true.  The on-board processor is executing instructions
> from host memory.  However, this is read only in quiescent (waiting for
> host connect or target reconnect) mode, so shouldn't be a problem for
> suspend.  On the 875 and later, we host the scripts in on-chip memory so
> they shouldn't be troubling main memory when idling.

The problem was a 875, the script was writing some kind of status or
whatever at regular interval to host memory. Might be specific to the
script used by the MacOS driver tho...

Ben.

Re: [PATCH] SCSI midlayer power management

[Nigel Cunningham]

Howdy again.

On Fri, 2004-08-13 at 08:36, Nigel Cunningham wrote:
> I'm not pretending to understand the issues you're talking about, but I
> do have a question that might possibly be helpful: Pages can be marked
> with the Nosave flag, so that they're not saved in the image and not
> overwritten when we copy the old kernel back. Would using Nosave help
> here at all?

Having read the rest of the thread, I can see that's probably not
helpful :>

Nigel
-- 
Nigel Cunningham
Christian Reformed Church of Tuggeranong
PO Box 1004, Tuggeranong, ACT 2901

Many today claim to be tolerant. But true tolerance can cope with others
being intolerant.

Re: [PATCH] SCSI midlayer power management

[Nigel Cunningham]

Hi.

On Fri, 2004-08-13 at 05:34, James Bottomley wrote:
> Er, well this is a huge problem then.  Even if DMA were stopped, the
> registers for all these locations need to be altered to change the
> location of the DMA mboxes.  This isn't just a SCSI problem, it's a
> general device problem (most devices having mboxes programmed by
> register).  If we can't rely on the resuming kernel setting up these
> registers for us to exactly what they were in the resume image, then
> we're in a bit of trouble.
> 
> Architecturally what you are trying to do is to re POST the SCSI card. 
> Except it's the kernel's job to POST it, so the kernel init code needs
> to be re-run.  I assume that's what the pci suspend/resume calls are
> supposed to do?

I'm not pretending to understand the issues you're talking about, but I
do have a question that might possibly be helpful: Pages can be marked
with the Nosave flag, so that they're not saved in the image and not
overwritten when we copy the old kernel back. Would using Nosave help
here at all?

Nigel
-- 
Nigel Cunningham
Christian Reformed Church of Tuggeranong
PO Box 1004, Tuggeranong, ACT 2901

Many today claim to be tolerant. But true tolerance can cope with others
being intolerant.

Re: [PATCH] SCSI midlayer power management

[Nathan Bryant]

James Bottomley wrote:
> On Thu, 2004-08-12 at 16:37, Pavel Machek wrote:
> 
>>Can't you simply reuse bootup code? It will no longer be __init,
>>but it should make suspend/resume functions quite simple.
> 
> 
> Unfortunately, no that simply.
> 
> Bootup is all about allocating these areas and initialising the card. 
> Resume will be about initialising the card knowing the existing areas
> (and the data about the existing areas will have to be part of our
> persistent data on suspend).
> 
> So, modifying the bootup to do something like
> 
> if (in_resume)
> 	addr = read from suspend image

Maybe not that complex. The "suspend image" has become the running 
kernel by the time we receive the resume request, right? So we maybe 
just look at the dma base address that we already have in our 
card-private data structure. If that's all that's needed the S3-resume 
is more likely to just work for disk suspend.

But in general yes you are right you need to separate the bootup code 
into "allocate structures" and "program the card for those locations" 
sections. Personally I think this approach can ultimately be more 
maintainable than saving and restoring registers.

> else
> 	addr = dma_alloc_coherent(...)
> 
> may work.
> 
> 
>>>to pick three drivers to do this for, that would be aic7xxx, aic79xx and
>>>sym_2?
>>
>>No idea, only SCSI host I owned was some 8-bit isa thing....
> 
> 
> Well, someone who's interested needs to pick a driver.  It's usually
> easier to persuade everyone to add the feature if there's an example to
> copy...

Look at my aic7xxx patch. Maybe a little messy but it works. Eh it kind 
of uses a mix of a save/restore registers technique (inherited from 
Justin Gibbs' code) and my own "reuse-the-boot-code" approach that I 
needed to fill in the gaps in his resume code, which was untested

> 
> James
> 
> 
> 

Re: [PATCH] SCSI midlayer power management

[Pavel Machek]

Hi!

> > Can't you simply reuse bootup code? It will no longer be __init,
> > but it should make suspend/resume functions quite simple.
> 
> Unfortunately, no that simply.
> 
> Bootup is all about allocating these areas and initialising the card. 
> Resume will be about initialising the card knowing the existing areas
> (and the data about the existing areas will have to be part of our
> persistent data on suspend).

You can simply free those areas during suspend, so that resume will be
same as powerup....

Essentially if you can do insmod and rmmod, you can "simply" emulate
rmmod during suspend and emulate "insmod" during resume...

> > > to pick three drivers to do this for, that would be aic7xxx, aic79xx and
> > > sym_2?
> > 
> > No idea, only SCSI host I owned was some 8-bit isa thing....
> 
> Well, someone who's interested needs to pick a driver.  It's usually
> easier to persuade everyone to add the feature if there's an example to
> copy...

Make it aic7xxx then... Someone already worked on that one.

									Pavel
-- 
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!

Re: [PATCH] SCSI midlayer power management

[James Bottomley]

On Thu, 2004-08-12 at 16:37, Pavel Machek wrote:
> Can't you simply reuse bootup code? It will no longer be __init,
> but it should make suspend/resume functions quite simple.

Unfortunately, no that simply.

Bootup is all about allocating these areas and initialising the card. 
Resume will be about initialising the card knowing the existing areas
(and the data about the existing areas will have to be part of our
persistent data on suspend).

So, modifying the bootup to do something like

if (in_resume)
	addr = read from suspend image
else
	addr = dma_alloc_coherent(...)

may work.

> > to pick three drivers to do this for, that would be aic7xxx, aic79xx and
> > sym_2?
> 
> No idea, only SCSI host I owned was some 8-bit isa thing....

Well, someone who's interested needs to pick a driver.  It's usually
easier to persuade everyone to add the feature if there's an example to
copy...

James

Re: [PATCH] SCSI midlayer power management

[Nathan Bryant]

James Bottomley wrote:
> On Thu, 2004-08-12 at 16:26, Pavel Machek wrote:
> 
>>Yes.
> 
> 
> Well, that makes the suspend and resume functions rather complex. 
> They're not going to be coded simply if we have to save and restore the
> register state of the cards and reinitialise them.  I assume if you had
> to pick three drivers to do this for, that would be aic7xxx, aic79xx and
> sym_2?

S3 suspend-to-RAM has the same requirement. When the system enters S3, 
the PCI slot loses all physical power and the card's registers are 
completely gone.

So I've already had to fix up the aic7xxx driver to, among other things, 
reset the DMA base addres on resume. This should help for swsusp. 
Although maybe there are other requirements for swsusp... I think for 
swsusp we have to cope with a resume request that seems to come out of 
nowhere.

> 
> James
> 
> 
> 

Re: [PATCH] SCSI midlayer power management

[Pavel Machek]

Hi!

> > Yes.
> 
> Well, that makes the suspend and resume functions rather complex. 
> They're not going to be coded simply if we have to save and restore the
> register state of the cards and reinitialise them.  I assume if you
> had

Can't you simply reuse bootup code? It will no longer be __init,
but it should make suspend/resume functions quite simple.

> to pick three drivers to do this for, that would be aic7xxx, aic79xx and
> sym_2?

No idea, only SCSI host I owned was some 8-bit isa thing....

								Pavel

-- 
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!

Re: [PATCH] SCSI midlayer power management

[James Bottomley]

On Thu, 2004-08-12 at 16:26, Pavel Machek wrote:
> Yes.

Well, that makes the suspend and resume functions rather complex. 
They're not going to be coded simply if we have to save and restore the
register state of the cards and reinitialise them.  I assume if you had
to pick three drivers to do this for, that would be aic7xxx, aic79xx and
sym_2?

James

Re: [PATCH] SCSI midlayer power management

[Pavel Machek]

Hi!

> > Ok, but what happens on next resume? If coherent mbox is at exactly
> > same place at every boot I guess it could even work, but...
> 
> Er, well this is a huge problem then.  Even if DMA were stopped, the
> registers for all these locations need to be altered to change the
> location of the DMA mboxes.  This isn't just a SCSI problem, it's a
> general device problem (most devices having mboxes programmed by
> register).  If we can't rely on the resuming kernel setting up these
> registers for us to exactly what they were in the resume image, then
> we're in a bit of trouble.
> 
> Architecturally what you are trying to do is to re POST the SCSI card. 
> Except it's the kernel's job to POST it, so the kernel init code needs
> to be re-run.  I assume that's what the pci suspend/resume calls are
> supposed to do?

Yes.
								Pavel

-- 
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!

Re: [PATCH] SCSI midlayer power management

[James Bottomley]

On Thu, 2004-08-12 at 15:11, Pavel Machek wrote:
> Ok, but what happens on next resume? If coherent mbox is at exactly
> same place at every boot I guess it could even work, but...

Er, well this is a huge problem then.  Even if DMA were stopped, the
registers for all these locations need to be altered to change the
location of the DMA mboxes.  This isn't just a SCSI problem, it's a
general device problem (most devices having mboxes programmed by
register).  If we can't rely on the resuming kernel setting up these
registers for us to exactly what they were in the resume image, then
we're in a bit of trouble.

Architecturally what you are trying to do is to re POST the SCSI card. 
Except it's the kernel's job to POST it, so the kernel init code needs
to be re-run.  I assume that's what the pci suspend/resume calls are
supposed to do?

James

Re: [PATCH] SCSI midlayer power management

[Pavel Machek]

Hi!

> > Even read-only access could hurt.... That DMA engine is going to get
> > very unhappy if we change data from under it, right?
> 
> But we're not planning to change this area of memory (it's a driver
> allocated coherent mbox) until we power off the box, right? so it should
> be just like a reboot today.

Ok, but what happens on next resume? If coherent mbox is at exactly
same place at every boot I guess it could even work, but...
								Pavel
-- 
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!

Re: [PATCH] SCSI midlayer power management

[Patrick Mansfield]

On Thu, Aug 12, 2004 at 09:41:00AM -0400, Nathan Bryant wrote:
> James Bottomley wrote:
> 
> >Why?  We don't do a bus reset on boot, why should we need to do one on
> >resume?  For FC, the equivalent, a LIP Reset can be rather nasty on a
> >SAN and should be avoided.
> > 
> >
> that can be host specific. aic7xxx does a bus reset on boot, so i 
> preserved this on resume.

It can be changed via the "no_reset" option, that should probably be
honored for resume.

> don't know why they do it, but they do.

I used to use no_reset, as well as bios no reset (or is it no spin up?) to
decrease boot time, until a crash hung the bus and the system could not
boot. I suppose letting the bios reset would have gotten around the
problem, I can't remember.

-- Patrick Mansfield

Re: [PATCH] SCSI midlayer power management

[James Bottomley]

On Thu, 2004-08-12 at 09:14, Pavel Machek wrote:
> Even read-only access could hurt.... That DMA engine is going to get
> very unhappy if we change data from under it, right?

But we're not planning to change this area of memory (it's a driver
allocated coherent mbox) until we power off the box, right? so it should
be just like a reboot today.

James

Re: [PATCH] SCSI midlayer power management

[Nathan Bryant]

Nigel Cunningham wrote:

>The host adapter isn't in the device's chain of parents?
>
It does seem to be, if you look in sysfs. Ok maybe I'm pointing out the 
obvious ;)

Re: [PATCH] SCSI midlayer power management

[Nathan Bryant]

James Bottomley wrote:

>Why?  We don't do a bus reset on boot, why should we need to do one on
>resume?  For FC, the equivalent, a LIP Reset can be rather nasty on a
>SAN and should be avoided.
>  
>
that can be host specific. aic7xxx does a bus reset on boot, so i 
preserved this on resume.

don't know why they do it, but they do.

>The slight problem is probably going to be knowing that we may need to
>spin up devices (for internal ones) before resuming operation.
>  
>
that's easy for system suspend, but somewhat harder for device suspend

Re: [PATCH] SCSI midlayer power management

[Pavel Machek]

Hi!

> > Some hosts will continuously DMA to memory iirc.. I remember having a
> > problem with 53c8xx on some macs when transitionning from MacOS to Linux
> > because of that.
> 
> I think you're thinking of the scripts engine?  on pre 53c875 chips,
> yes, this is true.  The on-board processor is executing instructions
> from host memory.  However, this is read only in quiescent (waiting for
> host connect or target reconnect) mode, so shouldn't be a problem for
> suspend.  On the 875 and later, we host the scripts in on-chip memory so
> they shouldn't be troubling main memory when idling.

Even read-only access could hurt.... That DMA engine is going to get
very unhappy if we change data from under it, right?


								Pavel
-- 
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!

Re: [PATCH] SCSI midlayer power management

[James Bottomley]

On Wed, 2004-08-11 at 19:36, Benjamin Herrenschmidt wrote:
> Some hosts will continuously DMA to memory iirc.. I remember having a
> problem with 53c8xx on some macs when transitionning from MacOS to Linux
> because of that.

I think you're thinking of the scripts engine?  on pre 53c875 chips,
yes, this is true.  The on-board processor is executing instructions
from host memory.  However, this is read only in quiescent (waiting for
host connect or target reconnect) mode, so shouldn't be a problem for
suspend.  On the 875 and later, we host the scripts in on-chip memory so
they shouldn't be troubling main memory when idling.

> We need to properly quisce the host, but that's a per host driver thing
> and shouldn't be too difficult.
> 
> Regarding suspend-to-disk, it's fairly easy for the sd driver not to
> spin down the disk for S4 (only for S3). However, we will still probably
> do at least a bus reset when waking up...

Why?  We don't do a bus reset on boot, why should we need to do one on
resume?  For FC, the equivalent, a LIP Reset can be rather nasty on a
SAN and should be avoided.

The slight problem is probably going to be knowing that we may need to
spin up devices (for internal ones) before resuming operation.

James

Re: [PATCH] SCSI midlayer power management

[Benjamin Herrenschmidt]

> Hmm, and it can not be handled by "just remember why you were
> suspended", because it is one suspend, two resumes...
> 
> Yes, I agree that argument will be usefull. Just who does all the
> driver updating? ;-).

At this point, no driver cares, so it's just a matter of fixing the
prototypes, I'll leave that to somebody more familiar with sed :)

Ben.

Re: [PATCH] SCSI midlayer power management

[Nigel Cunningham]

Hi.

On Thu, 2004-08-12 at 08:48, Nathan Bryant wrote: 
> Just to speculate about what would be required for swsusp: you probably 
> need to be using a SCSI LLD that properly implements pci suspend/resume, 
> which implies you need to make sure the card's DMA state machine is 
> flushed and idle before suspend completes. I've got a patch that fixes 
> this much up for aic7xxx. And my other midlayer-level patch may also 
> help... What happens during resume is interesting. I think maybe the 
> problem is not what the drive is expecting, but what the card's state 
> engine is expecting when it tries to map commands to command buffers in 
> DMA space.  Maybe you need to suspend the LLD from the context of the 
> kernel that is doing the image load, and then resume from the context of 
> the kernel that was just loaded.

I fully agree. That's what I'm doing at the moment; it's been a while
since I looked at swsusp though, so can't say anything about Pavel &
Patrick's implementation.

> >With my 'device tree' code, I'm getting the struct dev of the device
> >we're using via the struct block_device in the swap_info struct.
> >
> Right, though you also need to get the host adapter's struct device, if 
> you're not already doing so, that is. Many IDE host drivers don't bother 
> with suspend/resume callbacks at the pci_driver level, but SCSI needs 
> callbacks because the BIOS usually doesn't handle things for us.

The host adapter isn't in the device's chain of parents?

Nigel 

-- 
Nigel Cunningham
Christian Reformed Church of Tuggeranong
PO Box 1004, Tuggeranong, ACT 2901

Many today claim to be tolerant. But true tolerance can cope with others
being intolerant.

Re: [PATCH] SCSI midlayer power management

[Pavel Machek]

Hi!

> > Actually, the answer is to most intents and purposes "yes".  You are
> > technically correct: there's no way to disable DMA in SCSI.  However,
> > once a device is quiesced, it has no outstanding commands, so there will
> > be no outstanding DMA to that device.  When all devices on a host have
> > been quiesced, then there will be no DMA at all going on *except* if the
> > user initiates any via another interface (like sending a device probe or
> > doing a unit scan).  The guarantee should be strong enough for swsusp to
> > proceed, but we can look at quiescing a host properly (however, we'd
> > need to move to a better host state model than we currently possess).
> 
> Some hosts will continuously DMA to memory iirc.. I remember having a
> problem with 53c8xx on some macs when transitionning from MacOS to Linux
> because of that.
> 
> We need to properly quisce the host, but that's a per host driver thing
> and shouldn't be too difficult.
> 
> Regarding suspend-to-disk, it's fairly easy for the sd driver not to
> spin down the disk for S4 (only for S3). However, we will still probably
> do at least a bus reset when waking up...
> 
> Pavel: That's one of the reason I wanted an argument to resume() too so
> drivers can make a difference between the immediate wakeup that happens
> for writing the image to disk, vs. the real wakeup on resume. In the first
> case, SCSI can avoid the bus reset, and any kind of re-configuring, in the 
> second case, the full stuff might be necessary. 

Hmm, and it can not be handled by "just remember why you were
suspended", because it is one suspend, two resumes...

Yes, I agree that argument will be usefull. Just who does all the
driver updating? ;-).
								Pavel
-- 
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!

Re: [PATCH] SCSI midlayer power management

[Pavel Machek]

Hi!

> >I tried it on an OSDL machine and could suspend (suspend 2), but only
> >resume as far as copying back the original kernel. The problem then
> >looked to me like it was request ids not matching what the drive was
> >expecting (but I'm ignorant of scsi, so might be completely wrong
> >there).
> > 
> >
> I saw "no match for command buffer" interrupt storms when I was fixing 
> up aic7xxx for S3. The problem was due to not reprogramming the address 
> of our SCB's on resume. Needed to tell the card the base address for all 
> the DMA structures.
> 
> Just to speculate about what would be required for swsusp: you probably 
> need to be using a SCSI LLD that properly implements pci suspend/resume, 
> which implies you need to make sure the card's DMA state machine is 
> flushed and idle before suspend completes. I've got a patch that fixes 
> this much up for aic7xxx. And my other midlayer-level patch may also 
> help... What happens during resume is interesting. I think maybe the 
> problem is not what the drive is expecting, but what the card's state 
> engine is expecting when it tries to map commands to command buffers in 
> DMA space.  Maybe you need to suspend the LLD from the context of the 
> kernel that is doing the image load, and then resume from the context of 
> the kernel that was just loaded.

Ideally, suspended driver should have no state at all. Like if I send
card to suspend with 2.6.8, and when I send it to suspend in 2.6.11,
it should be in same state.

> Sounds like this is why Pavel is asking about DMA. So he'll need to 
> manage calling the host adapter's suspend callbacks, not just 
> generic_scsi_suspend. DMA base addresses are likely to change when you 
> load the new kernel image

sysfs should call host adapter's suspend callbacks... It should work
today.
								Pavel
-- 
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!

Re: [PATCH] SCSI midlayer power management

[Benjamin Herrenschmidt]

> Actually, the answer is to most intents and purposes "yes".  You are
> technically correct: there's no way to disable DMA in SCSI.  However,
> once a device is quiesced, it has no outstanding commands, so there will
> be no outstanding DMA to that device.  When all devices on a host have
> been quiesced, then there will be no DMA at all going on *except* if the
> user initiates any via another interface (like sending a device probe or
> doing a unit scan).  The guarantee should be strong enough for swsusp to
> proceed, but we can look at quiescing a host properly (however, we'd
> need to move to a better host state model than we currently possess).

Some hosts will continuously DMA to memory iirc.. I remember having a
problem with 53c8xx on some macs when transitionning from MacOS to Linux
because of that.

We need to properly quisce the host, but that's a per host driver thing
and shouldn't be too difficult.

Regarding suspend-to-disk, it's fairly easy for the sd driver not to
spin down the disk for S4 (only for S3). However, we will still probably
do at least a bus reset when waking up...

Pavel: That's one of the reason I wanted an argument to resume() too so
drivers can make a difference between the immediate wakeup that happens
for writing the image to disk, vs. the real wakeup on resume. In the first
case, SCSI can avoid the bus reset, and any kind of re-configuring, in the 
second case, the full stuff might be necessary. 

Ben.

Re: [PATCH] SCSI midlayer power management

[Nathan Bryant]

Nigel Cunningham wrote:

>Hi.
>
>On Wed, 2004-08-11 at 23:13, Nathan Bryant wrote:
>  
>
>>>>ACPI S1 and S4/swsusp are untested, but I think there should be no
>>>>regressions with S1. To do S1 properly, we probably need to tell the
>>>>drive to spin down, and I don't know what the SCSI command is for
>>>>that... For S4, the call to scsi_device_quiesce might pose a problem for
>>>>the subsequent state dump to disk. But I'm not sure swsusp ever worked
>>>>for SCSI.
>>>>        
>>>>
>
>I tried it on an OSDL machine and could suspend (suspend 2), but only
>resume as far as copying back the original kernel. The problem then
>looked to me like it was request ids not matching what the drive was
>expecting (but I'm ignorant of scsi, so might be completely wrong
>there).
>  
>
I saw "no match for command buffer" interrupt storms when I was fixing 
up aic7xxx for S3. The problem was due to not reprogramming the address 
of our SCB's on resume. Needed to tell the card the base address for all 
the DMA structures.

Just to speculate about what would be required for swsusp: you probably 
need to be using a SCSI LLD that properly implements pci suspend/resume, 
which implies you need to make sure the card's DMA state machine is 
flushed and idle before suspend completes. I've got a patch that fixes 
this much up for aic7xxx. And my other midlayer-level patch may also 
help... What happens during resume is interesting. I think maybe the 
problem is not what the drive is expecting, but what the card's state 
engine is expecting when it tries to map commands to command buffers in 
DMA space.  Maybe you need to suspend the LLD from the context of the 
kernel that is doing the image load, and then resume from the context of 
the kernel that was just loaded.

Sounds like this is why Pavel is asking about DMA. So he'll need to 
manage calling the host adapter's suspend callbacks, not just 
generic_scsi_suspend. DMA base addresses are likely to change when you 
load the new kernel image

>>answer is NO. For purposes of not suspending the drivers, I haven't 
>>looked into how swsusp would see which host adapter owns which drive, 
>>but some of the required information seems to be present in sysfs.
>>    
>>
>
>With my 'device tree' code, I'm getting the struct dev of the device
>we're using via the struct block_device in the swap_info struct.
>  
>
Right, though you also need to get the host adapter's struct device, if 
you're not already doing so, that is. Many IDE host drivers don't bother 
with suspend/resume callbacks at the pci_driver level, but SCSI needs 
callbacks because the BIOS usually doesn't handle things for us.

Nathan

Re: [PATCH] SCSI midlayer power management

[Nigel Cunningham]

Hi.

On Wed, 2004-08-11 at 23:13, Nathan Bryant wrote:
> >>ACPI S1 and S4/swsusp are untested, but I think there should be no
> >>regressions with S1. To do S1 properly, we probably need to tell the
> >>drive to spin down, and I don't know what the SCSI command is for
> >>that... For S4, the call to scsi_device_quiesce might pose a problem for
> >>the subsequent state dump to disk. But I'm not sure swsusp ever worked
> >>for SCSI.

I tried it on an OSDL machine and could suspend (suspend 2), but only
resume as far as copying back the original kernel. The problem then
looked to me like it was request ids not matching what the drive was
expecting (but I'm ignorant of scsi, so might be completely wrong
there).

> answer is NO. For purposes of not suspending the drivers, I haven't 
> looked into how swsusp would see which host adapter owns which drive, 
> but some of the required information seems to be present in sysfs.

With my 'device tree' code, I'm getting the struct dev of the device
we're using via the struct block_device in the swap_info struct.

Nigel
-- 
Nigel Cunningham
Christian Reformed Church of Tuggeranong
PO Box 1004, Tuggeranong, ACT 2901

Many today claim to be tolerant. But true tolerance can cope with others
being intolerant.

Re: [PATCH] SCSI midlayer power management

[Nathan Bryant]

Pavel Machek wrote:

> No, I do not need PIO. I'll probably need host
> controller support, too, but even w/o it it should
> work acceptably. Thanks for the answers.

Well, take note of what James wrote. We do guarantee that there will be 
no further user-initiated disk activity after generic_scsi_suspend. (For 
the disk but not the rest of the bus.) This doesn't apply to 
ioctl's/special requests but as James has rightly pointed out this is 
probably not an issue for realistic intents and purposes. It may 
actually be an advantage.

Re: [PATCH] SCSI midlayer power management

[Pavel Machek]

Hi!

> >swsusp will then resume disk and write the image, that should not be 
> >a
> >problem. Is it guaranteed that after generic_scsi_suspend() no DMA is
> >going on?
> > 
> >
> No. Remember that DMA works differently under SCSI than it does under 
> IDE. SCSI DMA is a host controller feature, whereas under IDE it is 
> enabled/disabled at the drive level and the drives have special 
> knowledge of DMA. Since generic_scsi_suspend() is the device level 
> suspend routine, it is called before the host controller's suspend 
> routine, (due to depth first traversal of device tree), which is 
> responsible for disabling the PCI slot. Only after the host 
> controller is suspended will there be no DMA, but if your real 
> question is "can I generically control a SCSI disk with PIO for 
> software suspend" then the answer is NO. For purposes of not 

No, I do not need PIO. I'll probably need host
controller support, too, but even w/o it it should
work acceptably. Thanks for the answers.
-- 
64 bytes from 195.113.31.123: icmp_seq=28 ttl=51 time=448769.1 ms         

Re: [PATCH] SCSI midlayer power management

[Nathan Bryant]

James Bottomley wrote:
> On Wed, 2004-08-11 at 10:21, Alan Cox wrote:
> 
>>In addition we are not doing SCSI target so multi-initiator is ok.
>>One question James - what are the rules for power management with
>>SCSI when we provide termpwr to a shared bus ?

Well there are (at least) two reasons to do shared bus: IP over SCSI and 
cluster filesystems. At least for cluster filesystems I think the answer 
is "don't suspend your machine" for the foreseeable future - what 
happens if your machine goes to sleep while holding cluster FS locks, 
etc. I doubt any CFS vendors are going to bother to support this any 
time soon, since it's a HA/server feature and power management is really 
a desktop/laptop feature.

Re: [PATCH] SCSI midlayer power management

[James Bottomley]

On Wed, 2004-08-11 at 10:21, Alan Cox wrote:
> In addition we are not doing SCSI target so multi-initiator is ok.
> One question James - what are the rules for power management with
> SCSI when we provide termpwr to a shared bus ?

I don't believe the spec addresses that.

The rules we use in HA are that each bus entity supplies termpwr (i.e.
all initiators and targets) and each terminator draws its power from the
bus).  Thus the bus is properly terminated until the last device powers
down.

You have to jumper this specially on most SCSI cards (especially if
they're at the end of the bus and have internal termination).

James

Re: [PATCH] SCSI midlayer power management

[Alan Cox]

On Mer, 2004-08-11 at 14:37, James Bottomley wrote:
> Actually, the answer is to most intents and purposes "yes".  You are
> technically correct: there's no way to disable DMA in SCSI.  However,
> once a device is quiesced, it has no outstanding commands, so there will
> be no outstanding DMA to that device. 

In addition we are not doing SCSI target so multi-initiator is ok.
One question James - what are the rules for power management with
SCSI when we provide termpwr to a shared bus ?

Re: [PATCH] SCSI midlayer power management

[James Bottomley]

On Wed, 2004-08-11 at 04:53, Alan Cox wrote:
> That was something Mark Lord reported higher level I suspect - which is
> that the scsi path is disabled before the sync cache command is sent so
> the command is always errored before it hits the drive

Yes, this one's my fault.  I have patches to fix this, I never got
around to merging them.  Randy was on at me yesterday about this too. 
I'll dig them out.  (The problem is that last sync is sent in SDEV_DEL
state, so it gets rejected).

James

Re: [PATCH] SCSI midlayer power management

[James Bottomley]

On Wed, 2004-08-11 at 08:13, Nathan Bryant wrote:
> No. Remember that DMA works differently under SCSI than it does under 
> IDE. SCSI DMA is a host controller feature, whereas under IDE it is 
> enabled/disabled at the drive level and the drives have special 
> knowledge of DMA. Since generic_scsi_suspend() is the device level 
> suspend routine, it is called before the host controller's suspend 
> routine, (due to depth first traversal of device tree), which is 
> responsible for disabling the PCI slot. Only after the host controller 
> is suspended will there be no DMA, but if your real question is "can I 
> generically control a SCSI disk with PIO for software suspend" then the 
> answer is NO. For purposes of not suspending the drivers, I haven't 
> looked into how swsusp would see which host adapter owns which drive, 
> but some of the required information seems to be present in sysfs.

Actually, the answer is to most intents and purposes "yes".  You are
technically correct: there's no way to disable DMA in SCSI.  However,
once a device is quiesced, it has no outstanding commands, so there will
be no outstanding DMA to that device.  When all devices on a host have
been quiesced, then there will be no DMA at all going on *except* if the
user initiates any via another interface (like sending a device probe or
doing a unit scan).  The guarantee should be strong enough for swsusp to
proceed, but we can look at quiescing a host properly (however, we'd
need to move to a better host state model than we currently possess).

James

Re: [PATCH] SCSI midlayer power management

[Nathan Bryant]

Pavel Machek wrote:

>Hi!
>
>  
>
>>This proposed patch implements enough power-management support within
>>the SCSI midlayer to get ACPI S3 working on my system. Changes as follows:
>>
>>* Add generic_scsi_{suspend,resume} methods to scsi.c
>>* Add suspend and resume callbacks to the scsi_driver structure, and
>>implement those callbacks in sd.c
>>* In sd.c, we call sd_shutdown on suspend, in order to synchronize the
>>write-back cache.
>>* In sd.c, we call sd_rescan from sd_resume in order to ensure that
>>drives have spun up and avoid passing not ready errors back to the block
>>layer.
>>* In generic_scsi_suspend, we call scsi_device_quiesce before calling
>>the scsi_driver suspend callback. We resume from quiesce state in
>>reverse order in generic_scsi_resume.
>>
>>ACPI S1 and S4/swsusp are untested, but I think there should be no
>>regressions with S1. To do S1 properly, we probably need to tell the
>>drive to spin down, and I don't know what the SCSI command is for
>>that... For S4, the call to scsi_device_quiesce might pose a problem for
>>the subsequent state dump to disk. But I'm not sure swsusp ever worked
>>for SCSI.
>>    
>>
>
>swsusp will then resume disk and write the image, that should not be a
>problem. Is it guaranteed that after generic_scsi_suspend() no DMA is
>going on?
>  
>
No. Remember that DMA works differently under SCSI than it does under 
IDE. SCSI DMA is a host controller feature, whereas under IDE it is 
enabled/disabled at the drive level and the drives have special 
knowledge of DMA. Since generic_scsi_suspend() is the device level 
suspend routine, it is called before the host controller's suspend 
routine, (due to depth first traversal of device tree), which is 
responsible for disabling the PCI slot. Only after the host controller 
is suspended will there be no DMA, but if your real question is "can I 
generically control a SCSI disk with PIO for software suspend" then the 
answer is NO. For purposes of not suspending the drivers, I haven't 
looked into how swsusp would see which host adapter owns which drive, 
but some of the required information seems to be present in sysfs.

Now, I don't know how disabling DMA is supposed to behave under SATA. 
There may be something extra we have to do that isn't really standard 
SCSI... SATA is a weird beast and if somebody is using libata it seems 
it might not be possible to use PIO, since enhanced mode might have to 
be enabled at the BIOS. But I don't know this stuff, ask jgarzik.

>Anyway, you should try swsusp, preferably on some IDE notebook first
>and prefereably -mm one, to get feel how it works. It should be
>possible/easy to make it work with SCSI...
>  
>
Does it depend on IDE PIO for some reason? Perhaps related to writing 
out a consistent memory image? I'm not sure if I'm motivated to do 
anything about that ;)

>>This might help SATA drives, too, but I seem to remember that the SATA
>>layer doesn't properly emulate the SYNCHRONIZE_CACHE command.
>>
>>Comments, anybody? Can this be applied upstream? I think it's a step in
>>the right direction.
>>    
>>
>
>Looks good to me.
>								Pavel
>  
>
Thanks.
Nathan

Re: [PATCH] SCSI midlayer power management

[Nathan Bryant]

Alan Cox wrote:

>On Mer, 2004-08-11 at 00:56, Nathan Bryant wrote:
>  
>
>>This might help SATA drives, too, but I seem to remember that the SATA 
>>layer doesn't properly emulate the SYNCHRONIZE_CACHE command.
>>    
>>
>
>That was something Mark Lord reported higher level I suspect - which is
>that the scsi path is disabled before the sync cache command is sent so
>the command is always errored before it hits the drive
>  
>
That applies to shutdown, I have no idea whether it also applies to suspend.

Thanks
Nathan

Re: [PATCH] SCSI midlayer power management

[Alan Cox]

On Mer, 2004-08-11 at 00:56, Nathan Bryant wrote:
> This might help SATA drives, too, but I seem to remember that the SATA 
> layer doesn't properly emulate the SYNCHRONIZE_CACHE command.

That was something Mark Lord reported higher level I suspect - which is
that the scsi path is disabled before the sync cache command is sent so
the command is always errored before it hits the drive

Re: [PATCH] SCSI midlayer power management

[Pavel Machek]

Hi!

> This proposed patch implements enough power-management support within
> the SCSI midlayer to get ACPI S3 working on my system. Changes as follows:
> 
> * Add generic_scsi_{suspend,resume} methods to scsi.c
> * Add suspend and resume callbacks to the scsi_driver structure, and
> implement those callbacks in sd.c
> * In sd.c, we call sd_shutdown on suspend, in order to synchronize the
> write-back cache.
> * In sd.c, we call sd_rescan from sd_resume in order to ensure that
> drives have spun up and avoid passing not ready errors back to the block
> layer.
> * In generic_scsi_suspend, we call scsi_device_quiesce before calling
> the scsi_driver suspend callback. We resume from quiesce state in
> reverse order in generic_scsi_resume.
> 
> ACPI S1 and S4/swsusp are untested, but I think there should be no
> regressions with S1. To do S1 properly, we probably need to tell the
> drive to spin down, and I don't know what the SCSI command is for
> that... For S4, the call to scsi_device_quiesce might pose a problem for
> the subsequent state dump to disk. But I'm not sure swsusp ever worked
> for SCSI.

swsusp will then resume disk and write the image, that should not be a
problem. Is it guaranteed that after generic_scsi_suspend() no DMA is
going on?

Anyway, you should try swsusp, preferably on some IDE notebook first
and prefereably -mm one, to get feel how it works. It should be
possible/easy to make it work with SCSI...

> This might help SATA drives, too, but I seem to remember that the SATA
> layer doesn't properly emulate the SYNCHRONIZE_CACHE command.
> 
> Comments, anybody? Can this be applied upstream? I think it's a step in
> the right direction.

Looks good to me.
								Pavel
-- 
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!

[PATCH] SCSI midlayer power management

[Nathan Bryant]

[-- Attachment #1: Type: text/plain, Size: 1389 bytes --]


[resend] oops, sorry, forgot to send the patch!

Hi,

This proposed patch implements enough power-management support within
the SCSI midlayer to get ACPI S3 working on my system. Changes as follows:

* Add generic_scsi_{suspend,resume} methods to scsi.c
* Add suspend and resume callbacks to the scsi_driver structure, and
implement those callbacks in sd.c
* In sd.c, we call sd_shutdown on suspend, in order to synchronize the
write-back cache.
* In sd.c, we call sd_rescan from sd_resume in order to ensure that
drives have spun up and avoid passing not ready errors back to the block
layer.
* In generic_scsi_suspend, we call scsi_device_quiesce before calling
the scsi_driver suspend callback. We resume from quiesce state in
reverse order in generic_scsi_resume.

ACPI S1 and S4/swsusp are untested, but I think there should be no
regressions with S1. To do S1 properly, we probably need to tell the
drive to spin down, and I don't know what the SCSI command is for
that... For S4, the call to scsi_device_quiesce might pose a problem for
the subsequent state dump to disk. But I'm not sure swsusp ever worked
for SCSI.

This might help SATA drives, too, but I seem to remember that the SATA
layer doesn't properly emulate the SYNCHRONIZE_CACHE command.

Comments, anybody? Can this be applied upstream? I think it's a step in
the right direction.

Applies to scsi-misc-2.6

Nathan


[-- Attachment #2: scsi-powermanage.patch --]
[-- Type: text/x-patch, Size: 3765 bytes --]

===== drivers/scsi/scsi.c 1.145 vs edited =====
--- 1.145/drivers/scsi/scsi.c	2004-06-19 10:38:34 -04:00
+++ edited/drivers/scsi/scsi.c	2004-08-10 19:31:45 -04:00
@@ -60,6 +60,7 @@
 #include <scsi/scsi_cmnd.h>
 #include <scsi/scsi_dbg.h>
 #include <scsi/scsi_device.h>
+#include <scsi/scsi_driver.h>
 #include <scsi/scsi_eh.h>
 #include <scsi/scsi_host.h>
 #include <scsi/scsi_tcq.h>
@@ -1175,6 +1176,40 @@
 #define register_scsi_cpu()
 #define unregister_scsi_cpu()
 #endif /* CONFIG_HOTPLUG_CPU */
+
+#ifdef CONFIG_PM
+int generic_scsi_suspend(struct device *dev, u32 state)
+{
+	int err;
+	struct scsi_device *sdev = to_scsi_device(dev);
+	struct scsi_driver *drv = to_scsi_driver(dev->driver);
+
+	err = scsi_device_quiesce(sdev);
+	if (err)
+		return err;
+
+	if (drv->suspend)
+		return drv->suspend(dev, state);
+
+	return 0;
+}
+
+int generic_scsi_resume(struct device *dev)
+{
+	int err;
+	struct scsi_device *sdev = to_scsi_device(dev);
+	struct scsi_driver *drv = to_scsi_driver(dev->driver);
+
+	if (drv->resume) {
+		err = drv->resume(dev);
+		if (err)
+			return err;
+	}
+
+	scsi_device_resume(sdev);
+	return 0;
+}
+#endif /*CONFIG_PM*/
 
 MODULE_DESCRIPTION("SCSI core");
 MODULE_LICENSE("GPL");
===== drivers/scsi/scsi_priv.h 1.33 vs edited =====
--- 1.33/drivers/scsi/scsi_priv.h	2004-06-16 11:45:44 -04:00
+++ edited/drivers/scsi/scsi_priv.h	2004-08-09 20:07:35 -04:00
@@ -92,6 +92,10 @@
 static inline void scsi_log_completion(struct scsi_cmnd *cmd, int disposition)
 	{ };
 #endif
+#ifdef CONFIG_PM
+extern int generic_scsi_suspend(struct device *dev, u32 state);
+extern int generic_scsi_resume(struct device *dev);
+#endif
 
 /* scsi_devinfo.c */
 extern int scsi_get_device_flags(struct scsi_device *sdev,
===== drivers/scsi/scsi_sysfs.c 1.52 vs edited =====
--- 1.52/drivers/scsi/scsi_sysfs.c	2004-07-28 23:59:10 -04:00
+++ edited/drivers/scsi/scsi_sysfs.c	2004-08-10 17:43:04 -04:00
@@ -187,8 +187,12 @@
 }
 
 struct bus_type scsi_bus_type = {
-        .name		= "scsi",
-        .match		= scsi_bus_match,
+	.name		= "scsi",
+	.match		= scsi_bus_match,
+#ifdef CONFIG_PM
+	.suspend	= generic_scsi_suspend,
+	.resume		= generic_scsi_resume,
+#endif
 };
 
 int scsi_sysfs_register(void)
===== drivers/scsi/sd.c 1.154 vs edited =====
--- 1.154/drivers/scsi/sd.c	2004-06-19 10:38:40 -04:00
+++ edited/drivers/scsi/sd.c	2004-08-10 19:33:15 -04:00
@@ -110,6 +110,10 @@
 
 static int sd_probe(struct device *);
 static int sd_remove(struct device *);
+#ifdef CONFIG_PM
+static int sd_suspend(struct device *, u32);
+static int sd_resume(struct device *);
+#endif
 static void sd_shutdown(struct device *dev);
 static void sd_rescan(struct device *);
 static int sd_init_command(struct scsi_cmnd *);
@@ -126,6 +130,10 @@
 	},
 	.rescan			= sd_rescan,
 	.init_command		= sd_init_command,
+#ifdef CONFIG_PM
+	.suspend		= sd_suspend,
+	.resume			= sd_resume,
+#endif
 };
 
 /* Device no to disk mapping:
@@ -1549,7 +1557,21 @@
 	
 	scsi_release_request(sreq);
 	printk("\n");
-}	
+}
+
+#ifdef CONFIG_PM
+static int sd_suspend(struct device *dev, u32 state)
+{
+	sd_shutdown(dev);
+	return 0;
+}
+
+static int sd_resume(struct device *dev)
+{
+	sd_rescan(dev);
+	return 0;
+}
+#endif /*CONFIG_PM*/
 
 /**
  *	init_sd - entry point for this driver (both when built in or when
===== include/scsi/scsi_driver.h 1.2 vs edited =====
--- 1.2/include/scsi/scsi_driver.h	2003-11-12 09:15:46 -05:00
+++ edited/include/scsi/scsi_driver.h	2004-08-10 18:02:45 -04:00
@@ -13,6 +13,10 @@
 
 	int (*init_command)(struct scsi_cmnd *);
 	void (*rescan)(struct device *);
+#ifdef CONFIG_PM
+	int (*suspend)(struct device *dev, u32 state);
+	int (*resume)(struct device *dev);
+#endif
 };
 #define to_scsi_driver(drv) \
 	container_of((drv), struct scsi_driver, gendrv)

[PATCH] SCSI midlayer power management

[Nathan Bryant]

Hi,

This proposed patch implements enough power-management support within 
the SCSI midlayer to get ACPI S3 working on my system. Changes as follows:

* Add generic_scsi_{suspend,resume} methods to scsi.c
* Add suspend and resume callbacks to the scsi_driver structure, and 
implement those callbacks in sd.c
* In sd.c, we call sd_shutdown on suspend, in order to synchronize the 
write-back cache.
* In sd.c, we call sd_rescan from sd_resume in order to ensure that 
drives have spun up and avoid passing not ready errors back to the block 
layer.
* In generic_scsi_suspend, we call scsi_device_quiesce before calling 
the scsi_driver suspend callback. We resume from quiesce state in 
reverse order in generic_scsi_resume.

ACPI S1 and S4/swsusp are untested, but I think there should be no 
regressions with S1. To do S1 properly, we probably need to tell the 
drive to spin down, and I don't know what the SCSI command is for 
that... For S4, the call to scsi_device_quiesce might pose a problem for 
the subsequent state dump to disk. But I'm not sure swsusp ever worked 
for SCSI.

This might help SATA drives, too, but I seem to remember that the SATA 
layer doesn't properly emulate the SYNCHRONIZE_CACHE command.

Comments, anybody? Can this be applied upstream? I think it's a step in 
the right direction.

Applies to scsi-misc-2.6

Nathan