Re: Disk write cache (Was: Hyper-Threading Vulnerability)

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Robert Hancock]

Richard B. Johnson wrote:
> Then I suggest you never use such a drive. Anything that does this,
> will end up replacing a good track with garbage. Unless a disk drive
> has a built-in power source such as super-capacitors or batteries, what
> happens during a power-failure is that all electronics stops and
> the discs start coasting. Eventually the heads will crash onto

If the power to the drive is truly just cut, then this is basically what 
will happen. However, I have heard, for what it's worth, that in many 
cases if you pull the AC power from a typical PC, the Power Good signal 
from the PSU will be de-asserted, which triggers the Reset line on all 
the buses, which triggers the ATA reset line, which triggers the drive 
to finish writing out the sector it is doing. There is likely enough 
capacitance in the power supply to do that before the voltage drops off.

> the platter. Older discs had a magnetically released latch which would
> send the heads to an inside landing zone. Nobody bothers anymore.

Sure they do. All current or remotely recent drives (to my knowledge, 
anyway) will park the heads properly at the landing zone on power-off. 
If the drive is told to power off cleanly, this works as expected, and 
if the power is simply cut, the remaining energy in the spinning 
platters is used like a generator to provide power to move the head 
actuator to the park positon.

-- 
Robert Hancock      Saskatoon, SK, Canada
To email, remove "nospam" from hancockr@nospamshaw.ca
Home Page: http://www.roberthancock.com/

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Eric D. Mudama]

On 5/16/05, Robert Hancock <hancockr@shaw.ca> wrote:
> If the power to the drive is truly just cut, then this is basically what
> will happen. However, I have heard, for what it's worth, that in many
> cases if you pull the AC power from a typical PC, the Power Good signal
> from the PSU will be de-asserted, which triggers the Reset line on all
> the buses, which triggers the ATA reset line, which triggers the drive
> to finish writing out the sector it is doing. There is likely enough
> capacitance in the power supply to do that before the voltage drops off.

Yes, but as you said this isn't a power loss event.  It is a hard
reset with a full write cache, which all drives on the market today
respond to by flushing the cache.

According to the spec the time to flush can exceed 30s, so your PSU
better have some honkin caps on it to ensure data integrity when you
yank the power cord out of the wall.

--eric

RE: Disk write cache (Was: Hyper-Threading Vulnerability)

[Lincoln Dale (ltd)]

 

> -----Original Message-----
> From: John Stoffel [mailto:john@stoffel.org] 
> Sent: Wednesday, 18 May 2005 11:49 PM
> To: Lincoln Dale (ltd)
> Cc: Eric D. Mudama; Robert Hancock; linux-kernel
> Subject: RE: Disk write cache (Was: Hyper-Threading Vulnerability)
> 
> >>>>> "Lincoln" == Lincoln Dale \(ltd\) <Lincoln> writes:
> 
> Lincoln> why don't drive vendors create firmware which reserved a 
> Lincoln> cache-sized (e.g. 2MB) hole of internal drive space 
> somewhere 
> Lincoln> for such an event, and a "cache flush caused by hard-reset"
> Lincoln> simply caused it to write the cache to a fixed (contiguous) 
> Lincoln> area of disk.
> 
> Well, if you're losing power in the next Xmilliseconds, do 
> you have the time to seek to the cache holding area and 
> settle down the head (since you could have done a seek from 
> the edge of the disk to the middle), start writing, etc? 

I believe its possible.
rationale:

 [1] ATX power specification, (google finds this for me at
http://www.formfactors.org/developer%5Cspecs%5CATX12V_1_3dg.pdf)
     section 3.2.11 (Voltage Hold-up time) states:

	The power supply should maintain output regulation per Section
3.2.1 despite a loss of input
	power at the low-end nominal range-115 VAC / 57 Hz or 230 VAC /
47 Hz-at maximum
	continuous output load as applicable for a minimum of 17 ms.

     the assumption here is that T6 in figure 5 does de-assert the
POWER_OK signal early in that "minimum of 17ms".
     the spec (unfortunately) only calls for >=1msec.

     once again, i see that there could be a market for a combination of
p/s & peripherals that could make use of it.
     lets say that we DO have 17msec.

 [2] Hard drive response times
     picking a 'standard' high-end hard drive (Maxtor Atlas 10K V scsi
disk):

	average seek + rotional latency is measured at 7.6msec.
	transfer rates at beginning of disk are 89.5MB/s at end of disk
are 53.9MB/s.
      (source
http://www.storagereview.com/articles/200411/200411028D300L0_2.html)

     allowing 8msec for seek time, and writing at the 'slow' side of the
disk, writing 2MB
     could take ~37msec (2 / 53.9).  allow 50% overhead here - and we
have 55msec.

     55 + 8 = 63 msec.

ok - 63msec doesn't fit into 17msec -
but as i say, a combination of p/s and/or larger caps (and/or more
innovative design by a case or p/s manufactuer which creates a dedicated
peripheral power bus)

> Seems better to have a cache sized flash ram instead where 
> you could just keep the data there in case of power loss.  
> 
> But that's expensive, and not something most people need...

indeed, and that is what MS have been targeting. (flash isn't that
expensive, but flash write times are..).



cheers,

lincoln.


> 
> John
> 

RE: Disk write cache (Was: Hyper-Threading Vulnerability)

[John Stoffel]

>>>>> "Lincoln" == Lincoln Dale \(ltd\) <Lincoln> writes:

Lincoln> why don't drive vendors create firmware which reserved a
Lincoln> cache-sized (e.g. 2MB) hole of internal drive space somewhere
Lincoln> for such an event, and a "cache flush caused by hard-reset"
Lincoln> simply caused it to write the cache to a fixed (contiguous)
Lincoln> area of disk.

Well, if you're losing power in the next Xmilliseconds, do you have
the time to seek to the cache holding area and settle down the head
(since you could have done a seek from the edge of the disk to the
middle), start writing, etc?  Seems better to have a cache sized flash
ram instead where you could just keep the data there in case of power
loss.  

But that's expensive, and not something most people need...

John

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Stephan Wonczak]

Paul Zimmerman wrote:
> On May 17, 2005, at 21:41:39, Kyle Moffett wrote:
> 
>>I've seen some articles recently on a micro-punchcard technology that  uses
>>grids of thousands of miniature needles and sheets of polymer plastic
> 
> 
> Bwa-ha-ha! That's rich. You should have saved that one for next April
> 1st!
> Does it use micro-relay logic to drive the micro-punchcard reader? Or
> does it have nano-technology vacuum tube logic circuits?
> 
> Good one.

    No, actually. That one's for real. See:

    http://www.zurich.ibm.com/st/storage/millipede.html

    Looks like it will be the next generation of storage after rotating
discs.
    (*grumble* ... forgot to hit 'reply all'. Sorry!)

		Stephan Wonczak
		
     "I haven't lost my mind; I know exactly where I left it."
     "The meaning of my life is to make me crazy"

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Richard B. Johnson]

On Wed, 18 May 2005, Paul Zimmerman wrote:

> On May 17, 2005, at 21:41:39, Kyle Moffett wrote:
>> I've seen some articles recently on a micro-punchcard technology that  uses
>> grids of thousands of miniature needles and sheets of polymer plastic
>
> Bwa-ha-ha! That's rich. You should have saved that one for next April
> 1st!
> Does it use micro-relay logic to drive the micro-punchcard reader? Or
> does it have nano-technology vacuum tube logic circuits?
>
> Good one.
>
> -Paul

Actually carbon nano tubes, vacuum tubes not needed! May need a
"filament" transformer, though ^:


Cheers,
Dick Johnson
Penguin : Linux version 2.6.11.9 on an i686 machine (5537.79 BogoMips).
  Notice : All mail here is now cached for review by Dictator Bush.
                  98.36% of all statistics are fiction.

RE: Disk write cache (Was: Hyper-Threading Vulnerability)

[Lincoln Dale (ltd)]

Eric,

> On 5/16/05, Robert Hancock <hancockr@shaw.ca> wrote:
> > If the power to the drive is truly just cut, then this is basically 
> > what will happen. However, I have heard, for what it's 
> worth, that in 
> > many cases if you pull the AC power from a typical PC, the 
> Power Good 
> > signal from the PSU will be de-asserted, which triggers the 
> Reset line 
> > on all the buses, which triggers the ATA reset line, which triggers 
> > the drive to finish writing out the sector it is doing. There is 
> > likely enough capacitance in the power supply to do that 
> before the voltage drops off.
> 
> Yes, but as you said this isn't a power loss event.  It is a 
> hard reset with a full write cache, which all drives on the 
> market today respond to by flushing the cache.
> 
> According to the spec the time to flush can exceed 30s, so 
> your PSU better have some honkin caps on it to ensure data 
> integrity when you yank the power cord out of the wall.

why don't drive vendors create firmware which reserved a cache-sized
(e.g. 2MB) hole of internal drive space somewhere for such an event, and
a "cache flush caused by hard-reset" simply caused it to write the cache
to a fixed (contiguous) area of disk.

the same drive firmware on power-on could check that area and 'write
back' the data to the correct locations.

all said and done, why wouldn't a vendor (lets just say "Maxtor" :) )
implement something like this and market it as a feature?
i'd happily spend a few extra bucks for something that given a modern
PSU providing a few Hz of power (e.g. 50msec) provided higher data
reliability in case of power failure..


cheers,

lincoln.

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Paul Zimmerman]

On May 17, 2005, at 21:41:39, Kyle Moffett wrote:
>I've seen some articles recently on a micro-punchcard technology that  uses
>grids of thousands of miniature needles and sheets of polymer plastic

Bwa-ha-ha! That's rich. You should have saved that one for next April
1st!
Does it use micro-relay logic to drive the micro-punchcard reader? Or
does it have nano-technology vacuum tube logic circuits?

Good one.

-Paul

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today - it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Eric D. Mudama]

On 5/16/05, Matthias Andree <matthias.andree@gmx.de> wrote:
> I've heard that drives would be taking rotational energy from their
> rotating platters and such, but never heard how the hardware compensates
> the dilation with decreasing rotational frequency, which also requires
> changed filter settings for the write channel, block encoding, delays,
> possibly stepping the heads and so on. I don't believe these stories
> until I see evidence.

I'm pretty sure that most drives out there will immediately attempt to
safely retract or park the heads the instant that a power loss is
detected.  There's too much potential damage that can occur if the
heads aren't able to safely retract to a landing zone or ramp, that
trying to save "one more block of cached data" just isn't worth the
risk.

--eric

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Eric D. Mudama]

On 5/15/05, Mark Lord <lkml@rtr.ca> wrote:
> There's your clue.  The drive LEDs normally reflect activity
> over the ATA bus (the cable!). If they're not on, then the drive
> isn't receiving data/commands from the host.

Mark is correct, activity indicators are associated with bus activity,
not internal drive activity.

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Kyle Moffett]

On May 17, 2005, at 09:15:52, Bill Davidsen wrote:
> What would be ideal is some cache which didn't depend on power to  
> maintain
> state, like core (remember core?) or the bubble memory which spent  
> almost
> a decade being just slightly too {slow,costly} to replace disk. There
> doesn't seem to be a cost effective technology yet.

I've seen some articles recently on a micro-punchcard technology that  
uses
grids of thousands of miniature needles and sheets of polymer plastic  
that
can be melted at somewhat low temperatures to create or remove  
indentations
in the plastic.  The device can read and write each position at a  
very high
rate, and since there are several thousand bits per position, with  
one bit
for each needle, the bandwidth is enormous.  (And it scales linearly  
with
the size of the device, too!)  Purportedly these grids can be easily  
built
with slight modifications to modern semiconductor etching  
technologies, and
the polymer plastic is reasonably simple to manufacture, so the  
resultant
cost per device is hundreds of times cheaper than today's drives.   
Likewise,
they have significantly higher memory density than current hardware  
due to
fewer relativistic and quantum effects (no magnetism).


Cheers,
Kyle Moffett

-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCM/CS/IT/U d- s++: a18 C++++>$ UB/L/X/*++++(+)>$ P+++(++++)>$
L++++(+++) E W++(+) N+++(++) o? K? w--- O? M++ V? PS+() PE+(-) Y+
PGP+++ t+(+++) 5 X R? tv-(--) b++++(++) DI+ D+ G e->++++$ h!*()>++$  
r  !y?(-)
------END GEEK CODE BLOCK------

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Bill Davidsen]

On Mon, 16 May 2005, Alan Cox wrote:

> > flashes its cache to NVRAM, or uses rotational energy to save its cache
> > on the platters, please name brand and model and where I can download
> > the material that documents this behavior.
> 
> I am not aware of any IDE drive with these properties.

I'm not sure I know of a SCSI drive which does that, either. It was a big
thing a few decades to use rotational energy to park the heads, but I
haven't seen discussion of save to nvram. Then again, I haven't been
looking for it.

What would be ideal is some cache which didn't depend on power to maintain
state, like core (remember core?) or the bubble memory which spent almost
a decade being just slightly too {slow,costly} to replace disk. There
doesn't seem to be a cost effective technology yet.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 332 bytes --]

On Mon, 16 May 2005 10:33:30 EDT, Jeff Garzik said:

> Upon power loss, at least one ATA vendor's disks try to write out as 
> much data as possible.

Does the firmware for this vendor's disks have enough smarts to reserve that
last little bit of power to park the heads so it's not actively writing when
it finally loses entirely?

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Richard B. Johnson]

On Mon, 16 May 2005, Jeff Garzik wrote:

> Matthias Andree wrote:
>> On Sun, 15 May 2005, Mikulas Patocka wrote:
>>
>>
>>> Note that disk can still ignore FLUSH CACHE command cached data are small
>>> enough to be written on power loss, so small FLUSH CACHE time doesn't
>>> prove disk cheating.
>>
>> Have you seen a drive yet that writes back blocks after power loss?
>>
>> I have heard rumors about this, but all OEM manuals I looked at for
>> drives I bought or recommended simply stated that the block currently
>> being written at power loss can become damaged (with write cache off),
>> and that the drive can lose the full write cache at power loss (with
>> write cache on) so this looks like daydreaming manifested as rumor.
>
> Upon power loss, at least one ATA vendor's disks try to write out as
> much data as possible.
>
> 	Jeff

Then I suggest you never use such a drive. Anything that does this,
will end up replacing a good track with garbage. Unless a disk drive
has a built-in power source such as super-capacitors or batteries, what
happens during a power-failure is that all electronics stops and
the discs start coasting. Eventually the heads will crash onto
the platter. Older discs had a magnetically released latch which would
send the heads to an inside landing zone. Nobody bothers anymore.

Many high-quality drives cache data. Fortunately, upon power loss
these data are NOT attempted to be written. This means that,
although you may have incomplete or even bad data on the physical
medium, at least the medium can be read and written. The sectoring
has not been corrupted (read destroyed).

If you think about the physical process necessary to write data to
the medium, you will understand that without a large amount of
energy storage capacity on the disk, it's just not possible.

To write a sector, one needs to cache the data in a sector-buffer
putting on a sector header and trailing CRC, wait for the write-
splice from the previous sector (could be almost one rotation),
then write data and sync to the sector. If the disc is too slow,
these data will be underwrite the sector. Also, if the disc
was only 5 percent slow, the clock recovery on a subsequent
read will be off by 5 percent, outside the range of PLL lock-in,
so you write something that can never be read, a guaranteed bad block.

Combinations of journalizing on media that can be completely flushed,
and ordinary cache-intensive discs can result in reliable data
storage. However a single ATA or SCSI disk just isn't a perfectly
reliable storage medium although it's usually good enough.

Cheers,
Dick Johnson
Penguin : Linux version 2.6.11 on an i686 machine (5537.79 BogoMips).
  Notice : All mail here is now cached for review by Dictator Bush.
                  98.36% of all statistics are fiction.

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Alan Cox]

> I have heard rumors about this, but all OEM manuals I looked at for
> drives I bought or recommended simply stated that the block currently
> being written at power loss can become damaged (with write cache off),
> and that the drive can lose the full write cache at power loss (with
> write cache on) so this looks like daydreaming manifested as rumor.

IBM drives definitely used to trash the sector in this case. They newer
ones either don't or recover from it presumably because people took that
to be a drive failure and returned it. Sometimes the people win ;)

> flashes its cache to NVRAM, or uses rotational energy to save its cache
> on the platters, please name brand and model and where I can download
> the material that documents this behavior.

I am not aware of any IDE drive with these properties.

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Alan Cox]

On Sul, 2005-05-15 at 16:00, Mikulas Patocka wrote:
> There are rumors that some disks ignore FLUSH CACHE command just to get
> higher benchmarks in Windows. But I haven't heart of any proof. Does
> anybody know, what companies fake this command?

The specification was intentionally written so that his command has to
do what it is specified to or be unknown and thus error and not be in
the ident info.

That was done by people who wanted to be very sure that any vendor who
tried to shortcut the command would have "sue me" written on their
forehead.

There are problems with a few older drives which have a write cache but
don't support cache commands.

Alan

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Jeff Garzik]

Matthias Andree wrote:
> On Sun, 15 May 2005, Mikulas Patocka wrote:
> 
> 
>>Note that disk can still ignore FLUSH CACHE command cached data are small
>>enough to be written on power loss, so small FLUSH CACHE time doesn't
>>prove disk cheating.
> 
> 
> Have you seen a drive yet that writes back blocks after power loss?
> 
> I have heard rumors about this, but all OEM manuals I looked at for
> drives I bought or recommended simply stated that the block currently
> being written at power loss can become damaged (with write cache off),
> and that the drive can lose the full write cache at power loss (with
> write cache on) so this looks like daydreaming manifested as rumor.

Upon power loss, at least one ATA vendor's disks try to write out as 
much data as possible.

	Jeff

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Mark Lord]

Gene Heskett wrote:
>
>>There's your clue.  The drive LEDs normally reflect activity
>>over the ATA bus (the cable!). If they're not on, then the drive
>>isn't receiving data/commands from the host.
>
> That was my theory too Mark, but Jeff G. says its not a valid 
> indicator.  So who's right?

If the LEDs are connected to the controller on the motherboard,
then they are a strict indication of activity over the cable
between the drive and controller (if they function at all).
But it is possible for software to leave those LEDs permanently
in the "on" state, depending on the register sequence used.

If the LEDs are on the drive itself, they may indicate transfers
over the connector (cable) -- usually always the case -- or they
could indicate transfers to/from the media.

Cheers

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Matthias Andree]

On Sun, 15 May 2005, Mikulas Patocka wrote:

> Note that disk can still ignore FLUSH CACHE command cached data are small
> enough to be written on power loss, so small FLUSH CACHE time doesn't
> prove disk cheating.

Have you seen a drive yet that writes back blocks after power loss?

I have heard rumors about this, but all OEM manuals I looked at for
drives I bought or recommended simply stated that the block currently
being written at power loss can become damaged (with write cache off),
and that the drive can lose the full write cache at power loss (with
write cache on) so this looks like daydreaming manifested as rumor.

I've heard that drives would be taking rotational energy from their
rotating platters and such, but never heard how the hardware compensates
the dilation with decreasing rotational frequency, which also requires
changed filter settings for the write channel, block encoding, delays,
possibly stepping the heads and so on. I don't believe these stories
until I see evidence.

These are corner cases that a vendor would hardly optimize for.
If you know a disk drive (not battery-backed disk controller!) that
flashes its cache to NVRAM, or uses rotational energy to save its cache
on the platters, please name brand and model and where I can download
the material that documents this behavior.

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Gene Heskett]

On Sunday 15 May 2005 22:32, Mark Lord wrote:
> >took place on another list, and wrote a test gizmo that copied a
> >large file, then slept for 1 second and issued a sync command.  No
> >drive led activity until the usual 5 second delay of the
> > filesystem had expired.  To me, that indicated that the sync
> > command was being
>
>There's your clue.  The drive LEDs normally reflect activity
>over the ATA bus (the cable!). If they're not on, then the drive
>isn't receiving data/commands from the host.
>
>Cheers

That was my theory too Mark, but Jeff G. says its not a valid 
indicator.  So who's right?

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
99.34% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2005 by Maurice Eugene Heskett, all rights reserved.

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Gene Heskett]

On Sunday 15 May 2005 22:24, Mikulas Patocka wrote:
>On Sun, 15 May 2005, Gene Heskett wrote:
>> >There is a large amount of yammering and speculation in this
>> > thread.
>>
>> I agree, and frankly I'm just another  of the yammerers as I don't
>> have the clout to be otherwise.
>>
>> >Most disks do seem to obey SYNC CACHE / FLUSH CACHE.
>> >
>> > Jeff
>>
>> I don't think I have any drives here that do obey that, Jeff.  I
>> got curious about this, oh, maybe a year back when this discussion
>> first took place on another list, and wrote a test gizmo that
>> copied a large file, then slept for 1 second and issued a sync
>> command.  No drive led activity until the usual 5 second delay of
>> the filesystem had expired.  To me, that indicated that the sync
>> command was being returned as completed without error and I had my
>> shell prompt back long before the drives leds came on.  Admittedly
>> that may not be a 100% valid test, but I really did expect to see
>> the leds come on as the sync command was executed.
>>
>> I also have some setup stuff for heyu that runs at various times
>> of the day, reconfigureing how heyu and xtend run 3 times a day
>> here, which depends on a valid disk file, and I've had to use
>> sleeps for guaranteeing the proper sequencing, where if the sync
>> command actually worked, I could get the job done quite a bit
>> faster.
>>
>> Again, probably not a valid test of the sync command, but thats
>> the evidence I have.  I do not believe it works here, with any of
>> the 5 drives currently spinning in these two boxes.
>
>Note, that Linux can't send FLUSH CACHE command at all (until very
> recent 2.6 kernels). So write cache is always dangerous under
> Linux, no matter if disk is broken or not.
>
>Another note: according to posix, sync() is asynchronous --- i.e. it
>initiates write, but doesn't have to wait for complete. In linux,
> sync() waits for writes to complete, but it doesn't have to in
> other OSes.
>
>Mikulas
>
Humm, I'm getting the impression I should rerun that test script if I 
can find it.  I believe the last time I tried it, I was running a 
2.4.x kernel, right now 2.6.12-rc1.

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
99.34% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2005 by Maurice Eugene Heskett, all rights reserved.

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Mark Lord]

 >took place on another list, and wrote a test gizmo that copied a
 >large file, then slept for 1 second and issued a sync command.  No
 >drive led activity until the usual 5 second delay of the filesystem
 >had expired.  To me, that indicated that the sync command was being

There's your clue.  The drive LEDs normally reflect activity
over the ATA bus (the cable!). If they're not on, then the drive
isn't receiving data/commands from the host.

Cheers

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Mikulas Patocka]

On Sun, 15 May 2005, Gene Heskett wrote:

> >There is a large amount of yammering and speculation in this thread.
>
> I agree, and frankly I'm just another  of the yammerers as I don't
> have the clout to be otherwise.
>
> >Most disks do seem to obey SYNC CACHE / FLUSH CACHE.
> >
> > Jeff
>
> I don't think I have any drives here that do obey that, Jeff.  I got
> curious about this, oh, maybe a year back when this discussion first
> took place on another list, and wrote a test gizmo that copied a
> large file, then slept for 1 second and issued a sync command.  No
> drive led activity until the usual 5 second delay of the filesystem
> had expired.  To me, that indicated that the sync command was being
> returned as completed without error and I had my shell prompt back
> long before the drives leds came on.  Admittedly that may not be a
> 100% valid test, but I really did expect to see the leds come on as
> the sync command was executed.
>
> I also have some setup stuff for heyu that runs at various times of
> the day, reconfigureing how heyu and xtend run 3 times a day here,
> which depends on a valid disk file, and I've had to use sleeps for
> guaranteeing the proper sequencing, where if the sync command
> actually worked, I could get the job done quite a bit faster.
>
> Again, probably not a valid test of the sync command, but thats the
> evidence I have.  I do not believe it works here, with any of the 5
> drives currently spinning in these two boxes.

Note, that Linux can't send FLUSH CACHE command at all (until very recent
2.6 kernels). So write cache is always dangerous under Linux, no matter if
disk is broken or not.

Another note: according to posix, sync() is asynchronous --- i.e. it
initiates write, but doesn't have to wait for complete. In linux, sync()
waits for writes to complete, but it doesn't have to in other OSes.

Mikulas

> --
> Cheers, Gene
> "There are four boxes to be used in defense of liberty:
>  soap, ballot, jury, and ammo. Please use in that order."
> -Ed Howdershelt (Author)
> 99.34% setiathome rank, not too shabby for a WV hillbilly
> Yahoo.com and AOL/TW attorneys please note, additions to the above
> message by Gene Heskett are:
> Copyright 2005 by Maurice Eugene Heskett, all rights reserved.
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Jeff Garzik]

Gene Heskett wrote:
> I don't think I have any drives here that do obey that, Jeff.  I got 
> curious about this, oh, maybe a year back when this discussion first 
> took place on another list, and wrote a test gizmo that copied a 
> large file, then slept for 1 second and issued a sync command.  No 
> drive led activity until the usual 5 second delay of the filesystem 
> had expired.  To me, that indicated that the sync command was being 
> returned as completed without error and I had my shell prompt back 
> long before the drives leds came on.  Admittedly that may not be a 
> 100% valid test, but I really did expect to see the leds come on as 
> the sync command was executed.

> Again, probably not a valid test of the sync command, but thats the 
> evidence I have.  I do not believe it works here, with any of the 5 
> drives currently spinning in these two boxes.

Correct, that's a pretty poor test.

	Jeff

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Gene Heskett]

On Sunday 15 May 2005 11:29, Jeff Garzik wrote:
>On Sun, May 15, 2005 at 11:21:36AM -0400, Gene Heskett wrote:
>> On Sunday 15 May 2005 11:00, Mikulas Patocka wrote:
>> >On Sun, 15 May 2005, Tomasz Torcz wrote:
>> >> On Sun, May 15, 2005 at 04:12:07PM +0200, Andi Kleen wrote:
>> >> > > > > However they've patched the FreeBSD kernel to
>> >> > > > > "workaround?" it:
>> >> > > > > ftp://ftp.freebsd.org/pub/FreeBSD/CERT/patches/SA-05:09
>> >> > > > >/ht t5.patch
>> >> > > >
>> >> > > > That's a similar stupid idea as they did with the disk
>> >> > > > write cache (lowering the MTBFs of their disks by
>> >> > > > considerable factors, which is much worse than the power
>> >> > > > off data loss problem) Let's not go down this path
>> >> > > > please.
>> >> > >
>> >> > > What wrong did they do with disk write cache?
>> >> >
>> >> > They turned it off by default, which according to disk
>> >> > vendors lowers the MTBF of your disk to a fraction of the
>> >> > original value.
>> >> >
>> >> > I bet the total amount of valuable data lost for FreeBSD
>> >> > users because of broken disks is much much bigger than what
>> >> > they gained from not losing in the rather hard to hit power
>> >> > off cases.
>> >>
>> >>  Aren't I/O barriers a way to safely use write cache?
>> >
>> >FreeBSD used these barriers (FLUSH CACHE command) long time ago.
>> >
>> >There are rumors that some disks ignore FLUSH CACHE command just
>> > to get higher benchmarks in Windows. But I haven't heart of any
>> > proof. Does anybody know, what companies fake this command?
>> >
>> >From a story I read elsewhere just a few days ago, this problem
>> > is
>>
>> virtually universal even in the umpty-bucks 15,000 rpm scsi server
>> drives.  It appears that this is just another way to crank up the
>> numbers and make each drive seem faster than its competition.
>>
>> My gut feeling is that if this gets enough ink to get under the
>> drive makers skins, we will see the issuance of a utility from the
>> makers that will re-program the drives therefore enabling the
>> proper handling of the FLUSH CACHE command.  This would be an
>> excellent chance IMO, to make a bit of noise if the utility comes
>> out, but only runs on windows.  In that event, we hold their feet
>> to the fire (the prefereable method), or a wrapper is written that
>> allows it to run on any os with a bash-like shell manager.
>
>There is a large amount of yammering and speculation in this thread.

I agree, and frankly I'm just another  of the yammerers as I don't 
have the clout to be otherwise.

>Most disks do seem to obey SYNC CACHE / FLUSH CACHE.
>
> Jeff

I don't think I have any drives here that do obey that, Jeff.  I got 
curious about this, oh, maybe a year back when this discussion first 
took place on another list, and wrote a test gizmo that copied a 
large file, then slept for 1 second and issued a sync command.  No 
drive led activity until the usual 5 second delay of the filesystem 
had expired.  To me, that indicated that the sync command was being 
returned as completed without error and I had my shell prompt back 
long before the drives leds came on.  Admittedly that may not be a 
100% valid test, but I really did expect to see the leds come on as 
the sync command was executed.

I also have some setup stuff for heyu that runs at various times of 
the day, reconfigureing how heyu and xtend run 3 times a day here, 
which depends on a valid disk file, and I've had to use sleeps for 
guaranteeing the proper sequencing, where if the sync command 
actually worked, I could get the job done quite a bit faster.

Again, probably not a valid test of the sync command, but thats the 
evidence I have.  I do not believe it works here, with any of the 5 
drives currently spinning in these two boxes.

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
99.34% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2005 by Maurice Eugene Heskett, all rights reserved.

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Tomasz Torcz]

On Sun, May 15, 2005 at 11:21:36AM -0400, Gene Heskett wrote:
> >FreeBSD used these barriers (FLUSH CACHE command) long time ago.
> >
> >There are rumors that some disks ignore FLUSH CACHE command just to
> > get higher benchmarks in Windows. But I haven't heart of any proof.
> > Does anybody know, what companies fake this command?
> >
> >From a story I read elsewhere just a few days ago, this problem is 
> virtually universal even in the umpty-bucks 15,000 rpm scsi server 
> drives.  It appears that this is just another way to crank up the 
> numbers and make each drive seem faster than its competition.

 Probably you talking about this: http://www.livejournal.com/~brad/2116715.html
It has hit Slashdot yesterday.

-- 
Tomasz Torcz                 "God, root, what's the difference?"
zdzichu@irc.-nie.spam-.pl         "God is more forgiving."

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Mikulas Patocka]

On Sun, 15 May 2005, Gene Heskett wrote:

> On Sunday 15 May 2005 11:00, Mikulas Patocka wrote:
> >On Sun, 15 May 2005, Tomasz Torcz wrote:
> >> On Sun, May 15, 2005 at 04:12:07PM +0200, Andi Kleen wrote:
> >> > > > > However they've patched the FreeBSD kernel to
> >> > > > > "workaround?" it:
> >> > > > > ftp://ftp.freebsd.org/pub/FreeBSD/CERT/patches/SA-05:09/ht
> >> > > > >t5.patch
> >> > > >
> >> > > > That's a similar stupid idea as they did with the disk write
> >> > > > cache (lowering the MTBFs of their disks by considerable
> >> > > > factors, which is much worse than the power off data loss
> >> > > > problem) Let's not go down this path please.
> >> > >
> >> > > What wrong did they do with disk write cache?
> >> >
> >> > They turned it off by default, which according to disk vendors
> >> > lowers the MTBF of your disk to a fraction of the original
> >> > value.
> >> >
> >> > I bet the total amount of valuable data lost for FreeBSD users
> >> > because of broken disks is much much bigger than what they
> >> > gained from not losing in the rather hard to hit power off
> >> > cases.
> >>
> >>  Aren't I/O barriers a way to safely use write cache?
> >
> >FreeBSD used these barriers (FLUSH CACHE command) long time ago.
> >
> >There are rumors that some disks ignore FLUSH CACHE command just to
> > get higher benchmarks in Windows. But I haven't heart of any proof.
> > Does anybody know, what companies fake this command?
> >
> From a story I read elsewhere just a few days ago, this problem is
> virtually universal even in the umpty-bucks 15,000 rpm scsi server
> drives.  It appears that this is just another way to crank up the
> numbers and make each drive seem faster than its competition.

I've just made test on my Western Digical 40G IDE disk:

just writes without flush cache: 1min 33sec
same access pattern, but flush cache after each write: 20min 7sec (and
disk made more noise)
(this testcase does many 1-sector writes to the same or adjacent sectors,
so cache helps here a lot)

So it's likely that this disk honours cache flushing.

(but the disk contains another severe bug --- it corrupts it
cache-coherency logic when 256-sector accesses are being used --- I
asked WD about it and got no response. 256 is represented as 0 in IDE
registers --- that's probably where the bug came from).

I've also heard a lot of rumors about ignoring cache flush --- but I mean,
have anybody actually proven that some disk corrupts data this way? i.e.:

make a program that does repeatedly this:
write some sector
issue flush cache command
send a packet about what was written where

... and turn off machine while this program runs and see if disk contains
all the data from packets.

or

write many small sectors
issue flush cache
turn off power via ACPI
on next reboot see, if disk contains all the data


Note that disk can still ignore FLUSH CACHE command cached data are small
enough to be written on power loss, so small FLUSH CACHE time doesn't
prove disk cheating.

Mikulas

> My gut feeling is that if this gets enough ink to get under the drive
> makers skins, we will see the issuance of a utility from the makers
> that will re-program the drives therefore enabling the proper
> handling of the FLUSH CACHE command.  This would be an excellent
> chance IMO, to make a bit of noise if the utility comes out, but only
> runs on windows.  In that event, we hold their feet to the fire (the
> prefereable method), or a wrapper is written that allows it to run on
> any os with a bash-like shell manager.
>
> >Mikulas
> >-
> >To unsubscribe from this list: send the line "unsubscribe
> > linux-kernel" in the body of a message to majordomo@vger.kernel.org
> >More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >Please read the FAQ at  http://www.tux.org/lkml/
>
> --
> Cheers, Gene
> "There are four boxes to be used in defense of liberty:
>  soap, ballot, jury, and ammo. Please use in that order."
> -Ed Howdershelt (Author)
> 99.34% setiathome rank, not too shabby for a WV hillbilly
> Yahoo.com and AOL/TW attorneys please note, additions to the above
> message by Gene Heskett are:
> Copyright 2005 by Maurice Eugene Heskett, all rights reserved.
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Jeff Garzik]

On Sun, May 15, 2005 at 11:21:36AM -0400, Gene Heskett wrote:
> On Sunday 15 May 2005 11:00, Mikulas Patocka wrote:
> >On Sun, 15 May 2005, Tomasz Torcz wrote:
> >> On Sun, May 15, 2005 at 04:12:07PM +0200, Andi Kleen wrote:
> >> > > > > However they've patched the FreeBSD kernel to
> >> > > > > "workaround?" it:
> >> > > > > ftp://ftp.freebsd.org/pub/FreeBSD/CERT/patches/SA-05:09/ht
> >> > > > >t5.patch
> >> > > >
> >> > > > That's a similar stupid idea as they did with the disk write
> >> > > > cache (lowering the MTBFs of their disks by considerable
> >> > > > factors, which is much worse than the power off data loss
> >> > > > problem) Let's not go down this path please.
> >> > >
> >> > > What wrong did they do with disk write cache?
> >> >
> >> > They turned it off by default, which according to disk vendors
> >> > lowers the MTBF of your disk to a fraction of the original
> >> > value.
> >> >
> >> > I bet the total amount of valuable data lost for FreeBSD users
> >> > because of broken disks is much much bigger than what they
> >> > gained from not losing in the rather hard to hit power off
> >> > cases.
> >>
> >>  Aren't I/O barriers a way to safely use write cache?
> >
> >FreeBSD used these barriers (FLUSH CACHE command) long time ago.
> >
> >There are rumors that some disks ignore FLUSH CACHE command just to
> > get higher benchmarks in Windows. But I haven't heart of any proof.
> > Does anybody know, what companies fake this command?
> >
> >From a story I read elsewhere just a few days ago, this problem is 
> virtually universal even in the umpty-bucks 15,000 rpm scsi server 
> drives.  It appears that this is just another way to crank up the 
> numbers and make each drive seem faster than its competition.
> 
> My gut feeling is that if this gets enough ink to get under the drive 
> makers skins, we will see the issuance of a utility from the makers 
> that will re-program the drives therefore enabling the proper 
> handling of the FLUSH CACHE command.  This would be an excellent 
> chance IMO, to make a bit of noise if the utility comes out, but only 
> runs on windows.  In that event, we hold their feet to the fire (the 
> prefereable method), or a wrapper is written that allows it to run on 
> any os with a bash-like shell manager.


There is a large amount of yammering and speculation in this thread.

Most disks do seem to obey SYNC CACHE / FLUSH CACHE.

	Jeff

Re: Disk write cache (Was: Hyper-Threading Vulnerability)

[Gene Heskett]

On Sunday 15 May 2005 11:00, Mikulas Patocka wrote:
>On Sun, 15 May 2005, Tomasz Torcz wrote:
>> On Sun, May 15, 2005 at 04:12:07PM +0200, Andi Kleen wrote:
>> > > > > However they've patched the FreeBSD kernel to
>> > > > > "workaround?" it:
>> > > > > ftp://ftp.freebsd.org/pub/FreeBSD/CERT/patches/SA-05:09/ht
>> > > > >t5.patch
>> > > >
>> > > > That's a similar stupid idea as they did with the disk write
>> > > > cache (lowering the MTBFs of their disks by considerable
>> > > > factors, which is much worse than the power off data loss
>> > > > problem) Let's not go down this path please.
>> > >
>> > > What wrong did they do with disk write cache?
>> >
>> > They turned it off by default, which according to disk vendors
>> > lowers the MTBF of your disk to a fraction of the original
>> > value.
>> >
>> > I bet the total amount of valuable data lost for FreeBSD users
>> > because of broken disks is much much bigger than what they
>> > gained from not losing in the rather hard to hit power off
>> > cases.
>>
>>  Aren't I/O barriers a way to safely use write cache?
>
>FreeBSD used these barriers (FLUSH CACHE command) long time ago.
>
>There are rumors that some disks ignore FLUSH CACHE command just to
> get higher benchmarks in Windows. But I haven't heart of any proof.
> Does anybody know, what companies fake this command?
>
>From a story I read elsewhere just a few days ago, this problem is 
virtually universal even in the umpty-bucks 15,000 rpm scsi server 
drives.  It appears that this is just another way to crank up the 
numbers and make each drive seem faster than its competition.

My gut feeling is that if this gets enough ink to get under the drive 
makers skins, we will see the issuance of a utility from the makers 
that will re-program the drives therefore enabling the proper 
handling of the FLUSH CACHE command.  This would be an excellent 
chance IMO, to make a bit of noise if the utility comes out, but only 
runs on windows.  In that event, we hold their feet to the fire (the 
prefereable method), or a wrapper is written that allows it to run on 
any os with a bash-like shell manager.

>Mikulas
>-
>To unsubscribe from this list: send the line "unsubscribe
> linux-kernel" in the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at  http://www.tux.org/lkml/

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
99.34% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2005 by Maurice Eugene Heskett, all rights reserved.

Disk write cache (Was: Hyper-Threading Vulnerability)

[Mikulas Patocka]

On Sun, 15 May 2005, Tomasz Torcz wrote:

> On Sun, May 15, 2005 at 04:12:07PM +0200, Andi Kleen wrote:
> > > > > However they've patched the FreeBSD kernel to "workaround?" it:
> > > > > ftp://ftp.freebsd.org/pub/FreeBSD/CERT/patches/SA-05:09/htt5.patch
> > > >
> > > > That's a similar stupid idea as they did with the disk write
> > > > cache (lowering the MTBFs of their disks by considerable factors,
> > > > which is much worse than the power off data loss problem)
> > > > Let's not go down this path please.
> > >
> > > What wrong did they do with disk write cache?
> >
> > They turned it off by default, which according to disk vendors
> > lowers the MTBF of your disk to a fraction of the original value.
> >
> > I bet the total amount of valuable data lost for FreeBSD users because
> > of broken disks is much much bigger than what they gained from not losing
> > in the rather hard to hit power off cases.
>
>  Aren't I/O barriers a way to safely use write cache?

FreeBSD used these barriers (FLUSH CACHE command) long time ago.

There are rumors that some disks ignore FLUSH CACHE command just to get
higher benchmarks in Windows. But I haven't heart of any proof. Does
anybody know, what companies fake this command?

Mikulas